I run the crawler, about 16 hours a night, making 478 requests, very slow.I wonder why?When my crawler wasn't running on a messy cloud, it was ok, but Google banned it, so I used cralera, but the efficiency was worrying.
0 Votes
Nestor Toledo posted
about 7 years ago
AdminBest Answer
I run the crawler, about 16 hours a night, making 478 requests, very slow.I wonder why?When my crawler wasn't running on a messy cloud, it was ok, but Google banned it, so I used cralera, but the efficiency was worrying.
0 Votes
Nestor Toledo posted about 7 years ago Admin Best Answer
See solution 1) from: https://support.scrapinghub.com/a/forums/topics/22000010595
0 Votes
1 Comments
Nestor Toledo posted about 7 years ago Admin Answer
See solution 1) from: https://support.scrapinghub.com/a/forums/topics/22000010595
0 Votes
Login to post a comment