I'm trying to scrape toysrus.com. Also i build scrapy spider for it and the problem is next:
It is work perfect without proxy. It scrape 100% pages and very fast.
When i'm trying to scrape through proxy(crawlera) through
time(a couple of times successfully and couple vice versa) it raise timeout(at all works not stable) and i don't know what to
do with this. I'm using next headers:
I'm trying to scrape toysrus.com. Also i build scrapy spider for it and the problem is next:
It is work perfect without proxy. It scrape 100% pages and very fast. When i'm trying to scrape through proxy(crawlera) through time(a couple of times successfully and couple vice versa) it raise timeout(at all works not stable) and i don't know what to do with this. I'm using next headers:If it needs i can provide api key(maybe you can look at logs).
Can you help me?
2 Votes
0 Comments
Login to post a comment