When i'm using proxies i sometimes i got timeout error
a
a_voskresenskyi
started a topic
over 6 years ago
I'm trying to scrape toysrus.com. Also i build scrapy spider for it and the problem is next:
It is work perfect without proxy. It scrape 100% pages and very fast.
When i'm trying to scrape through proxy(crawlera) through
time(a couple of times successfully and couple vice versa) it raise timeout(at all works not stable) and i don't know what to
do with this. I'm using next headers:
a_voskresenskyi
I'm trying to scrape toysrus.com. Also i build scrapy spider for it and the problem is next:
It is work perfect without proxy. It scrape 100% pages and very fast. When i'm trying to scrape through proxy(crawlera) through time(a couple of times successfully and couple vice versa) it raise timeout(at all works not stable) and i don't know what to do with this. I'm using next headers:If it needs i can provide api key(maybe you can look at logs).
Can you help me?
2 people have this question