I have been trying to transfer my local scraping to cloud.
The site I've been working on is sahibinden [dot] com.
Locally I can scrape with scrapy-fake-useragent.
But in scrapinghub I can't get it to scrape. it responds with 200, sometimes 301 but never gives out any results. I have simplified the spider, I have tried everything that occurred to me but alas, I have to ask for help.
Hello,
I have been trying to transfer my local scraping to cloud.
The site I've been working on is sahibinden [dot] com.
Locally I can scrape with scrapy-fake-useragent.
But in scrapinghub I can't get it to scrape. it responds with 200, sometimes 301 but never gives out any results. I have simplified the spider, I have tried everything that occurred to me but alas, I have to ask for help.
Attachments (2)
sahbot.py
1.31 KB
logsahbot16.txt
4.2 KB
0 Votes
0 Comments
Login to post a comment