videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic

inconsistent behaviour

For some reason every other run of the spider with the same settings returns different results. Project -https://app.scrapinghub.com/p/396740/jobs


In details it  crawls different number of pages and returns different number of items.

I assume the issue is in ip used for crawl?

Running from PC returns more items and crawl statistics is consistent.

Login to post a comment