I have i c50 crawlera plan that runs at 50 threads and quite basic spider.
But it runs impossibly slow. Like 25 requests per minute.
The only thing is that i load 250k start urls from google sheet. Can this influence the speed? Autothrotle is disabled of course.
Billy John
I have i c50 crawlera plan that runs at 50 threads and quite basic spider.
But it runs impossibly slow. Like 25 requests per minute.
The only thing is that i load 250k start urls from google sheet. Can this influence the speed? Autothrotle is disabled of course.
1 person has this question