Start a new topic
Answered

Settings for different plans

 

CRAWLERA_ENABLED = True

CRAWLERA_APIKEY = "KEY"


CRAWLERA_PRESERVE_DELAY = 3


CONCURRENT_REQUESTS = 10

CONCURRENT_REQUESTS_PER_DOMAIN = 10

AUTOTHROTTLE_ENABLED = False

DOWNLOAD_TIMEOUT = 600


Hello everyone.
Tell me please what I need to change in these settings because I see `
429` error in my terminal, I use C10 plan.

Thank you very much.


Best Answer

It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay


Which account are you using? From what I can see your Crawlera account is suspended.

Nestor, sorry, what do you think about my settings? All good for C10?
Answer

It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay

Login to post a comment