Settings for different plans

Posted over 5 years ago by aserhii

Post a topic
Answered
a
aserhii

 

CRAWLERA_ENABLED = True

CRAWLERA_APIKEY = "KEY"


CRAWLERA_PRESERVE_DELAY = 3


CONCURRENT_REQUESTS = 10

CONCURRENT_REQUESTS_PER_DOMAIN = 10

AUTOTHROTTLE_ENABLED = False

DOWNLOAD_TIMEOUT = 600


Hello everyone.
Tell me please what I need to change in these settings because I see `
429` error in my terminal, I use C10 plan.

Thank you very much.

0 Votes

nestor

nestor posted over 5 years ago Admin Best Answer

It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay

0 Votes


3 Comments

Sorted by
nestor

nestor posted over 5 years ago Admin Answer

It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay

0 Votes

a

aserhii posted over 5 years ago

Nestor, sorry, what do you think about my settings? All good for C10?

0 Votes

nestor

nestor posted over 5 years ago Admin

Which account are you using? From what I can see your Crawlera account is suspended.

0 Votes

Login to post a comment