CRAWLERA_APIKEY = "KEY"
CRAWLERA_PRESERVE_DELAY = 3
CONCURRENT_REQUESTS = 10
CONCURRENT_REQUESTS_PER_DOMAIN = 10
AUTOTHROTTLE_ENABLED = False
DOWNLOAD_TIMEOUT = 600
Hello everyone.Tell me please what I need to change in these settings because I see `429` error in my terminal, I use C10 plan.
Thank you very much.
It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay
Which account are you using? From what I can see your Crawlera account is suspended.
aserhii
CRAWLERA_APIKEY = "KEY"
CRAWLERA_PRESERVE_DELAY = 3
CONCURRENT_REQUESTS = 10
CONCURRENT_REQUESTS_PER_DOMAIN = 10
AUTOTHROTTLE_ENABLED = False
DOWNLOAD_TIMEOUT = 600
Hello everyone.
Tell me please what I need to change in these settings because I see `429` error in my terminal, I use C10 plan.
Thank you very much.
It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstnestor
Which account are you using? From what I can see your Crawlera account is suspended.
aserhii
nestor
It's fine except for CRAWLERA_PRESERVE_DELAY which is a True or False setting: https://scrapy-crawlera.readthedocs.io/en/latest/settings.html#crawlera-preserve-delay
-
Crawlera 503 Ban
-
Amazon scraping speed
-
Website redirects
-
Error Code 429 Too Many Requests
-
Bing
-
Subscribed to Crawlera but saying Not Subscribed
-
Selenium with c#
-
Using Crawlera with browsermob
-
CRAWLERA_PRESERVE_DELAY leads to error
-
How to connect Selenium PhantomJS to Crawlera?
See all 389 topics