How to avoid 429 Client Error: Internal Server Error using crawlera?
Is this just an issues of there aren't enough proxy IPs? How can I avoid exceeding the rate-limit in a given period of time?
Best Answer
n
nestor
said
over 5 years ago
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
1 Comment
nestor
said
over 5 years ago
Answer
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
madhulika
Hi,
How to avoid 429 Client Error: Internal Server Error using crawlera?
Is this just an issues of there aren't enough proxy IPs? How can I avoid exceeding the rate-limit in a given period of time?
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
nestor
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
-
Crawlera 503 Ban
-
Amazon scraping speed
-
Website redirects
-
Error Code 429 Too Many Requests
-
Bing
-
Subscribed to Crawlera but saying Not Subscribed
-
Selenium with c#
-
Using Crawlera with browsermob
-
CRAWLERA_PRESERVE_DELAY leads to error
-
How to connect Selenium PhantomJS to Crawlera?
See all 395 topics