How to avoid 429 Client Error: Internal Server Error using crawlera?
Is this just an issues of there aren't enough proxy IPs? How can I avoid exceeding the rate-limit in a given period of time?
0 Votes
nestor posted
over 6 years ago
AdminBest Answer
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
0 Votes
1 Comments
nestorposted
over 6 years ago
AdminAnswer
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
Hi,
How to avoid 429 Client Error: Internal Server Error using crawlera?
Is this just an issues of there aren't enough proxy IPs? How can I avoid exceeding the rate-limit in a given period of time?
0 Votes
nestor posted over 6 years ago Admin Best Answer
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
0 Votes
1 Comments
nestor posted over 6 years ago Admin Answer
You need to limit the number of concurrent request to what your plan allows. A C10 allows for up to 10 concurrent requests. If you exceed it, Crawlera will reject the request with a 429 code.
0 Votes
Login to post a comment