I had written it so that my code exited when getting a 403 error or the message contained forbidden but now it seems that 500 errors "Response status code does not indicate success: 500 (Internal Server Error)." are being generated which are too broad. Is there a simple way of determining if the plan limit has been reached and/or the account suspended so I don't keep sending requests?
Thanks
Best Answer
n
nestor
said
over 3 years ago
401 would be the correct way, aside from looking at your dashboard. Once you reach your limit, the account gets suspended and returns 401, no other code. I've had a look at your logs and don't see any 500s.
Yes, we do have a permanent fix but it's not released yet. I don't have an ETA, but it will possibly be next week. I'll post here when I have an update on the release date, but you can also subscribe to our status page https://status.scrapinghub.com/ since we'll post it there too.
1 person likes this
c
crapo
said
over 3 years ago
Hi nestor, any update?
nestor
said
over 3 years ago
Yes, you're right. I was able to reproduce it. Let me check and get back to you on that.
By the way, it should be 401 not 403. I'll edit my previous message to prevent confusion.
1 person likes this
c
crapo
said
over 3 years ago
Thanks for taking a look Nestor. Weird though; I'm getting 500
nestor
said
over 3 years ago
Answer
401 would be the correct way, aside from looking at your dashboard. Once you reach your limit, the account gets suspended and returns 401, no other code. I've had a look at your logs and don't see any 500s.
crapo
I had written it so that my code exited when getting a 403 error or the message contained forbidden but now it seems that 500 errors "Response status code does not indicate success: 500 (Internal Server Error)." are being generated which are too broad. Is there a simple way of determining if the plan limit has been reached and/or the account suspended so I don't keep sending requests?
Thanks
401 would be the correct way, aside from looking at your dashboard. Once you reach your limit, the account gets suspended and returns 401, no other code. I've had a look at your logs and don't see any 500s.
- Oldest First
- Popular
- Newest First
Sorted by Newest Firstcrapo
nestor
Yes, we do have a permanent fix but it's not released yet. I don't have an ETA, but it will possibly be next week. I'll post here when I have an update on the release date, but you can also subscribe to our status page https://status.scrapinghub.com/ since we'll post it there too.
1 person likes this
crapo
Hi nestor, any update?
nestor
Yes, you're right. I was able to reproduce it. Let me check and get back to you on that.
By the way, it should be 401 not 403. I'll edit my previous message to prevent confusion.
1 person likes this
crapo
Thanks for taking a look Nestor. Weird though; I'm getting 500
nestor
401 would be the correct way, aside from looking at your dashboard. Once you reach your limit, the account gets suspended and returns 401, no other code. I've had a look at your logs and don't see any 500s.
-
Crawlera 503 Ban
-
Amazon scraping speed
-
Website redirects
-
Error Code 429 Too Many Requests
-
Bing
-
Subscribed to Crawlera but saying Not Subscribed
-
Selenium with c#
-
Using Crawlera with browsermob
-
CRAWLERA_PRESERVE_DELAY leads to error
-
How to connect Selenium PhantomJS to Crawlera?
See all 395 topics