Start a new topic
Answered

Does a file download count as a separate request?

If I have a scraper set up to send a request to a URL, scrape a URL from the HTML that gets returned, and then download the file at the URL, does that count as one request towards Crawlera's monthly limit, or two?


Best Answer

Hello,


If you have to issue a separate request to download the content, then it would count as two different requests.

Crawlera accounts for all successful requests it delivers to your spider. If the requests are marked as banned or failed, they do not count.


Hope this is the info you were looking for.


Thanks!

Answer

Hello,


If you have to issue a separate request to download the content, then it would count as two different requests.

Crawlera accounts for all successful requests it delivers to your spider. If the requests are marked as banned or failed, they do not count.


Hope this is the info you were looking for.


1 person likes this
Login to post a comment