videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

Is any config there for crawlera to allow download large files(>100M)?

 Is any config there for crawlera to allow download large files(>100M)?


thanks so much


Best Answer

There is no specific config for this. The file size limit is 500MB, anything higher will return a 541 error code, see:  https://doc.scrapinghub.com/crawlera-errors.html#failure-errors-server-side 

1 Comment

Answer

There is no specific config for this. The file size limit is 500MB, anything higher will return a 541 error code, see:  https://doc.scrapinghub.com/crawlera-errors.html#failure-errors-server-side 

Login to post a comment