Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic
Answered

Is any config there for crawlera to allow download large files(>100M)?

 Is any config there for crawlera to allow download large files(>100M)?


thanks so much


Best Answer

There is no specific config for this. The file size limit is 500MB, anything higher will return a 541 error code, see:  https://doc.scrapinghub.com/crawlera-errors.html#failure-errors-server-side 

1 Comment

Answer

There is no specific config for this. The file size limit is 500MB, anything higher will return a 541 error code, see:  https://doc.scrapinghub.com/crawlera-errors.html#failure-errors-server-side 

Login to post a comment