Start a new topic
Answered

How can I know about the response in cloud

Some website will response different content in different IP. So I want to know the difference from my debug computer. How can I know about the response in cloud?


Best Answer

If it's less than 1MB you can store the full HTML as an item, if larger than 1MB you'll need an external storage like S3.

Otherwise, you can use the pagestorage addon: https://support.scrapinghub.com/support/solutions/articles/22000200403-page-storage-addon

1 Comment

Answer

If it's less than 1MB you can store the full HTML as an item, if larger than 1MB you'll need an external storage like S3.

Otherwise, you can use the pagestorage addon: https://support.scrapinghub.com/support/solutions/articles/22000200403-page-storage-addon

Login to post a comment