Some website will response different content in different IP. So I want to know the difference from my debug computer. How can I know about the response in cloud?
0 Votes
nestor posted
almost 6 years ago
AdminBest Answer
If it's less than 1MB you can store the full HTML as an item, if larger than 1MB you'll need an external storage like S3.
Some website will response different content in different IP. So I want to know the difference from my debug computer. How can I know about the response in cloud?
0 Votes
nestor posted almost 6 years ago Admin Best Answer
If it's less than 1MB you can store the full HTML as an item, if larger than 1MB you'll need an external storage like S3.
Otherwise, you can use the pagestorage addon: https://support.scrapinghub.com/support/solutions/articles/22000200403-page-storage-addon
0 Votes
1 Comments
nestor posted almost 6 years ago Admin Answer
If it's less than 1MB you can store the full HTML as an item, if larger than 1MB you'll need an external storage like S3.
Otherwise, you can use the pagestorage addon: https://support.scrapinghub.com/support/solutions/articles/22000200403-page-storage-addon
0 Votes
Login to post a comment