Zyte Support Center

Dear Support Team,

I encountered this error when using Scrapy Cloud to run my job:

Rejected message because it was too big: ITM {...

I am sure that the request is fired successfully. This error should be caused by large response, which is about 2M. But this is the size I need to store.

I followed the advices in the following link, i.e. enabling Page Storage addon, and configured a Amazon S3 storage using Feed Export:

https://support.scrapinghub.com/support/discussions/topics/22000009523

 However, the same error is throw. May i know how to solve this issue?

I have the same issue. My scraper produces just one result item, which is a 100MB JSON tree representing a website's data. This works fine locally for me.

What limits are enforced? Is there any way to use Scrapy Cloud with an item like this?