HI Chandan,
The error seems pretty straightforward. You spider is exceeding the storage capacity.
If you use Scrapy Units, that provides you with 2.5 GB of temporary storage during the time your spider runs.
Keep in mind that disk space is not persisted, though. You'll need to use other techniques in order to persist data between jobs.
What is the way to overcome this issue?
Hi Ihor
I'm sorry for the long delay in getting back at you.
The way to overcome this would be to assign more Scrapy Cloud units to your job. I can purchase as many scrapy units as you need.
Let me know if you still need help.
Chandan Kumar Ojha