Scrapy Cloud FAQ
Note: Portia is no longer available for new users. It has been disabled for all the new organisations from August 20, 2018 onward. Here is a diagram tha...
Wed, 3 Feb, 2021 at 8:21 AM
Scrapy Cloud jobs run in containers. These containers can be of different sizes defined by Scrapy Cloud units. A Scrapy Cloud provides: 1 GB of RAM 2.5G...
Thu, 27 Jan, 2022 at 5:40 PM
When you subscribe to Zyte, 1 Scrapy Cloud unit is given for free. This free unit has features such as data retention for 7 days and a run time limit for jo...
Wed, 3 Feb, 2021 at 8:24 AM
Yes, you can. To do that, you have to enable Scrapy's HTTP cache extension by setting HTTPCACHE_ENABLED to True in your project settings. The...
Wed, 3 Feb, 2021 at 8:24 AM
Scenario: You have e.g. a MongoDB server that you would like your spiders to write to, and you would like to open access to that server only from Scrapy Clo...
Wed, 3 Feb, 2021 at 8:25 AM
Scrapy Cloud is only offered as a hosted service. If you're interested in hosting your own version of Scrapy Cloud, get in touch and let us know.
Wed, 3 Feb, 2021 at 8:25 AM
Here are a few things that work differently on Scrapy Cloud, compared to a default Scrapy configuration: AutoThrottle extension is enabled, to crawl webs...
Wed, 3 Feb, 2021 at 8:26 AM
A common issue is not to receive your activation link when you subscribe to a new account in Zyte or when you forgot your password and trying to send re-ac...
Wed, 3 Feb, 2021 at 8:26 AM
Zyte will support eggs within the Zyte Dashboard through the end of 2016. After that, project dependencies outside of the Scrapy Cloud stack must be managed...
Wed, 3 Feb, 2021 at 8:27 AM
Here at Zyte we are big fans of Heroku. When people ask what Scrapy Cloud is about we sometimes tell people that "it's like Heroku, but for web cra...
Wed, 3 Feb, 2021 at 8:28 AM