Customizing Scrapy settings in Scrapy Cloud

You can customize the behavior of all Scrapy components -- including the core, extensions, pipelines, and spiders themselves -- via Scrapy settings. These settings are usually defined in your Scrapy project's settings.py file, but you can also manage them through Scrapy Cloud's web interface.

First, go to your project Dashboard and then go to the Spiders Settings page. There you can add or remove the Scrapy settings using the (+) or (x) buttons, as shown below:

Then you can customize your settings, click "Save" and that's it!

For a list of available built-in settings see the Built-in settings reference.