Start a new topic

Need help creating

I have created a which I am able to run as

    $ scrapy runspider

I want to deploy this to ths cloud.

I tried with shub as follows:

    $ shub deploy

and it ran for a while and said:

    Packing version e721061-master

    Created at /home/xxx/scrapy1/yyy

    Deploying to Scrapy Cloud project "zzz"

    {"status": "ok", "project": zzz, "version": "e721061-master", "spiders": 0}

    Run your spiders at:

I suspect my and scrapinghub.yml files are not ok.


        name = 'project',

        version = '1.0',

        packages = find_packages(),

        entry_points = {'scrapy': ['settings = test1.settings']},


I don't have a - so not sure about the settings

Please help!

1 Comment

Hi Sureshv,

Please check the Scrapy Project structure, it should be as below

    scrapy.cfg            # deploy configuration file

    tutorial/             # project's Python module, you'll import your code from here          # project items definition file      # project pipelines file       # project settings file

        spiders/          # a directory where you'll later put your spiders

Ensure that correct folders are mentioned in and the spider is also under the required folder.

To deploy a Scrapy project to Scrapy Cloud, navigate into the project’s folder and run:


where [TARGET] is either a project name defined in scrapinghub.yml or a numerical Scrapinghub project ID. If you have configured a default target in your scrapinghub.yml, you can leave out the parameter completely (as you have used).

Login to post a comment