Zyte Support Center

It goes against my company's security policy to put a cloud service ID string in version control, so this means that we can't commit the necessary `scrapinghub.yml` into version control without using something like git-crypt to encrypt the file first.

We were hoping that you could use environment variables within `scrapinghub.yml` but it appears this isn't supported.

Is there known pattern or best practice approach for reading project IDs from environment variables or otherwise, so as to not commit them to source control?

Hi Ned,

I believe that your asking about storing projectIds for the purpose of deploying your spiders to ScrapyCloud  correct?

If so, I have found that it is not a strict requirement to store the project Id in the scrapinghub.yml to have a spider deploy.

In my case we manage multiple projects with the same spider. To deploy, I specify the projectId after my depploy command (using shub). So for example, for project 123456, it would be:

shub deploy 123456

So long as your user has access to a project and you login to shub with the API key for that user, then you can deploy to any of those projects. This then allows you to store those ids however you need. For example, in the case of using Jenkins to deploy, you could save as a build parameter or stored secret (or password).