I want to read an API as an environment variable in ScrapingHub.
I have saved it in Spiders > Settings (in the screenshot), and then I have tried to extract it in Python:
But when I deploy it to the scrapy cloud, the variable api is None. It doesn't detect the environment variable.
In my scrapinghub.yml I only have the project name and the pointer to the requirements file. What am I doing wrong?
I believe the "Raw Settings" panel is not for environment variables, but for Spider settings, similar to what you can find in "settings.py" file.
I do something similar, and here's how it works for me:
1. Add a new line to "settings.py" like so:
OWM_API = os.environ.get('OWM_API')
2. Wherever you need to get this setting, you can fetch it from the spider's 'settings' attribute:
your_token = spider.settings['OWM_API'] ## or this one, if you're within the Spider class: your_token = self.settings['OWN_API']
Hope this helps!