Start a new topic

Can I access "Spider Settings" programmatically from the spider?

Every now and then I bump into the same problem: I want my spider to access webservices that I have a paid access to (with an API token that had a significant value) but the Python code is in a public repository, I can not put the API token there (in the code). The perfect solution would be exactly as it is done with the AWS service access parameters, to enter my credentials as "Spider Settings" manually (or through your interface), but access them FROM THE CODE OF THE SPIDER.

I dug through this knowledge base (which is really good, by the way) but didn't find how to do it (in Python), can you help me? An example couple of lines of code would be best.

1 Comment

I've already found the solution in the documentation and your answers to another question.

The solution is:

1. Set the required parameters through the web-interface (pay attention to the fact that you need to click 'Custom name' first);

2. In your callback function (where you need the token) include a line of code like:
    api_token = self.settings['API_TOKEN']

Login to post a comment