Zyte Support Center

Hey all,

I want to achieve the following:

Invoking an AWS Lambda function.

Situation:

- I can invoke the Lambda locally

- I'm using a ~/.aws/credentials file, where I store the AWS access and secret key

Challenge:

- I set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the Spider Settings UI

- However, when running the spider on Zyte, I get the following error: botocore.exceptions.NoCredentialsError: Unable to locate credentials

When I use

- os.environ["AWS_ACCESS_KEY_ID"] = <my_key>

- os.environ["AWS_SECRET_ACCESS_KEY"] = <my_key>

in the spider itself it works fine. But it is really against all best-practices to store the keys in the code.

Does anyone know why the environment variables are not picked up from the Spider Settings?

Best regards

I solved it now by reading the following:

shub_settings = json.loads(os.environ['SHUB_SETTINGS'])

aws_access_key_id = shub_settings['project_settings']['AWS_ACCESS_KEY_ID']