videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

403 Client Error: Forbidden for url: https://app.scrapinghub.com/api/settings/get.json?project=XXXX

Hello,


I'm trying to deploy to an existing Scrapy Cloud project with the following command:


shub image upload PROJECT_NAME -f PATH_TO_DOCKERFILE --username API_KEY --password ' '


The image builds and is pushed successfully, but then I get the following error:


requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://app.scrapinghub.com/api/settings/get.json?project=XXXXX


-----------------------


Full stack trace:


Building images.scrapinghub.com/project/PROJECT_ID:VERSION.
Steps: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9
The image images.scrapinghub.com/project/PROJECT_ID:VERSION build is completed.
Login to images.scrapinghub.com succeeded.
Pushing images.scrapinghub.com/project/PROJECT_ID:VERSION to the registry.
Layers: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14/14
The image images.scrapinghub.com/project/PROJECT_ID:VERSION pushed successfully.
Traceback (most recent call last):
  File "/usr/local/bin/shub", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/shub/image/upload.py", line 42, in cli
    async_, skip_tests, filename)
  File "/usr/local/lib/python3.7/site-packages/shub/image/upload.py", line 53, in upload_cmd
    apikey, insecure, async_)
  File "/usr/local/lib/python3.7/site-packages/shub/image/deploy.py", line 81, in deploy_cmd
    username, password, email)
  File "/usr/local/lib/python3.7/site-packages/shub/image/deploy.py", line 198, in _prepare_deploy_params
    metadata = list_mod.list_cmd(image_name, project, endpoint, apikey)
  File "/usr/local/lib/python3.7/site-packages/shub/image/list.py", line 66, in list_cmd
    settings = _get_project_settings(project, endpoint, apikey)
  File "/usr/local/lib/python3.7/site-packages/shub/image/list.py", line 101, in _get_project_settings
    req.raise_for_status()
  File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://app.scrapinghub.com/api/settings/get.json?project=PROJECT_ID


Best Answer
I figured out what was happening. I'm deploying using a configuration YAML (scrapinghub.yml) and I had to set my API key on this file too, passing it on the command line as above is not enough. That solved the issue.
1 Comment

Answer
I figured out what was happening. I'm deploying using a configuration YAML (scrapinghub.yml) and I had to set my API key on this file too, passing it on the command line as above is not enough. That solved the issue.
Login to post a comment