I've spent hours trying to find a solution to this.
When trying to do a shub deploy, this happens:
Entrypoint container is created successfully >>> Checking python dependencies awscli 1.10.52 has requirement botocore==1.4.42, but you have botocore 1.10.1. Warning: Pip checks failed, please fix the conflicts. {"message": "Dependencies check exit code: 1", "details": "Pip checks failed, please fix the conflicts", "error": "requirements_error"} {"status": "error", "message": "Requirements error"} Deploy log location: /var/folders/qc/dzs6cch91lq21cb50q2c7zhm0000gn/T/shub_deploy_e0o02o4v.log Error: Deploy failed: b'{"status": "error", "message": "Requirements error"}'
My requirements.txt looks like this:
Distance==0.1.3 nltk==3.2.5 Scrapy==1.5.0 tweepy==3.5.0 scrapinghub==2.0.3 setuptools==36.5.0 gensim==3.4.0 msgpack_python==0.5.6 PyDispatcher==2.0.5
My scrapinghub.yml looks like this:
project: 285313 requirements: file: requirements.txt stacks: default: scrapy:1.1-py3
It seems that the awscli version is being overrided somehow, as the awscli version that I have installed on my computer is updated.
Pip freeze, awscli==1.15.1 (on my computer), but when I do a shub deploy, it says that I have awscli==1.10.52
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
This worked, thank you!
I'm glad it worked.
Is there any way just to ignore such warnings?? (like adding a parameter in the .yml file that would take care of that)
No, it's an error that causes conflict when deploying. Otherwise how could the tool determine which version to deploy.
Movere
I've spent hours trying to find a solution to this.
When trying to do a shub deploy, this happens:
My requirements.txt looks like this:
My scrapinghub.yml looks like this:
It seems that the awscli version is being overrided somehow, as the awscli version that I have installed on my computer is updated.
Pip freeze, awscli==1.15.1 (on my computer), but when I do a shub deploy, it says that I have awscli==1.10.52
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstnestor
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
Movere
This worked, thank you!
nestor
I'm glad it worked.
moeen
Is there any way just to ignore such warnings?? (like adding a parameter in the .yml file that would take care of that)
nestor
No, it's an error that causes conflict when deploying. Otherwise how could the tool determine which version to deploy.
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 446 topics