Getting requirements error while doing shub deploy
v
vaibhav_9
started a topic
over 3 years ago
I am unable to resolve the version conflict between boto3, awscli, s3storage and botocore. I was somehow able to resolve the conflict between these four but i received another error as below
"watchtower 0.8.0 has requirement boto3<2,>=1.9.253, but you have boto3 1.8.1.
botocore 1.11.1 has requirement urllib3<1.24,>=1.20, but you have urllib3 1.25.10"
The requirements.txt has content as below :
boto3==1.8.1
pandas
unidecode
dateparser
pyes
mmh3
beautifulsoup4
scrapy==1.4.0
ftfy
scrapy-crawlera
elasticsearch==5.1.0
requests-aws4auth
html5lib==1.0b8
six
pytest
watchtower
logzio-python-handler
docker
scrapinghub-entrypoint-scrapy
requests==2.23.0
awscli==1.16.1
botocore==1.11.1
s3transfer==0.1.13
Kindly help to resolve the error
Best Answer
n
nestor
said
over 3 years ago
The first one is pretty simple, you are not specifying any watchtower version so it assumes the latest 0.8.0. Watchtower 0.8.0 has a requirement for boto3 to be higher than 1.9.253 and lower than 2. The version you are specifying for boto3 1.8.1 is lower than the one required by watchtower 0.8.0 so it generates a conflict, you need to either put a higher version for boto3 or specify a lower one for watchtower that fills that criteria.
The second one is similar but a little bit more hidden because you are not specifying urllib3, it is instead part of the 2.0 scrapinghub scrapy stack https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-2.0/requirements.txt#L74. Botocore 1.11.1 has a requirement of urllib3 between 1.20 and 1.24 but the stack predefines 1.25, so you would can specify a higher version for botocore to solve the conflict.
Note that you have a lot of libraries without a specified version, this is not recommended as it causes issues like this.
The first one is pretty simple, you are not specifying any watchtower version so it assumes the latest 0.8.0. Watchtower 0.8.0 has a requirement for boto3 to be higher than 1.9.253 and lower than 2. The version you are specifying for boto3 1.8.1 is lower than the one required by watchtower 0.8.0 so it generates a conflict, you need to either put a higher version for boto3 or specify a lower one for watchtower that fills that criteria.
The second one is similar but a little bit more hidden because you are not specifying urllib3, it is instead part of the 2.0 scrapinghub scrapy stack https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-2.0/requirements.txt#L74. Botocore 1.11.1 has a requirement of urllib3 between 1.20 and 1.24 but the stack predefines 1.25, so you would can specify a higher version for botocore to solve the conflict.
Note that you have a lot of libraries without a specified version, this is not recommended as it causes issues like this.
vaibhav_9
I am unable to resolve the version conflict between boto3, awscli, s3storage and botocore. I was somehow able to resolve the conflict between these four but i received another error as below
"watchtower 0.8.0 has requirement boto3<2,>=1.9.253, but you have boto3 1.8.1.
botocore 1.11.1 has requirement urllib3<1.24,>=1.20, but you have urllib3 1.25.10"
The requirements.txt has content as below :
boto3==1.8.1
pandas
unidecode
dateparser
pyes
mmh3
beautifulsoup4
scrapy==1.4.0
ftfy
scrapy-crawlera
elasticsearch==5.1.0
requests-aws4auth
html5lib==1.0b8
six
pytest
watchtower
logzio-python-handler
docker
scrapinghub-entrypoint-scrapy
requests==2.23.0
awscli==1.16.1
botocore==1.11.1
s3transfer==0.1.13
Kindly help to resolve the error
The first one is pretty simple, you are not specifying any watchtower version so it assumes the latest 0.8.0. Watchtower 0.8.0 has a requirement for boto3 to be higher than 1.9.253 and lower than 2. The version you are specifying for boto3 1.8.1 is lower than the one required by watchtower 0.8.0 so it generates a conflict, you need to either put a higher version for boto3 or specify a lower one for watchtower that fills that criteria.
The second one is similar but a little bit more hidden because you are not specifying urllib3, it is instead part of the 2.0 scrapinghub scrapy stack https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-2.0/requirements.txt#L74. Botocore 1.11.1 has a requirement of urllib3 between 1.20 and 1.24 but the stack predefines 1.25, so you would can specify a higher version for botocore to solve the conflict.
Note that you have a lot of libraries without a specified version, this is not recommended as it causes issues like this.
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstnestor
The first one is pretty simple, you are not specifying any watchtower version so it assumes the latest 0.8.0. Watchtower 0.8.0 has a requirement for boto3 to be higher than 1.9.253 and lower than 2. The version you are specifying for boto3 1.8.1 is lower than the one required by watchtower 0.8.0 so it generates a conflict, you need to either put a higher version for boto3 or specify a lower one for watchtower that fills that criteria.
The second one is similar but a little bit more hidden because you are not specifying urllib3, it is instead part of the 2.0 scrapinghub scrapy stack https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-2.0/requirements.txt#L74. Botocore 1.11.1 has a requirement of urllib3 between 1.20 and 1.24 but the stack predefines 1.25, so you would can specify a higher version for botocore to solve the conflict.
Note that you have a lot of libraries without a specified version, this is not recommended as it causes issues like this.
vaibhav_9
Thanks for the Help !!
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 453 topics