Hi I try to set up my script wiht boto3 to scrapinghub cloud, in deploy it does not work, I encounter some version error for pip install.
How can I solve it?
Oh not really, but I figured things out here, I put
botocore==1.5.8
boto3==1.4.4
and it works fine now.
Thanks a lot for your help!
Can you show the requirements file? Did you hardcode the versions in the requirements.txt?
No I just put boto3 in it, how could I know what version I should use for boto3?
You can add versions to the `requirements.txt` with boto3==10.0.0, does that help?
it says Could not find a version that satisfies the requirement boto3==10.0.0
Sorry that was an example. From the screen shot you sent you want
awscli==1.11.45
s3transfer==0.1.9
What is the code you use to upload to S3?
upload to S3? sorry you mean shub deploy?
I assumed you use boto and aws to upload your scraped items to S3.
Yingke Yu
Hi I try to set up my script wiht boto3 to scrapinghub cloud, in deploy it does not work, I encounter some version error for pip install.
How can I solve it?
Oh not really, but I figured things out here, I put
botocore==1.5.8
boto3==1.4.4
and it works fine now.
Thanks a lot for your help!
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstjwaterschoot
Can you show the requirements file? Did you hardcode the versions in the requirements.txt?
Yingke Yu
No I just put boto3 in it, how could I know what version I should use for boto3?
Yingke Yu
jwaterschoot
You can add versions to the `requirements.txt` with boto3==10.0.0, does that help?
Yingke Yu
it says Could not find a version that satisfies the requirement boto3==10.0.0
jwaterschoot
Sorry that was an example. From the screen shot you sent you want
awscli==1.11.45
botocore==1.5.8
s3transfer==0.1.9
What is the code you use to upload to S3?
Yingke Yu
upload to S3? sorry you mean shub deploy?
jwaterschoot
I assumed you use boto and aws to upload your scraped items to S3.
Yingke Yu
Oh not really, but I figured things out here, I put
botocore==1.5.8
boto3==1.4.4
and it works fine now.
Thanks a lot for your help!
1 person likes this
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 446 topics