Start a new topic
Answered

Read Data from excel before scraping.

I have an excel sheet that contains some urls. I want to read the file and scrap the data from the urls.
On local machine its simple and running perfectly. Is there a way on srapinghub cloud to read the data from the file.
Thanks


Best Answer

Hello,


Yes this can be done on Scrapy Cloud as well. The Excel file needs to be deployed to Scrapy cloud along with the project as given in https://support.scrapinghub.com/support/solutions/articles/22000200416-deploying-non-code-files

Then read the file (example also given in the article) and use it further as required. 


Regards,



Answer

Hello,


Yes this can be done on Scrapy Cloud as well. The Excel file needs to be deployed to Scrapy cloud along with the project as given in https://support.scrapinghub.com/support/solutions/articles/22000200416-deploying-non-code-files

Then read the file (example also given in the article) and use it further as required. 


Regards,


Hi,

I have already applied that, but it seems like the byte code generated by get_data() on an excel sheet may not be decoded. However, I did my solution by copying the data from excel to .txt file and its working perfectly using this approach.

Thanks

Hellow... thriveni
I tried this tutorial.

I need to read an xls file through pandas.

The name of my file is: xls_conformidade_gov_julho2020v1

When I try to do the tutorial I get the following error when I deploy my project.



FileNotFoundError: [Errno 2] No such file or directory: '/tmp/unpacked-eggs/__main__.egg/consulta_remedio/src/xls_conformidade_gov_julho2020v1.xls'


1 person likes this
Login to post a comment