Read Data from excel before scraping.

Posted about 5 years ago by srmd

Post a topic

Answered

srmd

I have an excel sheet that contains some urls. I want to read the file and scrap the data from the urls.
On local machine its simple and running perfectly. Is there a way on srapinghub cloud to read the data from the file.
Thanks

0 Votes

thriveni posted almost 5 years ago Admin Best Answer

Hello,

Yes this can be done on Scrapy Cloud as well. The Excel file needs to be deployed to Scrapy cloud along with the project as given in https://support.scrapinghub.com/support/solutions/articles/22000200416-deploying-non-code-files.

Then read the file (example also given in the article) and use it further as required.

Regards,

0 Votes

3 Comments

Rodrigo Robledo posted over 4 years ago

Hellow... thriveni
I tried this tutorial.

I need to read an xls file through pandas.

The name of my file is: xls_conformidade_gov_julho2020v1

When I try to do the tutorial I get the following error when I deploy my project.

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/unpacked-eggs/__main__.egg/consulta_remedio/src/xls_conformidade_gov_julho2020v1.xls'

1 Votes

srmd posted almost 5 years ago

Hi,

I have already applied that, but it seems like the byte code generated by get_data() on an excel sheet may not be decoded. However, I did my solution by copying the data from excel to .txt file and its working perfectly using this approach.

Thanks

0 Votes

thriveni posted almost 5 years ago Admin Answer

Hello,

Then read the file (example also given in the article) and use it further as required.

Regards,

0 Votes