IOError while deplying

Posted over 6 years ago by rafalf

Post a topic

Un Answered

rafalf

I have a piece of code in my spider that read a csv file and it fails

any solution

 File "/tmp/unpacked-eggs/__main__.egg/kompassdotcom/spiders/basic.py", line 40, in BasicSpider
    with open(company_path, "r") as f:
IOError: [Errno 2] No such file or directory: '/tmp/unpacked-eggs/__main__.egg/kompassdotcom/input/test_companies.csv'
{"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}
{"status": "error", "message": "Internal error"}

0 Votes

4 Comments

nestor posted over 6 years ago Admin

The article suggests to use:

import pkgutil   

data = pkgutil.get_data("myproject","resources/cities.txt")

It seems like you're trying to open() it. By using pkgutil, you're are already reading the data and there's no need to use open().

0 Votes

rafalf posted over 6 years ago

Yes I did.

the folder input aint created in build

if you have a look into attached 1.png, you'll see the structure of the project

0 Votes

nestor posted over 6 years ago Admin

Did you follow this guide: https://support.scrapinghub.com/support/solutions/articles/22000200416-deploying-non-code-files?

0 Votes

rafalf posted over 6 years ago

and here is my setup file

what I can see that is build/lib input directory is not mirrored

setup(
    name         = 'project',
    version      = '1.0',
    packages     = find_packages(),
    package_data={
        'project': ['input/*.csv']
    },
    entry_points = {'scrapy': ['settings = kompassdotcom.settings']},
    zip_safe=False,
)

Attachments (1)

1.png
27.2 KB

0 Votes