IOError: [Errno 2] No such file or directory:

Posted about 6 years ago by Michael Jester

Post a topic

Un Answered

Michael Jester

Hello,

I am trying to upload my local scrapy project to scraping hub. However when I try to run shub deploy, I get the following error (changed name to private_file for privacy reasons):

IOError: [Errno 2] No such file or directory: 'private_file.json'

I've tried to follow the instructions found here, but I still get the error.

Any help is greatly appreciated.

2 Votes

8 Comments

harvyjackk posted over 4 years ago

Most likely, the problem is that you're using a relative file path to open the file, but the current working directory isn't set to what you think it is.

It's a common misconception that relative paths are relative to the location of the python script, but this is untrue. Relative file paths are always relative to the current working directory, and the current working directory doesn't have to be the location of your python script.

You have three options:

Use an absolute path to open the file.
Generate the path to the file relative to your python script.
Change the current working directory before opening the file

0 Votes

nestor posted about 6 years ago Admin

import pkgutil

data = pkgutil.get_data("myproject","resources/private_file.json")
data = data.decode("UTF-8")

credentials = ServiceAccountCredentials.from_json_keyfile_dict(data, scope)

0 Votes

Michael Jester posted about 6 years ago

When I try to use from_json_keyfile_dict, I get the error

AttributeError: 'NoneType' object has no attribute 'get'

When I try from_json I get the error

ValueError: None could not be converted to unicode

0 Votes

nestor posted about 6 years ago Admin

Look at the last few lines of the build error, that should give you an idea of the error:

credentials = ServiceAccountCredentials.from_json_keyfile_name(data, scope)
  File "/app/python/lib/python2.7/site-packages/oauth2client/service_account.py", line 219, in from_json_keyfile_name
    with open(filename, 'r') as file_obj:
TypeError: coercing to Unicode: need string or buffer, NoneType found
{"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}

It's trying to a open a file. from_json_keyfile_name is expecting a file name (https://oauth2client.readthedocs.io/en/latest/source/oauth2client.service_account.html#oauth2client.service_account.ServiceAccountCredentials.from_json_keyfile_name), "data" is not a file.

I haven't used oauth2client before, but it should be either from_json or from_json_keyfile_dict instead.

0 Votes

Michael Jester posted about 6 years ago

When I try using that, I get the error

TypeError: coercing to Unicode: need string or buffer, NoneType found

Here's the python code that I use for the local file

credentials = ServiceAccountCredentials.from_json_keyfile_name(data, scope)

0 Votes

nestor posted about 6 years ago Admin

You're not supposed to use open(), at the bottom of the article it shows how you should use the file:

import pkgutil  

data = pkgutil.get_data("myproject","resources/cities.txt")

That means that "data" already is the contents of the file.

0 Votes

Michael Jester posted about 6 years ago

The structure matches, with one exception. I don't have the "resources" folder, I just have my .json file in the main directory. And yes, I've tried to use the open. I can post the code snippet if that's helpful.

0 Votes

nestor posted about 6 years ago Admin

Does the project folder structure matches the one in the article?

If so, are you trying to use open()?

0 Votes