Start a new topic

IOError: [Errno 2] No such file or directory:


I am trying to upload my local scrapy project to scraping hub. However when I try to run shub deploy, I get the following error (changed name to private_file for privacy reasons):

IOError: [Errno 2] No such file or directory: 'private_file.json'

I've tried to follow the instructions found here, but I still get the error.

Any help is greatly appreciated.

2 people have this question

Does the project folder structure matches the one in the article?

If so, are you trying to use open()?

The structure matches, with one exception. I don't have the "resources" folder, I just have my .json file in the main directory. And yes, I've tried to use the open. I can post the code snippet if that's helpful.

You're not supposed to use open(), at the bottom of the article it shows how you should use the file:

import pkgutil  

data = pkgutil.get_data("myproject","resources/cities.txt")

That means that "data" already is the contents of the file.

When I try using that, I get the error

TypeError: coercing to Unicode: need string or buffer, NoneType found

Here's the python code that I use for the local file


credentials = ServiceAccountCredentials.from_json_keyfile_name(data, scope)


Look at the last few lines of the build error, that should give you an idea of the error:

credentials = ServiceAccountCredentials.from_json_keyfile_name(data, scope)
  File "/app/python/lib/python2.7/site-packages/oauth2client/", line 219, in from_json_keyfile_name
    with open(filename, 'r') as file_obj:
TypeError: coercing to Unicode: need string or buffer, NoneType found
{"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}

It's trying to a open a file. from_json_keyfile_name is expecting a file name (, "data" is not a file.

I haven't used oauth2client before, but it should be either from_json or from_json_keyfile_dict instead.

When I try to use from_json_keyfile_dict, I get the error

AttributeError: 'NoneType' object has no attribute 'get'

When I try from_json I get the error

ValueError: None could not be converted to unicode

import pkgutil

data = pkgutil.get_data("myproject","resources/private_file.json")
data = data.decode("UTF-8")

credentials = ServiceAccountCredentials.from_json_keyfile_dict(data, scope)

Most likely, the problem is that you're using a relative file path to open the file, but the current working directory isn't set to what you think it is.


It's a common misconception that relative paths are relative to the location of the python script, but this is untrue. Relative file paths are always relative to the current working directory, and the current working directory doesn't have to be the location of your python script.


You have three options:


  • Use an absolute path to open the file.
  • Generate the path to the file relative to your python script.
  • Change the current working directory before opening the file


Login to post a comment