The directory '/root/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/root/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting setuptools>=38.2.5
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/root/.cache/pip/wheels/a3/63/e3/a02e38a020e587e1cb7b263c1f26ebb218fd3d87f77584662b'
Consider using the `--user` option or check the permissions.
The problem is that pendulum tries to force installation with some custom flags: '--ignore-installed' and '--no-user', which require access to '/root/.cache' folder, and root access is forbidden for users.
One option is to make a fork of this library and update the logic for installation. Another option is to downgrade to a version of pendulum which used to work. And last but not least, replace pendulum with another library that doesn't force those flags.
Edit: Even when I am trying to redeploy exactly the same version that is currently on scrapinghub, I am getting this error. Looks like some problem with permissions on container.
L
Lukasz Jaworowski
said
almost 5 years ago
More info about bug: All requirements are ok except pendulum.
After removing pendulum from requirements deployment process is finished, but there is import error after (some spiders are using pendulum). So problem is not solved, because we still need pendulum for our spiders. Before weekend it was ok, we're using pendulum since last year.
nestor
said
almost 5 years ago
Answer
The problem is that pendulum tries to force installation with some custom flags: '--ignore-installed' and '--no-user', which require access to '/root/.cache' folder, and root access is forbidden for users.
One option is to make a fork of this library and update the logic for installation. Another option is to downgrade to a version of pendulum which used to work. And last but not least, replace pendulum with another library that doesn't force those flags.
Lukasz Jaworowski
Hi, I made a little change to code (its working on my local scrapy), but while deploying to scrapinghub i am getting this error:
{"status": "error", "message": "Requirements error"}
Here is the full log.
Login succeeded
Building an image:
Step 1/12 : FROM scrapinghub/scrapinghub-stack-scrapy:1.4
# Executing 2 build triggers...
Step 1/1 : ENV PIP_TRUSTED_HOST $PIP_TRUSTED_HOST PIP_INDEX_URL $PIP_INDEX_URL
---> Using cache
Step 1/1 : RUN test -n $APT_PROXY && echo 'Acquire::http::Proxy \"$APT_PROXY\";' >/etc/apt/apt.conf.d/proxy
---> Using cache
---> 24d6293b6b61
Step 2/12 : ENV PYTHONUSERBASE /app/python
---> Using cache
---> 61cdf4ed7547
Step 3/12 : ADD eggbased-entrypoint /usr/local/sbin/
---> Using cache
---> 92017487150e
Step 4/12 : ADD run-pipcheck /usr/local/bin/
---> Using cache
---> a8a333f93303
Step 5/12 : RUN chmod +x /usr/local/bin/run-pipcheck
---> Using cache
---> b1bd6a21e692
Step 6/12 : RUN chmod +x /usr/local/sbin/eggbased-entrypoint && ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/start-crawl && ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/scrapy-list && ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/shub-image-info && ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/run-pipcheck
---> Using cache
---> 24fb77a956ac
Step 7/12 : ADD requirements.txt /app/requirements.txt
---> Using cache
---> be3e85f65e23
Step 8/12 : RUN mkdir /app/python && chown nobody:nogroup /app/python
---> Using cache
---> f39f9e835f37
Step 9/12 : RUN sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE pip install --user --no-cache-dir -r /app/requirements.txt
---> Running in 640f6ab4299b
Collecting pdfminer==20140328 (from -r /app/requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/57/4f/e1df0437858188d2d36466a7bb89aa024d252bd0b7e3ba90cbc567c6c0b8/pdfminer-20140328.tar.gz (4.1MB)
Collecting pendulum==2.0.2 (from -r /app/requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/4a/97/c288b8c98e49b9e1ec67b28ada3f39b7bd786be11ca119244e2aca3a5cf4/pendulum-2.0.2.tar.gz (75kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
Complete output from command /usr/local/bin/python -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-OJLJb2 --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=38.2.5 wheel:
The directory '/root/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/root/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting setuptools>=38.2.5
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/root/.cache/pip/wheels/a3/63/e3/a02e38a020e587e1cb7b263c1f26ebb218fd3d87f77584662b'
Consider using the `--user` option or check the permissions.
----------------------------------------
Command "/usr/local/bin/python -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-OJLJb2 --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=38.2.5 wheel" failed with error code 1 in None
{"message": "The command '/bin/sh -c sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE pip install --user --no-cache-dir -r /app/requirements.txt' returned a non-zero code: 1", "details": {"message": "The command '/bin/sh -c sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE pip install --user --no-cache-dir -r /app/requirements.txt' returned a non-zero code: 1", "code": 1}, "error": "requirements_error"}
Requirements file:
The problem is that pendulum tries to force installation with some custom flags: '--ignore-installed' and '--no-user', which require access to '/root/.cache' folder, and root access is forbidden for users.
One option is to make a fork of this library and update the logic for installation. Another option is to downgrade to a version of pendulum which used to work. And last but not least, replace pendulum with another library that doesn't force those flags.
- Oldest First
- Popular
- Newest First
Sorted by Oldest FirstLukasz Jaworowski
Edit: Even when I am trying to redeploy exactly the same version that is currently on scrapinghub, I am getting this error. Looks like some problem with permissions on container.
Lukasz Jaworowski
More info about bug: All requirements are ok except pendulum.
After removing pendulum from requirements deployment process is finished, but there is import error after (some spiders are using pendulum). So problem is not solved, because we still need pendulum for our spiders. Before weekend it was ok, we're using pendulum since last year.
nestor
The problem is that pendulum tries to force installation with some custom flags: '--ignore-installed' and '--no-user', which require access to '/root/.cache' folder, and root access is forbidden for users.
One option is to make a fork of this library and update the logic for installation. Another option is to downgrade to a version of pendulum which used to work. And last but not least, replace pendulum with another library that doesn't force those flags.
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 446 topics