videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

Relative Import Error with no known parent package

Hello, Nestor helped me a lot earlier to deploy my project, but now I have another problem, this time with running custom scripts and importing spiders and other scripts with relative import.


I searched already but didn't find any topic talking about this. It's taking so much time for me to find a workaround so I please help me. 


My project is structured like this:   

.
├── runscripts
│   └── rodar_ps42.py
│   └── callps42.py
├── bin
│   ├── __init__.py
│   ├── quickstart.py
│   ├── items.py
│   ├── settings.py
│   └── spiders
│       ├── __init__.py
│       └── spider_url.py
│       
├── scrapy.cfg
├── scrapinghub.yml
├── requirements.txt
└── setup.py

    

After deploying, I try to run 'rodar_ps42.py', code:

from ..runscripts import callps42

callps42.callps42()

     

Then the 'callps42' function call the spider through '.spider_results' like this:  

from ..bin.spiders import spider_url
import time
from ..bin import quickstart, spreadsheet

def callps42():
    dados = spider_url.spider_results()
.
.
. 

   
 The 'spider_results' returns the spider results to 'dados' variable, and then the function continues to post the results to my database.


My problem is, when I run 'rodar_ps42.py' at Scrapy Cloud, the following Import Error appears:      

    from ..runscripts import callps42
ImportError: attempted relative import with no known parent package

 

Searched about this issue, tried to move the project around, removed some __init__.py  files, but nothing worked. 


I managed to put everything in setup.py file, and configuring the project like this.


I also looked at scrapy_price_monitor example and copied the import process from 'monitor'.py ... but then it does not find the module:

    from runscripts import callps42
ModuleNotFoundError: No module named 'runscripts'

  What do I need to do? Can't find anything about it, anywhere. If needed I can post more of my project.


 Thank you so much.


Best Answer

Hi, to import custom python scripts into your project please follow this guide: https://support.scrapinghub.com/support/solutions/articles/22000200394-running-custom-python-scripts

Basically they need to be declared in your setup.py 


Answer

Hi, to import custom python scripts into your project please follow this guide: https://support.scrapinghub.com/support/solutions/articles/22000200394-running-custom-python-scripts

Basically they need to be declared in your setup.py 

Hi, thank you for response. As I said before, I managed to put everything in setup.py file, and configuring the project like this. I even copied  the import process from 'monitor'.py on scrapy_price_monitor example.


I am trying a lot of different 'solutions' but after deploy correctly and run, it happens this problem with Relative Import Error with no known parent package... I really need some help now..

Thank you for everything

Login to post a comment