Start a new topic

Portia spider with custom feed exports

I'm trying to customize a Portia project to send the scraped data to another server by calling a REST API.

Basically, I want to create spiders following these steps:

1- Create the spider with Portia.

2- Publish the spider to Scrapy Cloud

3- When a scheduled job is completed, send the results in JSON format to the API.

How can I add a custom feed export to the Portia project? Does Portia overwrite all files when publishing changes?

Is it creating a custom feed export the right way to do it, or should I use a pipeline instead?


1 Comment

I have this problem too; it seems the documentation on setting up feed exports is a bit weak for portia / scrapy cloud users. I'd like to ftp to overwrite a file called feed.json or similar. Any help?

Login to post a comment