Start a new topic

Portia spider with custom feed exports

I'm trying to customize a Portia project to send the scraped data to another server by calling a REST API.


Basically, I want to create spiders following these steps:


1- Create the spider with Portia.

2- Publish the spider to Scrapy Cloud

3- When a scheduled job is completed, send the results in JSON format to the API.


How can I add a custom feed export to the Portia project? Does Portia overwrite all files when publishing changes?


Is it creating a custom feed export the right way to do it, or should I use a pipeline instead?


Best,


1 Comment

I have this problem too; it seems the documentation on setting up feed exports is a bit weak for portia / scrapy cloud users. I'd like to ftp to overwrite a file called feed.json or similar. Any help?

Login to post a comment