Start a new topic
Answered

Scrape JSON response

Is there a way to use Portia to scrape a JSON response?


Best Answer

Unfortunately, Splash cannot render that URL so it won't be possible to scrape with Portia. JSON responses are possible to scrape as long as Portia is able to render as HTML.


Yeah, it is possible to extract JSON response using Portia.

How is this done? I am getting no response from a url that returns json in chrome.

What's the URL?

Answer

Unfortunately, Splash cannot render that URL so it won't be possible to scrape with Portia. JSON responses are possible to scrape as long as Portia is able to render as HTML.

Is that why I can't seem to get any items out of this rather simple JSON response using regex? 


https://jsonplaceholder.typicode.com/posts/1


I do get the response to load from this url, but my annotations produce no items, it starts trying to extract and just keeps spinnin'. Is there some trick to annotations on straight JSON, or does everything need to be framed in HTML?

@rpm61 Enable JS in Portia and when you run the spider in Scrapinghub it should extract the JSON response.

Login to post a comment