videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic

add/remove command with selective scraping "SYNC SCRAPING

 "Job Search Board" for example

Can you add instructions to the scrapper:

-Scrape all the requested items except "these"

"These" 'items' are already in your database so you're not wanting to duplicate info.

 Add command

Remove any data from database that is no longer in search results

 Remove command

The intention is to have the database to synchronize with the website every time it performs a scrape

Login to post a comment