videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic

Go behind user / password

Hi


I´m new in scrapinghub.


Is it possible to go behind a user and password web page to which I log as a user - no as an admin? If so,  this is done with a spider? Can you recomend a pre design / created spider that I can buy / subscribe and deploy?


Also this webpage has more than one section with different url´s. To get the posts  from all sections when the post are created can I get the info with different spiders or I can get a single spider to do so?


In the image 1 the login process, Image 2 the first click to get into the post section, Image 3 the differnt sections, Image 4 an example from the post´s

Post.png
(342 KB)
Find.png
(311 KB)
piptalk.png
(637 KB)
Login.png
(1.59 MB)
Login to post a comment