Start a new topic

Integrate Scrapy + Selenium scraper to smart proxy manager and schedule daily tasks on prefect

Hi everyone, I am having trouble trying to figure out how to integrate smart proxy manager with my scraper which uses scrapy and selenium (I have provided a screenshot of the code for my scraper). What I aim to do is to schedule a daily task on prefect which runs my scraper to scrape data with IP rotation using zyte. Please help me understand how I should structure and integrate the three components: scraper and prefect and zyte. Any help is truly appreciated!! :)

Login to post a comment