videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic

using datetime in setting e.g feed_uri

 I have a spider which writes data to s3.

locally I specify the file to write to a date separated folder in s3.

I want to replicate the feed URI setting in scraping hub, is it possible to replicate the behavior below?


import  datetime
today = datetime.date.today()
FEED_URI = 's3://relayplay-prebackend/ev_scrapedata/' + today.strftime("%Y/%m/%d") + '/eventbrite_spider.csv'
Login to post a comment