# See also autothrottle settings and docs
DOWNLOAD_DELAY = 3
# Disable cookies (enabled by default)
COOKIES_ENABLED = False
DOWNLOADER_MIDDLEWARES = {
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,
'scrapy_user_agents.middlewares.RandomUserAgentMiddleware': 400,
}
# Enable and configure the AutoThrottle extension (disabled by default)
# See https://docs.scrapy.org/en/latest/topics/autothrottle.html
AUTOTHROTTLE_ENABLED = True
I have tried enabling and disabling the auto-throttle. It returned items the first time I disabled however has stopped retuning items since.
Hi all,
Sorry if this is a silly question.
I'm not getting any items returned when I run the spider on scrapy cloud. It works fine on my local machine.
spider.py file
scrapinghub.yml
setting.py
I have tried enabling and disabling the auto-throttle. It returned items the first time I disabled however has stopped retuning items since.
The log file is
I welcome any thoughts and help from the community!
Thanks,
Adi
0 Votes
0 Comments
Login to post a comment