No recent searches
Popular Articles
Sorry! nothing found for
Posted almost 6 years ago by Lilu Cao
Log opened.
[scrapy.log] Scrapy 1.4.0 started
[scrapy.utils.log] Scrapy 1.4.0 started (bot: newscrawl)
[scrapy.utils.log] Overridden settings: {'NEWSPIDER_MODULE': 'newscrawl.spiders', 'STATS_CLASS': 'sh_scrapy.stats.HubStorageStatsCollector', 'LOG_LEVEL': 'INFO', 'SPIDER_MODULES': ['newscrawl.spiders'], 'AUTOTHROTTLE_ENABLED': True, 'LOG_ENABLED': False, 'MEMUSAGE_LIMIT_MB': 950, 'BOT_NAME': 'newscrawl', 'TELNETCONSOLE_HOST': '0.0.0.0'}
[scrapy.middleware] Enabled extensions:
[scrapy.middleware] Enabled downloader middlewares:
[scrapy.middleware] Enabled spider middlewares:
[scrapy.middleware] Enabled item pipelines:
[scrapy.core.engine] Spider opened
[scrapy.extensions.logstats] Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
[news_reuters] Spider opened: news_reuters
TelnetConsole starting on 6023
[scrapy.core.engine] Closing spider (finished)
[scrapy.statscollectors] Dumping Scrapy stats:
[scrapy.core.engine] Spider closed (finished)
(TCP Port 6023 Closed)
Main loop terminated.
0 Votes
1 Comments
tejashri posted almost 6 years ago Admin
Hi,
Could you please try disabling the auto-throttle? Please set the value to False AUTOTHROTTLE_ENABLED': False
Login to post a comment
People who like this
This post will be deleted permanently. Are you sure?
Time (UTC)
Log opened.
[scrapy.log] Scrapy 1.4.0 started
[scrapy.utils.log] Scrapy 1.4.0 started (bot: newscrawl)
[scrapy.utils.log] Overridden settings: {'NEWSPIDER_MODULE': 'newscrawl.spiders', 'STATS_CLASS': 'sh_scrapy.stats.HubStorageStatsCollector', 'LOG_LEVEL': 'INFO', 'SPIDER_MODULES': ['newscrawl.spiders'], 'AUTOTHROTTLE_ENABLED': True, 'LOG_ENABLED': False, 'MEMUSAGE_LIMIT_MB': 950, 'BOT_NAME': 'newscrawl', 'TELNETCONSOLE_HOST': '0.0.0.0'}
[scrapy.middleware] Enabled extensions:
More[scrapy.middleware] Enabled downloader middlewares:
More[scrapy.middleware] Enabled spider middlewares:
More[scrapy.middleware] Enabled item pipelines:
More[scrapy.core.engine] Spider opened
[scrapy.extensions.logstats] Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
[news_reuters] Spider opened: news_reuters
TelnetConsole starting on 6023
[scrapy.core.engine] Closing spider (finished)
[scrapy.statscollectors] Dumping Scrapy stats:
More[scrapy.core.engine] Spider closed (finished)
(TCP Port 6023 Closed)
Main loop terminated.
0 Votes
1 Comments
tejashri posted almost 6 years ago Admin
Hi,
Could you please try disabling the auto-throttle? Please set the value to False AUTOTHROTTLE_ENABLED': False
0 Votes
Login to post a comment