2017-08-21 16:20:02 INFO Log opened.
2017-08-21 16:20:02 INFO [scrapy.log] Scrapy 1.4.0 started
2017-08-21 16:20:02 INFO [scrapy.utils.log] Scrapy 1.4.0 started (bot: scrapybot)
2017-08-21 16:20:02 INFO [scrapy.utils.log] Overridden settings: {'NEWSPIDER_MODULE': 'crawley_cloud.spiders', 'LOG_LEVEL': 'INFO', 'CONCURRENT_REQUESTS_PER_DOMAIN': 10, 'SPIDER_MODULES': ['crawley_cloud.spiders'], 'STATS_CLASS': 'sh_scrapy.stats.HubStorageStatsCollector', 'LOG_ENABLED': False, 'MEMUSAGE_LIMIT_MB': 950, 'COOKIES_ENABLED': False, 'TELNETCONSOLE_HOST': '0.0.0.0', 'AUTOTHROTTLE_ENABLED': True, 'DOWNLOAD_DELAY': 5, 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246'}
2017-08-21 16:20:02 INFO [scrapy.middleware] Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.debug.StackTraceDump',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.spiderstate.SpiderState',
 'scrapy.extensions.throttle.AutoThrottle',
 'sh_scrapy.extension.HubstorageExtension']
2017-08-21 16:20:02 INFO [scrapy.middleware] Enabled downloader middlewares:
['sh_scrapy.diskquota.DiskQuotaDownloaderMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats',
 'sh_scrapy.middlewares.HubstorageDownloaderMiddleware']
2017-08-21 16:20:02 INFO [scrapy.middleware] Enabled spider middlewares:
['sh_scrapy.diskquota.DiskQuotaSpiderMiddleware',
 'sh_scrapy.middlewares.HubstorageSpiderMiddleware',
 'scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2017-08-21 16:20:02 INFO [scrapy.middleware] Enabled item pipelines:
[]
2017-08-21 16:20:02 INFO [scrapy.core.engine] Spider opened
2017-08-21 16:20:02 INFO [scrapy.extensions.logstats] Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2017-08-21 16:20:02 INFO TelnetConsole starting on 6023
2017-08-21 16:21:02 INFO [scrapy.crawler] Received SIGTERM, shutting down gracefully. Send again to force 
2017-08-21 16:21:02 INFO [scrapy.core.engine] Closing spider (shutdown)
2017-08-21 16:21:02 INFO [scrapy.extensions.logstats] Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2017-08-21 16:22:02 INFO [scrapy.extensions.logstats] Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)