videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

Connection to the other side was lost in a non-clean fashion

The same excat project runs locally like a charm.

When deployed to SH i am getting [scrapy.core.scraper] Error downloading <GET http://www.guatecompras.gt/proveedores/consultaDetProvee.aspx?rqp=10&lprv=1>: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>] from start.


My settings are -

[scrapy.utils.log] Overridden settings: {'NEWSPIDER_MODULE': 'shupload.spiders', 'LOG_LEVEL': 'INFO', 'SPIDER_MODULES': ['shupload.spiders'], 'STATS_CLASS': 'sh_scrapy.stats.HubStorageStatsCollector', 'LOG_ENABLED': False, 'MEMUSAGE_LIMIT_MB': 950, 'TELNETCONSOLE_HOST': '0.0.0.0', 'BOT_NAME': 'shupload', 'MEMUSAGE_ENABLED': True, 'USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36'}

 


Any thoughts?


Best Answer

I can't open that website, they probably block access to certain countries. You might need a proxy service like Crawlera to access this website, unless they allow access only from Guatemala, because we don't have proxies from Guatemala.

1 Comment

Answer

I can't open that website, they probably block access to certain countries. You might need a proxy service like Crawlera to access this website, unless they allow access only from Guatemala, because we don't have proxies from Guatemala.

Login to post a comment