Start a new topic

Python code works on my computer doesn't work with AWS Lambda


I have Python web scraping code works well on my local machine (Ubuntu 20.04). But when I try to run it on AWS Lambda, I always get Proxy connection error:


[ERROR] ProxyError: HTTPSConnectionPool(host='', port=443): Max retries exceeded with url: /jobs?q=Software+Engineer&sort=date (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9d4156d3d0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
Traceback (most recent call last):
  File "/var/task/", line 16, in lambda_handler
  File "/var/task/", line 945, in scraping_job_stats
  File "/var/task/", line 607, in process_job_title_stats
    response = self.proxyAdapter.getURL(url)
  File "/var/task/", line 115, in getURL
    response = requests.get(
  File "/opt/python/requests/", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/opt/python/requests/", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/python/requests/", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/python/requests/", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/opt/python/requests/", line 559, in send
    raise ProxyError(e, request=request)END RequestId: 1ebf64b9-3f2e-457f-8e76-63bfde9d69c1
REPORT RequestId: 1ebf64b9-3f2e-457f-8e76-63bfde9d69c1	Duration: 131163.76 ms	Billed Duration: 131164 ms	Memory Size: 128 MB	Max Memory Used: 68 MB	Init Duration: 922.33 ms

Request ID

 I believe I setup the API token and zyte certificate properly. What could be the issues there?


1 Comment

The only thing different between my machine and AWS lambda is that I'm using python 3.10 and there is only python 3.9 available on AWS Lambda.

Login to post a comment