How to deploy scrapy-playwright to zyte > scrapy cloud?

Posted 3 months ago by arthurb

Post a topic
Un Answered
a
arthurb

   

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.8/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/downloader/middleware.py", line 42, in process_request
    defer.returnValue((yield download_func(request=request, spider=spider)))
  File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 824, in adapt
    extracted = result.result()
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 378, in _download_request
    return await self._download_request_with_retry(request=request, spider=spider)
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 397, in _download_request_with_retry
    page = await self._create_page(request=request, spider=spider)
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 296, in _create_page
    ctx_wrapper = await self._create_browser_context(
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 257, in _create_browser_context
    await self._maybe_launch_browser()
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 205, in _maybe_launch_browser
    self.browser = await self.browser_type.launch(**self.config.launch_options)
  File "/app/python/lib/python3.8/site-packages/playwright/async_api/_generated.py", line 14115, in launch
    await self._impl_obj.launch(
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_browser_type.py", line 95, in launch
    Browser, from_channel(await self._channel.send("launch", params))
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_connection.py", line 59, in send
    return await self._connection.wrap_api_call(
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_connection.py", line 514, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at /scrapinghub/.cache/ms-playwright/chromium-1134/chrome-linux/chrome
╔════════════════════════════════════════════════════════════╗
║ Looks like Playwright was just installed or updated.       ║
║ Please run the following command to download new browsers: ║
║                                                            ║
║     playwright install                                     ║
║                                                            ║
║ <3 Playwright Team                                         ║

 

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.8/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/downloader/middleware.py", line 42, in process_request
    defer.returnValue((yield download_func(request=request, spider=spider)))
  File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 824, in adapt
    extracted = result.result()
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 378, in _download_request
    return await self._download_request_with_retry(request=request, spider=spider)
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 397, in _download_request_with_retry
    page = await self._create_page(request=request, spider=spider)
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 296, in _create_page
    ctx_wrapper = await self._create_browser_context(
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 257, in _create_browser_context
    await self._maybe_launch_browser()
  File "/app/python/lib/python3.8/site-packages/scrapy_playwright/handler.py", line 205, in _maybe_launch_browser
    self.browser = await self.browser_type.launch(**self.config.launch_options)
  File "/app/python/lib/python3.8/site-packages/playwright/async_api/_generated.py", line 14115, in launch
    await self._impl_obj.launch(
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_browser_type.py", line 95, in launch
    Browser, from_channel(await self._channel.send("launch", params))
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_connection.py", line 59, in send
    return await self._connection.wrap_api_call(
  File "/app/python/lib/python3.8/site-packages/playwright/_impl/_connection.py", line 514, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: BrowserType.launch: Failed to launch chromium because executable doesn't exist at /ms-playwright/chromium/chrome-linux/chrome

  Any guides or ideas how to get the browsers installed? Seems I can't really get scrapy-playwright to work on scrapy cloud.

1 Votes


0 Comments

Login to post a comment