With the job console you can open a Unix shell directly into the container where your job is running. Once in the console, you can perform tasks such as:

  • tailing the Scrapy log (tail -f scrapy.log)
  • accessing the Scrapy Telnet Console with telnet localhost 6023
  • inspecting the files generated by your Scrapy process
  • use top to inspect CPU and memory consumption
  • ... and anything else you can do in a Unix console

To open the job console, click the Console tab:

Finally, a couple of things to keep in mind:

  • the job console is only available for running jobs
  • access is restricted to the directory of your scrapy project (in other words, you don't have root access)