When crawling at a decent scale, it can be hard to keep track of the health of your web crawlers. Crawlera provides a usage dashboard but it is used on demand, often when you experience issues and want to understand what happened, or when you need to check what you were billed for.
To stay on top of your crawlers health, you need a periodic, ongoing report that provides you the high level insights that allow you to spot the issues before your customers find them. This is why today we’re announcing a new feature in Crawlera: The Crawlera Weekly Digest.
The Crawlera Weekly Digest is a report sent by email every week with key insights of your Crawlera usage and traffic patterns.
The report includes 5 different graphs detailing the most frequent accounts, errors and websites. We encourage you to give a try and provide feedback. At this point, the report is disabled by default and you can enable it by going to the Account Settings → Notifications section and selecting the organizations from which you want to receive the weekly digest. The report will be sent on Monday mornings.
For more information, please check the Crawlera Weekly Digest article in our Support Center.
In the coming months we’ll be making this report better by:
introducing recommendations to proactively solve issues and improve your performance, based on how you use Crawlera today
including direct links from the report to the usage stats, so you can quickly drill down on the insights provided by the report
adding more useful graphs and insights based on your feedback
We hope you find this report useful and look forward to your feedback!