What is AI Error Analysis?
Checkly has an AI error analysis feature that is able to:- Analyzes your monitoring data to identify pattewrns and anomalies
- Provides intelligent recommendations for issue resolution
- Helps you understand complex system failures
Key Capabilities
Playwright Test Analysis
On each failed Browsr or Playwright Check result, Checkly AI analyzes the traces, script code, screenshots and script code to provide you with a detailed analysis of the failure. This includes:- End-user impact: Checkly AI analyzes the failure and provides you with a list of impacted users and features. The text used is generated by AI and can be enriched with organisation-specific domain knowledge using the Tag description editor the AI settings screen.
- Step summary: Checkly AI summarizes the steps in the check script, making it easier to understand what the check is doing and where it failed.
- Root cause: Checkly AI analyzes the failure and provides you with a root cause analysis, helping you quickly remediate the issue.
Tag Description Editor
Using the Tag Description Editor, you can add custom descriptions to your tags. Checkly AI uses these descriptions to provide more context and insights into failures and the impact on user groups, features and applications. This is very useful for organizations with specific domain knowledge and terminology embedded in their tags. For example, a tag likefeat:checkout can be enriched with a description like “This tag is used for the checkout feature of our e-commerce application, which allows users to purchase products online.”
Playwright Error Explainer
On each Playwright error, Checkly AI provides you with an error explanation. This includes:- A general explanation of the error.
- A suggestion on how to fix the error with a code snippet.
- A (set of) link(s) to the Playwright documentation for the error.
Use Cases
Incident Response
Incident Response
- Quick root cause analysis
- Automated incident documentation
- Intelligent escalation recommendations
- Post-incident learning and improvement
Performance Optimization
Performance Optimization
- Identify performance bottlenecks
- Suggest optimization strategies
- Monitor optimization effectiveness
- Track performance improvements
Proactive Monitoring
Proactive Monitoring
- Identify potential issues before they occur
- Suggest preventive measures
- Monitor system health trends
- Optimize monitoring strategies
Benefits
- Faster Resolution: Get insights and recommendations quickly
- Reduced Manual Work: Automate routine analysis tasks
- Better Understanding: Gain deeper insights into system behavior
- Proactive Management: Identify issues before they become problems
Data Privacy and Security
All AI features are designed to operate on the data you provide or the data generated by your checks. This data is already stored in Checkly, and we do not send any additional data to third-party AI providers. Any provided environment secrets are scrubbed before being sent to the AI provider. We do not train any custom models on your data. Any data used by the AI features is processed in real-time and not stored for future use. To address further privacy concerns, you can bring your own AI model and provider to Checkly. For more information, see the section below on built-in and custom models.Built-in and Custom Models
By default, Checkly uses OpenAI’s GPT-4.1 model for AI features. However, you can also bring your own model (BYOM) and provider to Checkly. To do this, you need to set up a custom AI provider in the AI settings screen.The Bring Your Own Model (BYOM) feature is available for Enterprise customers only.