Tracing and Web Vitals
For all your browser checks we automatically collect a comprehensive set of data like console logs and network requests to help you triage the root cause whenever your check might fail. We also snap a screenshot automatically whenever your script encounters an error.
Next to this, we also automatically collect Performance signals (Web Vitals) for each of the pages you visit in the browser session.
Checkly only collects tracing and web vitals data if your browser check meets the following criteria:
- It is based on Playwright.
- It runs on any runtime above 2021.06
For each page you visit, we automatically collect the following:
infolevel console messages.
- Any network requests, like
You can use this data to quickly find issues with the pages you visit. Use cases are:
- Pinpoint missing images or other resources: they will show a
404in the network tab.
Performance tracing with Web Vitals
For each page your script visits, we automatically collect a set of five Web Vitals. Web Vitals are user focused quality signals for web pages that indicate a good, ok or poor user experience.
First Contentful Paint
First Contentful Paint (FCP) measures the time from when the page initially starts loading to when any content is rendered on the page. This content can be anything: text, an image, an svg, etc. In practical terms, when you have a large FCP, a user will see a white screen for a long time, doubting whether the page works at all.
Largest Contentful Paint
Largest Contentful Paint is similar to FCP, but measures the time it took to render the largest visual item within the browser viewport. If you have a high LCP, your user might wait too long before the most useful part of your page has loaded. This makes it hard for the user to determine if your page is useful to them at all.
Cumulative Layout Shift
Cumulative Layout Shift (CLS) is an aggregate metric that signals screen elements “jumping around” as the page loads. You probably have experienced this when trying to click some element on screen and just as you click it, it jumps to another part of the page. Not great. A low CLS score means you have a stable page.
Total Blocking Time
Total Blocking Time (TBT) is a metric that reflect the time a web page is “blocked” from receiving any user input because the main rendering thread busy. A high TBT means a user experiences sluggish behaviour when interacting with your page.
Time to First Byte
Time to First Byte indicates how long it took for your server to respond with the first byte of content whenever a browser requests a page. TTFB can vary from run location to run location of course: when you have servers in the US and are requesting the page from Japan, expect a higher TTFB.
Web Vitals are a part Google Lighthouse, but the two are not the same: Web Vitals are a lighter, higher-level tool built on existing browser APIs. Being less resource-intensive than Lighthouse, Web Vitals can be reliably calculated for you on cloud resources every minute.
Why are some Web Vitals not reported?
Checkly automatically instruments your Playwright code to collect Web Vitals metrics. However, is some cases we will not be able to collect a good measurement. There are multiple reasons for this:
A Single Page App (SPA) like a React, Vue or Angular app only has one HTML page load event and then relies on route changes. The underlying library we use for collecting Web Vitals does not yet account for this. The folks over at Google are aware of this and posted an FAQ.
Cumulative Layout Shift (CLS) relies on layout events accumulating over time. If your script exits very fast, or triggers a redirect, there might not be enough time to actually “catch” this metric.
Some metrics require a small amount of interaction to effectively “trigger”. Clicking a button or other screen element in your Playwright script can help here.
We are constantly tweaking and updating our Web Vitals collection code, so expect improvements in the future!
Lab metrics vs. Field metrics
Checkly collects “lab” metrics for you. These are different from “field” metrics.
Lab metrics are collected in a controlled environment and generated by “synthetic” users, in this case your Browser check scripts. Lab metrics are useful to establish a baseline against which you can determine performance regressions or broken deploys before you go live.
Field metrics are based on real user traffic collected “in the wild”. These types of metrics are also known as Real User Monitoring (RUM) and can vary a ton based on the device, location and network quality of and end user.
Automatic screenshots on error
Whenever your Playwright script encounters an error, we will automatically snap a screenshot the moment the error occurs. Here is an example from real life!
- Your script clicks on a button using a selector
- For some reason, that button does not exist or is not clickable.
- Playwright waits for the button with the selector to appear. It does not and Playwright throws an error.
- Checkly automatically calls
page.screenshot()a screenshot. The screenshot indicates that that specific button was missing.
You can contribute to this documentation by editing this page on Github