Synthetic Monitoring With Checkly and Playwright Test

One of the most effective ways to monitor a critical user flow on a website—or monitor the operation of a critical API that other applications depended on—is to adopt synthetic monitoring. 

Synthetic monitoring is an approach to monitoring websites and applications that simulates the actions of real users via browser automation. It mirrors the actions that a visitor may take on your website, say browsing an online shop, adding items to a shopping cart, and then checking out. Being able to simulate critical user flows helps you quickly identify website issues before a user encounters them. 

Additionally, synthetic monitoring allows us to measure the performance of complex applications and also allows us to identify performance regressions over time.

We at Checkly believe that combining synthetic monitoring with Playwright will take your testing and monitoring efforts to the next level. So what is Playwright?

Better Together: Playwright Test and Synthetic Monitoring

Backed by Microsoft, Playwright is an open-source, cross-platform testing framework for end-to-end web application testing. It was designed to be fast, reliable, and robust, and its API supports modern rendering engines that include Chromium, Firefox, and WebKit. 

Playwright tests run on Windows, Linux, and macOS, locally or on your continuous integration pipeline, and can be executed in headed and headless browser mode. Playwright supports several programming languages, including TypeScript, JavaScript, Python, .NET, and Java. Playwright also has a growing community of users that support it, and the Playwright team is constantly adding new features.

• Related: Playwright Explained

// example of a Playwright Test spec file
const { expect, test } = require('@playwright/test')

test('Playwright home page', async ({ page }) => {
  const response = await page.goto(process.env.ENVIRONMENT_URL || 'https://checklyhq.com/')
  expect(response?.status()).toBeLessThan(300)
  expect(page).toHaveTitle('Code, test, and deploy synthetic monitoring at scale.')
  await page.screenshot({ path: 'homepage.jpg' })
})

Playwrights testing features and its provided test runner Playwright Test (PWT) are perfect tools for synthetic monitoring. Read more about it in the next section!

Why use synthetic monitoring with PWT?

So why should you combine Playwright and PWT with your synthetic monitoring efforts? For starters, combining the two provides a delightful developer experience. Writing end-to-end monitoring code with Playwright looks very similar to application code, and you can readily use languages you already know and love, like JavaScript and TypeScript.

Playwright also offers a useful high-level API, as well as powerful locators aiming to be as close to the end-user experience as possible rather than relying on test ids.

// Use Playwright’s user-first locators to interact with your application
await page.getByLabel('User Name').fill('John');
await page.getByLabel('Password').fill('secret-password');
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByText('Welcome, John!')).toBeVisible();

Playwright end-to-end tests are also resilient, thanks to auto-waiting and web-first assertions with automatic retries. Drop all these waitFor statements and let Playwright figure out when a UI element reaches its desired state.

// wait for the element with the text “Welcome” to be visible or time out
await expect(page.getByText('Welcome')).toBeVisible();

Playwright even offers a code-generation tool—Playwright Codegen—that generates test code from your onscreen demo test activity, like going through the steps of adding an item to an online shopping cart and then checking out.

And Playwright’s last selling point is the advanced debugging capabilities, including the creation of videos and traces that help you visually track down how and why checks fail. Traces are like “time travel debugging” including screenshot snapshots of failed checks, and network request logs, and they help you find performance bottlenecks and reasons for test failures.

After evaluating carefully, we decided that PWT is the perfect tool for Checkly’s browser checks and synthetic monitoring and helps thousands of Checkly users detect application issues early and often.

Best practices for using PWT for synthetic monitoring

After using PWT for over a year, we’ve gathered some tips and best practices for using Playwright and PWT for your synthetic monitoring efforts.

In no particular order:

Creating and evolving checks

Keep your end-to-end tests small and focused. Separate checks can test different parts of your web application; they’re easier to debug and understand. Plus, having smaller checks also reduces the risk of breaking them when introducing changes.

Tips for maintainability. It’s wise to avoid hard-coded DOM or CSS class selectors. Relying on a CSS class will most likely break with CSS updates, and higher-level locators such page.getByPlaceholder() as are a better option to mimic user experience. 

You also should avoid arbitrary timeouts to wait for elements and rely on PWT’s auto-awaiting and retries.

// Playwright click when this button is:
//
// - attached to the DOM
// - visible
// - stable (not animated)
// - receives events (not obscured by another element)
// - enabled
await page.getByRole('button').click();

Use page object models (POMs). We suggest that you use POMs, as that can reduce duplication, and a single POM can be used in many checks. Taking that approach also helps you maintain more readable, maintainable code, and also provides a high-level API for interfacing with changes.

// playwright-dev-page.js
const { expect } = require('@playwright/test');

exports.PlaywrightDevPage = class PlaywrightDevPage {
  constructor(page) {
    this.page = page;
    this.getStartedLink = page.locator('a', { hasText: 'Get started' });
    this.gettingStartedHeader = page.locator('h1', { hasText: 'Installation' });
  }

  async goto() {
    await this.page.goto('https://playwright.dev');
  }

  async getStarted() {
    await this.getStartedLink.first().click();
    await expect(this.gettingStartedHeader).toBeVisible();
  }
}


// example.spec.js
const { test } = require("@playwright/test");
const { PlaywrightDevPage } = require("./playwright-dev-page");

test("getting started should contain table of contents", async ({ page }) => {
  const playwrightDev = new PlaywrightDevPage(page);
  await playwrightDev.goto();
  await playwrightDev.getStarted();
});

Checks should live and evolve with your application. It’s a good idea to keep your check code in the same repository as your application code and leverage the monitoring as code workflow. This allows you to deploy your application and its monitoring at the same time.

Monitoring setup tips

Monitor your web apps and APIs continuously. You should ideally configure your end-to-end monitoring to continuously monitor your infrastructure on a 24/7 basis, and use a monitoring frequency that will allow you to detect issues quickly. The key question for choosing a frequency is “How long are you comfortable with not knowing about a production issue?”. The higher the check frequency, the shorter the time to detection (Mean time to Detection—MTTD) will be.

Regressions and incidents often happen during deployment. To avoid regressions and incidents, integrate synthetic tests with your CI/CD pipeline. Your existing CI/CD infrastructure can reuse your monitoring setup and test production/preview deployments when they happen. This approach also has the benefit of giving developers a fast feedback loop. 

Global business? Use global monitoring. If your company or organization has customers across the globe, it’s essential to ensure that you’re monitoring from as many global locations as possible. You may have different regional versions of your product that might need unique monitoring, and network conditions can vary from country to country and across continents. Use checks and monitors in regions where your customers are, as a global pool of customers requires a global monitoring infrastructure.

Alerting setup tips

Surfacing issues. Monitoring is only effective if you know about potential issues and problems that could impact your monitoring efforts. A single check failure is not always an indicator of an incident—it may be a momentary slowdown—so look at how retries are configured and set smart alerting thresholds, such as creating an alert when your check has multiple consecutive failures.

Separate alerts and checks into critical and secondary categories. Creating a system to triage your alerts into critical and non-critical categories will be a big help. A critical alert should be only used for a very small number of business-critical checks. Failures should be addressed immediately and reported to the wider on-call team, while a secondary tier of alerts should be reserved for lower-priority issues. We at Checkly (the company) monitor Checkly (the product) with over a hundred scheduled checks, but only a few are “mission critical”.

Identify performance trends and patterns over time. Rely on the Checkly dashboards to create data-rich status pages that will help you more effectively analyze performance metrics such as Google’s core web vitals over time. Export your data into a wider APM system for even more visibility and granularity into your monitoring efforts.

Summary

We believe that making Playwright—and PWT—vital parts of your synthetic monitoring efforts is the best option for developers. 

Playwright provides a great developers experience with high-level APIs and a coding approach that provides familiar, well-known patterns for TypeScript and JavaScript. Maintainable tests can be written and debugged easily, making your testing and monitoring efforts much more efficient. 

And finally, using Playwright and PWT with the Checkly monitoring as code (MaC) workflow and Checkly CLI allows your monitoring code to live right next to your application code, which helps blur the lines between development, operations, and QA.