Playwright Check Suites Are Now GA - But What Does That Mean For You?

There are only a few companies that successfully invest in actively monitoring real user flows in production. I’ve been puzzled by the state of the art for many years, because I’m an anxious developer that always needs to know that production is “all right”. How can it be okay for all of us to wait for error logs, thrown exceptions or customer complains to learn about production issues? Shouldn’t knowing that all the systems are operational at all time be the ultimate monitoring goal? And why’s no one actively testing and monitoring core application flows in production to ensure that all customers are having a good time?

The short answer: developers, QA engineers and SREs think that synthetic monitoring or “testing in production” is too hard. But is it really?

We at Checkly have some news, so buckle up!

Playwright has Taken over the End-to-end Testing Market

The teams at Microsoft started working on the Playwright end-to-end testing framework in February 2020. If you fast forward to today, the project took over the testing community, convincing with multiple-browser support, advanced error traces, built-in multi-threading, and an overall stellar developer experience.

Two million daily npm downloads can’t be wrong, can they?

Thanks to Microsoft, end-to-end testing and validating user flows with real browsers is a solved problem. So, here’s a thought; if developers and QA engineers adopt Playwright to simulate and test their critical application features in CI/CD to avoid regressions, why do we all stop there? Why is Playwright only used for pre-production testing?

If we now have a tool that helps us catch bugs before they go live, why don’t we take the test suites that are already implemented, replace clunky monitoring solutions, and bury the belief that “testing in production” is too hard?

Well, that’s exactly what we’ve been busy building and I’m excited to share that Playwright Check Suites just went into general availability.

Let me explain how they work and convince you to start testing in production today!

Playwright Check Suites — your Playwright-native Monitoring Solution

The core problem of existing monitoring solutions is that they don’t integrate naturally with existing developer workflows. The development teams need to learn domain-specific languages, master new tools and integrate unfamiliar workflows to get some basic end-to-end coverage implemented. Developers are lucky when they don’t need to log in somewhere to create, update or configure their general monitoring setup. Today’s monitoring landscape is anything but developer-first.

And isn’t it infuriating that siloed teams often double their workload just because testing and monitoring are done with different tools?

Here’s what our CEO, Hannes Lenke, thinks about this problem:

“We believe monitoring should match reality. That means that monitoring should include real browsers, real flows, and the exact same code and config you use in your end-to-end tests. Synthetic monitoring needs to become an extension of your test suite, not a separate system to manage.”

And that’s why we have built Playwright Check Suites; they’re a natural extension of your existing Playwright end-to-end test suite. Every Playwright project can now quickly become your most favorite monitoring solution.

With Playwright Check Suites, you can pick, choose, and group your existing tests to use them as synthetic monitors. Configure your monitoring infrastructure as code and use the CLI to transform on-demand end-to-end tests running in CI to active synthetic monitors testing and validating your apps from multiple locations.

And if one of your Playwright monitors fails, you’ll get alerted via your preferred alert channel (PagerDuty, Opsgenie, etc.). This way, your problem discovery becomes active, and instead of your users alerting you, you can fix issues before they even notice them.

And the best thing, all the beloved Playwright features will become your favorite monitoring features, too! But what makes Playwright such a great fit for monitoring?

When Testing Features Become your Favorite Monitoring Features

Regardless of whether youwant to test in multiple browsers, reuse browser and storage state to avoid running the same actions in every test, or perform visual regression testing, Playwright Check Suites allowyou to take your Playwright setup as is and transform it into synthetic monitoring.

You only have to decide what tests should run as distributed monitors and can then enjoy knowing that your core application flows work globally in production. At all times. Instead of waiting for your users to trigger exceptions and hit a wall, you’ll already be on top of it, resolving issues as quickly as possible.

But how can you know what’s broken when you rely on user-first production monitoring?

Debug Failed Monitors with Traces, Screenshots and Videos

Whenever a Playwright Check Suite is run you can access videos, screenshots or the holy grail of end-to-end test analysis; Playwright trace files in the Checkly web application. Forget about hunting down trace artifacts. Every Playwright Check Suite run records all the taken test steps, HTTP requests, JavaScript exceptions and logs; even DOM snapshots are available for every test step to evaluate the application state at the moment in time.

You can safely forget about calling customers to understand what they did to trigger an issue. Whenever your Playwright monitors fail, you’ll have all the information available to understand what actions were taken and what went wrong.

But what if you’re just the poor person who gets woken up by an alert? How do you know about the alert's root cause and if the issue has a direct user impact?

Analyze Failed Tests and Alerts With AI

To make alert and error analysis easier, Playwright Check Suites also enable you to analyze failed runs with AI. With the click of a button, AI can be your alert investigator.

Checkly AI will analyze your test code, the errors and the resulting trace files to give you hints on user impact and possible root causes. When you’ll receive a critical alert, you’ll take all the help you can get. Trust me!

Transform Your Tests to Monitoring and Scale Up!

Playwright Check Suites are a critical step to unify your testing and monitoring tool chain. End-to-end monitoring won’t rely on yet another tool, but be the natural extension of your existing testing workflows. Developers and QA engineers can maintain test validating real user flows while site-reliability engineers can reuse the exact same code to control the critical application monitoring and alerting.

Say goodbye to siloed teams and welcome cross-team collaboration!

We’re pretty excited about this shift! Try Playwright Checks Suites today and let us know what you think! And if you’re new to Playwright end-to-end testing, we’ve written countless end-to-end testing guides, too.