Skip to main content
We all do testing. We all test code as we write it, and before we release it. But somehow regressions, bugs, and even downtime still happen. Code that worked perfectly on our laptop somehow fails when it meets production, and in ways so obscure that simple integration tests, even canary deploys, don’t catch every failure. When we want to ensure reliable software, traditional testing methods often fall short at monitoring real-world scenarios. Enter Playwright and Checkly, two powerful tools that enable end-to-end (E2E) testing in production. In this guide, we’ll explore how these tools work together to provide robust monitoring and testing solutions, ensuring your application performs flawlessly for end users.

​
The Problem: Monitoring in a Complex World

Modern software systems are no longer simple. With the rise of cloud-native technologies, microservices, and distributed systems, applications have become more complex than ever. Front-end frameworks like React and Angular, each implementing modeling that’s more complex than an entire 2010 web application, have added further edge cases and wrinkles. Questions like “if this microservice goes down, will users still be able to add items to their cart?” are harder to answer than ever. a diagram showing the ideal SDLC The result? A flood of monitoring data—logs, metrics, and traces—that can be overwhelming. While observability tools have emerged to help manage this data, they often come with a hefty price tag and can be noisy, making it difficult to extract clear signals about whether your application is truly working. The core question developers and operations teams need to answer is: “Does my app still work?” This means ensuring that the application provides the expected business function correctly and in a timely manner, not just in pre-production environments but in production, where real users interact with it. a diagram showing the SDLC with real world feedback

​
Why Traditional QA Falls Short

Traditional QA processes, including unit tests and end-to-end tests, are essential but often stop at the pre-production stage. Once the application goes live, monitoring takes over, typically handled by a different team using different tools. This disconnect between QA and monitoring creates gaps in visibility and reliability. a diagram showing the ideal SDLC In the ideal SDLC, Playwright helps you test code pre-deployment, and make sure everything’s working A simple thought experiment is to ask:
  • Can we agree that no level of pre-deployment QA will catch every failure?
  • And if so, is it acceptable that everything not caught by QA will have to be found and reported by a user?
If your answers to these questions are ‘yes’ and ‘no’ respectively, it’s time to think about going beyond traditional QA. Testing in production is often treated as a meme, but it shouldn’t be. The reality is that production environments are fundamentally different from pre-production environments. Differences in data, security configurations, third-party services, and geographic distribution mean that testing in staging or UAT environments doesn’t guarantee success in production. The real dependencies of production apps In the real world, problems appear at lots of stages, and Playwright can also help you find these edge cases, with the power of Checkly

​
Enter Playwright: A Modern Browser Automation Tool

Playwright is Microsoft’s open-source browser automation framework that simplifies end-to-end testing. It supports multiple platforms and provides a robust API for interacting with web pages and APIs. Here’s why Playwright stands out:
  1. Web-First Locators and Assertions: Playwright uses web-first locators, making it easier to interact with dynamic web elements without relying on brittle CSS selectors.
  2. Automatic Retries and Waits: Playwright automatically retries actions and waits for elements to be visible, reducing flaky tests.
  3. Powerful Debugging Tools: Playwright provides traces, videos, and screenshots to help diagnose issues quickly.
  4. API Testing: Beyond browser automation, Playwright can also handle API testing, making it a versatile tool for end-to-end workflows.
  5. Detailed Assertions: No more do we call every 200 OK response a passing test. Playwright can make complex, programmatic assertions about the responses it receives, checking things like ‘every loaded record has a connected ID’ or ‘all buttons are set to be a brand-approved color.‘

​
Testing in Production with Playwright and Checkly

While Playwright excels at browser automation, Checkly takes it a step further, automating runs of Playwright tests and enabling Monitoring as Code. Playwright Check Suites let you run and schedule Playwright tests against production environments, providing real-time insights into your application’s health. Run a Playwright test every hour or every minute from locations all across the globe, all managed from a workflow that fits into your current CI/CD model, rather than requiring a cumbersome interface to manage monitors. Checkly Dashboard with multiple Playwright Check Suites

​
Key Features of Checkly:

  1. Global Test Execution: Run your Playwright scripts from multiple geographic locations to ensure your application performs well for users worldwide.
  2. Monitoring Cadence: Run tests every minute or every hour, and set a monitoring cadence that defends your SLA.
  3. Alerting and Notifications: Set up alerts via email, Slack, PagerDuty, or other channels to notify your team of failures.
  4. Connect Backend Traces: With Checkly Traces, harness the power of OpenTelemetry to see backend traces connected to each testing session.
  5. Code Workflow Integration: Manage your monitoring scripts as code, enabling version control, collaboration, and CI/CD integration.

​
A Practical Example: Monitoring an E-Commerce Workflow

Let’s say you’re running an e-commerce platform. Any regression, missing feature, or unavailability means users can’t check out on your site, so even a few minutes of a problem costs your business money. A critical user journey might involve:
  1. Logging into the application.
  2. Searching for a product.
  3. Adding the product to the cart.
  4. Checking out.
A typical web development process would have QA or developers run tests before deployment, and then ‘monitor’ deployments to make sure the site is still available after releases. Playwright enhances this pre-deployment testing by letting you simulate user behavior in detail. However, when features don’t work as expected in production, in a way that wasn’t detectable in a QA environment, you might end up relying on your users to tell you. By the time problems are identified, revenue (and trust) has already been lost. With Playwright, you can automate the process of testing your site, removing the need for a human to click through your pages. And with Checkly’s Playwright Check Suites, you can take your existing Playwright test suite, run it globally and on a cadence, meaning your monitoring will be the first place you’ll hear about problems with the critical user paths. Here’s how Checkly enhances the process of releasing new code:
  1. Write the Playwright Script: Automate the login, product search, and checkout steps. Run initial tests through the Checkly network to make sure it’s working from all your users’ geo-locations.
  2. Configure Monitoring Settings: Define how often the script should run (e.g., every 15 minutes) and from which locations (e.g., Ireland, Frankfurt).
  3. Set Up Alerts: Notify your team if a Playwright test fails, ensuring quick resolution of issues.
  4. Deploy to Production: Use Checkly’s CLI to deploy your Playwright tests and start monitoring your production environment.
With Checkly running your Playwright monitors, you can find and resolve issues before your users are affected.

​
What to Consider When Getting Started with Monitoring in Production

But what do you need to consider when you start globally monitoring in production? How do you make your Playwright tests geo-location aware? How can you avoid having your tests counted as real users? And should every test become a Playwright monitor? We covered all these questions in a recent webinar.
If you want to learn how to apply different Playwright settings depending on the execution environment, watch the recording. This guide will include essential snippets below.

​
Environment-Aware Playwright Tests

Once your Playwright tests run locally, in CI/CD, and as scheduled monitors, they need to be aware of their execution environment. A local Playwright test executed with npx playwright test needs to target localhost, a CI/CD test run via npx checkly pw-test or npx checkly test needs to target environment-specific preview URLs, and global production monitoring might need to be aware of its execution region. To differentiate your Playwright test runs in Checkly, use the provided environment variables.
environment.ts
export function getEnvironment(): Environment {
  // Playwright running via:
  // $ npx checkly pw-test
  // $ npx checkly test
  const isTest = process.env.CHECKLY_RUN_SOURCE?.startsWith("TEST");
  if (isTest) return "CI";

  const isMonitoring = process.env?.CHECKLY_RUN_SOURCE?.startsWith("TRIGGER") ||
    process.env?.CHECKLY_RUN_SOURCE?.startsWith("SCHEDULE") ||
    process.env?.CHECKLY_RUN_SOURCE?.startsWith("GROUP") ||
    process.env?.CHECKLY_RUN_SOURCE === "CLI_DEPLOY" ||
    process.env?.CHECKLY_RUN_SOURCE === "DEPLOYMENT" ||
    process.env?.CHECKLY_RUN_SOURCE === "SCHEDULE";

  if (isMonitoring) return "PROD";

  // Playwright running locally via:
  // $ npx playwright test
  return "DEV";
}
After knowing about your Playwright environment, adjust baseURL, retries, and any other additional configuration.
base-url.ts
/**
 * Evaluate Playwright's `baseURL` depending on the environment
 */
export function getBaseUrl(env: Environment) {
  const isDev = env === "DEV";
  if (isDev) return "http://localhost:3000";

  const isCI = env === "CI";
  if (isCI) return process.env.PREVIEW_URL;

  const isProd = env === "PROD";
  if (isProd) return "your-production-site.com";
}
If your site is geo-location aware, use process.env.CHECKLY_REGION to adjust your tests based on where your tests are executed. 
export function getLocationCountry() {
  if (process.env.CHECKLY !== "1") return null;
  if (process.env.CHECKLY_REGION?.startsWith("eu-central")) return "DE";
  if (process.env.CHECKLY_REGION === "eu-west-1") return "IE";
  // more location handling
  // ...

  return null;
}

​
Request Blocking to Avoid Incorrect Analytics

If you start running Playwright against production sites, you must ensure not to mess with possible client-side analytics. Luckily, request interception is quickly done in Playwright. 
page.route("**/*", (route) => {
  const blockedDomains = ["analytics.example.com", "tracking.example.com"];

  if (
    blockedDomains.some((domain) =>
      route.request().url().includes(domain),
    )
  ) {
    console.log("Blocking:", route.request().url());
    return route.abort();
  }

  return route.continue();
});

​
Monitoring Selection via Playwright Projects

Playwright test suites running over ten minutes are very common. And while it makes sense to run all tests before going live and deploying to production, a slow test suite is not ideal for monitoring because it will delay sending triggered alerts. If your Playwright monitoring takes ten minutes to finish, your users might already face production issues for ten minutes before you get notified. Playwright test selection for testing and monitoring To guarantee quick and timely alerts, select only the most critical user journeys to become Playwright monitors. Playwright Check Suites let you select tests based on Playwright projects and tags.
checkly.config.ts
export default defineConfig({
  checks: {
    playwrightChecks: [
      {
        name: "Playwright Search Monitoring",
        logicalId: "search-monitoring",
        // Select the tests defined in
        // the `Search` Playwright project
        pwProjects: ["Search"],
      },
      {
        name: "Playwright Login Monitoring",
        logicalId: "login-monitoring",
        // Select tests tagged as `@login`
        pwTags: ["@login"],
      },
    ],
  },
})

​
Solving the Cultural Problem: DevOps in Practice

One of the biggest challenges in monitoring and testing is the cultural divide between development and operations teams. By adopting monitoring as code, you bridge this gap. Developers can write and maintain monitoring scripts using the same tools and workflows they use for application code. This approach aligns with the core principles of DevOps, where operational problems are solved using development practices.

​
Conclusion

Testing in production is no longer a meme—it’s a necessity. With tools like Playwright and Checkly, you can automate end-to-end testing, monitor your application in real-time, and ensure it delivers a seamless experience to users. By treating monitoring as code, you empower your team to proactively address issues, reduce downtime, and improve overall reliability. Whether you’re running a small startup or a large enterprise, integrating Playwright and Checkly into your workflow can transform how you approach testing and monitoring. So, start testing in production today—your users will thank you!