Checkly Blog: Monitoring Insights & Trends

How Upstash Monitors Every Redis Replica with Checkly

Ilter Kavlak — Fri, 10 Jul 2026 00:00:00 GMT

This is a guest post by Ilter Kavlak, Site Reliability Engineer at Upstash. It's a companion to his post on the Upstash blog, which covers the internal side of the same monitoring stack.

There's a support ticket every SRE dreads: "is something wrong with my database?" The outage is bad enough. Worse is the possibility that the customer knew first.

At Upstash, we treat that scenario as two failures rather than one: the incident itself, and the uptime monitoring gap that let a customer beat us to it. We write a postmortem for the gap, too.

This post is about the layer we built to make sure that never happens: a fleet of Checkly checks that watches our platform from the outside, the way a customer would. One check per replica, running every minute from 18 locations around the world, all of it defined in Terraform. I'll walk through what we check, how a failing check reaches a human, and how we monitor a product where "reachable" and "working" are very different claims.

Why every replica gets its own uptime check

Upstash Redis is multitenant and multizone. Every database is backed by at least two replicas, and with the Prod Pack those replicas live in different availability zones. When a replica dies, the survivors elect the most up-to-date among themselves and the database keeps serving; customers notice nothing.

That resilience is exactly what makes replica-level monitoring easy to skip. If your checks only watch the database endpoint, a dead replica is invisible: the endpoint stays green while you quietly run with less redundancy than you promised your customers. And the next failure is no longer a non-event.

So our rule is blunt. A dead replica is an incident even when no customer can feel it. Every single replica gets its own uptime check, and losing any one of them wakes the oncall.

A couple hundred URL monitors, all in Terraform

Because Upstash Redis speaks HTTP alongside the Redis protocol, health checking stays simple. No Redis client, no connection pooling, no scripted scenario. Each replica exposes a /ping endpoint over HTTPS, and a 200 means that replica is serving its databases the same way it would serve a real client.

We use Checkly URL monitors for these, defined in Terraform. Each replica is one module block:

module "check-eu-central-1-r2" {
  source = "../modules/checkly_url_monitor"
  name              = "eu-central-1-r2"
  base_url          = "https://<replica-endpoint>.upstash.io/ping"
  frequency_minutes = 1
  locations         = var.default_locations
  alert_channel_ids = local.default_alert_channel_ids
}

This is Monitoring as Code, and it's the feature that made us commit to Checkly. Our monitoring configuration lives in the same repository as the rest of our infrastructure, and a single apply deploys or updates the whole fleet. By now, the file reads like a census: one block per replica, a couple of hundred in total, spanning AWS and GCP regions from Tokyo to São Paulo. When a new replica joins a cluster, a new block gets written, reviewed, and applied like any other infrastructure change. No monitor is ever created by clicking around a dashboard; if it isn't in git, it doesn't exist in Checkly.

That discipline would be optional at two checks. At two hundred, it's the only way to be sure nobody forgot one.

Redis isn't the only product in that file. Upstash Vector and Search replicas get the same per-replica treatment, except their checks go beyond a ping: each one upserts a record and runs a query against it, because an index that accepts connections but can't answer queries is not "up" in any sense a customer cares about. Keep that idea in mind for the QStash section below.

From 18 locations, every minute

The hardest-working parameter in that module block is locations. Upstash serves clients globally, with replicas placed close to their readers, so a replica answering from one vantage point proves very little. Every monitor runs from all 18 of the Checkly locations we've configured, covering North and South America, Europe, Africa, and Asia-Pacific. A routing problem that only affects clients in Singapore is a full outage for a client in Singapore, so our monitoring treats it as one.

Each monitor runs every minute, and a failed request is retried after a short backoff before anything alerts, since a single dropped packet somewhere on the public internet is no reason to wake a human. And we refuse to conflate slow with down: a response over five seconds marks the monitor degraded, and past ten it counts as failed. Two different signals, routed two different ways.

How a failing check reaches a human

Every monitor is wired to the same set of alert channels, also defined in Terraform:

Opsgenie pages the oncall on failure. This is the loud one.
Slack receives both failures and recoveries, giving the rest of the team context without waking anyone.
Degraded-state notifications land in a Slack channel of their own. Slowness deserves eyes but not a 3 a.m. page, and it shouldn't drown in a busy alerts channel either.

When a replica check fails, the oncall is looking at it within a couple of minutes, whatever the hour. None of that is remarkable. The habit worth writing down is how incidents end: we don't call anything resolved because internal dashboards look healthy. Resolved means the Checkly check is green again, from all 18 locations. Internal metrics can tell you a process is running; only the external check can tell you a client on the other side of the planet is actually being served. The second claim is the one customers experience, so it's the one that closes the incident.

QStash: proving messages actually get delivered

For QStash, our message queue and task scheduler, a health endpoint would prove almost nothing. QStash exists to deliver messages; it could happily return a 200 for every publish, deliver none of them, and a ping check would stay green the whole time.

So instead of a ping, QStash gets a Checkly API check that uses the product the way a customer would. The check publishes a real message through the QStash publish API and points it at an Opsgenie heartbeat endpoint, so every successful delivery resets the heartbeat's timer.

One message tests the whole pipeline, and the two ways it can fail are distinguishable:

The publish fails. QStash is unreachable or rejecting requests. The Checkly check turns red and pages the oncall.
The publish succeeds but nothing arrives. QStash accepted the request and then failed to deliver it. The heartbeat falls silent, Opsgenie notices the silence, and the page goes out.

Either way a human is investigating within minutes, and the alert already names which half of the flow broke before anyone opens a dashboard. That's a serious head start.

My favorite property of this check is that it's indistinguishable from a customer. Same publish API, same delivery path, same retries as real traffic. There is no separate test mode that could pass while production fails.

What Checkly doesn't cover, on purpose

External checks answer one question: would a customer be served right now, from where they are? Two other layers answer the questions they can't.

Internally, a federated Prometheus stack watches the clusters themselves and catches pod-level trouble. And in every region, a fleet of benchmark clients (deployed deliberately outside the clusters they measure) pushes a steady mix of reads and writes around the clock, recording every latency it observes. That fleet is our second, independent paging path, and its historical dashboards are where slow-moving problems like release regressions and noisy neighbors show up as trend lines long before they'd trip any single alert. Both layers are managed the same way as the Checkly monitors: Terraform modules in the same repository, one workflow for everything. I've written about that side of the stack in detail on the Upstash blog.

Layer	Question it answers	Who gets woken
Checkly per-replica checks (Redis, Vector, Search)	Is each replica serving traffic, from every region, right now?	Oncall
Internal cluster monitoring	Is the software healthy on the inside?	Oncall
Continuous latency benchmarks	Is each replica as fast as it should be?	Oncall on failure, internal alerts on drift
QStash end-to-end check	Does the product do its real job, end to end?	Oncall, with the broken half already identified

Every layer exists to cover a blind spot in the others. But when internal and external monitoring disagree, the external answer wins the argument, because it's the only one describing what customers actually experience.

The philosophy, in five sentences

Watch every replica, because redundancy you can't verify isn't redundancy. Watch from wherever your users actually are. Treat ‘slow’ as its own signal rather than a milder form of down. Test the thing the product promises to do, not just whether it answers. And don't declare victory until the outside world agrees.

None of this prevents outages; hardware still fails, zones still have bad days, and the internet remains the internet. What it buys is time and clarity: minutes from failure to a human engaged, an alert that names what broke, and a client's-eye definition of "fixed." The ticket asking "is something wrong?" should only ever get one reply: "yes, we know, and we're already on it."

Upstash runs a couple hundred Checkly monitors, every one of them defined in code. If you want the same workflow, start with Monitoring as Code: define your monitors in TypeScript with the Checkly CLI, or use the Terraform provider like Upstash does. Start monitoring for free.

]]>

Autoscaling Checkly Private Location Agents in Kubernetes with KEDA

Edvinas Janusevicius — Thu, 02 Jul 2026 00:00:00 GMT

Monitoring load is not always steady. A team might add a new batch of checks or run several ad hoc tests during a rollout. When that happens, your Private Location agents need to pick up more work at once.

If there aren’t enough agents available during a burst, checks start piling up in the queue, which can delay or disrupt check execution. But solving this by running a high number of agents around the clock has the opposite problem: most of that capacity sits idle until the next busy period.

Autoscaling solves both ends of that problem. You scale agents up when checks queue, scale back down when the burst clears, and stop guessing in between. This guide covers two things that landed together: the new Private Location metrics that show you whether your agents are running efficiently, and the KEDA setup that automatically scales your agent deployment.

How do you know if a Private Location has enough agents?

You can now monitor Private Location health with Checkly, so capacity stops being a guessing game. Open any Private Location in the UI and you'll find three metrics that tell you exactly how your agents are doing:

Queue duration: how long check runs wait before an agent picks them up. This is the number to watch. If it creeps up, your agents may be falling behind on work.
Agent count: how many agents are currently reporting for the location.
Scheduled check runs: how many runs are queued for the location right now.

These metrics are also available via our Prometheus v2 integration.

Together, these signals help you understand whether your agents are healthy and whether your Private Location has enough capacity. For example, if two agents are reporting but queue duration keeps climbing and scheduled runs are stacking up, you may be under-provisioned. If ten agents are reporting and queue duration stays near zero all day, you may be running more capacity than you need. Those are the two states autoscaling exists to fix, and now you can see which one you're in.

Scaling on the right signal

Checkly agents are stateless: they sit in your network, pull scheduled checks from a shared queue, run them, and send the results back. You can add or remove agents freely without losing or duplicating work. That's the property that makes scaling safe.

To develop a practical approach to autoscaling, we studied a range of large Private Location setups across customers, including DVAG, LinkedIn, Sherwin-Williams, and Auto Trader. Looking at how those fleets handle different checks and workloads helped us identify the signal that best reflects a location’s current demand.

For teams running agents on Kubernetes, that signal is checkly_private_location_check_runs. This gauge reports the number of check runs routed through a Private Location, broken down by state.

Two states matter for capacity:

queued: the check run is scheduled, but no agent has picked it up yet.
inflight: an agent is running it right now.

Add those two and you have the total live load on the location, which is the signal you want driving replica count. To use this metric, make sure Prometheus V2 metrics are enabled for your account.

ℹ️ If your existing KEDA config still uses

checkly_private_location_oldest_scheduled_check_run, consider switching to checkly_private_location_check_runs with the queued and inflight states. The older metric is useful for spotting backlog, but queued plus inflight gives KEDA a more direct signal for current load.

Setting up KEDA

KEDA (Kubernetes Event-Driven Autoscaling) reads a Prometheus metric and scales a Kubernetes deployment up or down based on that value. You'll need KEDA installed in the cluster, your agents deployed via the Checkly agent Helm chart, and Prometheus V2 enabled for your Checkly account.

Here's a ScaledObject with sensible defaults. Adjust the bounds to your workload:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: checkly-agent-autoscaler
spec:
  scaleTargetRef:
    namespace: <namespace_for_agent_deployment>
    name: <agent_deployment_name>
  minReplicaCount: 2
  maxReplicaCount: 10
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleUp:
          policies:
            - type: Pods
              value: 1
              periodSeconds: 60
        scaleDown:
          selectPolicy: Min
          policies:
            - type: Pods
              value: 1
              periodSeconds: 60
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus-k8s.monitoring.svc.cluster.local:9090
        metricName: checkly_private_location_check_runs
        threshold: "1"
        query: sum(checkly_private_location_check_runs{state=~"queued|inflight", private_location_slug_name="<slug>"})

The query targets a single Private Location using private_location_slug_name. If you run multiple Private Locations, create one ScaledObject for each. Because the slug name is immutable, renaming the location in the UI won’t affect your scaling config.

If you deploy agents with the Helm chart, template the ScaledObject alongside your chart values. This keeps the autoscaler tied to the agent deployment instead of managing it as a separate manifest.

For a Prometheus instance outside the cluster, add an authenticationRef that points to a TriggerAuthentication with the required credentials.

How many pods will you get?

Set the threshold to match your agent JOB_CONCURRENCY. This defines how many checks one agent pod can process at the same time. With the default JOB_CONCURRENCY of 1, use threshold: "1". That means KEDA will try to keep roughly one queued or in-flight check run per agent pod, within your min and max replica limits.

For example, with minReplicaCount: 2 and maxReplicaCount: 10:

Queued + in-flight check runs	Resulting pods
0	2 (idle floor)
1	2
3	3
7	7
20	10 (capped)

Your browser does not support the video tag.

Autoscaling in action: a burst of checks spikes the queue, the agent count scales from 1 to 11 to clear the backlog, then settles back to a right-sized floor.

A few rules of thumb for the bounds:

Keep minReplicaCount at 2 or higher. One agent is a single point of failure, and if it dies your whole location goes dark until a replacement spins up.
Set maxReplicaCount above your expected peak load, because check runs that pile up past the cap get dropped after the queue's six-minute TTL.
Leave threshold at "1" unless you've raised JOB_CONCURRENCY; packing more checks per pod can delay long-running checks.
If you want to scale to zero when a location is completely idle, set minReplicaCount: 0 and tune cooldownPeriod, which controls how long KEDA waits after load drops before scaling all the way down.

Don't kill checks mid-run

Scaling down means evicting pods, and an evicted pod might be running a check. Checkly handles this for you: an in-flight check on a terminating pod gets rerun on another agent after a 300-second timeout, so a scale-down never silently loses a result. To give a pod room to finish its current work before Kubernetes sends SIGKILL, set terminationGracePeriodSeconds above that window on the agent pod spec:

spec:
  template:
    spec:
      terminationGracePeriodSeconds: 330

Set it to match your longest check type. Most checks finish fast, but Playwright Check Suites can run up to an hour, so bump the grace period accordingly:

Check type	Maximum runtime
API, TCP, DNS, ICMP	30 seconds
Browser	4 minutes
Multistep	4 minutes
Playwright Check Suite	60 minutes

If you run Playwright Check Suites in this location, terminationGracePeriodSeconds can go as high as 1800.

What this gets you

Your checks stay reliable under load. When a deploy or a traffic spike floods the queue, agents scale up to clear it instead of letting runs wait. That means faster, more dependable detection from inside your own infrastructure.

You also avoid paying for idle capacity. Rather than keeping enough agents running all day to handle occasional peaks, you can maintain a smaller baseline and scale up only when needed. Across multiple Private Locations, that unused capacity can add up quickly.

Get started

Private Locations are available on Checkly's Team and Enterprise plans. If you're already running agents on Kubernetes, follow the Private Location Autoscaling guide for the complete setup, including how to verify that KEDA is receiving the metric. Enable Prometheus V2, apply in the ScaledObject, and run a batch of checks to see how your agent deployment scales with demand.

Already using Private Locations? Open a location in the Checkly UI to view its queue duration, agent count, and scheduled check runs. These metrics help you quickly identify whether you have too little or too much capacity.

New to Checkly? See pricing to find the plan that fits, or start for free to try the rest of the platform.

]]>

Rocky reads your stack, not just your check

Stefan Judis — Thu, 04 Jun 2026 00:00:00 GMT

You get an infrastructure alert. Something's off, so you task your agent to investigate, and it ends up grepping through the logs. You both wonder if there's customer impact.

There’s a simple rule: AI agents in the incident loop are only as good as the signal you feed them. And logs alone aren't a great signal.

Synthetic monitoring at least solves the impact question. When a check fails, something user-facing is broken. You know what broke, not why. OpenTelemetry traces have answered that why for a while, and Checkly supports them natively. But there was an unacceptable catch in the agentic world: a human still had to correlate the two by switching tabs, finding the right service, copying a trace ID, and walking the spans. By the time they had an answer, the next check had already run.

"We have a lot of traces collected, but people maybe don't know they exist… I think Checkly is a way to integrate traces into developers' workflow."
— Observability lead, leading public-safety tech company

Rocky AI, Checkly's agent, now does that investigation walk for you. If a check fails, Rocky reads your OpenTelemetry traces and walks the spans behind the failure. When you're on call, "Where do I even start looking?" becomes "Let me validate my agent's fix!".

Your browser does not support the video tag.

Rocky Analysis using OTEL traces

What Rocky AI reads, per check type

Good signal in, good answer out. So Rocky's context comes in three layers:

The failure itself (whatever the check type produces: request, response, headers, traceroute, assertions, test source, etc.)
A set of on-demand tools Rocky reaches for when the picture is incomplete (the last passing run, PCAP data for API checks, Playwright trace files for browser failures)
Context provided by a human (the check description field, the tags, the "add context" box in the RCA UI)

OTEL traces are the newest tool in that on-demand layer.

Source	API	Playwright	Browser	MultiStep	DNS	TCP	ICMP	Notes
Check Result Payload	✅	✅	✅	✅	✅	✅	✅
Check Description	✅	✅	✅	✅	✅	✅	✅	Description context
User Context	✅	✅	✅	✅	✅	✅	✅	UI text box
Tags	✅	✅	✅	✅	✅	✅	✅	Tag names
Last Passing Result	✅	✅	✅	✅	✅	✅	✅	On-demand tool
OTEL traces (new!)	✅	✅	✅	✅	✅	✅	✅	On-demand tool
PCAP	✅	-	-	-	-	-	-	API-Checks only
Playwright Trace	-	✅	✅	✅	-	-	-	On-demand tool

What Rocky AI does with OTEL traces

Once you set up traces, every check Checkly runs injects a W3C traceparent header (tagged tracestate: checkly=true) on its outgoing requests. Your backend's OpenTelemetry instrumentation continues that same trace, so your spans share the trace ID Checkly generated. When those traces are sent back to Checkly, it stitches them to the originating check run by trace ID. That's what puts your backend traces right alongside a failing check, giving Rocky the evidence to pinpoint the root cause.

A 500 error on an API or Playwright check might trace back to a downstream identity provider returning DNS errors, a Clickhouse server timing out, or a third-party payment API throwing 502s through your load balancer. The same check failure could have countless reasons, different stories in the spans, and multiple teams that own the fix.

Rocky classifies (INFRASTRUCTURE_ERROR, DOWNSTREAM_DEPENDENCY), names the offending service, and writes a root cause, user impact, and suggested fix. It all lands in whatever alert channel you already use, Slack, PagerDuty, a webhook.

There's no separate add-on. If your traces are flowing into Checkly, Rocky reads them.

Remember the agent you tasked at the top of this post? The one grepping logs? It can read this analysis too.

The same root cause analysis, for your coding agent

Rocky's analysis is also available via the Checkly CLI, and who loves using CLI tools? Your agent!

Make sure to install the Checkly skills, point your coding agent at a failing check, and ask it to investigate. It picks up the right CLI commands on its own and reads the same classification, root cause, user impact, and suggested fix. Including, when it's available, the OTEL trace Rocky walked.

Your browser does not support the video tag.

Agent Conversation using Rocky's Root Cause Analysis

That's the agentic layer in action. There's no magic. Just CLI commands and agents helping each other out with good signal. And what about you?

You don't need to post screenshots into Slack threads, or investigate by hand. You're only in charge of validating the fix! Welcome to the new world.

Wire up OTEL and inspect the why

If you're already on Checkly with OTEL flowing in, Rocky is already walking those traces. Enjoy! Open a failing check and read the analysis.

If you're not yet sending traces to Checkly, the setup docs get you there.

Without traces, Rocky's intelligence stops at the border of your app. With them, your check failures come with the why attached so you can stop hunting for it. And your agent will happily stop grepping logs. Win-win!

]]>

Two AI agents, one incident: Rocky AI comes to the terminal

Stefan Judis — Wed, 29 Apr 2026 00:00:00 GMT

A Playwright Check fails at 2 am. The login flow is broken. Until today, that alert triggered a human to get up, open the Checkly dashboard, copy Rocky AI root cause analysis (RCA), and then tell an agent to get to work.

There were two AI agents, one incident, and no way for them to talk to each other.

The extended checkly checks and new checkly rca CLI commands close that gap. Your coding agent can now pull Rocky AI's analysis into its ongoing work, read the diagnosis, and go fix the code. Everything happens in the same session.

There's no poor soul on call required to pass around information.

Meet the two agents

One sees production from the outside. The other sees the code from the inside.

Rocky AI is the one watching your production site. It knows about failed checks, error messages, monitoring scripts, logs, metrics, network waterfalls, packet captures, and Playwright traces. Its entire and only job is to come back with a classification, a plain-English root cause, a user-impact assessment, evidence, and a suggested fix.

Your coding agent, on the other hand, is the one handling your application. If it uses the agentic layer, it can query databases, tail logs, hit APIs, and poke at your entire infrastructure. It can see production from the inside, the way the code sees it.

What it couldn't reach, until now, was Rocky AI analysis itself. Rocky sits a layer above the raw signals. It's a specialist who's already analyzed everything your monitoring produces into one diagnosis. For your coding agent, that's gold. Your agent skips the evaluation and already knows the kind of bug plus the evidence to reason from. Without Rocky's analysis, your agent has to redo the triage from scratch, which only burns tokens and risks the wrong fix. Especially on production alerts, every minute counts.

Until now, those two agents have never met. Rocky AI analysis stayed inside the Checkly UI. Your coding agent had plenty of reach, except for your alert analysis. This changes now.

Rocky AI analysis, in your terminal

If you have automatic Rocky analysis enabled, the root cause analysis is already embedded in the check result. Your agent doesn't need a new command. checkly checks already returns the results:

# access the failing checks
npx checkly checks list --status failing

# access check results
npx checkly checks get <check-id>

# access the root cause analysis for the current error group
npx checkly checks get <check-id> --error-group <group-id>

The error group's output includes the root cause, user impact, cited evidence (HTTP traces, assertions, timings), and reference links. Everything Rocky already checked and analyzed is available for you and your agent.

When should you use `checkly rca` directly?

There are two cases where you reach for the dedicated rca command:

You've turned off automatic analysis to control your invocation quota, and you want to trigger RCA only for specific failures.
You want to analyze a past failure that never got an automatic analysis. It could be a flaky check from last week, a historical outage, anything predating when you enabled Rocky.

# Run a fresh analysis on an error group
npx checkly rca run --error-group <group-id> --watch

# Fetch a previously-triggered analysis by its RCA ID
npx checkly rca get <rca-id> --output json

The rca command supports different output formats: detail(default, human-friendly), md (good for PR descriptions), and json (the one agents love).

Not in the terminal? The same data is in the public API

There are three new endpoints to analyze your failing checks and start new Rocky analyses:

GET /v1/root-cause-analyses/{id} to retrieve an existing root cause analysis
POST /v1/root-cause-analyses/error-groups/{errorGroupId} to generate a new root cause analysis for a check error group
POST /v1/root-cause-analyses/test-session-error-groups/{testsessionErrorGroupId} to generate a new root cause analysis for a test session error group

From alert to PR — a fast and optimized agent loop

Here's the workflow end-to-end with an agent driving.

An alert is triggered. Rocky has already analyzed what's going on. It pulled the evidence, classified the failure, and wrote up the root cause. You hand it over to your agent, telling it only that "something's off".

From there, your agent already knows which commands to reach for. The Checkly skill taught it the playbook. It finds the failing check, accesses the analysis, and gets to work — pointed at the right surface, whether the fix lands in the check config, the production code, or infrastructure. The investigation is already done. Less guessing, quicker resolution.

See the full loop in motion. Stefan demos the end-to-end agentic workflow, from generating checks with coding agents to resolving incidents with Rocky AI, on the agentic workflow webinar.

Get started

Install or upgrade to the latest Checkly CLI:

npm install checkly@latest

Install the Checkly skills so your agent knows which commands to use:

npx checkly skills install

Tell your coding agent to investigate a failing check. With automatic analysis on, Rocky AI analysis comes inline:

I've received a Checkly alert. Can you investigate?

And that’s it!

Previously, there were two AI agents, one incident, and a human connecting them.

Now, there's a single prompt. That's the agentic layer in action. Specialized agents work together to resolve production issues as quickly as possible. Isn't this beautiful?

]]>

How to Monitor a Shopify Store with Playwright and Checkly

Vince Graics — Mon, 13 Apr 2026 00:00:00 GMT

This is a guest post by Vince Graics, Staff QA Engineer at World of Books.

If you're running a Shopify storefront and want reliable synthetic monitoring, you'll hit a wall. Shopify's bot detection doesn't care that your headless browser is friendly; it sees datacenter IPs and acts accordingly. Cart API calls get hit with 429 rate limits, Cloudflare challenge pages pop up mid-check, and you're left wondering whether the bug is in your code or in the platform fighting you.

At World of Books, a UK-based secondhand bookseller running on Shopify, we ran into all of this while setting up monitoring with Checkly. The good news: Shopify actually has a fix for the bot-protection problem. The bad news: almost nobody talks about it, and there are a couple of non-obvious traps along the way.

This guide covers how we solved three Shopify-specific problems: bot protection headers, consent pop-up interception, and checkout flow testing. The patterns here work for any Shopify store. Swap out the selectors, plug in your store's URLs, and you have a working setup to monitor your Shopify store with Playwright.

What makes Shopify stores hard to monitor?

Shopify stores introduce platform-level obstacles that standard Playwright tests don't account for.

Bot protection. Shopify evaluates incoming traffic based on IP reputation, headers, and behavioral patterns. Monitoring checks run from cloud infrastructure, and datacenter IPs get treated with suspicion. Sometimes you get a 429 Too Many Requests on a cart API call. Sometimes Cloudflare intercepts the request entirely and serves a challenge page. And sometimes everything looks like it loaded, the page shell renders, the layout is there, but the interactive parts are quietly broken. A search page shows zero product cards. A mini-cart notification never appears.

Privacy and consent interception. Shopify's privacy SDK fires on every page load, showing a cookie consent banner and blocking script execution until the user accepts. In a headless context, that banner just sits there, adding flakiness and wasted seconds to every run.

Checkout DOM isolation. Shopify's checkout runs on checkout.shopify.com with a completely different markup. The selectors you wrote for the storefront don't work on checkout, and the page structure can change between Shopify editions.

Each section below tackles one of these problems and builds on the solution.

How do you bypass Shopify bot protection?

Shopify provides a mechanism for authorized crawlers to identify themselves: HTTP signature headers. You register your crawler in the Shopify admin, and Shopify gives you three headers: Signature, Signature-Input, and Signature-Agent.

When your requests include these headers, Shopify's bot protection lets them through cleanly.

Creating your crawler signature in Shopify

From your Shopify admin:

Go to Online Store > Preferences
In the Crawler access section, click Create signature
Enter a descriptive name (e.g., "Checkly Monitoring")
Select the domain and an expiration period
Copy the Signature-Input and Signature values

The third header, Signature-Agent, is static; its value is always "<https://shopify.com>" (with quotes).

One thing to watch for: signatures have an expiration date. Set a calendar reminder to rotate them before they expire, or your checks will start failing again with zero code changes. When they expire, every check returns a challenge page and it looks like your entire store is down.

Storing headers as Checkly secrets

Add the three values as secrets in your Checkly account:

SHOPIFY_SIGNATURE
SHOPIFY_SIGNATURE_INPUT
SHOPIFY_SIGNATURE_AGENT

This keeps your credentials out of code and makes them available to all your browser checks at runtime.

Injecting the headers in Playwright

The simplest approach, and what you should try first:

await page.setExtraHTTPHeaders({
  'Signature': process.env.SHOPIFY_SIGNATURE!,
  'Signature-Input': process.env.SHOPIFY_SIGNATURE_INPUT!,
  'Signature-Agent': process.env.SHOPIFY_SIGNATURE_AGENT!,
});

setExtraHTTPHeaders adds these headers to every request the page makes. For many Shopify stores, this just works and you're done.

When you need scoped header injection

If your store uses third-party services that don't play nice with extra headers, setExtraHTTPHeaders can cause problems. At World of Books, we hit this with a third-party search provider: the extra headers on cross-origin requests appeared to interfere with its responses, and product listings silently came back empty. No errors, no warnings; just an empty product grid and a check that fails for reasons that take an hour to debug.

The fix is Playwright's page.route(), which scopes header injection to your own domain:

const crawlerHeaders: Record<string, string>
  = (process.env.SHOPIFY_SIGNATURE
    && process.env.SHOPIFY_SIGNATURE_INPUT
    && process.env.SHOPIFY_SIGNATURE_AGENT)
    ? {
        'Signature': process.env.SHOPIFY_SIGNATURE,
        'Signature-Input': process.env.SHOPIFY_SIGNATURE_INPUT,
        'Signature-Agent': process.env.SHOPIFY_SIGNATURE_AGENT,
      }
    : {};

await page.route('**/www.yourstore.com/**', route =>
  route.continue({
    headers: { ...route.request().headers(), ...crawlerHeaders },
  }),
);

Third-party requests pass through untouched. A few things to note:

All three headers are required together. If any are missing, skip the injection entirely; partial headers won't work with Shopify's validation. And spread existing headers first: route.continue() replaces headers entirely, so the spread preserves cookies, content types, and everything else the browser set. After your test, call page.unrouteAll({ behavior: 'ignoreErrors' }) to avoid route handler leaks between checks.

Scaling this with Playwright fixtures

Once you have more than a couple of checks, you don't want to repeat the header setup in every file. Playwright fixtures let you extend the base page object once and use it everywhere:

import { test as base } from '@playwright/test';

export const test = base.extend({
  page: async ({ page }, use) => {
    if (Object.keys(crawlerHeaders).length > 0) {
      await page.route('**/www.yourstore.com/**', route =>
        route.continue({
          headers: { ...route.request().headers(), ...crawlerHeaders },
        }),
      );
    }
    await use(page);
    await page.unrouteAll({ behavior: 'ignoreErrors' });
  },
});

Every check that imports test from this fixture gets crawler header injection automatically. No headers configured? The fixture is a no-op; your checks still work locally without Shopify credentials.

The consent banner problem is frustrating because it's invisible in manual testing (you click it once and forget it) but it breaks every automated check.

Playwright's page.addLocatorHandler() lets you register a handler that fires whenever a matching element appears. Think of it as a popup bouncer for your checks:

export const test = base.extend({
  page: async ({ page }, use) => {
    await page.addLocatorHandler(
      page.locator('[aria-label="Close popup"]'),
      async () => {
        await page.locator('[aria-label="Close popup"]').click();
      },
    );
    await use(page);
  },
});

This works well for stores with a single, consistent popup. If your store has multiple overlays (newsletter signups, region selectors), you can register additional handlers with different locators. The key is keeping the selectors stable; aria-label attributes tend to survive theme updates better than CSS classes.

How do you test Shopify checkout?

Checkout is where monitoring gets tricky, because you're crossing into checkout.shopify.com with its own DOM, its own selectors, and real money involved.

Setting up safe checkout testing with Shopify Flow

You can't just skip checkout monitoring; it's the flow that matters most to your business. But running real transactions in production sounds terrifying. Here's the approach my team at World of Books came up with, using Shopify Flow:

Get a company card
Create a dedicated test user on your production store
Set up a Flow automation: if this specific user purchases something, the workflow immediately refunds the order and cancels the trade
Run your checkout checks against this test user

You can now run regular checkout validations until you run out of the company card budget, and wait for the refunds. It's not elegant, but it works, and it tests the real checkout flow, not a staging approximation.

Deploying to Checkly

Once the checks pass locally, deploy with the Checkly CLI:

# Test against Checkly infrastructure first
checkly test --record

# Deploy
checkly deploy -f -o

Set your environment variables in the Checkly dashboard under the check group. The required ones: SHOPIFY_SIGNATURE, SHOPIFY_SIGNATURE_INPUT, SHOPIFY_SIGNATURE_AGENT. Add LOGIN_EMAIL, LOGIN_PASSWORD, and discount codes if you're monitoring authenticated or promotional flows.

Because everything lives in your repo as code, your Shopify monitoring ships through the same CI/CD pipeline as your application. That's the Monitoring as Code approach: your checks are versioned, reviewed in PRs, and deployed automatically. No clicking through a UI to update a threshold.

Troubleshooting common failures

Challenge page instead of storefront. Your crawler headers are missing or expired. Regenerate them in Shopify Admin and update the env vars in Checkly.

Login fails or loops. Test account credentials are wrong, the account is locked, or MFA got enabled. Use a dedicated test account that the store admin has allowlisted.

Discount code shows "not valid." The code expired, hit its usage limit, or targets the wrong region. Verify it in Shopify Admin > Discounts. Use unlimited test codes.

Checks pass locally, fail on Checkly. Almost always missing environment variables. Run checkly env ls to verify. Second most common cause: viewport mismatch. Set viewport: { width: 1920, height: 1080 } explicitly in your Playwright config.

Wrapping up

At World of Books, customer experience is the reason we monitor our Shopify storefront so closely. If search breaks, checkout stalls, or a pop-up blocks the flow, it's important that we know before our customers do. Shopify can be difficult to test with headless browsers, but once we added crawler-signature headers via Checkly secrets, used a Playwright fixture to centralize header injection and consent handling, and set up Shopify Flow to clean up checkout test orders, the setup became reliable. The checks now see the same storefront our customers see, without the 429s, Cloudflare challenges, or quiet failures that make Shopify monitoring so frustrating.

If your checks keep mysteriously failing, check your headers.

Get started:

]]>

Checkly Playwright Reporter: A Cloud Dashboard for Your Playwright Tests

Pırıl Kavlak,María de Antón — Wed, 01 Apr 2026 00:00:00 GMT

The Checkly Playwright Reporter is an npm package that sends the results of npx playwright test to Checkly as a cloud test session, including traces, screenshots, videos, and full debugging context. Run your Playwright suite in CI or locally, and every result gets a persistent, shareable home in Checkly with AI-powered analysis, richer trace-derived views, and a direct path to production monitoring.

It does not replace Playwright. It makes the output of Playwright much easier to work with.

Today, the reporter is available at no additional cost on all Checkly plans.

Why we built this

Checkly has supported Playwright for years. You could write Playwright tests, validate them with npx checkly test, and deploy them as production monitors with npx checkly deploy. That workflow is powerful for production monitoring.

But most teams run npx playwright test hundreds of times a week in GitHub Actions, GitLab CI, or locally, and those results never reach Checkly. The evidence ends up scattered across CI logs, local trace files, and one-off HTML reports. That's manageable for an individual, but it breaks down for a team.

The Checkly Playwright Reporter helps close that gap by giving teams a shared place to review Playwright runs in Checkly.

From there, teams can run Playwright Check Suites with npx checkly pw-test, and when a test is important enough, npx checkly deploy can take it into production monitoring.

But it is not only about having a place to persist your test results. Once those results are in Checkly, Rocky AI can help analyze failed runs directly in the UI.

When enabled, Rocky AI can include AI-assisted root cause analysis and visibility into recurring failures across runs.

What you get beyond a hosted report

Today, there are several ways to get Playwright test results into the cloud, including Microsoft's own Playwright Workspaces. But most of them stop at hosting: they take the same local Playwright HTML report and put it behind a URL. That solves the sharing problem. It does not solve the understanding problem.

The Checkly Playwright Reporter goes further:

Filterable network view — every request and response captured from the trace, organized so you can focus on the failures instead of scanning a raw list.
Filterable browser console — browser console output pulled into a cleaner view, including an errors-only filter.
Automatic secret scrubbing — secrets can be scrubbed before upload, including support for the CHECKLY_SECRET_ prefix and sensitive header redaction.
System performance graphs — CPU and memory usage tracked throughout the test run, so you can spot whether a flaky or failing test might be tied to resource pressure.
Terminal progress and summary table — a clearer terminal view of what passed, failed, flaked, or got skipped across projects.
Automatic git detection — branch, commit, and author metadata can be attached to sessions automatically in CI and local git environments.
History across runs — instead of treating every run as an isolated local report, you can review patterns over time and see when the same failures keep coming back.

Hosting a report gives you a link. A reporting system gives you answers.

How to set up the Checkly Playwright Reporter

You need:

Node.js 18 or newer
Playwright 1.40 or newer
a Checkly account
your CHECKLY_API_KEY and CHECKLY_ACCOUNT_ID

Install the package:

npm install --save-dev @checkly/playwright-reporter

Add it to your playwright.config.ts:

import { defineConfig } from '@playwright/test'
import { createChecklyReporter } from '@checkly/playwright-reporter'

export default defineConfig({
  reporter: [
    ['list'],
    createChecklyReporter(),
  ],
  use: {
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
    trace: 'on',
  },
})

Set credentials:

export CHECKLY_API_KEY=your_api_key
export CHECKLY_ACCOUNT_ID=your_account_id

Then run your suite as usual:

npx playwright test

When the run finishes, you get a Checkly test session URL in the terminal. That is the entire setup. No cloud infrastructure to provision, no storage accounts to link, no role assignments to configure.

Try the Checkly Playwright Reporter

If your team already uses Playwright, this is the fastest way to stop losing useful test context after the run.

Start here:

Frequently Asked Questions

Yes. The reporter is free on all Checkly plans today, including the free tier. As we learn from usage and continue improving the product, pricing may evolve over time. Our goal is to keep it highly competitive relative to other reporting tools and make it easy for teams to get started.

If you are already using the reporter and want to share feedback, get in touch! We'd love to hear from early users helping shape the product.

Yes. The reporter works with local and CI runs, as long as the Checkly credentials are available.

No. The reporter uploads test runs as Checkly test sessions. Production monitoring is the next step in Checkly’s Playwright monitoring workflow, where teams define Playwright Check Suites in a checkly.config.ts file and deploy them as monitors.

]]>

One CLI, Two Audiences: How We Built for Agents and Humans

Stefan Judis — Thu, 26 Mar 2026 00:00:00 GMT

Half of the Checkly CLI users are already coding agents. This is not a prediction — it's what the data shows today.

Since February, more and more agents have been using the CLI to manage and configure their Checkly monitoring setups. Right now, we're at 50% human and 50% agentic CLI users. And we predict that by the end of 2026, it won't be humans using the CLI; the agents will have taken over.

The terminal became the primary interface for AI agents doing real work in the Checkly ecosystem. Configuring monitoring, resolving incidents, and communicating outages; all this can be done automatically.

The CLI became the new agentic layer. So we asked: how do you build for two audiences at once?

The full agentic loop — from alert to resolution

To show what this looks like in practice, here's a real scenario: a Playwright check detects that the storefront's login flow is broken. The old way would be to manually open the Checkly dashboard, click through results, and switch between the IDE, alert information, and Git logs to figure out what's off. We now hand these tasks to the agent with a single prompt.

I just received a monitoring alert in Checkly about a broken login flow. Can you check what's up, investigate a fix, and communicate the issue?

It's a simple but powerful prompt. Here's what happens next:

The agent discovers Checkly capabilities. With the installed Checkly Skill, it pulls in the context it needs to act. Available commands, your account information and plan, investigation playbooks, and best practices; all these things become accessible through checkly skills.
The agent investigates. It calls npx checkly checks to get the current state of all monitors, identify the failing check, and analyze the Playwright error. It reads the check results, correlates them with your codebase, and additional context sources. Eventually, it finds the root cause.
The agent communicates. Before fixing anything, it tries to open an incident so customers know what's happening, but the CLI rejects the command and prompts for confirmation. Agents can't just declare a public incident without approval. Once confirmed, the incident goes live: "Login flow is broken, users cannot log in."
The agent provides a fix and deploys. It applies the fix, commits, and pushes. Your existing CI/CD pipeline takes it from there — the agent waits for the deployment to complete.
The agent verifies. Once deployed, it triggers the Checkly checks to confirm the fix actually works.
The agent resolves. If all the ad hoc tests pass, the incident is resolved with a summary of what happened and what was done. The status page goes green.

This might sound like future music, but you can see this flow in action below.

Coding agents can handle the full incident life cycle today. Detect, understand, communicate, fix, verify, resolve; all this can happen in a single agent session. All actions are enabled through the CLI. There's no need for you to open a monitoring dashboard. There's no context switching. Sit back and let the agent be the incident manager.

But how does this work in detail? How did the agent know what to do?

Checkly Skills: teach your agent how to handle your monitoring

Your agent didn't hallucinate, fetch docs, or browse the internet. It was tasked by the Checkly skill to rely on the built-in documentation accessible via npx checkly skills.

You can install the Checkly skill with a single command:

npx checkly skills install

This command drops a SKILL.md file into your project that any coding agent (Claude Code, Cursor, Codex, Copilot) can pick up. Everybody is raging about skills right now, but most skills have a fundamental flaw. Most skills are static. If you look at all the skills listed on skills.sh, you'll find that the majority of them are on-off documentation that you need to update manually.

The Checkly skill file is different because it's only a thin wrapper around the checkly skills command.

Run `npx checkly skills communicate` for the full protocol details.

### `npx checkly skills initialize`
Learn how to initialize and set up a new Checkly CLI project from scratch.

### `npx checkly skills configure`
Learn how to create and manage monitoring checks using Checkly constructs and the CLI.

#### `npx checkly skills configure api-checks`
Api Check construct (`ApiCheck`), assertions, and authentication setup scripts

Yes, you read that right, we've put optimized agent documentation right into the CLI itself. Required monitoring context ships within the tooling you use to manage your monitoring. Every CLI version update brings updated documentation, new workflows, and current best practices. There's no web search, no stale docs, no external files to maintain. Whenever you update the CLI, your agent gets smarter.

But we didn't stop there. Skills also use progressive disclosure across three levels:

Overview — npx checkly skills lists all available action categories: initialize, configure, investigate, communicate and manage.
Action — npx checkly skills investigate returns the detailed guide for that category — which commands to run, what to look for, and how to triage.
Reference — npx checkly skills configure api-checks returns construct-level documentation for any resource.

The built-in Checkly skills save tokens and tell your agent exactly what to do. No guessing. No hallucinations.

This single command transforms your agent into an up-to-date Checkly expert. But what can your agent handle for you now that it knows every corner of the Checkly ecosystem?

Structured output: the agent's dashboard

If you're facing issues and alerts, as a human, you head into Checkly, open the dashboard, click through the check results, and scan the graphs. And while you could task your agent to open the web app with a browser, this process couldn't be more inefficient. We released new CLI commands to enable the agent to access structured output, replacing the good old Checkly dashboard you know and love.

Read commands support multiple output formats optimized for agentic workflows:

npx checkly checks list --output json
npx checkly checks get 12345 --output md
npx checkly checks stats --output table

It's the same data you'll find in your Checkly web app, but rendered in text. Agents get parseable data they can reason about.

The new read commands cover the full monitoring setup:

checks list : All checks with current status. Filter by name (--search), tag (--tag), or check type (--type).
checks get <id> : Configuration, recent results, error groups, and analytics for a single check. Customize with --stats-range, --group-by, --metrics, and --filter-status.
checks stats : Availability, response times, and key metrics across multiple checks at once.

# Stats for all production API checks over the last 7 days
npx checkly checks stats --range=last7Days --tag=production --type=API

By exposing the monitoring data via the CLI, your agent now understands what an alert means and how to investigate. With access to your codebase, it can get to work and provide a fix.

Incident lifecycle through the CLI

Once the agent (and you) decide that you're facing a real issue, it's time to communicate it. The CLI covers the full incident lifecycle on your status pages:

npx checkly status-pages list          # discover your status pages
npx checkly status-pages get <id>      # inspect a specific page and its services
npx checkly incidents create           # open a new incident
npx checkly incidents update <id>      # post a progress update
npx checkly incidents resolve <id>     # close the incident

The same communication loop a human on-call would run: detect, communicate, resolve. But this time, you'll be faster, don't have to leave your terminal, and can supervise your agent to do the dirty work.

But isn't it dangerous to let your agent handle incidents? We thought so, too!

Guardrails: safe autonomy

Providing agents read-only commands was an easy decision, but what if wrong reasoning could have real consequences? What if the agent accidentally opens an incident, updates your monitors without you reviewing, or destroys your entire Checkly project, thinking it's "absolutely right"?

These actions and commands are deep in the danger zone, and you definitely don't want to wake up your teammates because your agent misconfigured something while you were vibing the time of your life.

We thought about how to handle these situations and decided to implement agent guardrails. If your agent decides to declare a public incident or deploy your monitoring, they need to ask you for confirmation first.

Critical commands implement an agent confirmation protocol. The CLI detects agent usage by checking for environment variables that coding agents set automatically (like CLAUDE_CODE, CURSOR_SESSION, or CODEX). If we detect that an agent is running a critical command, the CLI returns an exit code 2 with a JSON envelope:

{
  "status": "confirmation_required",
  "command": "incidents create",
  "changes": [
    "Will create incident \\"Login flow broken\\" on status page \\"Acme Store\\"",
    "Severity: major"
  ],
  "confirmCommand": "checkly incidents create --title=\\"Login flow broken\\" ... --force"
}

This forces the agent to present the changes to the user, wait for approval, and only then run the provided confirmCommand.

And what if the agent gets it wrong? What if it misdiagnoses the root cause or opens an incident for a false positive? That's exactly why the confirmation protocol exists. The agent proposes, you approve. A misdiagnosis becomes a learning moment, not an incident. The human stays in the loop for every action that has real-world consequences.

CLIs are the new agent interface

CLIs were built for humans first. But with a few design patterns, they become agent-ready. Here's what we learned building for both audiences:

Built-in documentation. Don't make agents search the web or rely on external docs. Ship agent-optimized documentation inside the CLI itself. When the tooling updates, the docs update as well. Minimize stale knowledge and hallucinations.
Structured output. Every read command should support machine-readable formats (--output json, --output md). Agents don’t care about colorful output, and humans shouldn't have to read raw JSON. Support both worlds.
Human/agent detection. Some CLI flows need to behave differently depending on who's running them. Detect the caller through environment variables and adapt — interactive prompts for humans, structured JSON responses, and additional command information for agents.
Agent guardrails. Critical commands need a confirmation protocol. Enforce human approval. Autonomy without oversight is just a liability.

These patterns aren't Checkly-specific. They're how every CLI should think about agent users. One CLI, two audiences — and the design choices made for agents make things better for everyone.

Half of our CLI users are already agents. Keep them coming!

Get started

Upgrade to the latest Checkly CLI:

npm install checkly@latest

Install the Checkly skill:

npx checkly skills install

Or, if you don’t have Checkly set up yet. Tell your agent the following:

Run 'npx checkly skills initialize' and follow the instructions.

It will figure out the rest.

]]>

Checkly and the Agentic Software Layer

Hannes Lenke — Thu, 26 Mar 2026 00:00:00 GMT

November 24th, the Opus 4.5 release turned around the entire tech industry. This was the moment when agents became capable. Capable enough to write solid staff-level code. Capable enough to reason about alerts, investigate root causes much faster than most engineers, and set up the reliability layer faster.

For me, this feels like an iPhone moment on steroids; the adoption of AI is accelerating much faster than any adoption curve I’ve seen over the past few decades. This rapid growth is fueled by early adopters who experienced their eureka moments early on, followed by many startup CEOs who have been actively developing prompting software during the holidays, making significant progress 10x quicker than ever before.

Now, AI is a topic in every customer conversation. While it took years to adopt DevOps, AI is on track to outpace that by a wide margin; in fact, it is already surpassing it.

The rise of AI agents introduced a new user of developer tools, one that doesn't click buttons, doesn't read dashboards, and doesn't care about your beautifully designed settings page. Agents live in the terminal. They reason over text. Over the last few years, we introduced Monitoring as Code (MaC), a workflow that enables developers to own reliability; in text. Accidentally, MaC is exactly what agents need.

You can see where we are going here. The work we did to make Checkly great for developers turns out to be the groundwork for making Checkly great for agents.

But "accidentally ready" isn’t a strategy - it was an opportunity.

Three Layers of Software Interfaces

Every software platform serves its users through interfaces. Historically, there have been two, sometimes three:

UIs are built for humans. They're visual, interactive, optimized for comprehension and exploration. Dashboards, forms, drag-and-drop builders. These are how people interact with software.

APIs are built for machines. They're structured, predictable, and designed for integration between systems. CI/CD pipelines, webhooks, Terraform providers. This is how humans have automated software.

CLIs are built to enable engineers to scale and automate their interactions with APIs. Especially, but not only for developer tools.

Turns out CLIs are great interfaces and became the agentic layer, and this is new. AI agents don't need pixel-perfect UIs or REST endpoints designed for static integrations. They need CLIs that output structured data in Markdown or JSON. They need progressive self-discovery, a way to explore capabilities without reading documentation. They need confirmation protocols for dangerous actions.

Most monitoring platforms were built for layer one. Some invested in layers two and three. Checkly is the first to double down on a new and redefined third layer. An application layer optimized for agents and for the modern software factory.

Why Code & CLI-First Platforms Win

Here's the thing about Monitoring as Code: it was already agent-friendly before agents existed.

When your monitoring lives in .ts files alongside your application code, an AI agent can read it, reason about it, generate new tests and checks, and deploy them the same way it works with any other code in your repository. There's no proprietary DSL to learn, no UI to scrape, no brittle browser automation required.

An agent working with the Checkly CLI can open your __checks__ directory, understand what's being monitored, identify gaps, write new checks using standard Playwright syntax, run npx checkly test to validate against our global infrastructure, and npx checkly deploy to ship the entire monitoring setup. Every step happens in the terminal, in code, using tools the agent already knows.

Legacy monitoring platforms trapped behind proprietary UIs and frameworks can't offer this. They'll bolt on a chatbot and call it AI. That's not the agentic layer: that's an AI-colored coat of paint.

Making the Implicit Explicit

Being code & CLI-first gave Checkly the agentic head start. But a head start doesn’t finish the race. We've shipped a new set of capabilities designed for the speed of this new world and purpose-built for agents.

Teaching your Agent Testing and Monitoring

Your agent understands how to configure Checkly today, because it can read the checkly.config and check.ts files. It also includes Checkly code in its training data, and if nothing else works, it can still fetch the docs to troubleshoot. But we thought, what if we could make it even easier?

Agent Skills are an open standard supported by Claude Code, Cursor, OpenAI Codex, and GitHub Copilot. Skills are providing agents with specialized knowledge on demand. Most skills out there are static files, though. We thought we could do better than providing some Markdown files!

Install the Checkly skill and see the magic happen:

npx checkly skills install

The Checkly SKILL.md file is your agents entry point to learn how to set up, configure and handle Checkly testing and monitoring. However, the static file doesn't include all the Checkly information. The Checkly skill is only a thin wrapper around the new npx checkly skills command. The new command that teaches your agent where to find construct documentation, other CLI commands, and monitoring workflows right in the CLI itself using npx checkly skills.

Once your agent discovers the npx checkly skills it can access agent-optimized docs to initialize, configure and manage the entire testing and monitoring setup. There's no relying on training data, no docs browsing, no context window compacting. Agents access the right docs when they need them.

But what does this mean in practice?

Creating your Monitoring Checks

The big difference between the agentic CLI layer and built-in product AI is that agents already understand and manage your codebase. They read and write files all day. With the right tools, coding agents have all the context they need to automatically set up and test your monitoring.

Your API endpoints, your login flows, your business logic; agents know about all of it. Task them to set up the monitoring and see your agent doing the work! In the previous world, you didn't have the time and energy to manage monitors for every page, endpoint, or application flow; now you do because your agent never gets tired.

Now, your agent can create, test, and deploy your entire monitoring!

More monitors also mean more alerts. Could your agent handle these, too?

Investigating Failures and Monitoring Results

If you received a monitoring alert in the old world, you usually log in to Checkly to understand what check failed and what triggered the alert. And while your agent could log in via a browser, it shouldn't need to do that to understand your monitoring setup.

A new set of read commands lets agents query and analyze your monitoring setup directly from the terminal:

# List all production API checks
npx checkly checks list --tag=production --type=API --output=json 

# Get 7-day stats for a check, grouped by location
npx checkly checks get 12345 --stats-range=last7Days --group-by=location 

# Analytics across all checks tagged "critical" 
npx checkly checks stats --tag=critical --range=last7Days

Every command supports --output table|json|md — table for humans, JSON and Markdown for programmatic agent consumption. Metric defaults adapt to check type: response time percentiles for API checks, Web Vitals for browser checks, latency and packet loss for ICMP.

With these new commands, your agent can access all the monitoring information you usually look at in the Checkly dashboard. Pair this ability with code access and let your agent figure out the issue.

Detecting issues is only one part of the story. Could your agent communicate issues, too?

Acting On Incidents

Agents can manage the full incident lifecycle on your status pages.

npx checkly incidents create --status-page-id <id> --title="API degradation" --severity=minor

npx checkly incidents update <id> --message="Root cause identified"

npx checkly incidents resolve <id>

Write commands implement a confirmation protocol designed for agent safety. When an agent runs a destructive or public-facing command, the CLI returns a JSON envelope describing the changes and provides a confirmation command.

This enforced confirmation is critical. Agents that can take action without guardrails are a liability. Agents with structured confirmation flows are a force multiplier.

If you put all these features together, your agent can handle your entire incident management while you drink your coffee. Don't believe me? Here's a video showing it entirely.

The Agentic Layer is Here

The shift to AI agents isn't coming. It's happening now. 50% percent of our active CLI users are already agents.

Development teams are using Claude Code, Cursor, and Copilot to write, test, and ship code. The tools in their stack that can't participate in that workflow will be replaced by tools that can.

Checkly is built for this moment. Upgrade your CLI, install the skill, and point your agent at your monitoring.

npm install checkly@latest
npx checkly skills install

… or, if you don’t have Checkly set up yet. Tell your agent the following:

Run 'npx checkly skills initialize' and follow the instructions.

… and it will figure out the rest.

]]>

12,479 Applications, Zero Ghosting: A Look at Checkly's 2025 Hiring

Lauren Singh — Thu, 19 Mar 2026 00:00:00 GMT

Hiring is one of the most human things a company does, and one of the least transparent. We want to change that. Here's an honest look at what our hiring actually looked like in 2025: the volume, the speed, and the funnel.

12 job postings. 24 people joined the team. 100% of applicants received a response.

12,479 applications. 12 job postings. A lot of trust placed in us.

In 2025, we received 12,479 applications across 12 job postings, an average of 1,040 applicants per posting. It's worth noting that some postings covered roles we hired for multiple times: for example, we brought on three Enterprise Account Executives from a single posting. Engineering, product, and design postings attracted the most interest, averaging around 1,105 applicants each, while go-to-market postings averaged 909.

Our most competitive posting was Senior Backend Engineer, with a total of 4,578 applicants.

Roughly 1 in 58 applicants progressed to interviews

We try to be thoughtful about who we move forward, and honest about the odds. Of 12,479 applicants, 217 progressed to the Hiring Manager interview stage, that's approximately 1.7%, or roughly 1 in 58. That's a tight funnel, and we don't shy away from saying so. What we will say is this: every single person who applied heard back from us. No ghosting, no black hole. We know that's not the norm, and we think it should be. Job searching is hard enough without silence on the other end. What made the difference for those who progressed? A genuine fit with the role, relevant experience for a developer tools company, and clear, async-friendly communication. We're a remote-first team, how someone writes and communicates matters.

Where our hires came from

Not every hire looks the same, and neither does every pipeline. Here's how our 24 hires broke down by source:

Actively applied: 60%
Referrals from the team: 20%
Sourced by a recruiter: 20%

The majority of our hires came through inbound applications, which means a well-written application genuinely matters. That said, referrals and sourcing play a real role too, and we're proud that team referrals account for a meaningful share of who joins us.

What candidates said about the experience

We ask every candidate who goes through our process to share feedback. In 2025, when asked how likely they were to recommend Checkly as an employer, the average score was 4.25 out of 5.

Here's what a couple of candidates had to say:

"The experience was overwhelmingly positive as I enjoyed speaking with all the stakeholders. Checkly is a place of high integrity that operates with the utmost professionalism."

"I am glad that even though I was not selected to continue the process, I got interviewed anyway. This is a good approach from the company and I hope you continue like that. The application form was unique and my time was respected. Thank you."

We don't share this to pat ourselves on the back, we share it because we think transparency should run both ways. Candidates trust us with their time, and we want to be accountable to that.

From application to offer: 22.5 days (median)

We know job searching is stressful, so we try to move quickly. Our median time from application to offer was 22.5 days, and our fastest? Just 4 days. Here's how the stages broke down:

The challenge stage, which applied to Engineering, Design, Support, and Customer Solutions roles only, naturally adds time, but it's an important part of how we assess real-world problem solving. Candidates are given up to 10 days to complete it, and we aim to make it focused and respectful of their time. If you're applying for a Product or GTM role, your process will typically be faster as there's no take-home challenge involved.

83% offer acceptance rate

In 2025, 24 offers were accepted, an 83% offer acceptance rate. We take this as a signal that the process works. By the time we get to an offer, both sides have had enough real conversation to feel confident. We hope that continues to be true.

What's next

Sharing this data publicly is something we'll keep doing. Hiring is a two-way street, candidates invest real time and energy in applying, and the least we can do is be honest about what that process looks like from our side.

Looking ahead, 2026 is shaping up to be a big year for our go-to-market team. If you're in sales, solutions engineering or customer success and you're excited about the developer tools space, check out our open roles or join our talent community, we'd love to hear from you. And if you've been through our process, whether you joined us or not, we're always open to feedback. Hiring should feel human on both sides, and we're committed to keeping it that way.

]]>

Expanding Uptime Monitoring Down The Stack: ICMP Monitors Are Now Available In Checkly

Susa Tünker,Pırıl Kavlak — Tue, 10 Mar 2026 00:00:00 GMT

When we started building Checkly's uptime monitoring suite, the goal was to give engineering teams complete visibility across every layer of their stack, from application down to network, in one place. URL, TCP, DNS, and Heartbeat monitors covered a lot of that ground. But one fundamental piece was missing: the ability to simply ping a host and know if it's reachable.

We're now closing that gap by adding ICMP monitors to Checkly's uptime monitoring suite, available on all plans today, alongside URL, TCP, DNS, and Heartbeat.

What is ICMP monitoring?

ICMP monitors send scheduled ping requests to any host or IP address to measure reachability, latency, and packet loss. You can set assertions on any of these to define exactly what "healthy" means for each host.

No application logic. No open ports required. ICMP is a low-level monitor that provides a direct network-level signal, independent of what is running on the machine.

What you can monitor with ICMP requests

Infrastructure without an HTTP endpoint

Databases, message queues, internal services, bare-metal servers, routers, and network devices don't serve web traffic, but they still need monitoring. TCP monitors cover many of these cases, but ICMP goes one level lower: no open port required. For hosts where no ports are intentionally exposed, ICMP is the right layer to monitor at. If a host can be pinged, it can be monitored in Checkly without adding extra tools to your stack.

Private networks and internal hosts

ICMP monitors are useful for hosts that aren't exposed to the public internet, making them valuable for teams monitoring internal infrastructure alongside their public-facing checks. This is possible via Checkly Private Locations, with ICMP support coming soon.

Incident triage

When an alert fires, the first question is always: Is the machine down, or is the service down? ICMP gives you that answer immediately. If ICMP fails, it's a network issue. If ICMP passes and your HTTP check fails, the server is up, but the service is broken. The right team gets paged with the right context from the start.

Regional latency comparison

Run ICMP monitors from multiple locations to compare latency across regions. If one region is consistently slower than others, you can catch geographic performance issues before they affect users in that area.

Some failures don't show up as outages. Intermittent packet loss is one of them. One of our beta users ran into exactly this:

"We had a serious issue in which every third packet was dropped, and it went completely undetected until Checkly caught it at the network level. That's the kind of signal that stops a problem before it becomes a customer-facing incident."

You can set separate degraded and failure thresholds on packet loss, and add latency assertions to define when to trigger an alert.

Shipping with full support for Monitoring as Code

ICMP monitors are fully integrated into Checkly's Monitoring as Code workflows, available in the CLI from v7.1.0.

Define your ICMP monitors in code alongside your Playwright and API checks, version them in Git, and deploy through CI/CD:

import { Frequency, IcmpAssertionBuilder, IcmpMonitor } from "checkly/constructs"

new IcmpMonitor('cloudflare-dns-icmp', {
  name: 'Cloudflare DNS ICMP Monitor',
  activated: true,
  frequency: Frequency.EVERY_1M,
  maxPacketLossThreshold: 20,
  degradedPacketLossThreshold: 10,
  request: {
    hostname: '1.1.1.1',
    pingCount: 20,
    assertions: [
      IcmpAssertionBuilder.latency('avg').lessThan(100),
      IcmpAssertionBuilder.latency('max').lessThan(200),
    ]
  }
})

Checkly Tip

Or use an AI agent - copy this prompt to get started:
"Create an ICMP monitor in Checkly for [your-hostname-or-ip]. Ping every minute from us-east-1 and eu-west-1. Set a degraded alert at 10% packet loss and a failure alert at 20% packet loss or average latency over 200ms."

ICMP monitors are also available in Terraform and Pulumi:

Terraform (v1.18.0+): checkly_icmp_monitor

Pulumi (v2.8.0): checkly.IcmpMonitor

Pricing and availability

ICMP monitors are priced as uptime monitors: flat-rate per monitor, available on all plans today. Check out our pricing page for full details.

Get started

Try ICMP monitors in your Checkly dashboard or read the documentation to learn more about configuration, assertions, and alerting.

Questions? Find us in the Checkly community on Slack→

]]>

We Turned Our WireShark Wizard Into a Markdown File

Tim Nolet — Wed, 04 Mar 2026 00:00:00 GMT

Rocky AI — Checkly’s AI agent — is now Generally Available. We developed Rocky AI over the last ~6 to 8 months. This is an aeon in AI-years.

During this period, we learned a ton. About AI, but mostly about how to fit them into an existing SaaS product, not just another chat widget. This is my ramble…

Something with AI

Like almost everyone in SaaS-land, we were pondering doing “something with AI” at least since late 2024. We went with some hunches, looked at competitors, talked to customers, hacked on some prototypes. But it was hard to pinpoint a shippable MVP. We were probably going too broad and promising a bit too much.

Save me time, and no magic please

So we did a hard reset on our AI ambitions. Instead of shipping the whole kitchen sink, what if we focus on one time consuming task in the critical path of our users? One repetitive, but critical taks that came to mind is triaging a failed Playwright Check test. Specifically, could we use LLMs to quickly create an analysis of a Playwright trace file?

We immediately bumped into two challenges:

A Playwright trace file can easily be 100Mb+. We need to parse it, remove unneeded data and turn it into a plain text representation. Counterintuitively, giving MORE data to an LLM does doet always guarantee better results. Or it just blows up the input token limit.
When the data wrangling done, how do we guide the LLM to make sense of the data? Sure, it can do some basic summary by itself, but we want to give it guidelines on what is important to look at, what things are typically related. We essentially want these ingrained triaging steps that (senior) engineers hold in their heads to made explicit.

Over the months we expanded our AI analysis skills to cover all our check types (Playwright, HTTP, TCP, DNS, ICMP etc.). We took the same approach each time:

Look at how we can use artifacts like logs, traces etc. to build up a body of evidence for a specific failure root cause. Data wrangle it into submission so an LLM can use it.
Capture the skill of analyzing and interpreting said artefacts into a semi-structured markdown file.

This is now our Rocky AI Root Cause Analysis Agent. We did bump into some other learnings along the way, so I dumped them into a nice listicle. 👇

#1. Just do what the humans do

Again, this might be blatantly obvious, but we are using AI to just do things humans can do. Example: we literally codified the Wireshark ICMP and PCAP analysis skills of one our engineers into a markdown file that now powers our AI driven ICMP and PCAP analysis. This looks a bit like this:

icmp-rca-skill.md# ICMP Analysis Skill

## Classifications
The root cause can only be classified using the following classifications:

### INFRASTRUCTURE_ERROR
External issues like network errors, packet loss, host unreachable, DNS failures, firewall blocks, or routing issues. Examples: 100% packet loss, DNS resolution failure, ICMP destination unreachable

### CONFIGURATION_ERROR
etc.

## Artifacts

### ICMP_REQUEST
Description: ICMP request properties including hostname, target IP, IP family, and ping count. etc.

## Common Failure Patterns

### PACKET_LOSS
Symptoms: High packet loss percentage, Packets sent but not received, Packet loss threshold exceeded
Causes: Network congestion along the route, Firewall dropping ICMP packets, Target host overloaded or rate limiting ICMP, Intermittent network issues
Analysis: Check packet loss percentage vs configured thresholds, Examine individual ping results for patterns, Compare results across different monitoring locations

### DNS_RESOLUTION_FAILURE
etc.

#2. Data wrangling is still the thing

Even with extended context windows and increased quota’s on input tokens, we still do a ton of data wrangling. As mentioned above, Playwright trace files easily go over 100Mb, a network PCAP file parsed to text can also be very large. These are all data sources want the LLM to take into account.

Currently, we solve this problem by pre-parsing, filtering and summarising these large assets in tool calls. The tool returns the key data from these assets back to the LLM.

#3. Model updates are almost a free lunch

Just the last ~6 months, the upgrade from OpenAI GPT-4.1 to GPT-5.1 (let’s not mention the dreadful GPT-5) was night and day: better answers, more reliable tool calls, faster responses, less processing errors, better schema output. You can see similar patterns for the Opus 4.5 to Opus 4.6 and the Gemini upgrades.

#4. Multi model is like multi cloud. It kinda sucks.

While building our Bring-Your-Own-Model (BYOM) feature, we allowed users to bring Gemini and Anthropic models next to our default OpenAI models. That was a mistake. Turns out that swapping models is quite hard right now if you want to keep some form of quality control and not have dedicated tests, fixtures and code paths.

This is even true with wrapper libraries like the Vercel AI SDK, which we use. In some way it reminds of how Terraform could in theory make your infra “multi cloud”. In practice, it really can’t.

#5. Don’t always default to chat

Obviously, chat is the de-facto UI paradigm for LLMs and generative AI. However, in our RCA case, it would be silly to force a user to go to some chat UI and asked Rocky “please analyse this failing check”. We can ask that question for the user, even when they are sleeping and just ship the initial analysis to their inbox. Chat can be a follow up.

We are totally not done with expanding Rocky AI and adding more tools and features to its RCA skills. Watch this space.

]]>

Introducing Rocky AI to General Availability

Dan Giordano — Wed, 04 Mar 2026 00:00:00 GMT

After months of being available in Beta for our app users, Rocky AI is now generally available to all users and plans. Rocky AI is Checkly’s AI agent that works around the clock, 24/7, to make sure your application’s reliability is optimal. In this first release, Rocky AI ships with the ability to run continual Analysis on test and check failures, giving your teams AI-powered root cause analysis, impact analysis, and more.

Since day 1 of the AI craze, Checkly has made it easy to work with your favorite LLMs and coding agents like Claude Code, Cursor, and OpenAI, allowing you to craft and manage your monitoring right from your IDE. Now, we’re bringing that power right into the Checkly workflow with Rocky AI.

Root Cause Analysis

When a test or monitor fails, Rocky AI analyzes multiple vectors across your application, including logs, traces, and more to understand what exactly caused the failure; and how to best fix it.

For large teams with hundreds of monitors across their apps and services, Rocky AI can help engineers understand the context of the check itself, even if they didn’t create it, reducing cognitive overload in the heat of an outage. For individuals and smaller engineering teams, Rocky AI will cut down on hours of back and forth, working to replicate the problem, and other triage works that delay teams from getting a quick fix back in production.

User Impact

Rocky AI also tries to understand and surface who exactly is impacted during an outage. Are users impacted from logging into the application? Is a core workflow being blocked? Is it a network failure that causes the test to fail, but user impact is actually minimal? Rocky AI can help you get the answers you need, faster than humanly possible, so you can get your application back up and running.

Automation

For teams with heavy alerting systems, Rocky AI can tap in and automatically send its findings inside any alert, right to your inbox or Slack channel. You'll get highly relevant context to the downtime, directly inside your workflow, allowing you to break out, fix it, and get back to work.

Getting Started

Rocky AI Error Analysis is available in Checkly Resolve today. To get started with Rocky AI right now in Checkly, just head over to app.checklyhq.com and click Rocky AI Analysis.

Or read through our documentation to get a better grasp of Rocky AI

What’s Next

With Rocky AI hitting general available with the ability Analyze results, we are only getting started. We will be adding other capabilities and skills to Rocky AI over time, to bring you native AI powered test generation, alert configuring, and more - right inside the Checkly dashboard. If you are a heavy user of LLMs or Agents, then check out Rocky AI or start using Checkly with tools like Claude Code today.

]]>

The Current State of Content Negotiation for AI Agents (Feb 2026)

Stefan Judis — Thu, 19 Feb 2026 00:00:00 GMT

The web was built for humans, but now the agents are taking over.

Humans look at a web page and see content rendered by their browser. AI agents see 180,000 tokens of nav bars, footers, and div soup — burning through their context window on junk that makes them slower and stupider.

The web needs to evolve, and we as developers are driving the shift. AI agents like Claude Code, Cursor, Codex, and Gemini are how we interact with documentation, CLIs, and products today. Every time the robots fetch a page, they parse markup that adds no value.

Cloudflare and Vercel recently published posts about solving the context waste problem with good old content negotiation, and the Accept header.

Both posts explain the concept well, but they're missing a crucial piece: which agents currently send these headers? And what's the best way to support the agentic web?

Let's fill those gaps.

The Problem: HTML Is Expensive and not Agent-friendly

When an AI agent fetches a URL, it doesn't see what you see. It sees this:

<div class="wrapper">
  <nav class="site-nav" aria-label="Main">
    <ul class="nav-list">
      <li class="nav-item"><a href="/docs">Docs</a></li>
      <!-- 47 more nav items -->
    </ul>
  </nav>
  <main class="content-wrapper layout-default">
    <article class="docs-content" id="main-content">
      <h1 class="title heading-1">Getting Started</h1>
      <!-- finally, some actual content -->

All those <div> wrappers, class names, ARIA labels, and semantic markup have zero value for an LLM. But every character counts against its context window.

The numbers speak for themselves:

Cloudflare measured an 80% token reduction when serving markdown instead of HTML
Vercel saw page content drop from 500KB to 2KB — a 99.6% reduction

The web wasn't always this heavy. But today, actual content only makes up a fraction of what gets sent over the wire. Users with limited mobile data feel it. So do people on slow connections. And now agents, paying by the token, feel it too.

The Solution: Content Negotiation

HTTP has had a mechanism for this since HTTP/1.1. It's called content negotiation, and it works through the Accept request header.

When a client makes a request, it can say, "Hey! I'd prefer markdown, but I'll take HTML if that's all you've got.":

Accept: text/markdown, text/html

If the server supports it, it responds with clean Markdown. If not, the client receives the HTML and handles it. This approach is simple, backwards-compatible, and elegant. And it's been part of HTTP since forever.

Here's what that approach looks like in practice, using curl:

# Request the Checkly docs with a proper `Accept` header
curl <https://www.checklyhq.com/docs/concepts/monitoring-as-code/> \\
  -H "Accept: text/markdown"

If you request the Checkly docs with a proper Accept: text/markdown header, you get structured markdown that an LLM can actually use instead of a massive HTML document.

Format	Size / Used Tokens
HTML	615.4 KB / 180,573
Markdown	2.3 KB / 478

That's a 99.6% reduction in size and 99.7% fewer tokens — independently measured on our own docs, and strikingly close to Vercel's numbers. And the best thing: it's the same content, but at a fraction of the cost.

We at Checkly believe agents are our next customers. When someone asks their agent, "How do I set up a Playwright Check Suite with Checkly?" we want them to receive clean, structured context, not a soup of HTML tags that burn through their context window. Our docs, of course, are at the center of supporting the agentic web.

What Agents Are Actually Sending The Required Accept Header?

Here's the one-million-dollar question that Cloudflare and Vercel didn't cover in their posts: which agents use content negotiation today?

To answer this question, we let common agents request the URL https://httpbin.org/headers with their native webfetch tools. The page responds with the received request headers in JSON format. Easy to parse and analyze.

❯ Can you fetch <https://httpbin.org/headers>. What are the displayed headers?

⏺ Fetch(<https://httpbin.org/headers>)
  ⎿  Received 254 bytes (200 OK)

Here are the request headers each agent sends (Accept and User-Agent for fun):

✅ "Fetch" in Claude Code (v2.1.38) requests Markdown

Accept: text/markdown, text/html, */*

User-Agent: axios/1.8.4

✅ "WebFetch" in Cursor (2.4.28) requests Markdown

Accept: text/markdown,text/html;q=0.9,application/xhtml+xml;q=0.8,application/xml;q=0.7,image/webp;q=0.6,*/*;q=0.5

User-Agent: Mozilla/5.0 ... Chrome/139.0.0.0 Safari/537.36

✅ "webfetch" in OpenCode (1.2.5) requests Markdown

Accept: text/markdown;q=1.0, text/x-markdown;q=0.9, text/plain;q=0.8, text/html;q=0.7, */*;q=0.1

User-Agent: Mozilla/5.0 ... Chrome/139.0.0.0 Safari/537.36

❌ "unknown tool" in OpenAI Codex 260203.1501 (523) doesn't request Markdown

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9

User-Agent: Mozilla/5.0 ... ChatGPT-User/1.0; +https://openai.com/bot

❌ "WebFetch" in Gemini CLI (0.28.2) doesn't request Markdown

Accept: */*

User-Agent: GoogleAgent-URLContext

❌ "fetch_webpage" in GitHub Copilot (GPT-5.2-Codex) doesn't request Markdown

Accept: text/html,application/xhtml+xml, ...

User-Agent: Mozilla/5.0 ... Code/1.109.3 ... Electron/39.3.0

❌ "read_url_content" in Windsurf (1.9552.21) doesn't request Markdown

Accept: */*

User-Agent: colly - <https://github.com/gocolly/colly>

Key findings

Only 3 out of 7 agents request markdown — Claude Code, Cursor, and OpenCode are the only ones asking for text/markdown.
Gemini CLI and Windsurf don't negotiate at all — */* means "give me anything." No markdown preference, no optimization.
OpenCode and Cursor use the q factor — they set explicit quality factors (e.g., text/markdown;q=1.0, text/html;q=0.7) to prioritize markdown over HTML. The q parameter is HTTP's way of expressing preference as a weight between 0 and 1. Claude Code skips q values and relies on the header value order instead.

The agents requesting markdown are already giving their users an advantage. What about the ones that are not, though? They're burning tokens on HTML that could have been avoided. We don't know why they haven't adopted content negotiation yet, but we expect them to follow suit soon. The benefits are too obvious to ignore.

How do agents actually process fetched content?
We can't know exactly what closed-source agents do under the hood, but from what's publicly documented Claude Code uses a two-stage architecture: the main conversation calls WebFetch, the response is converted to markdown, and a smaller, faster model summarizes it before injecting it into your conversation.
This agent-side optimization is impressive — yet completely unnecessary. Converting HTML to markdown, truncating content, and spinning up secondary models to summarize the content is a lot of work to retrieve available online content. It's on us, the people running servers, creating products, and building the web, to meet agents where they are. Content negotiation has been in HTTP since 1997. When someone requests markdown, we all need to "just respond with markdown".

We at Checkly believe text/markdown will become the standard for agent-to-server communication. The direction is clear, even if not all agent tools are yet on board. If your docs don't respect the Accept header, you're forcing agents to parse HTML, resulting in burned tokens, slow responses, and degraded-quality answers.

Beyond Headers: The Full Picture

Content negotiation is an important step toward improving agentic workflows, but it's not the end goal. Ideally, agents shouldn't need to search the web at all. The right context should be provided upfront, either by you as the agent user or by us as the developer tooling provider.

There are three emerging patterns that complete the picture:

llms.txt

Another approach gaining traction is the use of llms.txt files. These file points agents to (markdown) versions of key pages, similar to how robots.txt guides crawlers. It's a good complement to content negotiation: the Accept header optimizes how content is served, while llms.txt helps agents discover what content exists in the first place.

We at Checkly opted for serving our doc’s key pages on checklyhq.com/llms.txt because it’s most likely that agents want to set up or control the monitoring infrastructure, or help a user using the platform.

llms.txt# Checkly Docs

## Docs

- [Changing your email or password in Checkly](https://checklyhq.com/docs/admin/changing-your-email-password.md): Learn how to change your email address or password in your Checkly account
- [Creating an API key in Checkly](https://checklyhq.com/docs/admin/creating-api-key.md): Learn how to create and manage user and service API keys for the Checkly API and CLI
- ...

Agent skills

Skills are packaged, agent-readable instructions that tell an AI agent how to use a specific tool or product — no web search required. They go a step further than content negotiation: in addition to publishing our documentation online, we also released a first agent skill to provide executable guidance.

Install the Checkly monitoring skill with:

npx skills add checkly/checkly-cli

Once installed, your favorite agent knows how to create checks, configure alerts, and deploy monitoring infrastructure — not because it read our docs or searched the web, but because we gave it structured and up-to-date instructions designed for the new world of AI agents.

Agent-friendly Developer Platforms

Content negotiation with the Accept header, llms.txt and skills are the obvious choices to improve the agent experience today, but we're already working on the next iteration.

Here's what's coming to the Checkly Platform and Monitoring as Code:

Markdown-first CLI output

AI agents love CLIs, and we're improving the output of CLI commands like checkly test and checkly deploy to be structured and parseable. When an AI agent runs your checks, it should get clean, context-efficient results — not output designed for human eyes scanning a terminal. If the agent is facing issues, the CLI should provide context to help it get unstuck.

We're also exploring a new checkly skills command that will provide all the required context and documentation inside the CLI itself to get your agent back on track.

Markdown-first Core Features

Checkly's monitoring platform is designed to deliver critical information when your infrastructure experiences issues or outages. So far, you, as a human, needed to know what's broken in your infrastructure and since when. This will change — very soon, it won't be you asking for critical information about your platform's health; it will be your agent, supervised by you.

As a result, our API responses and platform core features will become token-efficient and agent-friendly. Watch the space! We've got many things in the works to make "Hey Agent, I received an alert. Can you fix it?" a reality.

Wrapping Up

The web has a new class of visitors. They don't have eyes, they pay by the token, and they're how developers discover and interact with your product.

Not every agent requests markdown today, but the ones that do are already giving their users a better experience — faster responses, lower costs, higher quality answers. The direction is clear.

Here's what you can do right now:

Using agents? — test what headers your agent tools send. Some agents optimize for you. Others don't. Know which one you're using. Consider a switch to save tokens.
Building for the web? — support Accept: text/markdown. Agents are your next users. Serve them well, or watch them struggle with your docs and product.

If you're monitoring your sites, apps, and infrastructure while riding the agentic train, check out Checkly and Monitoring as Code. It's AI-ready by design. Get started with AI-powered monitoring.

]]>

Introducing: Checkly Agent Skills

Stefan Judis — Wed, 18 Feb 2026 00:00:00 GMT

AI coding agents are excellent at writing code. Ask Claude Code, Codex, or Cursor to add a feature, and it just works. At Checkly, we were ready for the new agentic world from the start! Monitoring as Code means your entire monitoring setup lives in your repository. API Checks, Browser Checks, alert channels, status pages; everything is defined in code, managed with the Checkly CLI, and version-controlled like any other part of your stack. Agents can write, review, and ship your monitoring infrastructure!

In 2026, we believe AI agents are our new power users. And when developers ask their agents to set up end-to-end production monitoring, we want agents to generate code that not only runs but also follows best practices.

Today, we're releasing the first agent skill for Checkly, which includes best practices and patterns that teach your AI agent to control your monitoring the way Checkly engineers do.

What are agent skills?

Agent skills are a standardized format for giving AI agents specialized knowledge. Think of them as documentation written for machines. Skills are structured instructions that agents discover and load on demand.

A skill is a directory containing a SKILL.md file with instructions, plus optional reference files for detailed documentation:

monitoring/
├── SKILL.md              # Core instructions (~1K tokens)
└── references/           # Loaded on demand
    ├── api-checks.md
    ├── browser-checks.md
    ├── alert-channels.md
    └── ...

The skill format works with Claude Code, Cursor, Codex, and other agents supporting the standard. Instead of hoping your agent learned the right patterns from training data, you give it explicit instructions to discover and follow.

What's inside the Checkly skill

The new Checkly monitoring skill covers everything you need to control your monitoring setup as code:

Construct References
Construct	What it covers
API Checks	HTTP assertions, authentication, setup scripts
Browser Checks	Playwright-based E2E monitoring
Playwright Checks	Multi-browser Playwright test suites for production monitoring
MultiStep Checks	Complex user flow API monitoring
Monitors	TCP, URL, DNS, and Heartbeat monitors
Check Groups	Organizing checks with shared configuration
Alert Channels	Email, Phone, Slack, and webhook notifications
Supporting Constructs	Status pages, dashboards, maintenance windows

Each construct type has its own reference file with examples and documentation links your agent can use directly.

How agents use the skill

When you ask your agent to set up monitoring, it discovers the Checkly skill and loads the relevant context. Need an API Check? It pulls in the API Check reference. Setting up alerts? It loads the alert channel documentation. The agent treats these files as the source of truth rather than improvising and hallucinating forward.

For example, the skill teaches your agent to:

Follow project conventions: Put checks in src/__checks__/, use .check.ts for monitoring construct definitions and .spec.ts for Playwright code
Configure checks properly: Set appropriate frequencies, choose the right runtime version, and configure locations that make sense for your users
Set up alerting: Wire up Slack, email, or phone alerts with the right escalation patterns

Token-efficient by design

A year ago, we shipped the checkly.rules.md file that agents could load entirely. This file holds about 4,000 tokens of context, whether you needed it or not. It was a good first step, but the ecosystem has evolved toward more token-efficient approaches to avoid degraded performance and context rot.

The Agent skills format uses progressive disclosure so that agents load only what they need for each task:

Stage	What is loaded
Discovery	Skill name and description (~100 tokens)
Activation	Core Skill.md instructions (~1,000 tokens)
Specific Tasks	Individual reference files (on demand)

Your agent starts with consuming ~100 tokens just to know the skill exists. When monitoring becomes relevant, it loads ~1,000 tokens of core guidance. Detailed references about API Checks, Playwright Check Suites, alert channels, status pages, etc., are only loaded when the agent actually needs them.

Checkly Rules	Checkly Monitoring Skill
~4,000 tokens (always)	~100–1,000 tokens (typical)

This approach leads to faster responses, lower costs, and context that scales with your monitoring task.

Get started

Install the Checkly skills with the Skills CLI:

npx skills add checkly/checkly-cli

That's it. Your agent then has access to Checkly's monitoring patterns and will use them automatically when you work on your monitoring infrastructure.

Try asking your agent to:

"Set up monitoring for my API endpoints."
"Configure Slack alerts for Playwright checks."
"Update the core monitoring locations and add eu-central-1."

Your agent is now a Checkly monitoring expert!

What's next

The monitoring skill is just the start. We're working on adding more skills:

Playwright — Best practices for generating reliable, maintainable Playwright end-to-end test code
Resolution — Patterns for incident analysis, debugging failing checks, and resolving core issues

We're also exploring embedding agent context directly into the Checkly CLI itself. Imagine running npx checkly skills browser-check or npx checkly skills alert-channel to get the exact context your agent needs at the right time. No external installation will be required because we believe CLI tooling should be self-contained and self-documenting for humans and agents alike.

Try it out

The Checkly agent skill is available now. Install it, point your agent at your project, and let it set up production-ready monitoring.

Do you have questions or feedback? Find us on our community Slack.

]]>

How to Write a Cover Letter That Actually Helps You Get the Job

Kaylie Boogaerts — Mon, 01 Dec 2025 00:00:00 GMT

Cover letters are supposed to help you shine, but most of them blur together into the same polite, forgettable paragraphs.

The intention is good (“I want them to notice me!”), but the execution… not so much. So, here’s a simple, honest guide to writing a cover letter that actually works, especially if you’re applying to Checkly.

Spoiler: shorter is better. And authenticity in this AI era is better than perfect polished perfection.

Hi, I’m Kaylie, Director of People at Checkly. I’ve read a lot of cover letters, and I want to help you write one that actually works.

What most cover letters end up being

If you only knew how many times I’ve read the same sentence, slightly reworded, by hundreds of candidates…

A list of everything the job post mentions, with “I am excited about…” pasted in front.
Obviously AI-generated text that sounds nice but says absolutely nothing.
Claims about experience that don’t match the CV or LinkedIn profile at all.

None of this helps. It doesn’t tell us who you are or why you care about the role. And honestly? You’re wasting your own time writing something that isn’t going to move the needle.

Which brings me to…

What makes your cover letter stand out

Cover letters are totally optional at Checkly. But when they’re good, they can make your application stand out in a sea of hundreds.

Here’s what a great cover letter looks like:

1. Keep it concise and specific

When we publish a new role, we get hundreds of applications, sometimes over a thousand! We read through every single one, but that also means that we don’t have time for essays. A sharp, 3-paragraph cover letter beats a 1.5-page essay every time.

**2. Tell us why you want this role**

Not why you want a job. Not why you want “a new challenge.”

Why this role? What part of it sparks something for you? We want to hire people who choose us on purpose. If something about the company or role lights you up, say it.

Give us a story: a small, concrete example from your experience that shows you can do something this role needs.

Not a list of traits. Not “I am a great communicator.”

Something like: “In my last role, I rebuilt our onboarding flow from scratch and cut ramp-up time by 40%. I’d love to bring this kind of structured thinking to Checkly as you scale.”

That one example gives us far more than three paragraphs of adjectives ever could.

Why you want to work here, and
Why you think you can be great at this role.

That’s it.

If you can answer those clearly and briefly, you’re already way ahead of most applicants.

Extra: If you want real inspiration, check out the exact cover letter I submitted with my application.

]]>