Observability with OpenTelemetry and Checkly

Observability isn't just a buzzword; it's a vital compass guiding us through the maze of system health and performance. As we’ve adopted microservice architectures, the ability to know ‘what is currently happening in our system’ has diminished as our operational resilience has increased. We find services scattered among a maze of interconnections and interdependencies. And even the logs that used to guide are now scattered throughout this maze. Among the stars guiding this journey, OpenTelemetry twinkles particularly brightly, offering a comprehensive toolkit designed to provide deep insights into our applications. It's not just about monitoring; it's about understanding, a crucial distinction that OpenTelemetry and its community, including experts like our very own Nočnica from her upcoming webinar, champion passionately.

What is OpenTelemetry? A Friendly Introduction

Imagine OpenTelemetry as a multi-tool for your observability needs – versatile, reliable, and inclusive. At its heart, it's a collection of tools and standards, including the OTLP (OpenTelemetry Protocol), language agents for all the major programming languages, and an efficient collector tool for data management.

But here's the quirky part: OpenTelemetry decides to play it cool by not stepping on the toes of existing observability platforms. It focuses on gathering and transmitting data, leaving the storage, dashboards, and even some testing types to other players in the observability arena. It's like that friendly neighbor who's always there to help but knows when to give you your space.

What is Telemetry Data?

In its most abstract, telemetry data is data about any system that you can't observe directly. Think of it as any system where you can't understand how it's running with a local 'top' command. As microservice architectures have increased in complexity, we expect our telemetry data to tell us about the health both of the underlying infrastructure and the codebase that's running our service, and present that data in an aggregated format that makes it easy to understand when displayed on a dashboard UI.

What is OpenTelemetry Observability?

Founded as a merger of two previous projects — OpenCensus and OpenTracing back in 2019 — OpenTelemetry is a collection of closely related open source projects, all under the umbrella of the Cloud Native Computing Foundation. The intent was to build a unified system for observation of requests across our stack, including a context propagation standard so that a request handled by multiple microservices can be tracked throughout its journey. The OpenTelemetry project includes:

• Instrumentation libraries for all common backend languages and many frameworks
• The ‘collector’ for routing, filtering, and collating data
• The protocol for sending observability information, the OpenTelemetry protocol (OTLP)

How Does OpenTelemetry Observability Work?

OpenTelemetry Observability works by adding observer instrumentation to your system, generally with one of the OpenTelemetry language projects that automatically emit information about how your code is handling requests and performing in general.

OpenTelemetry Components

APIs

OpenTelemetry APIs for each of its supported languages allow a standard system for formatting trace, span, metric and log data from a code library. Essentially the API allows the measurement of these metrics directly from code. The API excludes any components needed to transmit that telemetry data to a collector or other data repository, making the API the best tool for instrumenting a code library or other component, since only the core application code needs to handle sending telemetry data.

For more information, see Instrumenting Libraries in the OpenTelemetry Documenation.

SDKs

If you want to both instrument your code and send it somewhere, you’ll use the OpenTelemetry SDK. This includes code to batch and send data to an endpoint, generally an OpenTelemetry Collector.

Collector

The OpenTelemetry project doesn’t include any system to store or display your telemetry data. That’s left to either an open source tool or a SaaS metrics service like Chronosphere. But before your data is sent on to a data backend, you’ll want to gather, filter, and batch that data before sending it out. For that, the OpenTelemetry Collector is here. Collectors can be hosted at multiple places within your network or run as an external service, and serve as a collection point for data before it’s sent.

Traces, Metrics, Logs & Monitoring

The three main signal types of telemetry are all part of the OpenTelemetry project. While Tracing is the project’s home base (after all, it grew out of the ‘OpenTracing project), logs and metrics are increasing the quality of coverage. Currently logs have the most limited coverage, which can seem surprising since surely we need to send our logs somewhere for analysis, but consider: you already have a way to send your logs someplace. Rather than the OpenTelemetry SDK’s adding new methods for sending logs, the recommendation is that you use your existing logging pipeline to send logs to your OpenTelemetry dashboard of choice.

The Trending Topic: OpenTelemetry's Rising Stardom

Our observability landscape is witnessing a trend that's hard to ignore – a rapid shift towards OpenTelemetry. Picture this: just a year apart, Kubecon's Observability day events transformed from half-filled rooms to a buzzing hub packed with industry leaders and technical masterminds. It's not just about jumping on the bandwagon; it's about recognizing a solution that promises flexibility, control, and a breath of fresh air from the stifling grip of vendor lock-in.

A Confluence of Insights: Nočnica Meets Gartner in the World of Observability

*"In the fast-evolving observability landscape, it's not just about collecting data; it's about making sense of it in the most efficient and flexible way,"* notes Nočnica, an observability enthusiast and beacon of knowledge in her insightful blog. Nica is set to expand on this narrative, illustrating how OpenTelemetry serves as a crucial puzzle piece in the broader picture of observability.

Complementing Nočnica’s practitioner's viewpoint, a recent Gartner report provides a strategic perspective: "The need to collect, ingest and analyze the volume of telemetry generated by modern workloads is giving rise to new, pipeline-based mechanisms that ensure support personnel have the right data at the right time." This insight highlights the critical role of OpenTelemetry in addressing the intricate demands of contemporary IT environments.

By weaving together Nica's hands-on experiences with the strategic analysis provided by Gartner, we gain a multifaceted understanding of the current state and the promising future of observability. This blend of perspectives underscores the importance of adopting a holistic approach to monitoring and analysis, one that OpenTelemetry is well-equipped to facilitate.

Embracing the Odyssey with Checkly and OpenTelemetry

As we navigate this observability odyssey, it's clear that OpenTelemetry, with its comprehensive data collection prowess, is a guiding star. But even stars need a constellation to shine brighter. Enter Checkly – your perfect co-traveler in this journey, filling the gaps with frontend monitoring prowess and alerting mechanisms that OpenTelemetry, in its humility, steps away from.

Charting the Course Forward

Embarking on the OpenTelemetry journey is not just about adopting a new tool; it's about embracing a mindset, a culture of openness, flexibility, and continuous learning. It's about joining hands with a community that's enthusiastic, supportive, and, let's admit it, a bit geeky in the best possible way. Whether you're fine-tuning performance, ensuring reliability, or simply seeking deeper operational insights, OpenTelemetry, along with allies like Checkly, is here to transform your observability landscape.

So, why not grab a cup of your favorite brew, tune into Nica's upcoming webinar, and join us on this fascinating journey? After all, in the world of observability, every insight, every metric, and every trace is a part of the story – your story.