Cloud Parts

Observability

Logs, metrics and traces that tell you what your system is doing.

14 technologies on this page ↓
Live model
What you’re watchingTwo services emit logs, metrics and traces. The collector routes each kind to its own store, and now and then a metric crosses its threshold and fires an alert.

How it works

Observability means emitting enough telemetry to answer questions about production you didn’t think to ask in advance. Logs are timestamped records of events. Metrics are numbers aggregated over time, like requests per second or p95 latency. Traces follow one request across every service it touches. A collector such as OpenTelemetry gathers all three and ships them to storage, dashboards and alerting.

Think of it as

A car’s gauges, trip log and dashcam. The gauges show how it’s doing now, the log shows what happened, and the dashcam shows one whole journey.

Reach for it when

  • Every production system, from the first day
  • Alerting on symptoms users feel: errors, latency, saturation
  • Finding the slow step in a request that crosses many services

Think twice when

  • Debug-level logging of everything in production; the bill grows fast
  • Alerting on every metric; page people only for what needs a human

A real example

A jump in p95 latency fires an alert. The trace of one slow request shows 1.8 s inside a single database query, and that service’s logs show the slowdown began right after a deploy.

Technologies

Specific products, and what each does differently.

Amazon CloudWatchAWSAWS’s built-in monitoring service: metrics from every AWS service, logs, alarms and dashboards, and now traces, OpenTelemetry and PromQL, billed per metric, per GB and per alarm.Better StackCommercialUptime checks, on-call paging and status pages, plus logs, traces, metrics and errors in a ClickHouse-based store, sold as one SaaS billed per responder and per GB.ClickStackOpen sourceClickHouse’s open-source observability stack: an OpenTelemetry collector, ClickHouse for storage and the HyperDX UI for search, dashboards and alerts on logs, traces, metrics and session replays.DatadogCommercialA hosted monitoring platform: an agent on every host ships metrics, logs and traces to Datadog’s SaaS, which bills per host, per GB and per metric.GrafanaOpen sourceDashboards and alerts over data that stays where it is: Prometheus, SQL databases, cloud monitoring and more.OpenTelemetryOpen sourceThe vendor-neutral standard for producing and shipping traces, metrics and logs, with SDKs for most languages and a Collector to route the data.

Also used for observability

Products of other kinds that are often used this way, grouped by what they are.

What each provider calls it

AWS
CloudWatch (metrics, Logs, Application Signals; X-Ray traces in the CloudWatch console)
Google Cloud
Cloud Logging, Monitoring, Trace
Azure
Azure Monitor, Application Insights
Open source & others
OpenTelemetry, Prometheus, Grafana, Jaeger, GreptimeDB, ClickStack (ClickHouse with the HyperDX UI and an OpenTelemetry collector); agents: Fluentd, Fluent Bit, Logstash, Telegraf, Vector; hosted: Datadog, Better Stack, Sentry (errors and traces)

See it in a real stack

Architectures that use observability

IoT telemetry pipelineDevices stream readings into Kinesis; Lambda writes them to TimescaleDB for Grafana, and raw data is archived to S3.Amazon Kinesis Data StreamsAWS LambdaTimescaleDBAmazon S3Grafana

In code · python

from opentelemetry import trace, metrics
tracer = trace.get_tracer("api")
latency = metrics.get_meter("api").create_histogram("http.server.request.duration", unit="s")

with tracer.start_as_current_span("GET /wells/{id}") as span:   # trace
    span.set_attribute("well.id", 42)
    log.info("fetching well status", extra={"well_id": 42})    # log
    latency.record(0.0385, {"http.route": "/wells/{id}"})        # metric

Worth exploring

WebsitePlatform tooling landscapePlatformEngineering.orgA filterable map of tools for building an internal developer platform. Each tool sits in one of five “planes” (developer control, integration and delivery, resource, observability, security) and a category such as IaC, CD pipeline, secrets or developer portal, with a one-line description. Vendors can submit their own tools, so treat it as a list of what exists rather than a ranking.
All resources →

Often confused with

Time-series databasesCI/CD pipelinesAlertingLogsMetricsDistributed tracing
Where it came from1960–2026

The word comes from control theory: in 1960 Rudolf Kálmán described a system as observable if its internal state can be inferred from its outputs. Running software started with simpler questions. Nagios, first released as NetSaint in 1999, ran checks against hosts and services and sent an alert when something failed and again when it recovered. Graphite, and StatsD, which Etsy released in early 2011, let engineers send and chart almost any number they cared about.

Distributed systems outgrew that. Google’s 2010 Dapper paper described how one request could touch many services, written by different teams in different languages across thousands of machines, and how sampling made it cheap enough to trace requests in production. In 2012 SoundCloud, with hundreds of services and thousands of instances, started Prometheus because its StatsD and Graphite setup couldn’t slice metrics by instance or endpoint.

“Observability” became the name for the goal: being able to answer questions about production you didn’t think to ask in advance, rather than only watching checks and dashboards someone set up beforehand. Metrics, logs and traces became known as its three pillars. In 2019 the OpenTracing and OpenCensus projects merged into OpenTelemetry, a vendor-neutral way to produce all three, which graduated from the CNCF in May 2026.

Why it’s built this way

Follow requests, not just machines
When one user action crosses dozens of services, per-host graphs can show that something is slow but not where. A trace records each step of one request, with timings, so the slow hop stands out.
Sample instead of recording everything
Dapper’s designers needed tracing to be always on with low overhead, so they recorded only a fraction of requests. Sampling is still how tracing stays affordable at scale.
Dimensions on every data point
Labels such as instance, endpoint and status let you slice the same data in ways nobody planned for, which is the difference between monitoring known failures and exploring new ones.
Standardise instrumentation, not the backend
Instrumenting code is the expensive part. A vendor-neutral standard means you instrument once and can change where the data goes without touching application code.

Timeline

  1. 1960

    Rudolf Kálmán defines observability in control theory

  2. 1999

    NetSaint, later renamed Nagios, is released

  3. 2010

    Google publishes the Dapper paper on distributed tracing

  4. 2012

    Prometheus begins at SoundCloud

  5. 2019

    OpenTracing and OpenCensus merge into OpenTelemetry

  6. 2026

    OpenTelemetry graduates from the CNCF in May

Sources22

Improve this page on GitHub