Structured logging

Logs written as named fields (logfmt or JSON) instead of free text, so a machine can filter, group and join them.

How it works

An unstructured log line is a sentence for a person: Payment for order 9124 declined after 812 ms (card_declined). To ask “which orders were declined in the last hour, by reason?” you need a regular expression for every way any developer ever phrased it, and the query breaks when someone rewords the message. A structured log keeps the message short and fixed and puts each variable part in a named field. In logfmt that’s level=error msg="payment declined" order_id=9124 reason=card_declined duration_ms=812; in JSON it’s the same as one object per line. The log store can index the fields, so the question becomes a filter on reason grouped by hour, with no parsing, and adding a field never breaks an old query. The most valuable fields join logs to everything else: trace_id and span_id from the current trace, a request ID, the service name and its version. OpenTelemetry’s log data model fixes names for the common parts (Timestamp, SeverityText, SeverityNumber, Body, Attributes, TraceId, SpanId, Resource) so that agents and stores agree on them. Most languages have a structured logger: pino for Node.js, zap and the standard library’s log/slog (since Go 1.21) for Go, structlog for Python and Serilog for .NET. The price is size and readability: JSON lines are longer than a sentence and harder to scan in a terminal.

Think of it as

A form instead of a letter. Letters say the same things in a different order every time; a form has the same boxes in the same places, so a clerk can sort a thousand of them by one box in minutes.

Reach for it when

Any service whose logs go to a central store, which in practice means every production service
Counting, grouping or alerting on what’s in the logs: errors by customer, slow calls by endpoint
Joining logs to traces: put trace_id and span_id in every line
Many teams and languages writing to one store, where field names need to match

Think twice when

A small script or CLI whose output only a person reads in a terminal
Dumping whole request bodies or objects as fields: size, cost and leaked personal data grow fast
Field names that change per call, such as one key per user; keep the names fixed and put the variable part in the value

A real example

An on-call engineer needs every declined payment for customer 7781 in the last day. With plain text lines she searches for “declined” and reads 4,000 matches by eye, because the customer ID appears in three different phrasings. With JSON lines in CloudWatch Logs, which discovers JSON fields by itself, filter reason = "card_declined" and customer_id = 7781 | fields @timestamp, order_id, trace_id returns 12 lines, and each trace_id opens the full trace of that request.

The same event, three ways#

One declined payment, written as a sentence, as logfmt and as JSON, and what it takes to answer “how many declines per reason in the last hour?”

Format	The line	Counting declines by reason
Plain text	`2026-10-06 09:14:03 ERROR Payment for order 9124 declined after 812 ms (card_declined)`	A regular expression to pull out the reason; it misses lines another developer worded differently
logfmt	`time=2026-10-06T09:14:03Z level=error msg="payment declined" order_id=9124 reason=card_declined duration_ms=812`	Group by the `reason` field; easy to read in a terminal too, but every value is a string
JSON	`{"time":"2026-10-06T09:14:03Z","level":"error","msg":"payment declined","order_id":9124,"reason":"card_declined","duration_ms":812}`	Group by `reason`; numbers stay numbers and objects can nest, at the cost of a longer line

Fields once, on every line (Go’s slog)#

A child logger carries the request’s fields, so every line it writes can be joined to the trace without repeating them.

logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))

// fields added here appear on every line this logger writes
reqLog := logger.With("service", "payments", "trace_id", traceID)

reqLog.Error("payment declined", "order_id", 9124, "reason", "card_declined")
// {"time":"2026-10-06T09:14:03.120345678Z","level":"ERROR","msg":"payment declined",
//  "service":"payments","trace_id":"4bf92f35...","order_id":9124,"reason":"card_declined"}

In code · json

{
  "timestamp": "2026-10-06T09:14:03.120Z",
  "level": "error",
  "msg": "payment declined",
  "service": "payments",
  "version": "2.14.0",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "order_id": 9124,
  "customer_id": 7781,
  "reason": "card_declined",
  "duration_ms": 812
}

Often confused with

Logs Correlation IDs Distributed tracing

Where it came from2009–2023

For decades log lines were sentences, and getting facts back out of them was a parsing job. Logstash’s grok filter exists to “parse arbitrary text and structure it” with patterns. Syslog itself tried to add fields: RFC 5424 (2009) defined structured data elements such as [exampleSDID@32473 iut="3" eventSource="Application"], but most programs kept writing free text.

Application developers took a simpler route: write the fields yourself. At Heroku the router and internal components logged key=value pairs, and in 2013 Brandur Leach wrote up the style under the name it had internally, logfmt, noting that Splunk recommended the same format. Libraries made fields the default: Serilog for .NET, built “from the ground up to record structured event data”, and structlog for Python both started in 2013; in 2016 came pino, a JSON logger for Node.js, and Uber’s zap for Go, both built for speed.

Standards followed. OpenTelemetry declared its log data model stable in 2022, with fields for the trace and span IDs so a log record can point at the trace it belongs to. In August 2023 Go 1.21 added log/slog to the standard library, because large programs were pulling in several structured loggers through their dependencies and needed a common one.

Why it’s built this way

A fixed message with the variable parts in fields: With free text, every log line is a small design decision: brackets here, parentheses there. With fields the developer just adds whatever matters, adding one more never breaks a query, and statistics such as the p95 of a duration field come straight from the log search.
logfmt for people and machines alike: Brandur Leach described logfmt as readable enough for a person and easy enough for a parser, “even while not being optimal for either”. JSON gives up some readability in exchange for types and nesting, and is what most log stores ingest natively.
One data model that any format maps onto: OpenTelemetry’s log data model was designed so that existing log formats can be mapped onto it unambiguously, and it adds trace context fields so logs, traces and metrics can be joined.

Timeline

2009
RFC 5424 adds structured data elements to syslog
2013
Serilog and structlog repositories start; logfmt is written up
2016
pino for Node.js and Uber’s zap for Go appear
2022
OpenTelemetry declares its log data model stable
2023
Go 1.21 adds log/slog to the standard library

Sources9

Improve this page on GitHub