Fluentd

A CNCF graduated log collector that tags each event, routes it by tag through plugins, and buffers it until the destination accepts it.

What it is: Logs
Used for: Observability
Type: Open source
Made by: Cloud Native Computing Foundation
License: Apache 2.0
Facts checked: 6 October 2026

What it is

Fluentd is a daemon you run on every server (as a forwarder) or on a few central machines (as an aggregator). Every event it handles has three parts: a tag such as nginx.access that says where it came from, a timestamp, and a record, which is a JSON object. <source> blocks bring events in (tail a file, listen for HTTP or syslog, or accept the forward protocol from other Fluentd and Fluent Bit agents), <filter> blocks change or drop them, and <match> blocks send every event whose tag matches a pattern to an output. In front of each output sits a buffer: events collect into chunks, in memory or in files on disk, and a chunk is sent when it is full or its time slot closes. If the destination is down, Fluentd retries with exponential backoff, by default for up to 72 hours.

Its docs describe it as C underneath with a thin Ruby layer, and almost everything is a plugin: more than 1,000 external plugins are published as Ruby gems. Because of Ruby’s global interpreter lock one process uses about one CPU core, so busy hosts run several worker processes. Fluentd is free under Apache 2.0, with no paid edition; ClearCode maintains it and ships the fluent-package installers, and sells support. Its sibling Fluent Bit, written in C, does a similar job in about 450 KB of memory against Fluentd’s 60 MB or more, and speaks the same forward protocol, so the two can be mixed in one pipeline.

Why teams pick it

Tag-based routing keeps one config readable with many inputs and outputs: a <match> pattern decides where each kind of event goes
Retries with backoff ride out long outages, and since v1.19 file buffers move chunks that still fail to a backup directory instead of throwing them away
A very large plugin catalogue; fluent-package bundles outputs for S3, Kafka, Elasticsearch, OpenSearch and Prometheus, plus an OpenTelemetry (OTLP) plugin
Vendor-neutral and fully Apache 2.0 under the CNCF, with LTS packages: the v6 LTS series is supported until the end of 2027

Watch out for

Heavier than its alternatives: more than 60 MB per process plus a Ruby runtime and gem dependencies. The Fluent Bit docs note that cloud providers have switched from Fluentd to Fluent Bit
Ruby’s global interpreter lock limits one process to about one CPU core; the single-process tuning guide covers up to about 5,000 messages a second, beyond that you configure multi workers
A memory buffer loses whatever it holds if Fluentd crashes; only a file buffer survives a restart, and you have to choose it per output
Built for logs: metrics and traces only arrive through plugins, unlike the OpenTelemetry Collector, which handles all three natively
Most plugins are external gems maintained outside the core project, so their quality and release pace vary

Alternatives

What all logs have in common →

LogstashOpen sourceElastic’s JVM-based pipeline that takes events in, parses them with filters such as `grok`, and sends them on, most often to Elasticsearch.

Workers ObservabilityCommercialCloudflare’s built-in logging and tracing for Workers: turn it on in `wrangler.jsonc` and every invocation, `console.log` and binding call is stored and queryable in the dashboard, or exported over OpenTelemetry.

How it fits with other technologies

What Fluentd uses

What uses Fluentd

Tail an nginx log, tag it, and ship hourly files to S3 through a disk buffer · bash

cat > fluent.conf <<'EOF'
<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/fluent/nginx-access.pos   # remembers how far it has read
  tag nginx.access
  <parse>
    @type nginx
  </parse>
</source>

<filter nginx.**>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
  </record>
</filter>

<match nginx.**>
  @type s3
  s3_bucket my-app-logs
  s3_region eu-west-1
  path logs/nginx/
  <buffer tag,time>
    @type file                 # survives restarts; @type memory would not
    path /var/log/fluent/s3
    timekey 3600               # one object per hour
    timekey_wait 10m           # wait for late events before uploading
  </buffer>
</match>
EOF

fluentd --dry-run -c fluent.conf && fluentd -c fluent.conf

Choosing a telemetry agent#

All five read from many sources, change data in flight and write it to many destinations. They differ in runtime, in the shape of data they were built around, and in who steers them.

	Fluentd	Logstash	Telegraf	OpenTelemetry Collector	Vector
Written in	C and Ruby	Java and JRuby; downloads bundle a JDK	Go; one static binary	Go	Rust
Licence	Apache 2.0	Apache 2.0, plus `x-pack` features under the Elastic License	MIT	Apache 2.0	MPL 2.0
Steward	CNCF (graduated 2019)	Elastic	InfluxData	CNCF (graduated 2026)	Datadog (bought its maker, Timber, in 2021)
Built around	Log events: tag, time, JSON record	Events of any shape, mostly logs	Metrics: measurement, tags, fields, time	Traces, metrics and logs in OTLP	Logs and metrics
Config	Own syntax: `<source>`, `<filter>`, `<match>`	Own DSL: `input`, `filter`, `output`	TOML	YAML	YAML, TOML or JSON
Plugins	1,000+ external Ruby gems	200+ Ruby gems	400+ compiled in; others run as external programs via `execd`	Core and contrib distributions, or build your own with OCB	Built-in sources, transforms and sinks; VRL for custom logic
Buffer by default, and the durable option	Memory or file, chosen per output	Memory; persistent queue on disk is opt-in	Memory; disk buffer is experimental (since 1.32)	Memory; persistent queue with the `file_storage` extension	Memory (500 events per sink); disk buffer optional

Plugin counts are the projects’ own: Fluent Bit’s comparison page for Fluentd, the Logstash README, and InfluxData’s Telegraf docs (the Telegraf README still says 300+).
Vector’s README lists traces as “coming soon”; its data model has log and metric events only.

Fluentd or Fluent Bit?#

Both are CNCF graduated projects under the Fluentd umbrella, both are Apache 2.0, and both can act as a forwarder or an aggregator. Fluent Bit started in 2014 as a lightweight processor for embedded Linux and gateways.

	Fluentd	Fluent Bit
Runs on	Containers and servers	Embedded Linux, containers and servers
Language	C and Ruby	C
Memory	More than 60 MB	About 450 KB
Dependencies	A Ruby gem that depends on other gems	None, unless a plugin needs one
Plugins	Over 1,000 external plugins	Over 100 built-in plugins
Signals	Logs; OpenTelemetry through plugins	Logs, metrics, traces and profiles; native OTLP in and out

Figures from the Fluent Bit manual’s “Fluentd and Fluent Bit” page. Pick Fluentd when you need a plugin only it has; otherwise Fluent Bit is the lighter default for a per-node agent.

See it in a real stack

No architecture uses Fluentd yet. Drop it into the playground and wire it up yourself.Open playground →

Where it came from2011–2025

By 2011 a typical web company had logs in many formats on many servers, and wanted them in many places: files, databases, Hadoop, search engines. Every new pair of source and destination meant another script. Sadayuki Furuhashi, co-founder of Treasure Data, a cloud service for big data, made the first commit to Fluentd in June 2011 to build what the project calls a “Unified Logging Layer”: one place that collects events and hands them to whatever needs them.

As maintainer Kiyoto Tamura put it in 2014, with M inputs and N outputs there are M×N possible paths for log data; with a plugin for each input and each output, you need only M+N. Records became JSON, picked because almost everything can read it, not because it is fast.

Fluentd joined the CNCF in November 2016 and in April 2019 became its sixth graduated project, after Kubernetes, Prometheus, Envoy, CoreDNS and containerd. Its C sibling Fluent Bit, begun in 2014 by Eduardo Silva in the same team, has since replaced Fluentd at several cloud providers. ClearCode maintains Fluentd today, and in 2023 its fluent-package installers replaced Treasure Data’s td-agent.

Why it’s built this way

Every event is a tag, a time and a JSON record: JSON was chosen for how widely it is understood: databases and Hadoop could take it as it was. The tag let Fluentd route events without looking inside them.
Plugins on both sides: With an input plugin per source and an output plugin per destination, connecting M sources to N destinations takes M+N pieces instead of M×N custom paths.
A buffer in front of every output: Destinations go down. Collecting events into chunks that wait in a queue, in memory or on disk, and retrying with backoff keeps a slow or failed backend from losing logs or stalling the inputs.

Timeline

2011
First commit in June; v0.10.0 tagged in October
2014
Eduardo Silva starts Fluent Bit, a lightweight C sibling
2016
Accepted as a CNCF project in November
2017
Fluentd v1.0 released in December
2019
Graduates from the CNCF in April
2023
td-agent becomes fluent-package; td-agent 4 support ends in December
2025
fluent-package v6 LTS ships with Fluentd v1.19 in August

Sources40 · facts checked 6 October 2026

Improve this page on GitHub