Cloud Parts

Logstash

Elastic’s JVM-based pipeline that takes events in, parses them with filters such as grok, and sends them on, most often to Elasticsearch.

What it is
Logs
Used for
Observability
Type
Open source
Made by
Elastic N.V.
License
Apache 2.0 (core and plugins); x-pack code under the Elastic License, included in the default download but not in the -oss builds
Facts checked
6 October 2026
Website Source code

What it is

Logstash is a server process that runs pipelines of three stages. Inputs receive events (from Beats and Elastic Agent, files, syslog, Kafka, S3 or a scheduled SQL query), filters parse and enrich them, and outputs send them on, most often to Elasticsearch. The best-known filter is grok, which turns an unstructured log line into named fields using about 120 built-in regular-expression patterns. Worker threads, one per CPU core by default, take events from a queue in batches of 125 and run them through the filters and outputs. That queue lives in memory unless you switch on the persistent queue, which writes events to disk (up to 1 GB by default) so they survive a restart; it is not replicated, so it does not survive losing the disk.

Logstash runs on the JVM: its core is Java and JRuby, plugins are Ruby gems (over 200 of them), and the downloads bundle a Temurin 21 JDK. It is free to use. The repository mixes licences: code outside the x-pack folder is Apache 2.0, code inside it is under the Elastic License, and the default download contains both; builds with -oss in the name contain only the Apache 2.0 parts. Elastic’s 2021 move of Elasticsearch and Kibana to SSPL and the Elastic License, and the AGPL option it added for them in 2024, did not apply to Logstash. Some features, such as managing pipelines centrally from Kibana, need a paid Elastic subscription.

Why teams pick it

  • Flexible parsing in one readable config: grok, date, mutate, drop, clone and geoip filters, with conditionals between them
  • The natural fit for Elasticsearch and Kibana; the beats input, and its successor elastic_agent, receive data from Elastic’s own shippers
  • Over 200 plugins, including Kafka, S3, JDBC and Redis, each a separate gem that is released on its own
  • The persistent queue gives at-least-once delivery through restarts and crashes, for inputs that acknowledge what they receive
  • The core stayed Apache 2.0 through all of Elastic’s licence changes

Watch out for

  • Heavy for running on every host: a JVM for which Elastic recommends a 4 to 8 GB heap in typical ingestion setups
  • The default memory queue loses in-flight events if the machine fails; the persistent queue is off by default, cannot protect inputs without acknowledgements (TCP, UDP), and is not replicated
  • Mixed licensing to keep track of: the default download includes Elastic License code, the -oss build drops every x-pack feature, and centralized pipeline management needs a subscription
  • Grok patterns are regular expressions, so parsing a new log format means writing and testing patterns by hand

Alternatives

What all logs have in common →
FluentdOpen sourceA CNCF graduated log collector that tags each event, routes it by tag through plugins, and buffers it until the destination accepts it.Workers ObservabilityCommercialCloudflare’s built-in logging and tracing for Workers: turn it on in `wrangler.jsonc` and every invocation, `console.log` and binding call is stored and queryable in the dashboard, or exported over OpenTelemetry.

How it fits with other technologies

Parse web server logs from Beats, then index them and copy them to Kafka · bash

cat > /etc/logstash/conf.d/web.conf <<'EOF'
input {
  beats { port => 5044 }
}
filter {
  grok { match => { "message" => "%{COMBINEDAPACHELOG}" } }
  date { match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ] }   # use the log’s own time
}
output {
  elasticsearch { hosts => ["https://es.internal:9200"] }
  kafka { bootstrap_servers => "kafka:9092" topic_id => "web-access" codec => json }
}
EOF

# keep queued events on disk across restarts (off by default)
echo 'queue.type: persisted' >> /etc/logstash/logstash.yml

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/web.conf --config.test_and_exit
sudo systemctl restart logstash

Choosing a telemetry agent#

All five read from many sources, change data in flight and write it to many destinations. They differ in runtime, in the shape of data they were built around, and in who steers them.

FluentdLogstashTelegrafOpenTelemetry CollectorVector
Written inC and RubyJava and JRuby; downloads bundle a JDKGo; one static binaryGoRust
LicenceApache 2.0Apache 2.0, plus x-pack features under the Elastic LicenseMITApache 2.0MPL 2.0
StewardCNCF (graduated 2019)ElasticInfluxDataCNCF (graduated 2026)Datadog (bought its maker, Timber, in 2021)
Built aroundLog events: tag, time, JSON recordEvents of any shape, mostly logsMetrics: measurement, tags, fields, timeTraces, metrics and logs in OTLPLogs and metrics
ConfigOwn syntax: <source>, <filter>, <match>Own DSL: input, filter, outputTOMLYAMLYAML, TOML or JSON
Plugins1,000+ external Ruby gems200+ Ruby gems400+ compiled in; others run as external programs via execdCore and contrib distributions, or build your own with OCBBuilt-in sources, transforms and sinks; VRL for custom logic
Buffer by default, and the durable optionMemory or file, chosen per outputMemory; persistent queue on disk is opt-inMemory; disk buffer is experimental (since 1.32)Memory; persistent queue with the file_storage extensionMemory (500 events per sink); disk buffer optional
  • Plugin counts are the projects’ own: Fluent Bit’s comparison page for Fluentd, the Logstash README, and InfluxData’s Telegraf docs (the Telegraf README still says 300+).
  • Vector’s README lists traces as “coming soon”; its data model has log and metric events only.

Which parts are open source#

Logstash is often lumped in with Elasticsearch’s licence changes. The repository’s LICENSE.txt says what applies today.

Core and plugins

Apache 2.0

Every file outside the x-pack folder is Apache 2.0 unless it says otherwise. Plugins live in their own repositories under logstash-plugins and ship as Ruby gems.

The `x-pack` folder

Elastic License

Elastic’s extra features, such as monitoring, central pipeline management and GeoIP database management, merged into the repository in April 2018. Their source is readable, but under the Elastic License, not an open-source licence.

Default download

both

Contains the Apache 2.0 code and the x-pack code. Free features are on by default; subscription features, such as centralized pipeline management, need a paid licence or a 30-day trial.

`-oss` download

Apache 2.0 only

Elastic still publishes these builds (9.5.4 in September 2026). They include only Apache 2.0 features.

2021 and 2024

no change

In January 2021 Elasticsearch and Kibana moved from Apache 2.0 to SSPL and the Elastic License, and in 2024 Elastic added AGPL as a third option for them. Elastic’s licensing FAQ says the changes covered only Elasticsearch and Kibana, and that Logstash and Beats remain Apache 2.0.

See it in a real stack

No architecture uses Logstash yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2009–2021

Jordan Sissel, who had worked at Google, Loggly and Heroku, started Logstash as an open-source project in 2009; the first prototype commit, by Pete Fritchman, is from August that year. The idea was a generic event processor: a log could be any timestamped piece of data, and inputs, filters and outputs could be combined freely. It was written in Ruby, with a short text config in three stages that made it easy to move data from any source to any destination.

Sissel says he designed Logstash from day one to work with Elasticsearch and Kibana, and the three became known as the ELK stack. In August 2013 he joined Elasticsearch, the company, which began selling support for Logstash the same day, alongside the release of Logstash 1.2.

JRuby became Logstash’s usual runtime for speed and for Java libraries, but crossing between Ruby and Java cost CPU and memory, so Elastic moved the internals to Java. A Java execution engine, previewed in 6.3 in 2018, became the default in 7.0 in 2019. When Elasticsearch and Kibana left Apache 2.0 in 2021, Logstash’s core stayed.

Why it’s built this way

Three stages: input, filter, output
Sissel wanted a generic system where any input, processing step and output could be composed. The simple model and text config proved a friendly way to get data from point A to point B.
Built for Elasticsearch
Sissel says he designed it from day one to work with Elasticsearch and Kibana: Logstash pulls events off systems, Elasticsearch stores and searches them, and Kibana charts the results.
Run on JRuby, then move the core to Java
JRuby gave Ruby code the JVM’s speed and its libraries. As more plugins were written in Java, compiling pipelines to JVM bytecode removed the cost of crossing between the two languages and cut startup and reload times.

Timeline

  1. 2009

    Jordan Sissel starts Logstash; first prototype commit in August

  2. 2011

    v1.0.0 tagged in May

  3. 2013

    Sissel joins Elasticsearch in August; Logstash 1.2 released

  4. 2018

    X-Pack source opens under the Elastic License and moves into the repository

  5. 2019

    Logstash 7.0 makes the Java execution engine the default

  6. 2021

    Elasticsearch and Kibana leave Apache 2.0; Logstash does not

Sources40 · facts checked 6 October 2026

Improve this page on GitHub