Cloud Parts

Apache Druid

An open-source real-time analytics database that makes Kafka and Kinesis events queryable within seconds and serves fast slice-and-dice queries to dashboards and APIs.

What it is
Real-time analytics databases
Type
Open source
Made by
Apache Software Foundation
License
Apache 2.0 (GPLv2 until February 2015)
Facts checked
6 October 2026
Website Source code

What it is

Druid stores each table, called a datasource, as segments: columnar files that each cover one time interval, such as an hour or a day, and ideally hold 300 to 700 MB. Every string column gets a dictionary, the encoded values, and a compressed bitmap for each distinct value, so a filter becomes AND and OR operations on bitmaps. A cluster splits the work across services. The Overlord hands ingestion tasks to Middle Managers, which run each task in its own JVM; for Kafka and Kinesis a supervisor keeps tasks reading partitions, tracking offsets so every event is ingested exactly once, and new rows are queryable as soon as a task has indexed them. Tasks then publish finished segments to deep storage, which Druid doesn’t provide: S3, Google Cloud Storage, Azure Blob Storage or HDFS. The Coordinator assigns segments to Historical servers, which download and cache them on local disk, and Brokers split each query across Historicals and running tasks and merge the results. A PostgreSQL or MySQL database holds segment and task metadata, and ZooKeeper handles leader election and service discovery. Rollup is on by default: rows with the same dimension values and the same truncated timestamp are stored as one row.

You query with Druid SQL or native JSON queries over HTTP; batch loads and rewrites use SQL INSERT and REPLACE statements run by the multi-stage query engine, and segments kept only in deep storage can still be queried, more slowly. Druid 38.0.0, released on 1 October 2026, runs on Java 21 or 25. Druid is free under Apache 2.0. Imply, founded in 2015 by Druid’s original creators, sells Imply Polaris, a managed Druid service on AWS: Starter projects (up to 25 GB) start at $100 a month and Standard at $600, with usage billed by the hour per project size, for example $1.30 an hour for an A.01 project with 100 GB of capacity (list prices, checked 6 October 2026).

Why teams pick it

  • Events from Kafka or Kinesis are queryable within seconds, with exactly-once ingestion based on partition offsets or shard sequence numbers
  • Time partitioning plus bitmap indexes let filtered queries skip most data, and built-in sketches give fast approximate distinct counts and quantiles on high-cardinality columns
  • Rollup at ingestion can cut row counts by orders of magnitude for metrics and event counts
  • Deep storage keeps every segment, so lost servers are rebuilt by downloading again, and cold data can be queried from deep storage instead of adding Historicals
  • Scales by adding servers: the cluster rebalances segments by itself and is designed to keep serving through configuration changes and upgrades

Watch out for

  • Many moving parts: Coordinator, Overlord, Broker, Router, Historical and Middle Manager services, plus ZooKeeper, a metadata database and deep storage, each a JVM to size and tune
  • No updates of single records by primary key and no UPDATE statement: you overwrite whole time ranges with REPLACE or a reindexing job
  • Rollup is irreversible: once rows are combined at ingestion, individual events can’t be queried
  • Joins are limited: the native engine broadcasts every table except the leftmost, which must fit in memory; the Dart engine’s sort-merge joins are still experimental
  • Streaming and repeated ingestion leave many small or overlapping segments, so compaction has to be set up and watched

Alternatives

What all real-time analytics databases have in common →
Alibaba Cloud HologresCommercialAlibaba Cloud’s PostgreSQL-compatible real-time data warehouse, which takes streaming writes, answers analytical queries and serves key lookups from one copy of the data.Apache DorisOpen sourceAn open-source MPP analytical database, born at Baidu as Palo, that serves sub-second queries on fresh, updatable data through the MySQL protocol and also queries Iceberg, Hudi, Paimon and Hive tables in place.FireboltCommercialA cloud data warehouse built for low-latency, high-concurrency queries behind apps and dashboards, also free to self-host as Firebolt Core.PivotOpen sourceA young open-source analytics server in Rust that speaks the PostgreSQL protocol and queries Iceberg and Delta Lake tables straight from object storage.StarRocksOpen sourceAn open-source MPP analytical database with a vectorized engine that serves fast joins and real-time updates on its own tables, and also queries Iceberg, Hudi, Delta Lake and Hive tables in place.TinybirdCommercialA hosted ClickHouse platform that turns SQL queries into authenticated REST API endpoints, for analytics inside your product.

How it fits with other technologies

In the web console: load the sample Wikipedia edits with rollup, then query them · sql

-- Batch ingestion with rollup: one row per channel, page and minute
REPLACE INTO wiki_edits OVERWRITE ALL
SELECT
  TIME_FLOOR(TIME_PARSE("timestamp"), 'PT1M') AS __time,
  channel,
  page,
  COUNT(*) AS edits,
  SUM(added) AS chars_added
FROM TABLE(
  EXTERN(
    inputSource => '{"type":"http","uris":["https://druid.apache.org/data/wikipedia.json.gz"]}',
    inputFormat => '{"type":"json"}'
  )
) EXTEND ("timestamp" VARCHAR, channel VARCHAR, page VARCHAR, added BIGINT)
GROUP BY 1, 2, 3
PARTITIONED BY DAY
CLUSTERED BY channel;

-- Busiest channels in one hour of the sample day
SELECT channel, SUM(edits) AS edits, SUM(chars_added) AS chars_added
FROM wiki_edits
WHERE __time >= TIMESTAMP '2015-09-12 10:00:00'
  AND __time < TIMESTAMP '2015-09-12 11:00:00'
GROUP BY channel
ORDER BY edits DESC
LIMIT 5;

Druid, Pinot and ClickHouse#

The three open-source databases most often compared for user-facing analytics. All are Apache 2.0 licensed; they differ in how many pieces you run, how they pre-aggregate and how they handle changed rows.

Apache DruidApache PinotClickHouse
Built atMetamarkets, for ad analytics dashboards (2011)LinkedIn, for member-facing features such as Who Viewed ProfileYandex, for the Yandex.Metrica web analytics service
Cluster piecesSix service types, plus ZooKeeper, a metadata database and deep storageControllers, brokers and servers, coordinated by Apache Helix on ZooKeeperOne binary; replicated clusters add ClickHouse Keeper
Pre-aggregationRollup at ingestion, on by defaultStar-tree index of pre-aggregated values across several columnsIncremental materialized views that write into a second table
Changing rowsNo updates by key; overwrite whole time rangesNative upserts during stream ingestionMutations and lightweight deletes; not cheap row-level edits
Managed serviceImply PolarisStarTree CloudClickHouse Cloud
  • Compiled from each project’s documentation. It says nothing about speed; compare on your own data and queries.

Ways to run it#

Imply list prices, checked on 6 October 2026. Polaris also bills for data ingested and processed and for file storage.

Self-managed Druid

Apache 2.0

Free. Download a release or use the Kubernetes extension to run ingestion tasks as jobs. A single machine for the quickstart needs at least 6 GiB of RAM; production clusters typically run master, query and data servers separately.

Imply Polaris Starter

from $100/month

Managed Druid for proofs of concept: projects up to 25 GB, variable performance.

Imply Polaris Standard

from $600/month

Production projects up to 9.6 TB, a 99.9% uptime SLA and private networking. Projects are billed hourly by size, from $0.80 an hour (D.04) to $124.80 (A.96).

Imply Polaris Custom

contact sales

Unlimited project size, a 99.95% SLA and scheduled upgrades.

See it in a real stack

No architecture uses Apache Druid yet. Drop it into the playground and wire it up yourself.Open playground →

Benchmarks

Benchmark · ClickHouse Inc.ClickBenchHow fast analytical databases run 43 SQL queries over one 100-million-row table of web analytics events, on the same cloud machines.Includes Apache Druid · 7 caveats to read first
How to read a benchmark →
Where it came from2011–2026

Metamarkets sold ad analytics dashboards whose customers wanted to drill into billions of events along any combination of dimensions, in under a second. Over a year it tried Greenplum, InfoBright, MySQL and HBase. Pre-computing every aggregation into a key-value store kept queries fast, but as dimensions grew the batch jobs ran past 24 hours, far too late for fresh data. So in April 2011 Eric Tschetter introduced Druid, a distributed, in-memory OLAP store the company had built itself.

Metamarkets open-sourced Druid in October 2012 under GPLv2, with Netflix evaluating it for real-time monitoring. A 2014 SIGMOD paper by Fangjin Yang, Tschetter, Xavier Léauté, Gian Merlino and colleagues described the design: real-time nodes that hand off immutable segments to deep storage and historical nodes that serve them. The name comes from the shape-shifting druid class in role-playing games.

In February 2015 Druid switched to Apache 2.0 to invite community governance, and that year its creators founded Imply. Druid entered the Apache Incubator in 2018 and graduated in December 2019.

Why it’s built this way

Summarise rows at ingestion (rollup)
Grouping raw impressions by timestamp and the chosen dimensions made Metamarkets’ data set often about 100 times smaller, while still letting users slice it any way at query time.
Immutable segments in deep storage
Historical nodes only read data that never changes, so they give consistent reads and scan segments in parallel without locks. A permanent copy in S3 or HDFS means servers can be replaced by downloading segments again.
Bitmap indexes for every dimension value
Users filtered on arbitrary combinations of dimensions. Boolean operations on compressed bitmaps, a technique from search engines, answer those filters without scanning rows.
Separate node types for fresh and historical data
Real-time nodes keep recent events in a queryable in-memory index and hand off finished segments, so ingestion and queries over older data scale independently.

Timeline

  1. 2011

    Metamarkets introduces Druid in April

  2. 2012

    Open-sourced under GPLv2 in October

  3. 2014

    SIGMOD paper describes the architecture

  4. 2015

    Relicensed under Apache 2.0; its creators found Imply

  5. 2018

    Enters the Apache Incubator in February

  6. 2019

    Graduates to a top-level Apache project in December

  7. 2026

    Druid 38 drops Java 17 and ZooKeeper-based segment discovery

Sources31 · facts checked 6 October 2026

Improve this page on GitHub