cloudparts

HerdDBDormant

A SQL database in Java that runs inside an application’s JVM or as a replicated cluster, with Apache BookKeeper as its write-ahead log and ZooKeeper for metadata.

What it is
Relational databases
Type
Open source
Made by
Diennea
License
Apache 2.0
Status
Dormant. No release since 0.29.0 in March 2024; master has had one commit since (September 2025), and fixes opened in autumn 2026 are still unmerged pull requests.
Facts checked
24 October 2026
Website Source code

What it is

HerdDB is, in its own words, a key-value database with an SQL layer on top. Each table is a map from primary key to row; Apache Calcite parses and plans the SQL, and applications use a JDBC driver or a lower-level API. Every change goes to a transaction log first. An update by primary key then changes only an in-memory buffer, and at checkpoints, on a timer or under memory pressure, dirty pages are written to disk as immutable page files. Secondary indexes adapt the block range index (BRIN) idea. Tables are grouped into tablespaces, and transactions and subqueries can only span tables in the same tablespace. The isolation level is always read committed.

The same database runs in three ways, picked by the JDBC URL: in memory or on local disk inside the application’s own JVM (jdbc:herddb:local), as a standalone server, or as a cluster found through ZooKeeper. In a cluster, the tablespace is the unit of replication. Each has one leader, which writes the log to Apache BookKeeper, and any number of followers, which tail the log and replay it into their own local copy. A write is acknowledged once a quorum of BookKeeper servers (bookies) has it, and when a new leader takes over, BookKeeper fences out the old one so it can’t write again. ZooKeeper holds node discovery, tablespace metadata and the list of active log ledgers. A diskless-cluster mode also keeps data pages on the bookies, so database nodes need no persistent volume. Diennea built HerdDB for its EmailSuccess mail transfer agent, which keeps the state of every message in it, and Apache Pulsar Manager, now discontinued, used it as its default database.

Why teams pick it

  • Embeds in any JVM through a JDBC URL, and the same application can later point at a standalone server or a replicated cluster
  • Designed for fast writes and primary-key reads and updates across hundreds of tables
  • Shared-nothing replication with automatic leader failover, built on BookKeeper and ZooKeeper rather than its own consensus code

Watch out for

  • Dormant: the last release was 0.29.0 in March 2024, and recent fixes sit in open pull requests
  • Read committed only, transactions and subqueries confined to one tablespace, joins the docs call “quite limited”, and forward-only JDBC result sets
  • Old log ledgers are dropped after a time-to-live, so a follower that is down longer than that can’t catch up from the log
  • A cluster also means running ZooKeeper and BookKeeper

Good for

Specs

Compare with alternatives →
Data model
Relational tables, each a map from primary key to row
Query language
SQL, parsed and planned by Apache Calcite
Client protocol
JDBC driver, or a lower-level Java API
Secondary indexes
Block range indexes (BRIN-style)
Transactions
Across tables in one tablespace
Default isolation
Read committed only
Replication
One leader per tablespace logs to BookKeeper; followers replay the log
Storage engine
Transaction log, in-memory pages checkpointed to immutable page files
Written in
Java
Ways to run it
Embedded in a JVM, standalone server, or a cluster coordinated by ZooKeeper

Alternatives

What all relational databases have in common →Compare side by side →
SQLiteOpen sourceAlso good for inside the application processA complete SQL database in a library and a single file, with no server to run.CockroachDBCommercialA distributed SQL database that speaks the PostgreSQL wire protocol, replicates every slice of data with Raft and keeps transactions serializable across nodes and regions.FoundationDBOpen sourceAn ordered, distributed key-value store with strictly serializable ACID transactions across the whole cluster, built as a small core that other databases are layered on.NeonCommercialServerless Postgres that keeps data in a shared storage layer, so compute can scale to zero and a branch of the database is a pointer, not a copy.XataCommercialA platform that runs unmodified Postgres on Kubernetes with copy-on-write branches at the storage layer, scale to zero, and anonymized copies of production for development.PlanetScaleCommercialManaged MySQL on Vitess and managed Postgres, with schema changes shipped through branches and deploy requests.Alibaba Cloud PolarDBCommercialAlibaba Cloud’s cloud-native relational databases: MySQL- and PostgreSQL-compatible clusters whose compute nodes share one copy of the data on distributed storage, plus PolarDB-X, a distributed MySQL-compatible edition.VitessOpen sourceA proxy layer, born at YouTube, that shards MySQL across many servers while the app still sees one database.Cloudflare D1CommercialCloudflare’s serverless SQL database that speaks SQLite, runs each database inside a Durable Object, and bills by rows read and written rather than by the hour.

How it fits with other technologies

One JDBC driver, three ways to run · java

// jdbc:herddb:local                              in memory, inside this JVM
// jdbc:herddb:server:localhost:7000              a standalone server
// jdbc:herddb:zookeeper:localhost:2181/herd      a replicated cluster, found through ZooKeeper

// start an embedded server in this JVM, keeping data in ./dbdata
String url = "jdbc:herddb:server:localhost:7000?server.start=true&server.base.dir=dbdata";
try (Connection c = DriverManager.getConnection(url, "sa", "hdb");
     Statement s = c.createStatement()) {
  s.execute("CREATE TABLE messages (id string primary key, state int, updated timestamp)");
  s.executeUpdate("INSERT INTO messages VALUES ('m-1', 0, CURRENT_TIMESTAMP)");
  s.executeUpdate("UPDATE messages SET state = 1 WHERE id = 'm-1'");  // primary-key update: in memory plus a log write
}

See it in a real stack

No architecture uses HerdDB yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2016–2025

Diennea, the company behind the MagNews platform and the EmailSuccess mail transfer agent, needed a database for EmailSuccess that could take many fast writes across many tables, run transactions over several tables, and be used through SQL and JDBC, and it found that HBase and other big-data engines didn’t fit. It also wanted the database to run embedded inside the agent’s JVM yet span machines. The team had already used Apache BookKeeper and ZooKeeper in Majordodo, another Diennea open-source project, and built HerdDB on them. The repository opened in February 2016 and version 0.0.1 was tagged in July.

Enrico Olivelli presented HerdDB at Pulsar Summit 2020 and described its design in a series of posts on building a database with BookKeeper. Apache Pulsar Manager shipped it as its default embedded database. Releases continued until 0.29.0 in March 2024; since then the project has been mostly quiet.

Why it’s built this way

A key-value core with an SQL layer
Fast writes and primary-key access were the main need, while SQL and JDBC let users reuse what they knew and port existing applications.
BookKeeper as a shared write-ahead log
Server nodes never talk to each other; every update passes through BookKeeper, whose quorum writes make it durable and whose fencing guarantees only one leader can write a tablespace’s log.
Embeddable in the application’s JVM
EmailSuccess needed the database inside its own process, where each node reads its local data without going over the network.
Replicate by tablespace
Making the tablespace the unit of leadership and replication spreads load across nodes, at the cost of transactions that can’t cross tablespaces.

Timeline

  1. 2016

    Repository opens; version 0.0.1 is tagged in July

  2. 2020

    Presented at Pulsar Summit; design posts on building a database with BookKeeper

  3. 2024

    Version 0.29.0, the latest release, in March

  4. 2025

    One commit to master, moving publishing to Maven Central

Sources10 · facts checked 24 October 2026

Improve this page on GitHub