Cloud Parts

RocksDB

The embedded key-value engine Facebook forked from LevelDB: an LSM tree in a C++ library that many databases and stream processors use as their storage layer.

What it is
Key-value stores,LSM trees
Type
Open source
Made by
Meta
License
Dual: GPLv2 or Apache 2.0, your choice (BSD plus a patent grant until July 2017)
Facts checked
6 October 2026
Website Source code

What it is

RocksDB is not a server. It is a C++ library that your program links in: you open a directory and call Put, Get, Delete and iterators, with keys and values as arbitrary byte strings kept in sorted order. There is no network protocol, no SQL and no user accounts. Many threads in one process can share a database, but only one process may open it for writing; others can open it read-only or as a “secondary” that catches up on demand. Official Java bindings (RocksJava, over JNI) are why so many JVM systems embed it.

It stores data as a log-structured merge (LSM) tree. A write goes into the memtable, a sorted buffer in memory, and (optionally) into a write-ahead log file so it survives a crash. When the memtable fills, it is flushed to an immutable sorted file (an SST) in level 0. Background threads then compact files down through levels L1, L2 and so on, each typically ten times larger than the one above, merging them and dropping overwritten values and deletion markers. A read checks the memtable first, then the levels from newest to oldest, using Bloom filters to skip files that can’t hold the key. The compaction style and many tuning options trade write, read and space amplification against each other.

On top of that it adds column families (separate key spaces in one database), atomic multi-key WriteBatch writes, snapshots, optimistic and pessimistic transactions, merge operators and compaction filters. Those pieces made it the storage engine inside other systems: TiKV (under TiDB) keeps each node’s data and Raft log in RocksDB, ArangoDB is built on it, MyRocks puts it under MySQL, and Kafka Streams, Flink, Spark Structured Streaming and Quix Streams use it to hold local state that can outgrow memory.

Why teams pick it

  • Write-optimized: an LSM tree compresses better and writes less than an update-in-place B-tree; Facebook’s MyRocks used 50% less storage than compressed InnoDB for the same data
  • No network hop: an embedded store answers from local memory or flash, and lets an application keep far more state than fits in its heap
  • Proven as a building block: the storage layer of TiKV, ArangoDB and MyRocks, and the default persistent state store of Kafka Streams
  • Permissive licence choice (Apache 2.0 or GPLv2) and an active release cadence from Meta’s database engineering team

Watch out for

  • Only a storage engine: no server, replication, SQL or secondary indexes; systems like TiKV and CockroachDB built Raft replication and SQL on top themselves
  • One writer process per database directory, enforced with an OS lock
  • Tuning is a craft: when compaction can’t keep up, RocksDB deliberately stalls writes, and its own guide warns that finely tuned instances often regress when the workload or hardware changes
  • A C++ core behind foreign-function boundaries: JNI caps keys and values at 2^31 bytes in Flink, and the cost of crossing cgo was one reason CockroachDB replaced it with Pebble, written in Go

Alternatives

What all key-value stores have in common →
Amazon DynamoDBAWSAWS’s serverless key-value and document database, with single-digit-millisecond reads at any size.Amazon ElastiCacheAWSAWS’s managed Valkey, Redis OSS and Memcached, as node-based clusters or a serverless cache.Amazon MemoryDBAWSA Valkey- and Redis-compatible database that keeps everything in memory but loses no acknowledged write.FoundationDBOpen sourceAn ordered, distributed key-value store with strictly serializable ACID transactions across the whole cluster, built as a small core that other databases are layered on.MemcachedOpen sourceA simple, multithreaded in-memory key-value cache; clients hash keys across independent servers.RedisOpen sourceAn in-memory data-structure server: the default cache, and also queues, counters and leaderboards.ValkeyOpen sourceThe Linux Foundation’s BSD-licensed fork of Redis 7.2.4: an in-memory key-value server that Redis clients can talk to unchanged.Workers KVCommercialCloudflare’s global key-value store that keeps data in a few central stores and caches it in the data centers where it is read, so reads are fast and writes take up to a minute to show up everywhere.

How it fits with other technologies

Write, read and compact a database with the bundled `ldb` and `sst_dump` tools · bash

ldb --db=/tmp/test_db --create_if_missing put a1 b1
ldb --db=/tmp/test_db batchput a3 b3 a4 b4     # one atomic WriteBatch
ldb --db=/tmp/test_db get a1
ldb --db=/tmp/test_db scan                     # keys come back in sorted order

# merge SST files down the levels, recompressing on the way
ldb --db=/tmp/test_db compact --compression_type=bzip2

# look inside the immutable sorted files themselves
sst_dump --file=/tmp/test_db --command=scan

See it in a real stack

No architecture uses RocksDB yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2011–2021

By mid-2011 Dhruba Borthakur, a Facebook engineer, had spent five years building HDFS and HBase and wanted to serve low-latency queries, not just analytics. Data was moving to flash, where a read takes about 100 microseconds while a hop across the data-centre network adds about 50, so a database embedded in the application could be markedly faster than one reached over the network. Google had open-sourced LevelDB, a small C++ LSM library, in July 2011, and it topped the benchmarks. But Borthakur found those benchmarks used databases that fit in RAM: on one five times larger than memory, LevelDB’s single-threaded compaction caused frequent write stalls and huge 99th-percentile latency, and it couldn’t use all the I/O a flash card offered.

So in 2012 he forked LevelDB (version 1.5) and rebuilt it for servers: multi-threaded compaction, a choice of compaction styles, and pluggable compression, memtables, file formats and compaction filters. Facebook open-sourced RocksDB on November 21, 2013, saying it already managed close to a petabyte across its applications and ran random writes and bulk loads 10 times faster than LevelDB. In 2016 Facebook announced MyRocks, RocksDB as a MySQL storage engine, which let its user database tier store the same data in half the space of compressed InnoDB.

RocksDB first shipped under BSD plus a separate patent grant. In 2017 Apache Cassandra developers asked the Apache Software Foundation whether that grant was acceptable; the foundation then barred Facebook’s BSD-plus-Patents licence from its projects, and on July 15, 2017 RocksDB was relicensed under GPLv2 or Apache 2.0. By then it had become the default answer for “I need a fast local store”: CockroachDB used it from the start until it replaced it in 2020 with Pebble, its own RocksDB-inspired engine in Go, while TiKV, Kafka Streams, Flink and Spark still build on it.

Why it’s built this way

Fork LevelDB instead of adapting HBase or Cassandra
HBase and Cassandra were already LSM designs, but they were servers with their own management and deployment; making either an embeddable library would have been a large project. LevelDB was a simple C++ library and the fastest embedded store in open benchmarks, so it was the shortest path.
Multi-threaded compaction, built for flash
LevelDB’s single compaction thread couldn’t keep up with server write rates and stalled writers. Running compactions in parallel lets a database sustain high update rates and use the million-plus IOPS of striped flash cards.
Make the amplification trade-off tunable
Flash wears out with writes, so write amplification mattered as much as speed. Leveled and universal compaction, and the options behind them, let each application choose between write, read and space amplification instead of accepting one fixed balance.
Pluggable parts
Facebook’s workloads differed, so compression, memtable structures, file formats, merge operators and compaction filters can all be swapped without forking the engine; one team used a compaction filter to expire keys.

Timeline

  1. 2011

    Google open-sources LevelDB under a BSD-style licence, July 27

  2. 2012

    Dhruba Borthakur forks LevelDB 1.5 at Facebook

  3. 2013

    Facebook open-sources RocksDB, November 21

  4. 2016

    Flink 1.0 adds a RocksDB state backend; Facebook announces MyRocks for MySQL

  5. 2017

    Relicensed as GPLv2 or Apache 2.0 after the ASF bars Facebook’s BSD-plus-Patents licence

  6. 2020

    CockroachDB 20.2 makes Pebble its default engine, replacing RocksDB

  7. 2021

    Spark 3.2 adds a RocksDB state store for Structured Streaming

Sources20 · facts checked 6 October 2026

Improve this page on GitHub