cloudparts

SlateDB

An embedded key-value store in Rust, built as an LSM tree that keeps everything, including its write-ahead log, in object storage such as S3, GCS or Azure Blob Storage instead of on local disk.

What it is
Key-value stores,LSM trees
Type
Open source
Made by
SlateDB project (Commonhaus Foundation)
License
Apache 2.0
Facts checked
23 October 2026
Website Source code

What it is

SlateDB is a library, like RocksDB: your program opens a database at a path in a bucket and calls put, get, delete, scan and merge on byte keys and values. Underneath it is an LSM tree. Writes go to an in-memory write-ahead log buffer and a MemTable and return a WriteHandle straight away; a background flusher writes the WAL buffer to object storage when it fills or a flush interval passes, and await_durable() on the handle waits until that has happened. Full MemTables are frozen and flushed as level-0 SSTs, and compaction merges L0 files into range-partitioned sorted runs. A manifest file lists which SSTs make up the database and is updated atomically with conditional writes, which is also how SlateDB fences a stale “zombie” writer that thinks it still owns the database.

SSTs (sorted string tables) are immutable files of data blocks followed by an optional Bloom filter, an index block, an optional stats block and a FlatBuffers footer, with a CRC32 checksum on every block. SlateDB writes the same format twice for different jobs: WAL SSTs hold rows in sequence-number order for recovery and change data capture, while compacted SSTs hold rows in key order for reads. A get checks the MemTables, then L0, then the sorted runs, using Bloom filters, an in-memory block cache and an optional local disk cache to avoid object-store requests. There is one writer per database, but any number of DbReader instances on other machines can read it from a checkpoint, and compaction and garbage collection can run as separate processes. Transactions with snapshot isolation, merge operators, checkpoints and cheap clones are built in. Official bindings exist for Go, Java, Node.js and Python.

Why teams pick it

  • No disks to manage or replicate: durability, capacity and cross-zone availability come from the object store
  • Batching writes into WAL and SST objects keeps PUT costs down compared with one object per key
  • Readers, compactors and garbage collectors can run on other machines against the same bucket, and clones and checkpoints are cheap because SSTs are immutable
  • Used in production by Dropbox, ZeroFS, HelixDB, Opendata and others, according to the project

Watch out for

  • Object storage latency: the docs target workloads that tolerate 50–100 ms writes, or accept losing unflushed writes on a crash
  • Only one writer per database; partitioning, and multi-writer setups, are up to you
  • Pre-1.0 (0.17 in September 2026): storage formats are compatible between adjacent versions, but the API can still break
  • Built for key-value OLTP; the FAQ points analytical scans of columnar data to other systems, and its own benchmarks put read throughput well below RocksDB on NVMe because of slower p99 reads

Good for

Specs

Compare with alternatives →
Data model
Sorted byte keys and values
Query language
Library API: put, get, delete, scan, merge
Transactions
Transactions within one database
Default isolation
Snapshot isolation
How it scales
One writer per database; any number of readers on other machines
Replication
None of its own; durability comes from the object store
Storage engine
LSM tree with WAL and SSTs in object storage
Storage format
SST files in S3, GCS or Azure Blob Storage
Written in
Rust, with Go, Java, Node.js and Python bindings
Ways to run it
Embedded library over a bucket

Alternatives

What all key-value stores have in common →Compare side by side →
RocksDBOpen sourceAlso good for inside the application processThe embedded key-value engine Facebook forked from LevelDB: an LSM tree in a C++ library that many databases and stream processors use as their storage layer.PebbleOpen sourceAlso good for inside the application processCockroachDB’s storage engine: an embedded LSM-tree key-value library written in Go, modelled on RocksDB and built for what CockroachDB needs.

How it fits with other technologies

Open a database on an object store, write, wait for durability, read · rust

use slatedb::{Db, Error};
use slatedb::object_store::{ObjectStore, memory::InMemory};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Error> {
    // InMemory for the example; use an S3, GCS or Azure store in production
    let object_store: Arc<dyn ObjectStore> = Arc::new(InMemory::new());
    let db = Db::open("/tmp/test_kv_store", object_store).await?;

    // put() returns once the write is in the in-memory WAL and MemTable
    let handle = db.put(b"user:42", b"Ada").await?;
    handle.await_durable().await?;   // wait until the WAL SST is in the bucket

    assert_eq!(db.get(b"user:42").await?, Some("Ada".into()));

    let mut iter = db.scan("user:".."user;").await?;   // range scan
    while let Ok(Some(item)) = iter.next().await {
        println!("{:?} = {:?}", item.key, item.value);
    }
    db.close().await
}

See it in a real stack

No architecture uses SlateDB yet. Drop it into the playground and wire it up yourself.Open playground →

Worth exploring

Article· October 2026Prism: A geo-replicated data lake serving billions of <10 ms point lookupsHåkon Åmdal and Jeff Ling, GoldskyHow Goldsky stores each blockchain’s history once in object storage and serves it in every region: recent blocks in memory, Vortex files that stay compressed in RAM, a SlateDB index beside the data on Tigris, and Tigris’s TAG cache for in-region reads. It gives real latency splits by tier (under 1 ms hot, about 9 ms through the cache, 10 to 800 ms cold) and per chain. It was reviewed by Tigris’s CEO, so read it partly as a showcase for Tigris.
All resources →
Where it came from2024–2026

Object storage is cheap, extremely durable and readable from anywhere, and systems such as WarpStream and turbopuffer showed it could back online systems, not just data lakes. But requests take 50–100 ms, every PUT and GET is billed, and objects can only be replaced whole. SlateDB’s authors first tried running RocksDB’s storage API on top of S3 and found the mismatch too large: RocksDB relies on file locks, the filesystem cache and hard links for checkpoints, none of which object stores provide. They built a new LSM tree in async Rust that batches writes, caches reads and coordinates writers with conditional IF-MATCH writes on a manifest.

Chris Riccomini and Rohan Desai presented it as a cloud-native LSM on object storage at P99 CONF in 2024, and version 0.1.0 shipped in August 2024. Releases followed roughly every two months, adding transactions, merge operators, clones, change data capture and bindings for Go, Java, Node.js and Python, and the project joined the Commonhaus Foundation. It was never formally announced until a June 2026 post by Almog, Chris, Rohan and team, which said Dropbox, ZeroFS, HelixDB and Opendata already ran it in production and that 1.0 was near.

Why it’s built this way

Everything in object storage, including the WAL
RocksDB-Cloud keeps its WAL on local disk, Kafka or Kinesis. Putting the WAL in the bucket too removes a second system, at the cost of write latency.
A WAL separate from L0
Flushing writes often as L0 SSTs would create too many L0 files to track in memory. A WAL that is only read during recovery lets SlateDB flush frequently for lower durable-write latency without that cost.
One writer, fenced through the manifest
Object stores have no file locks. Conditional writes on the manifest let a new writer take over and stop a zombie writer, which also makes partitioning across nodes straightforward to build on top.
Immutable SSTs referenced by a manifest
Data files are only deleted when no manifest references them, so checkpoints, clones and key-range splits are metadata operations instead of copies.

Timeline

  1. 2024

    Repository created (March); talk at P99 CONF; 0.1.0 released (August)

  2. 2025

    Releases 0.5 to 0.10 add features every two months or so

  3. 2026

    First official announcement post (June); 0.17.0 released (September)

Sources9 · facts checked 23 October 2026

Improve this page on GitHub