Inside the application process

A database that is a library linked into the program, with no server to run, so a query is a function call instead of a network round trip.

Relational databases Key-value stores Query engines

What it asks of a system

An embedded database runs in the same process as the application and keeps its data in local files. There is nothing to install or operate, a lookup costs no network hop, and the data can ship with the program: phones, browsers, desktop apps, devices and command-line tools all use this. SQLite’s docs put it plainly: it does not compete with client/server databases, it competes with fopen(). The same shape shows up in key-value engines such as RocksDB, which other databases build on, and in analytical engines such as DuckDB, which run SQL over local or remote files inside a Python or Node process.

The trade-offs follow from having no server. Only processes on the same machine can reach the data, so sharing it needs replication or a server built on top. Concurrency is limited: SQLite allows many readers but one writer at a time per file. And the application owns backups, upgrades and crash safety. When you compare engines, check the file format’s stability across versions, how writes from several threads or processes are handled, its behaviour on power loss, the size of the library, and the languages it has bindings for.

Good at it · 15

Compare side by side

Relational databases

HerdDBDormantOpen sourceA `jdbc:herddb:local` URL runs the database in memory or on local disk inside the application’s own JVM.A SQL database in Java that runs inside an application’s JVM or as a replicated cluster, with Apache BookKeeper as its write-ahead log and ZooKeeper for metadata.

SQLiteOpen sourceA library under 900 KiB that the program calls directly, with the whole database in one portable file.A complete SQL database in a library and a single file, with no server to run.

TursoOpen sourceAn in-process Rust rewrite of SQLite that opens the same files, with `BEGIN CONCURRENT` so several writers can commit at once.A from-scratch Rust rewrite of SQLite with concurrent writes, plus libSQL (an open-contribution SQLite fork) and Turso Cloud, which hosts many small databases on S3.

NoSQL databases

RealmDeprecatedOpen sourceEach SDK runs Realm Core inside the app and persists the app’s own classes to a file on the device, with no server or ORM.An embedded object database for mobile apps, a replacement for SQLite and Core Data, whose cloud sync service MongoDB shut down in 2025.

Key-value stores

PebbleOpen sourceA pure-Go library that a program imports, with no server and no cgo boundary to cross on each call.CockroachDB’s storage engine: an embedded LSM-tree key-value library written in Go, modelled on RocksDB and built for what CockroachDB needs.

RocksDBOpen sourceA C++ library linked into the program, with no server or network hop, and official Java bindings over JNI.The embedded key-value engine Facebook forked from LevelDB: an LSM tree in a C++ library that many databases and stream processors use as their storage layer.

SlateDBOpen sourceA Rust library that keeps its write-ahead log and SSTs in object storage, so there is no server and no disk to manage.An embedded key-value store in Rust, built as an LSM tree that keeps everything, including its write-ahead log, in object storage such as S3, GCS or Azure Blob Storage instead of on local disk.

Document databases

SurrealDBCommercialThe same engine embeds in Rust, Go, JavaScript, Python or .NET programs, or runs in a browser on IndexedDB.A multi-model database in Rust that stores documents, graph edges, vectors and time series in one engine, queried with an SQL-like language, SurrealQL, and able to run embedded, as a server or as a cluster.

Time-series databases

ArcticDBCommercialA Python package with a C++ engine and no server: it reads and writes DataFrames straight to S3, Azure Blob or local LMDB.Man Group’s serverless DataFrame database: a Python library with a C++ engine that stores versioned pandas DataFrames directly in S3, Azure Blob Storage or local LMDB.

Vector databases

ChromaOpen source`chromadb.PersistentClient` runs inside the Python process, keeping data in SQLite and index files in a local folder.An open-source vector database for AI apps that runs inside a Python or JavaScript process, as a single server, or as a distributed service on object storage.

Query engines

Apache DataFusionOpen sourceA Rust library compiled into your program: SQL or DataFrame queries run in-process over files and Arrow data, with no server.A query engine library in Rust, built on Arrow, that other projects embed and extend to build their own databases and data tools.

chDBOpen sourceClickHouse compiled into a library: a query is a function call in your own process, with no server, socket or child process.The ClickHouse engine packaged as a library: run ClickHouse SQL inside a Python, Go, Rust or Node.js program, with no server.

DuckDBOpen sourceRuns inside your process as a library with no dependencies; a database is one file, or it queries Parquet and CSV where they are.An in-process analytical SQL database: a library with no server that queries Parquet, CSV, Iceberg and S3 directly.

PolarsOpen sourceA library inside your Python or Rust process with no server, holding tables in Arrow so they pass to PyArrow or DuckDB without copying.A DataFrame library with a Rust query engine underneath: you write Python (or Rust, SQL, Node.js, R) and it plans, optimises and runs the query on every core, streaming data bigger than memory.

RayforceOpen sourceA dependency-free C library you link into your program, with Python bindings, so queries are function calls.A small embeddable engine in pure C that runs columnar analytics, graph traversals and recursive Datalog queries in one optimised plan.

Sources

Why each technology is listed comes from the sources on its own page.