Cloud Parts

chDB

The ClickHouse engine packaged as a library: run ClickHouse SQL inside a Python, Go, Rust or Node.js program, with no server.

What it is
Query engines
Type
Open source
Made by
ClickHouse, Inc.
License
Apache 2.0
Facts checked
6 October 2026
Website Source code

What it is

chDB is ClickHouse compiled into a library that loads into your own process. A call such as chdb.query(sql, "DataFrame") goes straight into ClickHouse’s parser, planner and vectorised engine, with no server, socket or child process; the result comes back in any ClickHouse output format (CSV, JSON, Arrow, Parquet or a pandas DataFrame), handed to Python through a memoryview to avoid copying. By default it keeps nothing: tables live in memory and disappear with the process. Open a connection or session on a path and it keeps tables on disk in the same format as clickhouse-local, so the two can share a database directory. It reaches data where it sits through ClickHouse’s table functions, such as file(), s3(), url(), mysql() and postgresql(), and the Python() table function queries a pandas DataFrame or Arrow table in place.

The engine itself ships as chdb-core (26.9.0, September 2026), versioned like ClickHouse. The chdb Python package (4.4.0) sits on top and adds DataStore, a pandas-compatible API that records a chain of pandas calls and compiles it into one ClickHouse query when you ask for the result. Bindings exist for C/C++, Go, Rust, Node.js, Bun, Java, Ruby, .NET, Zig and WebAssembly, at very different stages of maturity. It is Apache 2.0 licensed and free; ClickHouse, Inc. took the project over in March 2024 and employs its creator. Version 4.4 adds chdb.durable, which keeps a database’s state in an S3, GCS or Azure bucket you own, with a lease that guarantees a single writer.

Why teams pick it

  • The full ClickHouse SQL dialect and function library from a pip install, with no server to run
  • Reads and writes 60+ formats, and queries S3, URLs, MySQL and PostgreSQL directly through ClickHouse table functions
  • SQL over pandas DataFrames and Arrow tables in place, or pandas-style code run on the engine through DataStore
  • The same SQL runs later on a ClickHouse server or ClickHouse Cloud, and on-disk databases are interchangeable with clickhouse-local
  • Apache 2.0, with a full-time team at ClickHouse, Inc.

Watch out for

  • The Python package supports Linux and macOS only; there are no native Windows wheels
  • One process can have only one active data path: switching to a different on-disk database means closing the first
  • Outside Python, C/C++, Node.js and Go, the bindings are marked early or experimental in the project’s own capability matrix
  • Python UDFs are scalar only, receive every argument as a string and are called once per row
  • One machine and one process, like DuckDB: it isn’t a shared server for many users

Alternatives

What all query engines have in common →
Amazon AthenaAWSAWS’s serverless SQL engine for data in S3: no cluster to run, and on-demand queries cost $5 per TB scanned.Apache Arrow AceroExperimentalOpen sourceThe streaming execution engine inside Arrow’s C++ library: you hand it an exact plan of nodes (scan, filter, project, aggregate, join, sort) and it streams Arrow record batches through them, with no SQL parser or optimizer of its own.Apache CalciteOpen sourceA Java framework with everything a database needs to understand a query (SQL parser, validator, relational algebra and a rule-based, cost-based optimizer) but no storage of its own, which Hive, Flink, Druid, Dremio, Beam and many other systems embed as their SQL front end and planner.Apache DataFusionOpen sourceA query engine library in Rust, built on Arrow, that other projects embed and extend to build their own databases and data tools.+1 Apache DataFusion CometApache HadoopOpen sourceThe project that made big-data clusters normal: a distributed file system (HDFS), a cluster scheduler (YARN) and the MapReduce engine.Apache HiveOpen sourceThe first SQL engine for Hadoop, and the source of the Hive Metastore, the table catalog that Trino, Spark, Impala and Flink still read.Apache ImpalaOpen sourceAn MPP SQL engine in C++ for interactive queries over Hive tables on HDFS, Kudu, Iceberg and object storage; built and mainly sold by Cloudera.Apache PigDormantOpen sourceA step-by-step scripting language, Pig Latin, that compiles to MapReduce, Tez or Spark jobs on Hadoop. Still released, but the project calls itself dormant.Apache SparkOpen sourceThe general-purpose distributed engine for large batch jobs, streaming and machine learning, in Python, SQL, Scala, Java or R.Apache ZeppelinOpen sourceA self-hosted web notebook where each paragraph runs through an interpreter: Spark, Flink, JDBC, Python and more. Maintained, but behind Jupyter and Databricks in reach.Cloudflare Basin SQLRenamedCommercialCloudflare’s serverless, distributed SQL engine for Apache Iceberg tables in Basin Catalog on R2, called R2 SQL until it became generally available on 1 October 2026, billed by the data it scans.DuckDBOpen sourceAn in-process analytical SQL database: a library with no server that queries Parquet, CSV, Iceberg and S3 directly.GizmoSQLOpen sourceA database server that wraps DuckDB (or SQLite) and serves it to many clients over Arrow Flight SQL, with TLS and authentication.GlareDBDormantOpen sourceA small, embeddable SQL engine in Rust for querying Parquet and CSV files locally, over HTTP or in S3 and GCS, from a CLI, Python or the browser. It began as a DataFusion-based federated query service and was rewritten in 2025 on its own engine, and it has had no release since June 2025.HotdataCommercialA managed SQL engine for AI agents, built in Rust on Apache DataFusion and Arrow: each agent gets short-lived “instant databases” to load data into, then runs read-only SQL, vector, full-text and geospatial queries over an HTTP API or CLI.IbisOpen sourceA Python dataframe API that compiles to SQL, so the same code runs on DuckDB on a laptop and on BigQuery, Snowflake, PostgreSQL or more than 20 other engines.PolarsOpen sourceA DataFrame library with a Rust query engine underneath: you write Python (or Rust, SQL, Node.js, R) and it plans, optimises and runs the query on every core, streaming data bigger than memory.Presto (PrestoDB)Open sourceFacebook’s distributed SQL engine, now governed by the Linux Foundation, moving its workers from Java to C++ on Velox. Trino is its 2019 fork.RayforceOpen sourceA small embeddable engine in pure C that runs columnar analytics, graph traversals and recursive Datalog queries in one optimised plan.Spice.ai OSSOpen sourceA single-binary SQL engine in Rust, built on DataFusion and Arrow, that federates queries across databases, warehouses and lakes, keeps hot datasets in a local accelerated copy, and adds search and LLM APIs on top.TrinoOpen sourceA distributed SQL engine that queries data where it lives, from Iceberg tables on S3 to a live PostgreSQL database, in one query.VeloxOpen sourceMeta’s open-source C++ execution engine library: it takes an already-optimized query plan and runs it with vectorized operators, and it is the engine inside Presto’s C++ workers (Prestissimo) and Apache Gluten’s native Spark.

How it fits with other technologies

SQL over Parquet on S3 and a pandas DataFrame, in one process · python

import chdb
import pandas as pd

# ClickHouse SQL straight over Parquet files, returned as a DataFrame
daily = chdb.query("""
    SELECT toDate(ts) AS day, count() AS views
    FROM s3('https://acme-raw.s3.amazonaws.com/clicks/2026-09-*.parquet', 'Parquet')
    GROUP BY day ORDER BY day
""", "DataFrame")

# Join it with an in-memory DataFrame through the Python() table function
targets = pd.DataFrame({"day": daily["day"], "goal": 50_000})
chdb.query("""
    SELECT d.day, d.views, t.goal, d.views >= t.goal AS hit
    FROM Python(daily) AS d JOIN Python(targets) AS t USING (day)
""", "Pretty").show()

# Keep tables between runs: a session on a directory
from chdb import session
sess = session.Session("./clicks_db")
sess.query("""
    CREATE TABLE IF NOT EXISTS clicks ENGINE = MergeTree ORDER BY ts AS
    SELECT * FROM s3('https://acme-raw.s3.amazonaws.com/clicks/2026-09-*.parquet', 'Parquet')
""")

chDB or DuckDB?#

Both run analytical SQL inside your process with no server. The difference is mostly where the engine came from and where you can take your queries next.

chDBDuckDB
EngineClickHouse, a server database, packaged as a libraryBuilt from scratch for embedding, at CWI from 2018
SQL dialectClickHouse SQLClose to PostgreSQL’s dialect
StorageIn memory by default; on-disk tables in the clickhouse-local formatIts own single-file database, or files in place
Formats and sources60+ ClickHouse formats; table functions for S3, URLs, MySQL, PostgreSQLParquet, CSV, JSON; extensions for S3, Iceberg, Delta Lake, PostgreSQL
DataFramesPython(df) table function; DataStore runs pandas code as SQLQueries pandas, Polars and Arrow objects by variable name
Next step upSame SQL on a ClickHouse server or ClickHouse CloudMotherDuck, or DuckDB’s own Quack server protocol
PlatformsLinux and macOS (Python)Linux, macOS and Windows
Licence and ownerApache 2.0; ClickHouse, Inc.MIT; DuckDB Foundation

See it in a real stack

No architecture uses chDB yet. Drop it into the playground and wire it up yourself.Open playground →

Benchmarks

Benchmark · ClickHouse Inc.ClickBenchHow fast analytical databases run 43 SQL queries over one 100-million-row table of web analytics events, on the same cloud machines.Includes chDB · 7 caveats to read first
How to read a benchmark →
Where it came from2023–2026

ClickHouse had long shipped clickhouse-local, a single binary that runs SQL over local and remote files without a server. But data scientists live in Python, and calling a separate binary through pipes meant starting a process for every query and copying every result. In February 2023, Auxten Wang wanted to read large annotation datasets from a Jupyter notebook while training computer-vision models, without waiting on slow Hive clusters. Over the Lunar New Year he built a prototype that linked ClickHouse’s 900,000 lines of C++ straight into a Python module.

Lorenzo Mangani of QXIP soon joined, adding Go, Node.js and Rust bindings, and the projects moved to a chdb-io GitHub organisation. ClickHouse, Inc. promoted chDB in its 23.7 release call and an August 2023 blog post. In March 2024 it announced that chDB was now part of ClickHouse and that Auxten would work on it full-time, with chDB tracking ClickHouse releases. chDB 4, announced in March 2026, split the engine into chdb-core and added DataStore, aimed at people who write pandas rather than SQL.

Why it’s built this way

Link the engine into the process instead of wrapping the binary
Running clickhouse-local through pipes added 10 to 50 ms of process start per query, copied every result and ruled out Python UDFs and SQL on DataFrames.
Hand results over in memory, not through stdout
Writing output into ClickHouse’s in-memory write buffer and exposing it through Python’s memoryview avoids copying results from C++ into Python objects.
Keep the clickhouse-local storage format
A chDB database directory can be opened by clickhouse-local and vice versa, so data moves between the two without export.

Timeline

  1. 2023

    Auxten Wang starts chDB in February; v0.1.0 in March, v1.0.0 in December

  2. 2024

    chDB becomes part of ClickHouse, Inc. in March

  3. 2025

    chDB 3.0 in January

  4. 2026

    chDB 4 adds the pandas-compatible DataStore; the engine becomes chdb-core

  5. 2026

    September: durable storage on object stores and a PostgreSQL extension

Sources16 · facts checked 6 October 2026

Improve this page on GitHub