Cloud Parts

ksqlDB

Confluent’s SQL server for Kafka: queries that run forever, turning topics into new topics and into tables you can look up.

What it is
Stream processing
Type
Commercial product
Made by
Confluent (an IBM company)
License
Confluent Community License (source-available, not OSI open source); Apache 2.0 until December 2018
Facts checked
5 October 2026
Website Source code

What it is

ksqlDB is a server, or a group of servers, that sits next to a Kafka cluster. You declare a STREAM or a TABLE over an existing topic and write SQL. A persistent query (CREATE STREAM AS SELECT or CREATE TABLE AS SELECT) is compiled into a Kafka Streams application that runs in the background on the servers, writing its results to a new topic and, for tables, keeping them in local state stores. Clients then ask in two ways: a push query (EMIT CHANGES) holds a connection open and streams every change, while a pull query returns the current value for a key and closes, like a database lookup. Servers with the same ksql.service.id share the work, and in interactive mode they agree on the statements to run through an internal command topic. ksqlDB can also run Kafka Connect connectors, embedded or on an existing Connect cluster.

The code is on GitHub under the Confluent Community License: you may use, change and redistribute it, but not offer it as a hosted service that competes with Confluent. The last standalone release, 0.29.0, came out in June 2023; ksqlDB now ships as part of Confluent Platform and as a managed service in Confluent Cloud, billed per Confluent Streaming Unit (CSU) per hour in clusters of 4 to 28 CSUs. For new work, Confluent Cloud’s docs now recommend its Flink service; ksqlDB “remains fully supported for existing applications”. Confluent has been part of IBM since March 2026.

Why teams pick it

  • Stream processing in SQL: filters, joins and windowed aggregations over Kafka topics without writing Java
  • Tables you can query: a pull query answers a key lookup from state kept up to date as events arrive
  • Push queries stream every change to a client over one long-lived HTTP connection
  • Runs on Kafka Streams, so it inherits its local state stores, changelog backups and exactly-once mode
  • Can create and manage Kafka Connect connectors, such as a JDBC source, from the same SQL

Watch out for

  • Not open source: the Confluent Community License forbids running it as a competing hosted service
  • Winding down: no standalone release since 0.29.0 (June 2023), and Confluent points new Confluent Cloud work to Flink
  • Pull queries are key lookups by default; scans need a setting, and the docs warn of deadlocks under many concurrent pull queries
  • Confluent Cloud limits: no user-defined functions, 40 persistent queries per cluster, no Enterprise clusters, and resizing goes through support
  • Kafka only: every source and result is a Kafka topic, and each persistent query adds internal topics and local state

Alternatives

What all stream processing have in common →
Apache FlinkOpen sourceThe most widely used open-source engine for stateful stream processing: keyed state, event time and exactly-once recovery from checkpoints.ArroyoOpen sourceA stream processing engine in Rust: you write the pipeline as SQL, and it keeps windows and joins running with checkpointed state.BytewaxOpen sourceA Python stream processing framework on a Rust engine built on Timely Dataflow; its company wound down in 2025 and the last release is from November 2024.Cloudflare Basin PipelinesRenamedCommercialCloudflare’s managed streaming ingestion service, called Cloudflare Pipelines until October 2026, that takes events over HTTP or a Workers binding, transforms them with SQL and writes them to R2 as Apache Iceberg tables, Parquet or JSON.DenormalizedArchivedOpen sourceAn embeddable stream processing library in Rust, with Python bindings, built on Apache DataFusion. Abandoned since December 2024 at version 0.0.13.FelderaOpen sourceAn incremental view maintenance engine in Rust: you write SQL tables and views, and it keeps every view up to date by processing only the changes, based on the award-winning DBSP theory.Kafka StreamsOpen sourceA Java library, part of Apache Kafka, that turns your own application into a stateful stream processor with no cluster of its own.PathwayCommercialA Python framework for streaming ETL and live RAG pipelines: you write table transformations in Python, and a Rust engine built on Differential Dataflow keeps the results up to date.Quix StreamsOpen sourceA pure-Python library for stateful stream processing on Kafka: a pandas-like Streaming DataFrame, RocksDB state backed up to Kafka, and no cluster of its own.Redpanda ConnectRenamedOpen sourceFormerly Benthos: one Go binary that runs a YAML pipeline of inputs, processors and outputs, with at-least-once delivery and no disk state of its own.RisingWaveOpen sourceA streaming database in Rust that speaks the PostgreSQL protocol: you write materialized views in SQL, and it keeps them up to date as events arrive, with all state in object storage.StreamFusionOpen sourceAn open-source accelerator for Apache Flink SQL: Flink still plans and coordinates the job, while the operators it can reproduce exactly run natively in Rust on Arrow and DataFusion.StreamlingCommercialGoldsky’s young Rust pipeline runtime: YAML pipelines of stateless SQL, TypeScript and plugin steps on Arrow, from Kafka or files into Postgres, ClickHouse or webhooks.Timely DataflowOpen sourceA Rust library for data-parallel dataflow with logical timestamps, loops and exact progress tracking; the engine underneath Materialize.

How it fits with other technologies

A stream, a table kept up to date by a persistent query, then pull and push queries · sql

-- register a stream over a topic (created with 6 partitions if missing)
CREATE STREAM pageviews (user_id VARCHAR KEY, page VARCHAR)
  WITH (KAFKA_TOPIC='pageviews', VALUE_FORMAT='JSON', PARTITIONS=6);

INSERT INTO pageviews (user_id, page) VALUES ('u-42', '/pricing');

-- persistent query: runs on the servers and keeps this table current
CREATE TABLE views_per_user AS
  SELECT user_id, COUNT(*) AS views
  FROM pageviews
  GROUP BY user_id;

-- pull query: the value right now, then the query ends
SELECT views FROM views_per_user WHERE user_id = 'u-42';

-- push query: stay connected and receive every change
SELECT user_id, views FROM views_per_user EMIT CHANGES;

-- views per user per one-minute window
CREATE TABLE views_per_minute AS
  SELECT user_id, COUNT(*) AS views
  FROM pageviews
  WINDOW TUMBLING (SIZE 1 MINUTE)
  GROUP BY user_id;

Ways to run it in 2026#

The same engine comes in three forms, and only two of them still get releases.

Standalone ksqlDB

frozen at 0.29.0

The confluentinc/ksqldb-server image and the docs at docs.ksqldb.io. The last release was 0.29.0 in June 2023. The GitHub repository still gets fixes and dependency updates, but on Confluent Platform branches.

Confluent Platform

self-managed

Ships as cp-ksqldb-server, versioned with the rest of the platform (7.x and 8.x lines in 2026). The code stays under the Confluent Community License; longer support comes with a paid Confluent Platform subscription.

Confluent Cloud

per CSU-hour

Managed clusters of 4, 8, 12, 16, 20, 24 or 28 CSUs; 8 CSUs or more get high availability. Prices vary by cloud and region and are shown in the console. Supported for existing apps, with Flink recommended for new ones.

See it in a real stack

No architecture uses ksqlDB yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2017–2026

Kafka Streams made stream processing a library, but you still had to write Java. In August 2017 Confluent co-founder Neha Narkhede announced KSQL, “a streaming SQL engine for Apache Kafka”, as a developer preview. A query such as counting errors per one-minute tumbling window became a CREATE TABLE ... AS SELECT statement, which KSQL turned into a Kafka Streams application that ran indefinitely.

KSQL started under Apache 2.0. In December 2018, Jay Kreps moved it and other Confluent components to the new Confluent Community License: the code could still be downloaded, changed and redistributed, but “you cannot create a KSQL-as-a-service offering”, a rule aimed at cloud providers. In November 2019 a release “so significant that we’re giving it a new name” became ksqlDB. It added pull queries, which read the current value from a materialized table, and connector management, so one system could ingest, process and serve data.

Confluent then bet on Apache Flink. It agreed to buy Immerok, a managed Flink start-up, in January 2023, and made Flink generally available in Confluent Cloud in March 2024. ksqlDB’s last standalone release, 0.29.0, came out in June 2023. IBM completed its acquisition of Confluent on 17 March 2026.

Why it’s built this way

SQL compiled to Kafka Streams, not a new engine
KSQL reused Kafka Streams’ partitioning, local state and fault tolerance, so a SQL statement became an ordinary Streams application and Confluent didn’t have to build a second runtime.
Streams and tables as SQL objects
They map directly onto Kafka Streams’ KStream and KTable: a stream is a sequence of facts, a table the latest value per key.
Push and pull queries
Applications need both a live feed of changes and request-and-response lookups. Serving lookups from the state stores saves copying results into a separate database.
A source-available licence
Confluent wanted to keep cloud providers from selling KSQL as a service while leaving the code free for everyone else to use, change and embed.

Timeline

  1. 2017

    KSQL announced as a developer preview in August

  2. 2018

    Moves from Apache 2.0 to the Confluent Community License in December

  3. 2019

    Renamed ksqlDB, adding pull queries and connector management

  4. 2023

    Confluent agrees to buy Immerok for Flink; last standalone release, 0.29.0

  5. 2024

    Flink becomes generally available in Confluent Cloud

  6. 2026

    IBM completes its acquisition of Confluent in March

Sources17 · facts checked 5 October 2026

Improve this page on GitHub