Quix Streams

A pure-Python library for stateful stream processing on Kafka: a pandas-like Streaming DataFrame, RocksDB state backed up to Kafka, and no cluster of its own.

What it is: Stream processing
Type: Open source
Made by: Quix Analytics Ltd.
License: Apache 2.0 (since the first open-source release in March 2023); Quix Cloud, the hosted platform, is proprietary
Facts checked: 6 October 2026

What it is

Quix Streams is a library you import into your own Python program (3.11 or newer), not a server. An Application joins a Kafka consumer group, reads and deserializes messages, runs each one through a StreamingDataFrame (a declarative chain of steps written like pandas: apply, filters, column assignments, windows, joins, group_by) and produces the results to output topics or sinks. Kafka hands out the input partitions, so you scale by starting more copies of the same app. Stateful steps keep data per message key in a local RocksDB store, one per partition, and every change also goes to a compacted changelog topic in Kafka. When a partition moves to another copy, that copy replays the changelog before it processes anything. Every 5 seconds by default the app takes a checkpoint: it flushes sinks and the producer, commits the input offsets and writes state to disk. Delivery is at-least-once by default; processing_guarantee="exactly-once" wraps each checkpoint in a Kafka transaction. Windows use event time, taken from the Kafka message timestamp or from a field you extract, and by default time moves forward separately for each key.

The library is Apache 2.0 and works with any Kafka-compatible broker, self-hosted or managed. Quix Analytics, founded by engineers from McLaren’s Formula 1 data platform, sells Quix Cloud for running these apps: hosted or bring-your-own Kafka (Confluent Cloud, Redpanda, Aiven), deployments on Kubernetes, projects tied to Git branches, pipelines defined in YAML, an online editor, logs and metrics. There is no public price list. The docs offer a time-limited free account after a call, and by October 2026 Quix’s website sold the platform mainly for hardware test and engineering data. The library itself is released every few weeks.

Why teams pick it

Pure Python, with no JVM or language bridge, so pandas, NumPy and ML libraries run inside the processing steps and debugging stays in one language
Kafka Streams-style design without the JVM: local RocksDB state, changelog topics for recovery, consumer groups for scaling, nothing extra to operate
Exactly-once from Kafka to Kafka with one setting; Quix’s docs put the cost at about 10% slower than at-least-once at worst
Tumbling, hopping and sliding windows by time or by message count, session windows since 3.27, and as-of, interval and lookup joins
Ready-made sources and sinks for S3, Iceberg, PostgreSQL, BigQuery, Kinesis, Google Pub/Sub, MQTT, Redis and more, plus Avro, Protobuf and JSON with a schema registry

Watch out for

Kafka is required, not optional: input, state backups and group_by repartitioning all go through Kafka topics, and each stateful app adds changelog topics
Parallelism is capped by the input topic’s partition count, and a copy that takes over partitions must rebuild their RocksDB state from the changelog first
Exactly-once covers Kafka output only; other sinks are flushed at each checkpoint and written again after a failure, and exactly-once output only becomes readable when the checkpoint commits
With the default per-key closing strategy, a key that stops sending keeps its last window open forever; closing_strategy="partition" fixes that but can drop out-of-order messages across keys
Quix Cloud has no published prices, doesn’t support a schema registry, and comes from a company now focused on hardware engineering; version 3.27 raised the minimum to Python 3.11

Alternatives

What all stream processing have in common →

Apache FlinkOpen sourceThe most widely used open-source engine for stateful stream processing: keyed state, event time and exactly-once recovery from checkpoints.

ArroyoOpen sourceA stream processing engine in Rust: you write the pipeline as SQL, and it keeps windows and joins running with checkpointed state.

BytewaxOpen sourceA Python stream processing framework on a Rust engine built on Timely Dataflow; its company wound down in 2025 and the last release is from November 2024.

Cloudflare Basin PipelinesRenamedCommercialCloudflare’s managed streaming ingestion service, called Cloudflare Pipelines until October 2026, that takes events over HTTP or a Workers binding, transforms them with SQL and writes them to R2 as Apache Iceberg tables, Parquet or JSON.

DenormalizedArchivedOpen sourceAn embeddable stream processing library in Rust, with Python bindings, built on Apache DataFusion. Abandoned since December 2024 at version 0.0.13.

FelderaOpen sourceAn incremental view maintenance engine in Rust: you write SQL tables and views, and it keeps every view up to date by processing only the changes, based on the award-winning DBSP theory.

Kafka StreamsOpen sourceA Java library, part of Apache Kafka, that turns your own application into a stateful stream processor with no cluster of its own.

ksqlDBCommercialConfluent’s SQL server for Kafka: queries that run forever, turning topics into new topics and into tables you can look up.

PathwayCommercialA Python framework for streaming ETL and live RAG pipelines: you write table transformations in Python, and a Rust engine built on Differential Dataflow keeps the results up to date.

Redpanda ConnectRenamedOpen sourceFormerly Benthos: one Go binary that runs a YAML pipeline of inputs, processors and outputs, with at-least-once delivery and no disk state of its own.

RisingWaveOpen sourceA streaming database in Rust that speaks the PostgreSQL protocol: you write materialized views in SQL, and it keeps them up to date as events arrive, with all state in object storage.StreamFusionOpen sourceAn open-source accelerator for Apache Flink SQL: Flink still plans and coordinates the job, while the operators it can reproduce exactly run natively in Rust on Arrow and DataFusion.

StreamlingCommercialGoldsky’s young Rust pipeline runtime: YAML pipelines of stateless SQL, TypeScript and plugin steps on Arrow, from Kafka or files into Postgres, ClickHouse or webhooks.Timely DataflowOpen sourceA Rust library for data-parallel dataflow with logical timestamps, loops and exact progress tracking; the engine underneath Materialize.

How it fits with other technologies

What Quix Streams uses

What uses Quix Streams

Average temperature per sensor per minute, Kafka to Kafka, exactly once · python

from datetime import timedelta

from quixstreams import Application
from quixstreams.dataframe.windows import Count, Mean

app = Application(
    broker_address="kafka-1:9092",
    consumer_group="sensor-averages",  # also names the state dir and changelog topics
    auto_offset_reset="earliest",
    processing_guarantee="exactly-once",
)

# messages are keyed by sensor id; use the reading's own time, in ms, as event time
readings = app.topic(
    "sensor-readings",
    value_deserializer="json",
    timestamp_extractor=lambda value, headers, ts, ts_type: value["ts_ms"],
)
averages = app.topic("sensor-averages-1m", value_serializer="json")

sdf = app.dataframe(readings)
sdf = sdf[sdf["temp_c"] > -50]  # drop obviously broken readings
sdf = (
    sdf.tumbling_window(timedelta(minutes=1), grace_ms=timedelta(seconds=10))
    .agg(avg_c=Mean("temp_c"), readings=Count())
    .final()  # one result per sensor per minute, once the window closes
)
sdf.to_topic(averages)  # {"start": ..., "end": ..., "avg_c": ..., "readings": ...}

if __name__ == "__main__":
    app.run()  # start more copies to spread the partitions

The library or Quix Cloud#

The same application code runs in both. Checked 2026-10-06.

	Quix Streams (library)	Quix Cloud
What it is	A Python package: `pip install quixstreams`	A hosted platform for building, deploying and monitoring Quix Streams apps
Kafka	Any Kafka-compatible broker you run or rent	Quix-hosted Kafka, or bring Confluent Cloud, Redpanda, Aiven or your own
Where apps run	Anywhere Python runs: a VM, a container, your own Kubernetes	Quix-managed Kubernetes, scaled by replicas, CPU and memory; dedicated infrastructure available
Extras	None; bring your own CI/CD and monitoring	Online editor, Git-linked environments, YAML pipelines, logs, metrics, live data views
Licence and price	Apache 2.0, free	Proprietary; no public price list, time-limited free account after booking a session

Quix Cloud doesn’t offer or let you configure a schema registry; apps can still use one directly.

Other ways to do the same job#

Kafka Streams

JVM

The design Quix Streams follows: a library with local RocksDB state, changelog topics and consumer-group scaling, but for Java and Scala, shipped with Apache Kafka.

Bytewax

community-maintained

Python on a Rust engine (Timely Dataflow) with its own cluster mode and SQLite recovery. The company behind it stopped work in 2025; the last release is from November 2024.

PyFlink and PySpark

JVM underneath

Python APIs over Flink and Spark, full distributed engines with clusters of their own. Your Python code runs next to a JVM, which Quix argues makes deploying and debugging harder.

Arroyo

SQL

A Rust engine where the pipeline is SQL instead of Python, with checkpoints to object storage and its own cluster, now developed by Cloudflare.

See it in a real stack

No architecture uses Quix Streams yet. Drop it into the playground and wire it up yourself.Open playground →

Where it came from2020–2026

Quix’s founders built the data platform at McLaren, where race decisions depend on high-frequency sensor data processed in milliseconds. They started Quix to give other teams that kind of real-time pipeline, began work on the library on 2 March 2020, and raised a $12.9 million Series A led by MMC Ventures in 2022, betting on Python as the language for analytics and machine learning.

On the company’s third birthday in March 2023 they open-sourced Quix Streams under Apache 2.0. That first version was written in C#, with Python bindings generated from ahead-of-time compiled .NET code, and was built around time-series telemetry: timestamped rows, stream contexts and buffers. Python users had to debug across two languages.

In November 2023 the 2.0 alpha replaced it entirely: pure Python, no .NET, a new Streaming DataFrame API and RocksDB state, close in spirit to Kafka Streams. Version 3.0 in October 2024 added branching and reading several topics in one app, and 2026 releases added session windows and moved the minimum to Python 3.11. Meanwhile the company’s own pitch shifted towards hardware test data and AI agents for engineers.

Why it’s built this way

A library, not a cluster: Like Kafka Streams, it runs inside your own process and leans on Kafka for partition assignment and state backups, so there is no separate processing cluster or orchestrator to run.
Rewrite in pure Python: The first version wrapped a C# core. The 2.0 release notes say dropping .NET made the library more stable, safe and easier to maintain, and Python users no longer debug through generated bindings.
A DataFrame-like API: The target users are data scientists and engineers who know pandas. Writing a stream job as column operations on an endless table keeps them in a familiar style while the library tracks state.
State in local RocksDB, backed up to Kafka: Keeping state next to the code keeps each step fast, and changelog topics reuse Kafka’s replication for durability, so the only infrastructure needed is the broker you already have.

Timeline

2020
Quix starts work on the library on 2 March
2022
$12.9 million Series A led by MMC Ventures
2023
Open-sourced in March as a C# library with Python bindings
2023
2.0 alpha in November: pure Python, Streaming DataFrames, RocksDB state
2024
3.0 in October adds branching and multiple input topics
2026
3.27 in September adds session windows and requires Python 3.11

Sources21 · facts checked 6 October 2026

Improve this page on GitHub