Cloud Parts

Redpanda

A Kafka-compatible streaming broker rewritten in C++ as one binary, with no JVM and no ZooKeeper.

What it is
Event streams
Used for
Pub/sub
Type
Commercial product
Made by
Redpanda Data
License
BSL 1.1 (each release becomes Apache 2.0 after four years); enterprise features under the Redpanda Community License
Facts checked
5 October 2026
Website Source code

What it is

Redpanda is a single C++ binary that speaks Kafka’s wire protocol, so Kafka clients from version 0.11 on connect to it unchanged. It is built on the Seastar framework: each thread is pinned to one CPU core, and threads pass messages to each other instead of sharing memory behind locks. Every partition is its own Raft group, with one leader and its followers; a write counts as committed once a majority of replicas have it in their log. Cluster metadata, such as topics and users, lives in a controller partition that all brokers replicate with Raft, so there is no ZooKeeper or separate controller to run. The same binary also serves a Schema Registry and an HTTP proxy. For production Redpanda asks for at least two physical cores, 2 GB of memory per core, and local NVMe disks with XFS or ext4.

The Community Edition is free under the Business Source License 1.1: you may run it for anything except selling Redpanda itself as a streaming or queuing service, and each release turns into Apache 2.0 four years after it ships. Several features need a paid enterprise licence (new clusters get a 30-day trial): Tiered Storage, which moves older log segments to S3, GCS or Azure storage; Iceberg topics, which also write a topic’s records as an Apache Iceberg table in Parquet; Cloud Topics (since v26.1), which skip local disks and write straight to object storage at 1 to 2 seconds of end-to-end latency; plus role-based access control, audit logs and continuous data balancing. Redpanda Cloud runs it for you as Serverless (multi-tenant, billed on uptime, partitions, data in and out, and storage), Dedicated, or BYOC in your own cloud account.

Why teams pick it

  • Kafka clients and tools work against it, and Redpanda tests the Java client, librdkafka, franz-go, Sarama, KafkaJS and others
  • One binary with Raft built in: no JVM, ZooKeeper, KRaft controllers or separate Schema Registry to deploy
  • The thread-per-core design is built to get high throughput and low tail latency out of fewer machines
  • Per-topic storage choices in one cluster: local disks, tiered, Iceberg tables, or diskless Cloud Topics for cheap, latency-tolerant data
  • BYOC clusters run in your own AWS, Azure or Google Cloud account, managed by Redpanda

Watch out for

  • Source-available, not open source: the BSL forbids offering Redpanda as a commercial streaming service, and code only becomes Apache 2.0 four years after release
  • Tiered Storage, Iceberg topics, Cloud Topics, RBAC, OIDC login and continuous balancing need an enterprise licence; once the 30-day trial ends they can’t be turned on without one
  • A few Kafka gaps: no request-rate quotas, one SCRAM mechanism per user, and no server-side KIP-890, so Kafka 4.x clients fall back to the older transaction protocol
  • Expects tuned hardware: at least two physical cores and 2 GB of memory per core, local NVMe disks with XFS or ext4, and 10 GigE networking
  • Enterprise and BYOC prices are quote-only on an annual commitment, and Serverless rates appear only in a price estimator

Alternatives

What all event streams have in common →
Amazon Kinesis Data StreamsAWSAWS’s serverless event stream, split into shards that each keep records in order.Apache FlussOpen sourceStreaming storage built for Flink: event logs and updatable tables in columnar Arrow format, with key lookups, and older data copied into Paimon, Iceberg, Hudi or Lance tables.Apache KafkaOpen sourceThe open-source distributed log that most event streaming is built on, or built to be compatible with.Apache PulsarOpen sourceA multi-tenant messaging and streaming platform from Yahoo that keeps brokers stateless and stores messages in Apache BookKeeper, so one topic can serve as a log, a pub-sub feed or a work queue.Cloudflare K2BetaCommercialCloudflare’s serverless event stream: a durable log stored in R2 that Workers and HTTP clients append to, read through independent subscriptions that lease out batches of records.WarpStreamCommercialA Kafka-compatible stream that keeps no data on disks: stateless Agents write straight to your object storage bucket.

How it fits with other technologies

Start a dev cluster with rpk, produce keyed records and read them as a group · bash

# three brokers in Docker, plus an rpk profile pointing at them
rpk container start -n 3

rpk topic create orders -p 6 -r 3
echo '{"total": 99}' | rpk topic produce orders -k customer-42

rpk topic consume orders -g billing -o start
rpk group describe billing

# with object storage and an enterprise licence configured:
rpk topic create clicks -c redpanda.storage.mode=tiered   # old segments go to object storage
rpk topic create logs -c redpanda.storage.mode=cloud      # Cloud Topic: no local disk

Apache Kafka, Redpanda and WarpStream#

All three speak the Kafka protocol, so the same clients connect to each. They differ in what a broker is, where the records sit, and what you trade for cost.

Apache KafkaRedpandaWarpStream
What you runJava brokers (Java 17+)One C++ binary per broker, one thread per coreStateless Go Agents; WarpStream hosts the control plane
Cluster metadataKRaft controllers, a Raft log; no ZooKeeper since 4.0A controller partition replicated by Raft across all brokersWarpStream’s cloud keeps which file holds which offsets
Where records liveBroker disks, copied to follower brokers; tiered storage optionalBroker disks, one Raft group per partition; Tiered Storage and Cloud Topics are enterprise optionsOnly in your object storage bucket; Agents have no disks
LatencySet by disks and broker-to-broker replication; no object store in the write pathSame model as Kafka, with Redpanda claiming up to 10x lower p99; Cloud Topics: 1 to 2 s end to endDefault p99: 500 ms to produce, 900 ms end to end; under 50 ms to produce with S3 Express and lightning topics
Cross-zone trafficReplication crosses zones; follower fetching can keep reads localSame; follower fetching and leader pinning reduce it, Cloud Topics replicate through object storageNone for produce or fetch: clients use Agents in their own zone
Adding capacityAdd brokers, then move partitions with the reassignment toolAdd brokers; continuous data balancing needs an enterprise licenceAdd or remove Agents; no data moves
LicenceApache 2.0BSL 1.1, plus a paid licence for enterprise featuresProprietary
Managed by someone elseAmazon MSK, Confluent Cloud, Azure Event Hubs (protocol) and othersRedpanda Cloud: Serverless, Dedicated, BYOCWarpStream BYOC (Agents in your account) or Serverless
  • Latency figures are each vendor’s own published numbers, checked 2026-10-05.

Ways to run Redpanda#

Community Edition

self-managed, free

The BSL-licensed binary on your own machines or Kubernetes. Everything except the enterprise features; you handle upgrades, disks and capacity.

Enterprise Edition

self-managed, licence key

The same binary with a licence key that unlocks Tiered Storage, Iceberg topics, Cloud Topics, RBAC, audit logs and continuous balancing.

Serverless

Redpanda’s cloud

Multi-tenant on AWS (GCP in beta), up to 100 MB/s of writes and 5,000 partitions, 99.9% SLA. Pay as you go on uptime, partitions, data in and out, and storage; new accounts get $100 of credit for 30 days.

Dedicated

Redpanda’s cloud

Single-tenant clusters on AWS, Azure or Google Cloud across several zones, up to 400 MB/s of writes, 99.99% SLA.

BYOC

your cloud account

Redpanda runs the cluster inside your AWS, Azure or Google Cloud account, up to 2 GB/s of writes, so data stays in your network. Sold on an annual commitment.

See it in a real stack

No architecture uses Redpanda yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2019–2026

By 2019 Kafka was the standard log for event streams, but running it meant a JVM cluster plus ZooKeeper. Alexander Gallego, who had earlier built the stream processor concord.io, wrote down a wish list for something better: “As easy to run as nginx. No dependencies. Ability to flush to disk with acks=-1. Leverage a huge and active ecosystem. It must be fast, really fast.” He wrote the first line of code on 7 January 2019. Kafka, he later argued, was designed when storage was expensive, and hardware had changed since.

His company, Vectorized, released Redpanda in November 2020 as source-available under the BSL. The goal was “to deter cloud providers from offering our work as a service”, while letting almost everyone else use it freely. Because it spoke Kafka’s protocol, existing clients and tools worked on day one. In 2022 the company renamed itself Redpanda Data and launched a managed cloud.

Redpanda has since grown from a broker into a platform: tiered storage, a connector framework through its 2024 purchase of Benthos (now Redpanda Connect), topics that double as Iceberg tables, and in 2026 diskless Cloud Topics.

Why it’s built this way

Kafka’s wire protocol instead of a new API
Gallego wanted to “leverage a huge and active ecosystem”. Speaking Kafka’s protocol meant every existing client, connector and tool worked without changes.
C++ on Seastar, one thread per core
Pinning each thread to a core and passing messages between threads avoids context switches and expensive locks. Gallego’s view was that Kafka had been designed for slower, costlier storage than today’s.
Raft everywhere, including metadata
One consensus algorithm for every partition and for the controller removed the need for ZooKeeper and let Redpanda ship as a single binary with no dependencies.
Source-available under the BSL
The licence lets anyone run it except as a competing hosted service, and each release becomes Apache 2.0 after four years, a model Redpanda credits MariaDB and CockroachDB with pioneering.

Timeline

  1. 2019

    Alexander Gallego writes the first line of code in January

  2. 2020

    Released as source-available under the BSL in November

  3. 2021

    Tiered Storage arrives in December

  4. 2022

    Vectorized becomes Redpanda Data; Redpanda Cloud launches in November

  5. 2024

    Buys Benthos and relaunches it as Redpanda Connect

  6. 2025

    Iceberg topics become generally available in 25.1

  7. 2026

    Version 26.1 adds Cloud Topics, written straight to object storage

Sources22 · facts checked 5 October 2026

Improve this page on GitHub