Skip to content

Redpanda

Summary

Redpanda (self-managed product now branded Redpanda Streaming) is a Kafka-API-compatible streaming platform written in C++ on the Seastar thread-per-core framework. It ships as a single binary with no JVM and no ZooKeeper or KRaft quorum, and uses Raft for every partition. The core is source-available under BSL 1.1. Tiered Storage, Cloud Topics, Iceberg Topics, Shadowing (DR), RBAC and OIDC are Enterprise-licensed. The current feature release is 26.2 (latest patch v26.2.3, 2026-09-17).

Overview

Redpanda is a drop-in Kafka replacement that re-implements the Kafka wire protocol in C++ on the Seastar reactor framework. Each topic partition is its own Raft group, and the controller that holds cluster metadata is also a Raft group, so no separate ZooKeeper or KRaft quorum is needed. Operationally it is one binary plus a node config file, with cluster settings managed through rpk. Since 2024 the company has built a broader platform around the broker: Redpanda Connect (the former Benthos), Iceberg Topics, Cloud Topics (object-storage-first topics), Shadowing for cross-region DR, the Oxla SQL engine, and an Agentic Data Plane for AI agents.

Key Facts

Attribute Detail
Website redpanda.com
GitHub redpanda-data/redpanda
Latest Version 26.2.3 (2026-09-17). Supported lines: 26.2, 26.1 (v26.1.18, 2026-09-24), 25.3 (v25.3.17, 2026-08-20)
Release cadence Three feature releases a year (YY.1/.2/.3), each supported about 12 months; frequent patches
Language C++ (Seastar); rpk and Redpanda Connect in Go
License Core: Redpanda BSL 1.1 (converts to Apache-2.0 four years after each release). Enterprise features: Redpanda Community License (RCL) plus a license key. 30-day built-in trial
Company Redpanda Data, Inc., founded by Alex Gallego. $100M Series D led by GV at a $1B valuation (April 2025)
Commercial Enterprise Edition (self-managed), Redpanda Cloud (BYOC, Dedicated, Serverless), Agentic Data Plane
Compatibility Kafka API (producer, consumer, AdminClient, transactions, ACLs), Confluent-compatible Schema Registry, Kafka Connect

Architecture at a Glance

The broker is a single process: Seastar shards own partitions, each partition is a Raft group, and object storage backs Tiered Storage, Cloud Topics and Iceberg Topics. See Explanation for the full diagram.

flowchart LR
    Clients["Kafka clients / Redpanda Connect"]
    subgraph Broker["Redpanda broker (x3+)"]
        Shards["Seastar shards<br/>(thread-per-core)"]
        Raft["Partition Raft groups<br/>+ controller Raft"]
        Extras["Schema Registry / HTTP Proxy<br/>Wasm transforms"]
    end
    Bucket["Object storage<br/>(S3 / GCS / Azure)"]
    Lake["Iceberg catalog<br/>(Glue, Unity, REST)"]
    Clients -->|"Kafka API :9092"| Shards
    Shards --> Raft
    Shards --> Extras
    Raft -->|"Tiered Storage / Cloud Topics"| Bucket
    Raft -->|"Iceberg Topics"| Lake

Evaluation

Pros Cons
Single binary. No JVM or ZooKeeper Source-available (BSL 1.1), not OSI open source. No offering as a "Streaming or Queuing Service"
Thread-per-core (Seastar) gives low tail latency Tiered Storage and all object-storage features are Enterprise-licensed, while Kafka's KIP-405 is free
Raft per partition: one consensus model, acks=all means fsync on a majority Smaller community and ecosystem than Apache Kafka
Cloud Topics (26.1) cut cross-AZ replication cost for latency-tolerant topics Some Kafka tooling expects JMX. Redpanda exposes Prometheus metrics instead
Iceberg Topics write lakehouse tables directly from the broker Vendor-controlled benchmarks. Verify on your own hardware
Wasm data transforms in-broker (free, GA since 24.1) New KIPs can land later than in Apache Kafka
Built-in Shadowing DR with identical offsets (Enterprise) Different ops mental model. Debugging Seastar needs familiarity

Use Cases

  • Kafka workload modernization: drop into existing Kafka pipelines for lower latency and TCO.
  • Low-latency fintech, gaming, ad tech: tail-latency-sensitive workloads.
  • Edge and single-node: runs on small ARM64 or x86_64 nodes without JVM overhead.
  • Streaming into the lakehouse: Iceberg Topics expose topics as Iceberg tables for Trino, Spark, Snowflake or Databricks.
  • Cost-optimized high-volume streams: Cloud Topics for logs, observability and ML feature feeds.
  • In-broker stream processing (Wasm): small per-record transforms without a separate Flink cluster.
  • Data integration and CDC: Redpanda Connect pipelines (Postgres, MySQL, MongoDB, SQL Server CDC and hundreds of connectors).

Licensing & Pricing

  • Community Edition (Redpanda BSL 1.1): free for production, including commercial use, except offering Redpanda to third parties as a streaming or queuing service. Each release becomes Apache-2.0 four years after its release date. It includes the Kafka API, Schema Registry, HTTP Proxy, SCRAM/mTLS, ACLs, Wasm transforms and rpk.
  • Enterprise Edition (RCL plus a license key): Tiered Storage, Remote Read Replicas, Cloud Topics, Iceberg Topics, Shadowing, Continuous Data Balancing, audit logging, RBAC/GBAC, OIDC and Kerberos, schema ID validation, Leader Pinning, FIPS mode, topic-deletion protection. The full trigger list is in Reference.
  • Redpanda Cloud: BYOC (data plane in your cloud account; BYOVPC GA in 26.1), Dedicated (single-tenant, Redpanda-operated), and Serverless (multi-tenant, pay-as-you-go; GA on AWS, GCP beta from 25.3). Pricing is quote- or consumption-based. No public list prices for self-managed Enterprise.
  • Redpanda Connect: Community and Certified connectors Apache-2.0, Enterprise connectors licensed.

Ecosystem

  • rpk: single CLI for cluster admin, topics, security, Schema Registry, transforms, Connect, Shadowing, SQL, benchmarks and debugging. See the command map.
  • Redpanda Console: web UI for topics, consumer groups, schemas, ACLs and Connect (redpanda-data/console, v3.12.0, 2026-09).
  • Redpanda Connect: the former Benthos, acquired May 2024. Declarative YAML pipelines, v4.111.0 (2026-09-24), with an mcp-server mode for AI agents.
  • Redpanda Operator and Helm chart: CRDs for clusters, topics, users, schemas, NodePools and Connect Pipelines (redpanda-data/redpanda-operator).
  • Wasm data transforms: Go (TinyGo), Rust, JavaScript and TypeScript SDKs.
  • Iceberg Topics: broker-native topic-to-Iceberg writer with REST catalog integrations.
  • Oxla SQL engine: acquired October 2025. Surfaced as rpk sql in 26.2.
  • Agentic Data Plane (ADP) and AI Gateway: Redpanda Cloud governance layer for AI agents, LLM and MCP traffic (announced October 2025, expanded 2026-02-18).

Compatibility & Requirements

Requirement Detail
Kafka API Producer, consumer, AdminClient, idempotence, transactions, ACLs
Kafka Connect Compatible. Runs as a separate process
Schema Registry Built-in, Confluent-compatible API (Avro, Protobuf, JSON Schema)
Object storage S3 (and S3-compatible), GCS, Azure Blob / ADLS Gen2
OS Linux (Debian/Ubuntu, RHEL/Fedora/Amazon Linux packages). macOS via Docker for dev
CPU x86_64 and ARM64. Give Redpanda whole cores
Storage Local NVMe for low latency. Object storage for the tiered or cloud tier
Memory 2 GB per core minimum
Kubernetes Operator chart requires Kubernetes 1.25 or later
Network TCP 9092 (Kafka), 9644 (Admin), 8081 (Schema Registry), 8082 (HTTP Proxy), 33145 (RPC). See ports

Latest Versions

  • 26.2 (2026-07-28): rpk sql for the Oxla SQL engine, stretch clusters with Operator dashboards, Operator Pipeline CRD for Redpanda Connect, Shadow Link role sync. Latest patch v26.2.3.
  • 26.1 (2026-03-31): "R1" adaptable engine, Cloud Topics GA, Group-Based Access Control, ranked Leader Pinning, BYOVPC GA, Operator NodePool.
  • 25.3 (2025-11-19): Shadowing DR, Cloud Topics beta, Serverless on GCP beta, SQL Server CDC connector.
  • 25.2 (2025-07-31) and 25.1 (2025-04-07): Iceberg Topics GA (25.1), then JSON Schema and Glue/Unity/Snowflake catalogs.
  • 24.x (2024): data transforms GA (24.1), rpk security command tree (24.1), Iceberg beta (24.3).

The full table with support end dates is in Reference. Track releases at GitHub releases and the release notes.

Alternatives

  • Apache Kafka: the reference implementation. Apache-2.0, broadest ecosystem, free tiered storage.
  • WarpStream (Confluent): S3-native Kafka API with stateless agents and no broker disks. Trades p99 latency for cost.
  • AutoMQ and Bufstream: other object-storage-backed Kafka-API implementations.
  • Apache Pulsar: segregated compute and storage, native multi-tenancy.
  • NATS: different protocol, lighter pub/sub. Not Kafka-compatible.
  • Side-by-side: Streaming Brokers Comparison and Messaging Patterns Comparison.

Migration & Lock-in

  • API-level lock-in: low. Kafka clients work unchanged. Moving data between Kafka and Redpanda needs replication (MirrorMaker 2 or Redpanda Connect), not just a binary swap.
  • Tooling lock-in: medium. rpk, Redpanda Console workflows, Wasm transforms and Shadowing have no Apache Kafka equivalent.
  • Tiered storage format is Redpanda-specific. Apache Kafka's KIP-405 plugins cannot read Redpanda's bucket layout. Plan a replication-based migration for long-retention topics.
  • Iceberg outputs are portable. They are standard Iceberg tables in your bucket and catalog.
  • License lock-in: object-storage features need a continuing Enterprise subscription for self-managed clusters.

Community Health

  • Vendor-driven project from Redpanda Data. Three feature releases a year, weekly Redpanda Connect releases.
  • External contributions accepted. A CLA is required (per CONTRIBUTING.md).
  • Community on Slack, GitHub issues, and Redpanda University.
  • Company signals: Series D (April 2025, $1B valuation), acquisitions of Benthos (2024) and Oxla (2025).

Topic Map

  • How-to Guides: install, cluster formation, Kubernetes, security setup, Tiered Storage, Iceberg, transforms, license, rolling upgrade, troubleshooting, Commands & Recipes.
  • Reference: release and support matrix, license matrix, ports, cluster and topic properties, rpk map, sizing, tuning, CVEs, hardening checklist.
  • Explanation: Seastar, Raft, controller, Tiered Storage, Cloud Topics, Iceberg, transforms, Shadowing, licensing model, security model.

Sources

Questions

  • For sustained 1M msg/s workloads, how does Redpanda's tail latency compare with Apache Kafka 4.x (KRaft plus tiered storage) on identical NVMe hardware?
  • What is the operational overhead of Wasm transforms at scale (CPU and memory per partition)? No independent figures found yet.
  • What produce latency do Cloud Topics deliver in practice on S3 vs S3 Express One Zone, and how does it compare with WarpStream and AutoMQ?
  • Does Continuous Data Balancing cause periodic latency spikes under sustained writes?
  • How does Iceberg schema evolution follow Schema Registry compatibility changes (field renames, type widening)?
  • When did Serverless reach GA on AWS? The exact date is not captured here.