Streaming Brokers: Kafka vs Redpanda vs Pulsar¶

Summary

A side-by-side comparison of the three log-structured event streaming platforms in this vault: Apache Kafka (Apache-2.0 reference implementation, KRaft-only since 4.0), Redpanda (Kafka-API broker in C++, BSL 1.1 core plus Enterprise features under the Redpanda Community License) and Apache Pulsar (Apache-2.0, stateless brokers over BookKeeper). Versions, licenses and features match the topic pages as of 2026-09-25. RabbitMQ Streams and NATS JetStream are lighter log options, covered briefly at the end.

Which One Should I Pick?¶

The flowchart gives a first cut. The sections below explain each branch.

flowchart TD
    Q1{"Need native multi-tenancy<br/>(tenants, namespaces) or built-in<br/>geo-replication without extra tools?"}
    Q2{"Team can run three tiers<br/>(brokers, bookies, metadata store)<br/>or buy a managed Pulsar?"}
    Q3{"Is a source-available license (BSL)<br/>acceptable, with Enterprise keys<br/>for tiered storage and DR?"}
    Q4{"Top priorities: single binary,<br/>no JVM, low tail latency?"}
    PULSAR["Apache Pulsar<br/>(4.0 LTS)"]
    KAFKA["Apache Kafka<br/>(4.3, KRaft)"]
    RP["Redpanda<br/>(26.2)"]
    KAFKA2["Apache Kafka<br/>(broadest ecosystem, free tiered storage)"]

    Q1 -->|"Yes"| Q2
    Q1 -->|"No"| Q3
    Q2 -->|"Yes"| PULSAR
    Q2 -->|"No"| KAFKA
    Q3 -->|"No"| KAFKA2
    Q3 -->|"Yes"| Q4
    Q4 -->|"Yes"| RP
    Q4 -->|"No"| KAFKA2

At a Glance¶

Dimension	Apache Kafka	Redpanda	Apache Pulsar
Latest version	4.3.1 (2026-06-25). 4.4.0 in release process	26.2.3 (2026-09-17)	4.2.4 (2026-08-03). LTS 4.0.13 (2026-08-03). 5.0.0-M2 preview
Language	Java (JVM), some Scala	C++ (Seastar)	Java (JVM, broker needs Java 21)
License	Apache-2.0	Core: BSL 1.1 (each release becomes Apache-2.0 after 4 years). Enterprise features: Redpanda Community License (RCL) plus license key	Apache-2.0
Architecture	Broker = compute + storage	Broker = compute + storage (single binary)	Stateless brokers + BookKeeper bookies (storage)
Consensus / replication	KRaft controller quorum (metadata, KRaft-only since 4.0) + ISR replication (data)	Raft per partition + Raft controller	BookKeeper quorum writes (ensemble E, write quorum Qw, ack quorum Qa). Metadata in ZooKeeper or Oxia (Oxia recommended from 5.0)
Multi-tenancy	ACLs + quotas (no native tenants)	ACLs + quotas. RBAC/GBAC are Enterprise	Tenants → namespaces → topics (native)
Geo-replication	MirrorMaker 2. Confluent Cluster Linking (commercial)	Shadowing, offset-preserving DR (Enterprise, 25.3+). MirrorMaker 2 also works	Built-in, per-namespace, asynchronous
Tiered storage	KIP-405: early access 3.6, production-ready 3.9. Needs a `RemoteStorageManager` plugin	Tiered Storage and Cloud Topics (GA 26.1). Enterprise-licensed	Native offloader (S3, GCS, Azure Blob, filesystem)
Schema registry	Separate: Confluent Schema Registry, Apicurio or Karapace	Built-in, Confluent-compatible API	Built-in, Pulsar-native (not Confluent-API compatible)
Stream processing	Kafka Streams, ksqlDB (Confluent Community License), Flink	Wasm data transforms in-broker (free, GA 24.1). Redpanda Connect pipelines. `rpk sql` (Oxla, 26.2)	Pulsar Functions (in broker or on workers)
Queue semantics	Share groups (KIP-932, production-ready 4.2)	Consumer groups (Kafka API)	Shared and Key_Shared subscriptions
Wire protocol	Kafka protocol	Kafka protocol (compatible)	Pulsar binary protocol. MoP (MQTT) and AoP (AMQP) plugins. KoP (Kafka) is archived
Operational complexity	Medium (KRaft removed ZooKeeper)	Low (single binary)	High (brokers + bookies + metadata store)
Managed offerings	Confluent Cloud (Confluent is an IBM subsidiary since 2026-03-17), Amazon MSK, Aiven, NetApp Instaclustr, Azure Event Hubs (Kafka endpoint)	Redpanda Cloud (BYOC, Dedicated, Serverless)	StreamNative Cloud, DataStax Astra Streaming (IBM), Tencent Cloud TDMQ for Pulsar

Redpanda licensing

The Redpanda Community Edition (BSL 1.1) is free for production use, except offering Redpanda to third parties as a streaming or queuing service. Tiered Storage, Remote Read Replicas, Cloud Topics, Iceberg Topics, Shadowing, RBAC and OIDC need an Enterprise license key. Apache Kafka's tiered storage (KIP-405) has no license restriction, although you must supply a storage plugin. See Redpanda: Licensing & Pricing.

When to Choose¶

Choose Kafka when¶

You need the broadest ecosystem (Connect, Streams, Flink, Debezium, hundreds of connectors, many managed services).
Your team already has Kafka operational expertise.
You want Apache-2.0 licensing with no restrictions, including free tiered storage.
You want queue-style consumption on the same platform (share groups, 4.2+).
You need Confluent's commercial features (Cluster Linking, RBAC, audit logs).

Choose Redpanda when¶

You want Kafka API compatibility with lower operational overhead.
Tail latency matters (fintech, gaming, ad tech).
You want a single binary with no JVM tuning.
The BSL terms fit your use case and you accept an Enterprise subscription for tiered storage, Cloud Topics, Iceberg Topics or Shadowing.
You want in-broker Wasm transforms or broker-native Iceberg Topics.

Choose Pulsar when¶

You need native multi-tenancy (tenants and namespaces with quotas and policies).
You need built-in geo-replication without external tooling.
Independent compute and storage scaling is a hard requirement.
You want queueing and streaming on the same topic (four subscription types).
You have the ops capacity for a three-tier deployment, or use a managed service.

Performance Comparison¶

Benchmark caveat

All published benchmarks are vendor-influenced (Confluent and Redpanda have published competing results). The figures below are indicative ranges from the topic pages, not controlled measurements. Run OpenMessaging Benchmark on your own hardware before sizing.

Metric	Kafka	Redpanda	Pulsar
Throughput reference point	~605 MB/s peak on 3 × `i3en.2xlarge`, RF=3 (Confluent OMB, 2020, vendor benchmark)	1 GB/s sustained on 3 × `i3en.6xlarge` (Redpanda OMB, vendor benchmark; results degraded with 50 producers in a 2023 rerun)	No neutral or project-published figure recorded
p99 produce latency	Depends on GC, batching and `acks` (no neutral figure recorded)	Around 80 ms at p99.99 at 1 GB/s on 3 nodes (Redpanda vendor benchmark, workload-specific)	Depends on bookie journal disks and GC (no neutral figure recorded)
Cold read from tiered storage	Adds object-store latency (KIP-405)	Adds object-store latency	Adds object-store latency (offloader)
Scaling	Linear with brokers	Linear with brokers	Brokers and bookies scale independently

Details: Kafka Reference: Benchmark Results, Redpanda Reference: Performance Characteristics, Pulsar Explanation: Performance Characteristics.

Cost Comparison¶

Factor	Kafka	Redpanda	Pulsar
Compute	JVM heap + OS page cache	Per-core Seastar shards (whole cores, 2 GB RAM per core minimum)	Broker heap and direct memory + bookie memory
Storage	Local disk (hot) + object storage (tiered, free)	Local NVMe (hot) + object storage (tiered, Enterprise)	Bookie journal and ledger disks (hot) + object storage (offload)
License	None	Enterprise subscription for object-storage features and DR	None
Ops headcount (qualitative)	Medium	Low	High

Migration Paths¶

From → To	Difficulty	Notes
Kafka → Redpanda	Low–medium	Clients work unchanged (same wire protocol). Data and offsets move by replication (MirrorMaker 2 or Redpanda Connect), not a binary swap. Tiered data cannot be read across systems
Redpanda → Kafka	Low–medium	Same as above in reverse. Redpanda's tiered-storage bucket layout is not readable by KIP-405 plugins, so replicate long-retention topics
Kafka → Pulsar	Medium–high	KoP is archived, so rewrite clients to the Pulsar SDK or use a vendor's Kafka-compatible service
Pulsar → Kafka	Medium–high	Rewrite clients. `Key_Shared` and individual acks have no exact Kafka equivalent (share groups come closest)

Other Log-Structured Options¶

These are not full competitors to the three brokers above, but they cover replayable streams at smaller scale:

RabbitMQ Streams (RabbitMQ 4.3.6, MPL 2.0): append-only, replicated streams and super streams with offset-based replay and retention by age or size. No tiered storage and no Kafka-scale ecosystem. A good fit when you already run RabbitMQ.
NATS JetStream (NATS 2.15.0, Apache-2.0): Raft-replicated streams built into the NATS server, with at-least-once delivery and exactly-once through Nats-Msg-Id dedup plus double ack. Suited to edge, multi-tenant and request-reply platforms rather than Kafka-style analytics pipelines.

See the Messaging Patterns Comparison for how all five systems map to log, queue and pub/sub patterns.