Messaging

Summary

Event streaming, message queues and pub/sub systems for asynchronous, decoupled and real-time architectures. The domain covers brokered systems that decouple producers and consumers, durable event streams that retain history for replay and analytics, and lightweight pub/sub layers inside distributed systems. It spans log-structured event stores (Kafka, Redpanda, Pulsar), a routing-rich queue broker (RabbitMQ) and a low-overhead pub/sub fabric with built-in persistence (NATS). Versions and licenses below match the topic pages as of 2026-09-25.

Domain Map

The map groups the five systems by native model. Redpanda implements the Kafka API. Pulsar, RabbitMQ Streams and NATS JetStream are separate protocols that also offer replayable logs.

flowchart LR
    subgraph LOG["Log-structured streaming"]
        KAFKA["Apache Kafka 4.3<br/>(KRaft, ISR)"]
        RP["Redpanda 26.2<br/>(Raft per partition)"]
        PULSAR["Apache Pulsar 4.0 LTS / 4.2<br/>(brokers + BookKeeper)"]
    end
    subgraph QUEUE["Queue broker"]
        RMQ["RabbitMQ 4.3<br/>(quorum queues, streams)"]
    end
    subgraph PUBSUB["Pub/sub fabric"]
        NATS["NATS 2.15<br/>(Core NATS + JetStream)"]
    end

    RP -->|"implements Kafka API"| KAFKA
    PULSAR -.->|"alternative architecture<br/>(KoP archived)"| KAFKA
    RMQ -.->|"Streams: replayable log"| LOG
    NATS -.->|"JetStream: replayable streams"| LOG
    NATS -.->|"lighter alternative for<br/>work queues and RPC"| RMQ

Topics

Topic	Style	Highlight	Latest version (2026-09-25)	License
Apache Kafka	Distributed log	The de facto standard event streaming platform. KRaft-only since 4.0. Share groups add queues (4.2)	4.3.1 (2026-06-25)	Apache-2.0
NATS	Pub/sub + JetStream	Single Go binary: Core NATS pub/sub and request-reply, JetStream streams, KV and Object Store, accounts for multi-tenancy, leaf nodes for edge	2.15.0 (2026-09-17)	Apache-2.0 (CNCF Incubating)
RabbitMQ	Multi-protocol broker	AMQP 0-9-1/1.0, MQTT and STOMP. Rich routing, quorum queues, streams, Khepri (Raft) metadata only since 4.3	4.3.6 (2026-09-16)	MPL 2.0
Redpanda	Kafka-compatible	C++ Kafka-API broker: single binary, no JVM, no ZooKeeper or KRaft, Raft per partition	26.2.3 (2026-09-17)	BSL 1.1 core. Enterprise features under the Redpanda Community License
Apache Pulsar	Tiered streaming	Stateless brokers over BookKeeper, native multi-tenancy, built-in geo-replication	4.2.4 (2026-08-03). LTS 4.0.13. 5.0.0-M2 preview	Apache-2.0

Comparisons

Comparison	Scope
Streaming Brokers Comparison	Kafka vs Redpanda vs Pulsar: licenses, consensus, tiered storage, geo-replication, migration, with a decision flowchart. Brief notes on RabbitMQ Streams and NATS JetStream
Messaging Patterns Comparison	Log-stream vs queue vs pub/sub across all five systems, with delivery guarantees

When to Use Which

Need	Reach For
Replayable event log, largest ecosystem, no license restrictions	Kafka
Kafka API with a single binary and low tail latency, BSL acceptable	Redpanda
Native multi-tenancy and built-in geo-replication	Pulsar
Lightweight pub/sub and request-reply, edge and IoT	NATS (Core)
Persistent streams, KV or object store without Kafka's footprint	NATS JetStream
Traditional work queues, complex routing, RPC	RabbitMQ
Per-message TTL, priorities, dead-lettering	RabbitMQ
Queue semantics on an existing Kafka platform	Kafka share groups (4.2+)

Decision Framework

When choosing a messaging system, evaluate along these axes:

1. Delivery guarantee. At-most-once, at-least-once or exactly-once? Kafka (and Redpanda through the Kafka API) and Pulsar offer transactional exactly-once inside the broker. NATS JetStream gives exactly-once through Nats-Msg-Id publish dedup plus double ack, while Core NATS is at-most-once with no acks or storage. RabbitMQ gives at-least-once with publisher confirms and consumer acks. See delivery guarantees.
2. Retention model. Do consumers need to replay history? Log-based systems (Kafka, Redpanda, Pulsar) retain by time or size. RabbitMQ queues remove a message once it is acknowledged, but RabbitMQ Streams retain messages after they are read. Core NATS stores nothing, while JetStream streams retain by limits, interest or work-queue policy.
3. Operational complexity. Redpanda and NATS are single-binary deploys. Kafka needs a KRaft controller quorum (ZooKeeper was removed in 4.0). Pulsar needs brokers, BookKeeper bookies and a metadata store (ZooKeeper or Oxia).
4. Latency profile. The NATS docs' single-machine nats bench example averages 50.87 µs per Core NATS request-reply round trip (NATS Explanation). Redpanda's vendor benchmark reports p99.99 around 80 ms at 1 GB/s on three nodes, well below Kafka on the same test, but an independent 2023 rerun found Redpanda's results degraded sharply with 50 producers instead of 4 (Redpanda Reference). Kafka and Pulsar optimize for throughput, and their tail latency depends on GC, batching and disks.
5. Multi-tenancy. Pulsar has native tenants and namespaces. NATS has accounts with strict subject isolation. Kafka and Redpanda use ACLs and quotas.
6. Ecosystem maturity. Kafka has the largest connector ecosystem (Kafka Connect, Debezium, hundreds of connectors). Redpanda reuses it through Kafka API compatibility. Pulsar's Kafka protocol handler (KoP) is archived, so Pulsar needs its own clients and Pulsar IO connectors.
7. Licensing. Kafka, Pulsar and NATS are Apache-2.0 and RabbitMQ is MPL 2.0. Redpanda's core is BSL 1.1, and its tiered storage, Cloud Topics, Iceberg Topics and Shadowing need an Enterprise license.

Cross-Cutting Concerns

• Exactly-once semantics: Kafka idempotent producer + transactions, Pulsar transactions, NATS JetStream Nats-Msg-Id dedup + double ack, RabbitMQ publisher confirms + idempotent consumers.
• Schema management: Confluent Schema Registry, Apicurio and Karapace for Kafka. Redpanda's built-in Confluent-compatible registry. Pulsar's built-in (Pulsar-native) registry.
• Observability: OpenTelemetry messaging spans, broker metrics (JMX for Kafka, Prometheus endpoints for Redpanda, RabbitMQ and NATS), NATS message tracing (2.11+).
• Storage tiers: Kafka tiered storage (KIP-405, production-ready since 3.9, needs a plugin), Pulsar offload to object storage, Redpanda Tiered Storage and Cloud Topics (Enterprise). JetStream uses replicated local storage (R1/R3/R5). RabbitMQ Streams use local segment files with no tiered storage.
• Client libraries: All five have official or mature clients for Go, Java and Python. Kafka and NATS have the broadest polyglot coverage.
• Managed services: Confluent Cloud (IBM since 2026-03), Amazon MSK, Aiven, Azure Event Hubs (Kafka endpoint), Redpanda Cloud, StreamNative Cloud, Synadia Cloud, CloudAMQP and Amazon MQ for RabbitMQ.

Kubernetes Operators

System	Operator or chart	CRD-driven	Notes
Kafka	Strimzi	Yes	CNCF Incubating. KRaft-only, kafka.strimzi.io/v1 CRDs
Redpanda	Redpanda Operator and Helm chart	Yes	CRDs for clusters, topics, users, schemas, NodePools and Connect Pipelines. Kubernetes 1.25+
Pulsar	Apache pulsar-helm-chart (reference path). StreamNative operators for its distribution	Chart: no. StreamNative: yes	Chart can deploy Oxia. Kubernetes 1.25+
NATS	Helm charts + NACK (NATS Controllers for Kubernetes)	Yes	Stream, Consumer, KeyValue, ObjectStore and Account CRDs
RabbitMQ	RabbitMQ Cluster Operator + Messaging Topology Operator	Yes	Topology Operator manages queues, exchanges, bindings and users

Related

• PostgreSQL, MySQL and CockroachDB: CDC sources into Kafka (Debezium, CockroachDB changefeeds)
• Kubernetes: where the operators above run
• OpenTelemetry and LGTM stack: messaging traces, Kafka-based ingest in Tempo
• MinIO and Ceph: S3-compatible targets for tiered storage and offload

Sources

• Apache Kafka documentation and downloads
• NATS documentation and nats-server releases
• RabbitMQ documentation and release information
• Redpanda documentation and licensing overview
• Apache Pulsar documentation and release notes
• Confluent Platform documentation

Open Questions

• How does Redpanda's tail latency compare with Kafka 4.x (KRaft, tiered storage) at more than 1M msgs/sec sustained on identical NVMe hardware?
• For multi-region active-active, is Pulsar's geo-replication operationally simpler than Kafka MirrorMaker 2 or Confluent Cluster Linking, or Redpanda Shadowing?
• What is the practical ceiling for NATS JetStream streams and consumers per cluster, and how does it compare with Kafka partition limits?
• What are the cost implications of Pulsar offload versus Kafka KIP-405 tiered storage at petabyte scale?
• How do RabbitMQ Streams compare with NATS JetStream and Kafka for ordered, replayable workloads (throughput, consumer-group semantics)?
• What is the TCO of self-hosted Kafka on Strimzi versus Redpanda (including Enterprise licensing) on equivalent hardware?
• How does Oxia behave in production (failure modes, backup and restore, sizing) for Pulsar clusters migrated from ZooKeeper with PIP-454? Oxia is supported since 3.3 and recommended from 5.0, but production experience reports are still scarce.