Difference between Kafka and RabbitMQ

When applications need to exchange messages, Apache Kafka and RabbitMQ are two of the most popular choices. Both move data between systems, but they are built on very different ideas and shine in different situations.

This guide compares them side by side: how they store and route messages, how consumers receive them, ordering, scaling and reliability, with Spring Boot code for both, an e-commerce example and clear advice on when to choose each.

1. Kafka and RabbitMQ at a Glance

The simplest way to remember the difference:

• Kafka is like a newspaper archive. Every article is written into an ordered record and kept for a set time. Any number of readers can read it, at their own pace, and come back to old articles later.
• RabbitMQ is like a post office. Each letter is sorted and delivered to the right mailbox, picked up by a recipient, and then it is gone.

Kafka (distributed log)
  Producer --> [ 0 | 1 | 2 | 3 | 4 | 5 ]  topic partition (kept for days)
                    ^           ^
              Consumer A    Consumer B     (each tracks its own position)

RabbitMQ (message broker with queues)
  Producer --> [ Exchange ] --routing--> [ Queue ] --> Consumer
                                         message removed after acknowledgement

2. What Is Apache Kafka?

Apache Kafka is a distributed event streaming platform designed to handle very high volumes of data in real time. It stores messages in an ordered, append-only log, split into partitions spread across brokers. Consumers read at their own pace and can re-read old messages.

• Typical uses: real-time analytics, event-driven microservices, event sourcing, log and metrics collection, large data pipelines.
• Message storage: kept on disk for a configurable time, whether or not they have been read, so many consumers can read them independently.
• Performance: very high throughput, often hundreds of thousands to millions of messages per second on a cluster.

Learn more in Kafka Introduction and Kafka Architecture.

3. What Is RabbitMQ?

RabbitMQ is a traditional message broker built around queues. Producers send messages to an exchange, which routes them to one or more queues using flexible rules; consumers receive messages from those queues and acknowledge them.

• Typical uses: task and job queues, background processing, request–reply between services, workflows that need complex routing.
• Message storage: a message is normally removed from its queue once a consumer acknowledges it.
• Performance: good throughput, typically tens of thousands of messages per second per node, with very low latency and strong per-message delivery control.
• Protocols: AMQP (its main protocol), plus MQTT and STOMP, which suits IoT and many programming languages.

4. Key Differences

5. Log vs Queue: How Messages Are Stored

5.1 Kafka: a log

Kafka appends every message to the end of a partition log and assigns it an offset. Reading does not remove anything. Each consumer group remembers its own offset, so one group can be at offset 5,000 while another is replaying from offset 0. Data is removed only by retention rules, such as “keep 7 days”.

5.2 RabbitMQ: queues

RabbitMQ holds messages in queues until a consumer receives and acknowledges them, then deletes them. Queues are designed to stay short: messages flow through quickly. If many messages pile up, RabbitMQ is doing more work than it was designed for.

6. How Messages Are Routed

Kafka routing is simple: the producer chooses a topic, and the message key decides the partition. Consumers subscribe to topics. Any filtering happens in the consumer.

RabbitMQ routing is one of its strongest features. Producers send to an exchange, and bindings decide which queues receive each message:

7. Push vs Pull: How Consumers Receive Messages

8. Ordering and Scaling

8.1 Ordering

• Kafka guarantees order within a partition. Messages with the same key go to the same partition, so all events of one order stay in order, even with many consumers.
• RabbitMQ delivers a queue’s messages in order, but with several competing consumers they may be processed out of order, and redeliveries after failures can change the order. Strict ordering usually means one consumer per queue, or RabbitMQ’s single-active-consumer feature.

8.2 Scaling

• Kafka scales by adding partitions and brokers. Within a consumer group, the number of partitions limits how many consumers can work in parallel.
• RabbitMQ scales consumers easily: just add more consumers to a queue. Scaling a single very busy queue across nodes is harder, though quorum queues, streams and sharding options help.

9. Delivery Guarantees and Reliability

Both can be configured for at-least-once delivery, the most common choice, where a message may occasionally be delivered twice but is not lost.

See Kafka Fault Tolerance for Kafka’s replication in detail.

10. Kafka in Spring Boot: Code Example

With Spring for Apache Kafka (spring-kafka), sending and receiving events takes only a few lines:

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      acks: all
    consumer:
      group-id: analytics-service
      auto-offset-reset: earliest

package com.example.shop.kafka;

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;
import org.springframework.stereotype.Service;

@Service
class PageViewPublisher {

    private final KafkaTemplate<String, String> kafka;

    PageViewPublisher(KafkaTemplate<String, String> kafka) {
        this.kafka = kafka;
    }

    void pageViewed(String userId, String productId) {
        // Key = user ID: all events of one user stay in order in one partition
        kafka.send("page-views", userId, "{\"userId\":\"" + userId + "\",\"productId\":\"" + productId + "\"}");
    }
}

@Component
class AnalyticsListener {

    // Each consumer group receives every event independently
    @KafkaListener(topics = "page-views", groupId = "analytics-service")
    void onPageView(String event) {
        System.out.println("Updating dashboards with " + event);
    }
}

@Component
class RecommendationListener {

    @KafkaListener(topics = "page-views", groupId = "recommendation-service")
    void onPageView(String event) {
        System.out.println("Updating recommendations with " + event);
    }
}

Both listeners receive every page view, because they belong to different consumer groups. A new service added next month can even read last week’s events, as long as they are still within the retention period.

11. RabbitMQ in Spring Boot: Code Example

With Spring AMQP (spring-boot-starter-amqp), you declare the exchange, queue and binding, then send and receive:

spring:
  rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest
    listener:
      simple:
        acknowledge-mode: auto   # ack after the listener returns, nack on exception
        prefetch: 10             # at most 10 unacknowledged messages per consumer
        default-requeue-rejected: false   # failed messages go to the dead-letter exchange

package com.example.shop.rabbit;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.QueueBuilder;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Component;
import org.springframework.stereotype.Service;

@Configuration
class RabbitConfig {

    @Bean
    TopicExchange ordersExchange() {
        return new TopicExchange("orders");
    }

    @Bean
    Queue paymentQueue() {
        return QueueBuilder.durable("order-payments")
                .quorum()                                    // replicated queue
                .deadLetterExchange("orders.dlx")            // failed messages go here
                .build();
    }

    @Bean
    Binding paymentBinding(Queue paymentQueue, TopicExchange ordersExchange) {
        // Receive every "order.created.*" message, for any region
        return BindingBuilder.bind(paymentQueue).to(ordersExchange).with("order.created.*");
    }
}

@Service
class OrderPublisher {

    private final RabbitTemplate rabbit;

    OrderPublisher(RabbitTemplate rabbit) {
        this.rabbit = rabbit;
    }

    void orderCreated(String orderId, String region) {
        rabbit.convertAndSend("orders", "order.created." + region, orderId);
    }
}

@Component
class PaymentWorker {

    // Run several instances: each message is processed by exactly one of them
    @RabbitListener(queues = "order-payments")
    void process(String orderId) {
        System.out.println("Processing payment for order " + orderId);
    }
}

Each order goes to one payment worker, even when several are running, and is removed from the queue once processed. If the worker throws an exception, the message is rejected and, because default-requeue-rejected is false, moved to the dead-letter exchange instead of being retried forever. Add a retry policy if you want a few attempts first.

12. Example: An E-Commerce Application

Imagine an online shop that uses both tools for what each does best:

• Kafka: every user activity, such as page views, searches, clicks and purchases, is sent to Kafka. The recommendation engine, analytics dashboards and fraud detection all read these events independently, and a new data-science team can replay last week’s events to train a model.
• RabbitMQ: when a customer places an order, a task is sent to RabbitMQ. One worker picks it up to process the payment, another creates a PDF invoice, and failed tasks are retried or moved to a dead-letter queue, so each order is handled exactly by one worker, reliably.

User activity --> [ Kafka: page-views, orders ] --> Analytics, Recommendations, Fraud
                                                    (each reads every event, can replay)

Order placed  --> [ RabbitMQ: orders exchange ] --> payment queue  --> Payment workers
                                               --> invoice queue  --> Invoice workers
                                                    (each task handled once, then removed)

13. When to Choose Kafka

• You need very high throughput, such as clickstreams, logs, metrics or IoT data.
• Many independent consumers must read the same events.
• You need to replay history, for example to rebuild a database or train a model.
• You are building event-driven microservices or event sourcing.
• You need stream processing (Kafka Streams) or data integration (Kafka Connect).
• Ordering per key matters at scale.

14. When to Choose RabbitMQ

• You need a task or job queue where each job is done once by one worker.
• You need complex routing by patterns or headers.
• You need per-message features: priorities, per-message expiry (TTL), delayed delivery or request–reply.
• Message volumes are moderate, and you want very low latency for each message.
• You need protocols such as AMQP, MQTT or STOMP, for example for IoT devices.
• You want a broker that is simple to start with for a small team.

15. Using Kafka and RabbitMQ Together

Many organizations run both. A common pattern:

• Kafka is the central event backbone: every important business event is published there, kept and available to any team.
• RabbitMQ handles work distribution inside a service or team: background jobs, retries with delays, and routing tasks to the right workers.

Connectors, such as Kafka Connect connectors for RabbitMQ, can move messages between the two when needed. Avoid running both without a clear reason, though: each adds operational work.

16. Common Myths

17. Summary

In simple words:

• Use Kafka when you need high throughput, real-time event streaming, long retention and the ability for many consumers to read and replay the same events independently.
• Use RabbitMQ when you need reliable task queues, flexible routing, per-message features such as priorities and delays, and traditional queue-based communication between services.

Both are powerful, mature and reliable when configured well. The right choice depends on whether your problem is mainly streaming events to many readers (Kafka) or distributing work to the right worker (RabbitMQ), and many systems use both.

18. Frequently Asked Questions (FAQ)

kafkaTemplate.send("page-views", userId, event);               // Kafka
rabbitTemplate.convertAndSend("orders", "order.created.eu", id); // RabbitMQ

# Kafka (single node, KRaft mode)
docker run -d --name kafka -p 9092:9092 apache/kafka:latest

# RabbitMQ with the management web UI at http://localhost:15672 (guest / guest)
docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:management