Apache Kafka Use Cases

Apache Kafka is a platform for handling real-time data. It is used when applications need to send, receive and process large amounts of data quickly and reliably. Kafka works like a central pipeline through which data flows between many different systems.

This guide explains Kafka’s most important use cases in simple terms, with small working examples for many of them: activity tracking, microservices events, change data capture, stream processing, log and metrics pipelines and more. It ends with a complete online-shopping example and an honest look at when Kafka is not the right tool. New to Kafka? Start with Kafka Introduction.

1. Why Do We Need Kafka?

Modern applications generate data continuously:

• Users click, search and browse websites and apps.
• Customers place orders and make payments.
• Servers write logs and metrics every second.
• Devices and sensors send readings.
• Microservices tell each other what has happened.

Sending all this data directly from each system to every other system quickly becomes slow, fragile and hard to manage: every new consumer needs a new connection, and if a receiver is down, data is lost.

Without Kafka: every system talks to every other system

   Orders ---> Analytics        Orders ---> Search
   Orders ---> Email            Payments -> Analytics
   Clicks ---> Analytics        Payments -> Fraud   ... (many fragile links)

With Kafka: one central pipeline

   Orders  ---+                 +---> Analytics
   Payments --+--> [ Kafka ] ---+---> Email
   Clicks  ---+                 +---> Search, Fraud, Data warehouse ...

Kafka solves this by acting as a durable, high-speed buffer and message hub. Producers write once; any number of consumers read independently, at their own pace, and can even re-read past data.

2. Kafka Use Cases at a Glance

3. Real-Time Data Streaming

Kafka is widely used to stream data in real time. Applications publish data continuously, and other systems can read it within milliseconds.

Examples: live order tracking, live stock prices, sports scores, delivery driver locations and in-app notifications.

Because Kafka stores the data, a consumer that was briefly offline does not miss anything: it continues from where it stopped.

4. Website Activity Tracking

This was Kafka’s original use case. Every user action, such as a page view, search or “add to cart”, becomes an event. Many teams then use the same stream: analytics, personalization, A/B testing and fraud detection.

package com.example.tracking;

import java.time.Instant;

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class ActivityTracker {

    private final KafkaTemplate<String, String> kafka;

    public ActivityTracker(KafkaTemplate<String, String> kafka) {
        this.kafka = kafka;
    }

    public void track(String userId, String action, String productId) {
        String event = """
                {"userId":"%s","action":"%s","productId":"%s","at":"%s"}
                """.formatted(userId, action, productId, Instant.now()).strip();

        // Key = user ID, so one user's actions stay in order
        kafka.send("user-activity", userId, event);
    }
}

The web application calls activityTracker.track(...) and moves on immediately; it does not wait for analytics or recommendations to process the event.

5. Microservices Communication

In a microservices architecture, services need to tell each other what has happened. With Kafka, a service publishes an event and does not need to know who listens. Services stay loosely coupled, and if one is down, its events wait safely in Kafka until it is back.

// order-service: publishes an event after saving the order
kafka.send("orders", order.getId(), orderPlacedJson);

// shipping-service: reacts in its own time
@KafkaListener(topics = "orders", groupId = "shipping-service")
public void onOrderPlaced(String orderPlacedJson) {
    shipmentService.prepareShipment(orderPlacedJson);
}

// notification-service: reacts to the same event independently
@KafkaListener(topics = "orders", groupId = "notification-service")
public void sendConfirmation(String orderPlacedJson) {
    emailService.sendOrderConfirmation(orderPlacedJson);
}

See Asynchronous Communication in Microservices for more patterns.

6. Event-Driven Architecture and Event Sourcing

Kafka fits event-driven systems naturally. Whenever something happens, such as order placed, payment successful or item shipped, an event is published, and every interested system reacts.

A step further is event sourcing: instead of storing only the current state, a service stores every change as an event. The current state can always be rebuilt by replaying the events, which also gives a complete audit history.

Topic "account-events" (key = account-42)

  offset 0: AccountOpened     balance  0
  offset 1: MoneyDeposited    +500    -> 500
  offset 2: MoneyWithdrawn    -120    -> 380
  offset 3: MoneyDeposited    +50     -> 430

Current balance = replay all events for account-42 = 430

Kafka’s ordered, durable log and per-key ordering make it a strong foundation for both. See Event-Driven Architecture.

7. Log Aggregation

Servers and applications produce huge amounts of logs. Kafka collects logs from many machines and delivers them to search and monitoring systems, such as Elasticsearch or OpenSearch, and to long-term storage.

• Log shippers such as Fluent Bit, Vector or Filebeat send log lines to a Kafka topic.
• Kafka buffers them, so a slow or offline search cluster does not cause lost logs.
• Several consumers use the same logs: search, security analysis and archiving.

This helps teams investigate errors, performance problems and suspicious behavior across hundreds of servers in one place.

8. Metrics and Monitoring

Similar to logs, applications and infrastructure produce metrics: request rates, response times, CPU and memory usage, business numbers such as orders per minute. Kafka can carry these metrics from many sources to monitoring, alerting and analytics systems, decoupling the producers from the tools that store and display them.

9. Change Data Capture (CDC)

Change data capture (CDC) turns every insert, update and delete in a database into an event, without changing the application. A popular open-source tool for this is Debezium, which runs on Kafka Connect and reads the database’s transaction log.

{
  "name": "shop-orders-cdc",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "postgres",
    "database.port": "5432",
    "database.user": "debezium",
    "database.password": "${file:/secrets/db.properties:password}",
    "database.dbname": "shop",
    "topic.prefix": "shop",
    "table.include.list": "public.orders,public.customers",
    "plugin.name": "pgoutput"
  }
}

# Register the connector with Kafka Connect
curl -X POST -H "Content-Type: application/json" \
     --data @postgres-orders-connector.json \
     http://localhost:8083/connectors

The password is read from a file through Kafka Connect’s FileConfigProvider (enabled in the Connect worker settings), so it is not stored in the connector configuration itself. Every change to the orders table now appears in the topic shop.public.orders. Typical uses: keeping a search index or cache in sync, feeding a data warehouse, and gradually splitting a monolith into microservices without touching its code. See Kafka Connect.

10. Real-Time Stream Processing

Kafka does not only move data; with Kafka Streams, a Java library, you can process it as it flows: filter, transform, join and aggregate. This example counts how many times each product is viewed per minute:

package com.example.streams;

import java.time.Duration;
import java.util.Properties;

import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.KeyValue;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Produced;
import org.apache.kafka.streams.kstream.TimeWindows;

public class ProductViewCounter {

    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "product-view-counter");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.StringSerde.class);
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.StringSerde.class);

        StreamsBuilder builder = new StreamsBuilder();

        // Input: key = product ID, value = view event (JSON)
        KStream<String, String> views = builder.stream("product-views");

        views.groupByKey()
             .windowedBy(TimeWindows.ofSizeWithNoGrace(Duration.ofMinutes(1)))
             .count()
             .toStream()
             .map((window, count) -> KeyValue.pair(window.key(), count))
             .to("product-views-per-minute", Produced.with(Serdes.String(), Serdes.Long()));

        KafkaStreams streams = new KafkaStreams(builder.build(), props);
        Runtime.getRuntime().addShutdownHook(new Thread(streams::close));
        streams.start();
    }
}

The results land in a new topic, ready for a live “trending products” widget. Similar code powers fraud detection (flag unusual payments within seconds), real-time pricing and alerting. See Kafka Streams.

11. Big Data, Analytics and Data Integration

Kafka is often the entry point for big data and analytics. It feeds data continuously into data lakes, data warehouses and processing engines such as Apache Spark and Apache Flink.

With Kafka Connect sink connectors, topics can be written to other systems with configuration instead of code, for example a JDBC sink that copies an orders topic into a reporting database, or a cloud-storage sink that archives events to object storage.

  Source connectors              Kafka               Sink connectors
  (bring data in)                                    (send data out)

  PostgreSQL (CDC) ----+                       +---> Data warehouse
  MySQL (CDC) ---------+--> [ topics ] --------+---> Elasticsearch / OpenSearch
  Files / APIs --------+                       +---> Cloud object storage

12. IoT and Sensor Data

Connected devices, such as vehicles, smart meters, factory machines and wearables, send continuous streams of readings. Kafka can ingest data from very large numbers of devices, usually through a gateway or an MQTT bridge, and make it available for:

• Real-time alerts, such as a temperature above a threshold.
• Analytics over long periods.
• Predictive maintenance, spotting failing machines before they break.

13. Kafka Use Cases by Industry

14. A Real-World Example: Online Shopping

Consider an e-commerce application. When a user places an order, several things must happen:

• The order service saves the order.
• The payment service processes the payment.
• The inventory service updates stock.
• The notification service sends a confirmation email.

14.1 Without Kafka

The order service calls each other service directly, one after another. If one of them is slow or down, the whole order process slows down or fails, and adding a new step means changing the order service.

14.2 With Kafka

1. The order service saves the order and publishes an OrderPlaced event to Kafka.
2. The payment service reads the event and processes the payment.
3. The inventory service reads the event and updates stock.
4. The notification service reads the event and sends the email.

                                +--> Payment Service      (group: payments)
Order Service --OrderPlaced-->  |
              [ topic: orders ] +--> Inventory Service    (group: inventory)
                                |
                                +--> Notification Service (group: notifications)

All services work independently. Kafka keeps the event safely even if a service is temporarily unavailable; that service simply catches up when it returns. Next month, an analytics or loyalty service can start reading the same topic without any change to the order service.

15. Benefits of Using Kafka

16. When Kafka Is Not the Right Choice

Kafka is powerful, but not always the best tool:

See Kafka vs RabbitMQ and Kafka vs JMS for detailed comparisons.

17. Summary

Apache Kafka is a powerful platform for handling real-time data and communication between systems. It is used for real-time streaming, website activity tracking, microservices communication, event-driven architecture and event sourcing, log and metrics pipelines, change data capture, stream processing, big data integration and IoT. By acting as a durable, scalable central pipeline, Kafka makes applications more reliable, scalable and easier to extend. If your system deals with high data volumes or real-time events that many systems need, Kafka is a strong choice; for small, simple or request–response needs, lighter tools may fit better. Kafka is not just a messaging system: it is a backbone for modern, data-driven applications. Continue with Kafka Architecture to see how it works inside.

18. Frequently Asked Questions (FAQ)

kafkaTemplate.send("orders", orderId, orderJson);          // publish

@KafkaListener(topics = "orders", groupId = "shipping-service")
void onOrder(String orderJson) { ... }                     // consume