Kafka Broker

A Kafka broker is a server that runs Apache Kafka and is responsible for receiving, storing and delivering messages. Brokers are the workhorses of every Kafka system: all data flows through them and is stored on them.

This guide explains what a broker does, how it handles requests and stores data on disk, how brokers share partition leadership, the most important configuration settings, how to run and inspect brokers, and how to add or remove them safely. For how brokers fit into the whole system, see Kafka Architecture.

1. What Is a Kafka Broker?

In simple words, a Kafka broker is the middleman between applications that send data (producers) and applications that read data (consumers). When an application sends a message to Kafka, a broker stores it safely on disk. When another application wants that message, a broker delivers it.

A Kafka system usually has several brokers working together as a cluster. Each broker is identified by a unique number, its node ID (or broker ID), and stores part of the data.

2. A Simple Example: The Post Office

Imagine Kafka as a postal system:

• Producers are people who send letters.
• Consumers are people who collect letters.
• A broker is a post office that stores the letters and hands them out.

When one post office is too busy, several post offices (brokers) share the work, and each keeps copies of important letters, so nothing is lost if one office closes. Unlike a real post office, a Kafka broker keeps the letters after delivery for a set time, so other people can read them too.

3. What a Broker Does

4. Key Features of a Kafka Broker

5. How a Broker Handles Requests

A broker handles thousands of client connections at once using a small set of thread pools:

Clients (producers / consumers)
        |
        v
 [ Network threads ]      read requests from sockets, send responses
        |                 (num.network.threads, default 3)
        v
 [ Request queue ]
        |
        v
 [ I/O threads ]          process requests: append to log, read from log
        |                 (num.io.threads, default 8)
        v
 [ Partition logs on disk / page cache ]

1. Network threads accept connections, read incoming requests and put them in a request queue.
2. I/O threads take requests from the queue and do the work: appending messages to the log or reading them.
3. For acks=all writes and long-polling fetches, requests can wait in purgatory until followers have copied the data or new data arrives, without blocking a thread.
4. Responses go back through the network threads to the client.

This design, plus batching and the operating system’s page cache, is what lets a single broker serve very high traffic.

6. How a Broker Stores Data on Disk

Each partition a broker hosts is a folder inside its log directory, named topic-partition. The folder contains segment files:

/var/lib/kafka/data/
  orders-0/
    00000000000000000000.log        <- messages, offsets 0 .. 52309
    00000000000000000000.index      <- maps offsets to file positions
    00000000000000000000.timeindex  <- maps timestamps to offsets
    00000000000000052310.log        <- active segment (new data goes here)
    00000000000000052310.index
    00000000000000052310.timeindex
    leader-epoch-checkpoint
    partition.metadata
  orders-1/
  payments-0/
  __consumer_offsets-12/

New messages always go to the newest, active segment. When it reaches a size or age limit (by default 1 GB, log.segment.bytes), the broker starts a new segment. Retention then deletes or compacts whole old segments, which is very cheap compared with deleting individual messages.

7. Partition Leaders and Followers on Brokers

Every partition has one leader replica and zero or more followers, each on a different broker. A broker is therefore usually a leader for some partitions and a follower for others:

            Broker 1     Broker 2     Broker 3
orders-0    LEADER       follower     follower
orders-1    follower     LEADER       follower
orders-2    follower     follower     LEADER

• Producers and consumers talk to the leader of each partition.
• Followers continuously fetch new data from the leader to stay in sync.
• The first replica in each partition’s list is the preferred leader. After a broker restart, Kafka moves leadership back to preferred leaders automatically (auto.leader.rebalance.enable=true), so the load stays balanced.

See Kafka Fault Tolerance for how leaders are replaced when a broker fails.

8. Broker and Controller Roles in KRaft

In KRaft mode, each Kafka node has one or both of two roles, set with process.roles:

Combined mode is convenient for development and small clusters. For production, dedicated controllers (usually three) and separate brokers are recommended, so heavy data traffic cannot slow down cluster management. See Kafka KRaft.

9. Important Broker Configuration

A broker is configured in server.properties. This example is a combined node in a three-node KRaft cluster:

# ---- Identity and role ----
node.id=1
process.roles=broker,controller
controller.quorum.voters=1@kafka1:9093,2@kafka2:9093,3@kafka3:9093

# ---- Network ----
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
advertised.listeners=PLAINTEXT://kafka1:9092
controller.listener.names=CONTROLLER
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT

# ---- Storage ----
log.dirs=/var/lib/kafka/data

# ---- Safe defaults for new topics ----
num.partitions=3
default.replication.factor=3
min.insync.replicas=2
offsets.topic.replication.factor=3

# ---- Retention ----
# Keep data for 7 days
log.retention.hours=168
# Start a new segment file every 1 GB
log.segment.bytes=1073741824

# ---- Threads ----
num.network.threads=3
num.io.threads=8

Many settings can be changed while the broker is running, without a restart, using kafka-configs.sh (dynamic broker configuration).

10. Listeners and Advertised Listeners

Listeners are the most common source of connection problems. Two settings matter:

• listeners: the network interfaces and ports the broker listens on.
• advertised.listeners: the address the broker tells clients to use for later connections.

1. Client connects to a bootstrap server:   kafka1:9092
2. Broker replies with cluster metadata:    "partition orders-1 is led by kafka2:9092"
3. Client connects directly to:             kafka2:9092   <- an advertised listener

If advertised.listeners contains an address the client cannot reach, such as a Docker-internal host name or localhost on a remote server, the first connection works but every later request fails. Use separate listeners for different networks, for example one for clients inside Docker and one for clients on your machine:

listeners=INTERNAL://:29092,EXTERNAL://:9092,CONTROLLER://:9093
advertised.listeners=INTERNAL://kafka1:29092,EXTERNAL://localhost:9092
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
inter.broker.listener.name=INTERNAL
controller.listener.names=CONTROLLER

11. Running a Kafka Broker

11.1 With Docker

The official image starts a single combined broker and controller in KRaft mode with sensible defaults:

docker run -d --name kafka -p 9092:9092 apache/kafka:latest

11.2 From the Kafka download

In KRaft mode, a new node’s storage must be formatted once with a cluster ID before the first start:

# 1. Generate a cluster ID (once per cluster)
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"

# 2. Format the log directories of this node
bin/kafka-storage.sh format -t "$KAFKA_CLUSTER_ID" -c config/server.properties

# 3. Start the broker
bin/kafka-server-start.sh config/server.properties

Every node in the same cluster must be formatted with the same cluster ID. Stop a broker gracefully with bin/kafka-server-stop.sh, so it can move leadership away before shutting down.

12. Inspecting Brokers from the Command Line

# Which brokers are reachable, and which API versions do they support?
bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092

# Disk usage of each partition on each broker
bin/kafka-log-dirs.sh --bootstrap-server localhost:9092 --describe

# Where are the leaders and replicas of a topic?
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders

# Show a broker's configuration
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 1 --describe --all

# Change a setting on a running broker (no restart)
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 1 --alter --add-config log.cleaner.threads=2

# KRaft: show the controller quorum
bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status

13. Inspecting Brokers with Java

The Admin client lets applications and tools inspect the cluster. This example lists the brokers, the active controller and one broker’s retention setting:

package com.example.kafka;

import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import org.apache.kafka.clients.admin.Admin;
import org.apache.kafka.clients.admin.AdminClientConfig;
import org.apache.kafka.clients.admin.Config;
import org.apache.kafka.clients.admin.DescribeClusterResult;
import org.apache.kafka.common.Node;
import org.apache.kafka.common.config.ConfigResource;

public class BrokerInspector {

    public static void main(String[] args) throws Exception {
        Properties props = new Properties();
        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

        try (Admin admin = Admin.create(props)) {
            DescribeClusterResult cluster = admin.describeCluster();

            System.out.println("Cluster ID: " + cluster.clusterId().get());
            System.out.println("Controller: " + cluster.controller().get());

            Collection<Node> brokers = cluster.nodes().get();
            for (Node broker : brokers) {
                System.out.printf("Broker %d at %s:%d (rack: %s)%n",
                        broker.id(), broker.host(), broker.port(), broker.rack());
            }

            // Read one broker setting
            ConfigResource broker1 = new ConfigResource(ConfigResource.Type.BROKER, "1");
            Map<ConfigResource, Config> configs = admin.describeConfigs(List.of(broker1)).all().get();
            System.out.println("log.retention.hours = "
                    + configs.get(broker1).get("log.retention.hours").value());
        }
    }
}

Cluster ID: 4L6g3nShT-eMCtK--X86sw
Controller: kafka2:9092 (id: 2 rack: null)
Broker 1 at kafka1:9092 (rack: eu-west-1a)
Broker 2 at kafka2:9092 (rack: eu-west-1b)
Broker 3 at kafka3:9092 (rack: eu-west-1c)
log.retention.hours = 168

14. Adding and Removing Brokers

14.1 Adding a broker

1. Start the new broker with a new, unique node.id, formatted with the cluster’s ID.
2. It joins the cluster, but existing partitions do not move automatically; only new topics use it.
3. Move some partitions onto it with kafka-reassign-partitions.sh, or a tool such as Cruise Control, to balance the load.

# 1. Generate a proposed plan that includes the new broker 4
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
  --topics-to-move-json-file topics.json --broker-list "1,2,3,4" --generate

# 2. Save the proposed plan as plan.json, then apply it
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
  --reassignment-json-file plan.json --execute

# 3. Check progress
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
  --reassignment-json-file plan.json --verify

14.2 Removing a broker

1. Move all its partitions to other brokers with a reassignment.
2. Wait until the reassignment has finished and every partition is fully in sync.
3. Stop the broker gracefully, and in KRaft mode unregister it from the cluster metadata.

14.3 Rolling restarts

To upgrade or reconfigure brokers without downtime, restart them one at a time, waiting until there are no under-replicated partitions before moving to the next broker.

15. Kafka Broker vs Kafka Cluster

Most production systems use a cluster of at least three brokers. See Kafka Cluster.

16. Sizing and Hardware Tips

• Disk: the most important resource. Size it for incoming data per day × retention days × replication factor, plus headroom. Several disks can be listed in log.dirs.
• Memory: a modest Java heap (often 4–8 GB) is enough; leave the rest of the RAM to the operating system’s page cache, which Kafka relies on heavily.
• Network: often the real limit; replication multiplies the traffic between brokers.
• CPU: matters most for compression, TLS encryption and many small requests.
• Separate machines: do not share broker disks with other busy applications.

Incoming data:       200 GB per day
Retention:           7 days
Replication factor:  3

Total storage = 200 GB x 7 x 3 = 4,200 GB
Across 3 brokers  = about 1,400 GB each, plus ~30% headroom -> ~1.8 TB per broker

17. Monitoring Brokers

Brokers expose these metrics through JMX. Tools such as a Prometheus JMX exporter and Grafana dashboards make them easy to watch and alert on.

18. Best Practices

1. Run at least three brokers, spread across racks or availability zones (broker.rack).
2. Set safe defaults: default.replication.factor=3 and min.insync.replicas=2.
3. Disable automatic topic creation in production and create topics deliberately.
4. Configure advertised.listeners carefully for every network clients use.
5. Use dedicated controllers for larger production clusters.
6. Leave most RAM to the page cache, with a moderate Java heap.
7. Enable security: TLS, authentication and ACLs.
8. Restart brokers one at a time, waiting for full replication between restarts.
9. Rebalance partitions after adding or before removing brokers.
10. Monitor under-replicated partitions, disk space and request latency.

19. Common Mistakes to Avoid

20. Summary

A Kafka broker is the core building block of Apache Kafka. It accepts messages from producers, stores them on disk in partition logs made of segment files, replicates them to other brokers, and delivers them to consumers. Brokers share the work by leading some partitions and following others, and in KRaft mode they can also act as controllers. Configure listeners and safe replication defaults carefully, run at least three brokers across zones, rebalance partitions when the cluster changes, and monitor replication, disk and latency. With these practices, brokers keep data fast, safe and scalable for modern microservices and event-driven systems. Continue with Kafka Cluster to see how brokers work together.

21. Frequently Asked Questions (FAQ)

total = daily data x retention days x replication factor
per broker = total / number of brokers, plus 30-50% headroom

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 1 --alter --add-config log.cleaner.threads=2

# Are all brokers reachable?
bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092

# Any partitions missing replicas?
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --under-replicated-partitions

# Is the KRaft controller quorum healthy?
bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status