Apache Kafka ZooKeeper

Apache Kafka is a popular platform for building real-time data pipelines and streaming applications. A Kafka cluster is made of several servers (brokers) that must agree on who is in the cluster, which broker leads each partition and how topics are configured. For most of Kafka's history, that coordination job was done by Apache ZooKeeper.

This tutorial explains what ZooKeeper is, exactly what it does for Kafka, how to explore Kafka's data inside ZooKeeper (including from Java), why Kafka replaced it with KRaft, and how to tell which mode your cluster uses.

1. What Is ZooKeeper?

Apache ZooKeeper is a separate open-source service for coordinating distributed systems. It stores small pieces of data in a tree of nodes called znodes, similar to folders and files, and keeps that data identical on several servers. Applications use it for things that are hard to get right on their own: knowing which servers are alive, electing a single leader, and sharing configuration.

• Znodes: each node has a path (for example /brokers/ids/1) and a small value.
• Ephemeral znodes: nodes that disappear automatically when the client that created them disconnects. Kafka uses them to detect failed brokers.
• Watches: clients can ask to be notified when a znode changes, so they react immediately.
• Ensemble: ZooKeeper runs on several servers that agree on every change by majority vote.

2. Role of ZooKeeper in Kafka

In ZooKeeper-based Kafka (versions before 4.0), ZooKeeper acts as the cluster's source of truth for metadata, the information about the cluster rather than the messages themselves:

3. What ZooKeeper Does Not Store

A common misunderstanding is that ZooKeeper stores Kafka messages or consumer progress. It does not:

• Messages: all records live in partition log files on the brokers' disks. ZooKeeper never sees them.
• Consumer offsets: since Kafka 0.9, consumer groups commit offsets to the internal Kafka topic __consumer_offsets. Only the very old Scala consumer kept offsets in ZooKeeper.
• Producer and consumer connections: modern clients connect only to brokers using bootstrap.servers. They never talk to ZooKeeper.

This is why ZooKeeper data is small (megabytes, not gigabytes) and why it could later be replaced without changing client applications. See Kafka Offset for how offsets work.

4. How Kafka and ZooKeeper Work Together

          [ ZooKeeper ensemble: zk1, zk2, zk3 ]
             stores metadata, elects the controller
                 ^            ^            ^
                 |            |            |
          Broker 1      Broker 2      Broker 3
         (controller)
                 ^            ^            ^
                 |            |            |
       Producers and consumers (connect to brokers only)

Kafka keeps its metadata in ZooKeeper under a set of well-known paths:

5. Controller Election Step by Step

One broker in the cluster acts as the controller: it decides partition leaders and pushes metadata to the other brokers. ZooKeeper makes sure there is exactly one:

1. When brokers start, each tries to create the ephemeral znode /controller.
2. ZooKeeper allows only one to succeed. That broker becomes the controller; the others set a watch on /controller.
3. If the controller crashes or loses its ZooKeeper session, its ephemeral znode is deleted automatically.
4. The watches fire, the remaining brokers race again, and a new controller is elected. /controller_epoch increases so old controllers are ignored.
5. The new controller reads all metadata from ZooKeeper and chooses new partition leaders where needed.

On large clusters, step 5 could take a long time because the new controller had to reload the metadata of every partition. This was one of the main reasons for creating KRaft.

6. ZooKeeper Ensemble and Quorum

In production, ZooKeeper runs as an ensemble of an odd number of servers. A change is accepted only when a majority (quorum) agrees, so the ensemble keeps working as long as most servers are alive:

If the ensemble loses its majority, Kafka brokers keep serving existing partitions for a while, but no metadata changes (new leaders, new topics) can happen until ZooKeeper recovers.

7. Example: Running Kafka with ZooKeeper

These commands work with Kafka 3.x downloads, which still include ZooKeeper. Kafka 4.0 and later no longer ship zookeeper-server-start.sh.

# 1. Start ZooKeeper (listens on port 2181 by default)
bin/zookeeper-server-start.sh config/zookeeper.properties

# 2. In another terminal, start a Kafka broker that uses this ZooKeeper
bin/kafka-server-start.sh config/server.properties

# 3. Create a topic named "orders" (tools talk to the broker, not to ZooKeeper)
bin/kafka-topics.sh --create --topic orders \
  --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

# 4. Check the topic
bin/kafka-topics.sh --describe --topic orders --bootstrap-server localhost:9092

The broker finds ZooKeeper through these settings in server.properties:

# Unique ID of this broker
broker.id=1

# ZooKeeper ensemble; the optional "/kafka" part (chroot) keeps Kafka data in its own folder
zookeeper.connect=zk1:2181,zk2:2181,zk3:2181/kafka

# How long ZooKeeper waits before treating a silent broker as dead
zookeeper.session.timeout.ms=18000

listeners=PLAINTEXT://:9092
log.dirs=/var/lib/kafka/data

8. Exploring Kafka Data in ZooKeeper

Kafka 3.x includes zookeeper-shell.sh, which lets you look inside ZooKeeper. This is useful for learning and troubleshooting; never edit Kafka's znodes by hand.

# List the IDs of live brokers
bin/zookeeper-shell.sh localhost:2181 ls /brokers/ids

# Show which broker is the controller
bin/zookeeper-shell.sh localhost:2181 get /controller

# Show the leader and in-sync replicas of partition 0 of "orders"
bin/zookeeper-shell.sh localhost:2181 get /brokers/topics/orders/partitions/0/state

[1, 2, 3]
{"version":2,"brokerid":1,"timestamp":"1767225600000"}
{"controller_epoch":4,"leader":2,"version":1,"leader_epoch":7,"isr":[2,3,1]}

The output shows three live brokers, broker 1 as the controller, and broker 2 leading partition 0 with all three replicas in sync.

9. Reading Kafka Metadata from Java

The official ZooKeeper Java client can read the same znodes. This program connects to ZooKeeper, lists live brokers and prints the current controller. It is a learning example; real applications should use Kafka's Admin client instead (see section 12).

<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.8.4</version>
</dependency>

package com.example.kafka;

import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

public class KafkaZooKeeperReader {

    public static void main(String[] args) throws Exception {
        CountDownLatch connected = new CountDownLatch(1);

        // Wait until the session is established before reading
        ZooKeeper zk = new ZooKeeper("localhost:2181", 15_000, event -> {
            if (event.getState() == Watcher.Event.KeeperState.SyncConnected) {
                connected.countDown();
            }
        });

        try {
            if (!connected.await(10, TimeUnit.SECONDS)) {
                throw new IllegalStateException("Could not connect to ZooKeeper");
            }

            List<String> brokerIds = zk.getChildren("/brokers/ids", false);
            System.out.println("Live brokers: " + brokerIds);

            for (String id : brokerIds) {
                byte[] data = zk.getData("/brokers/ids/" + id, false, null);
                System.out.println("  broker " + id + " -> "
                        + new String(data, StandardCharsets.UTF_8));
            }

            byte[] controller = zk.getData("/controller", false, null);
            System.out.println("Controller: "
                    + new String(controller, StandardCharsets.UTF_8));
        } finally {
            zk.close();
        }
    }
}

Live brokers: [1, 2, 3]
  broker 1 -> {"host":"kafka1","port":9092,"rack":null,...}
  broker 2 -> {"host":"kafka2","port":9092,"rack":null,...}
  broker 3 -> {"host":"kafka3","port":9092,"rack":null,...}
Controller: {"version":2,"brokerid":1,"timestamp":"1767225600000"}

10. Limitations of ZooKeeper

ZooKeeper served Kafka well for over a decade, but it brought real problems as clusters grew:

• Two systems to run: teams had to deploy, secure, monitor and upgrade both Kafka and a ZooKeeper ensemble, each with its own configuration and skills.
• Slow controller failover: a new controller had to reload all metadata from ZooKeeper, which could take minutes on clusters with hundreds of thousands of partitions.
• Limited scale: the number of partitions per cluster was practically limited to around 200,000.
• Metadata could get out of sync: the controller's in-memory view and ZooKeeper's data could briefly disagree, causing tricky bugs.
• Separate security setup: ZooKeeper needed its own authentication and TLS configuration, in addition to Kafka's.

11. ZooKeeper vs KRaft

KRaft (Kafka Raft) moves metadata management into Kafka itself. A small group of Kafka controller nodes store metadata in an internal, replicated log and elect a leader with the Raft protocol.

For a detailed comparison, see Kafka ZooKeeper vs KRaft.

12. Is My Cluster Using ZooKeeper or KRaft?

Applications do not need to know, but operators often do. Kafka's Admin client can tell you: the describeMetadataQuorum() call only works on KRaft clusters. On a ZooKeeper-based cluster it fails with an UnsupportedVersionException.

package com.example.kafka;

import java.util.Properties;
import java.util.concurrent.ExecutionException;

import org.apache.kafka.clients.admin.Admin;
import org.apache.kafka.clients.admin.AdminClientConfig;
import org.apache.kafka.clients.admin.QuorumInfo;
import org.apache.kafka.common.Node;
import org.apache.kafka.common.errors.UnsupportedVersionException;

public class KafkaModeCheck {

    public static void main(String[] args) throws Exception {
        Properties props = new Properties();
        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

        try (Admin admin = Admin.create(props)) {
            Node controller = admin.describeCluster().controller().get();
            System.out.println("Active controller id: " + controller.id());

            try {
                QuorumInfo quorum = admin.describeMetadataQuorum().quorumInfo().get();
                System.out.println("Mode: KRaft");
                System.out.println("Quorum leader: " + quorum.leaderId()
                        + ", voters: " + quorum.voters().size());
            } catch (ExecutionException e) {
                if (e.getCause() instanceof UnsupportedVersionException) {
                    // The metadata quorum API does not exist in ZooKeeper mode
                    System.out.println("Mode: ZooKeeper");
                } else {
                    throw e;
                }
            }
        }
    }
}

# On a ZooKeeper-based Kafka 3.x cluster
Active controller id: 1
Mode: ZooKeeper

# On a KRaft cluster
Active controller id: 3002
Mode: KRaft
Quorum leader: 3002, voters: 3

From the command line, the same check is bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status, which only succeeds on KRaft clusters.

13. Migrating from ZooKeeper to KRaft

Because Kafka 4.0 has no ZooKeeper support, every ZooKeeper-based cluster must move to KRaft before upgrading to 4.0. Kafka supports an online migration with no downtime for clients:

1. Upgrade to a recent 3.x release (3.9 is the final "bridge" release and the recommended starting point).
2. Deploy a KRaft controller quorum (usually 3 controllers) configured in migration mode.
3. Enable migration on the brokers and restart them one by one. The controllers copy all metadata from ZooKeeper.
4. Run in dual-write mode: the KRaft controllers manage the cluster while still writing metadata back to ZooKeeper, so you can roll back if needed.
5. Finalize: restart the brokers in pure KRaft mode and remove the ZooKeeper settings.
6. Decommission ZooKeeper, then upgrade to Kafka 4.x.

Producers and consumers keep working during the whole process because they only ever talk to brokers.

14. Best Practices

15. Summary

For most of its history, Kafka relied on ZooKeeper as its coordinator: brokers registered there, the controller was elected through it, and topic, partition, configuration and security metadata were stored in it. ZooKeeper never held messages or modern consumer offsets, and clients never connected to it directly. As clusters grew, running two systems and slow controller failover became real problems, so Kafka introduced KRaft, which keeps metadata inside Kafka itself. ZooKeeper mode was deprecated in Kafka 3.5 and removed in 4.0. Today, ZooKeeper knowledge is still useful for running and migrating older clusters, while all new Kafka clusters should use KRaft.

Frequently Asked Questions (FAQ)

props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");