00:00

Help Us Improve

We're sorry this tutorial wasn't helpful. Please tell us what we can do better.

Kafka MirrorMaker

Kafka MirrorMaker is a tool provided by Apache Kafka that helps copy (replicate) data from one Kafka cluster to another Kafka cluster. In simple words, it keeps topics in two different Kafka clusters in sync.

Think of MirrorMaker like a bridge between two Kafka clusters. Whatever messages are produced in the source cluster are automatically copied to the target cluster.

MirrorMaker is commonly used for:

  • Disaster recovery
  • Multi-data-center setups
  • Migrating data between Kafka clusters
  • Sharing data across teams or regions

Why Do We Need Kafka MirrorMaker?

In real-world systems, companies often run Kafka clusters in multiple locations (for example, one in India and another in the US). If one cluster goes down, data should still be available in another cluster.

Kafka MirrorMaker helps by continuously copying messages so that both clusters have the same data.

Key Features of Kafka MirrorMaker

1. Cross-Cluster Replication

MirrorMaker replicates Kafka topics from a source cluster to a target cluster. It works even if both clusters are in different data centers or regions.

2. Built on Kafka Consumers and Producers

Internally, MirrorMaker uses Kafka consumers to read messages and Kafka producers to write messages to another cluster.

3. Supports Multiple Topics

You can replicate one topic or multiple topics using patterns (for example, all topics starting with orders-).

4. Fault Tolerant

If a MirrorMaker instance fails, Kafka consumer group management ensures that another instance can continue replication without data loss.

5. Scalable

Multiple MirrorMaker instances can run in parallel to handle high volumes of data efficiently.

6. Offset Management

MirrorMaker maintains offsets, so messages are not duplicated or skipped during replication.

7. Near Real-Time Replication

Messages are copied almost in real time, making it suitable for backup and failover use cases.

Simple Example of Kafka MirrorMaker

Let’s understand this with a simple example.

Scenario

  • Source Cluster: Kafka Cluster A (India)
  • Target Cluster: Kafka Cluster B (US)
  • Topic: order-events

An application produces order-related messages to the order-events topic in Cluster A.

Kafka MirrorMaker is configured to:

  • Consume messages from order-events in Cluster A
  • Produce the same messages to order-events in Cluster B

Now, whenever a new order message is published in Cluster A, it automatically appears in Cluster B.

If Cluster A goes down, applications can still read order data from Cluster B without disruption.

Real-World Use Cases

  • Disaster Recovery: Keep a backup Kafka cluster ready
  • Geo-Replication: Share data across regions
  • Data Migration: Move topics from old cluster to new cluster
  • Analytics: Replicate production data to analytics cluster

MirrorMaker vs MirrorMaker 2

Kafka also provides MirrorMaker 2, which is an improved version built on Kafka Connect.

  • Better monitoring and configuration
  • Automatic topic and configuration syncing
  • More reliable offset handling

For new projects, MirrorMaker 2 is generally recommended.

Summary

Kafka MirrorMaker is a powerful and simple tool used to replicate Kafka topics from one cluster to another. It ensures data availability, supports disaster recovery, and enables multi-region Kafka architectures.

By copying messages in near real time, MirrorMaker helps build reliable, scalable, and fault-tolerant systems. For anyone working with distributed Kafka setups, MirrorMaker is an essential component.