00:00

Help Us Improve

We're sorry this tutorial wasn't helpful. Please tell us what we can do better.

Kafka Security

Kafka security is the set of mechanisms that protect Apache Kafka from unauthorized access, data leaks and misuse. Kafka often carries large volumes of business data between systems, so it must make sure that only trusted users and applications can produce, consume and manage data.

This guide explains encryption with TLS, authentication with SASL and certificates, authorization with ACLs, and securing broker traffic, with formatted broker settings, command-line steps, and Java and Spring Boot client examples. It builds on Kafka Broker and Kafka Cluster.

1. What Is Kafka Security?

In simple words, Kafka security answers three important questions:

QuestionSecurity areaKafka feature
Who are you?AuthenticationTLS client certificates or SASL (SCRAM, Kerberos, OAuth, PLAIN)
What are you allowed to do?AuthorizationAccess Control Lists (ACLs)
Is the data safe while traveling?EncryptionTLS on client and broker connections

Out of the box, Kafka has no security enabled: anyone who can reach a broker can read, write and delete data. Security is turned on through broker and client configuration.

A secured Kafka setup 
  Order Service ----TLS + login----+
                                   |        +---------------------------+
  Billing Service --TLS + login----+------> |  Kafka brokers             |
                                   |        |  1. check identity        |
  Admin tool -------TLS + login----+        |  2. check ACL permissions |
                                            |  3. serve request         |
                                            +---------------------------+

2. Key Features of Kafka Security

1. Authentication

Authentication verifies the identity of clients (producers, consumers or administrators) connecting to Kafka. Kafka supports several mechanisms:

  • TLS (mutual TLS): certificate-based authentication.
  • SASL/PLAIN: username and password; only safe together with TLS encryption.
  • SASL/SCRAM: a more secure username-password mechanism that never sends the password itself.
  • SASL/GSSAPI (Kerberos): enterprise authentication, often with Active Directory.
  • SASL/OAUTHBEARER: OAuth 2.0 tokens from an identity provider.

This ensures that only trusted users and applications can connect to Kafka.

2. Authorization

Authorization controls what an authenticated user is allowed to do. Kafka uses Access Control Lists (ACLs) to define permissions. For example, you can allow a user to:

  • Read messages from a topic.
  • Write messages to a topic.
  • Create or delete topics.

This prevents users from accessing or modifying data they are not permitted to use.

3. Encryption in transit

Kafka supports encryption using TLS (often still called SSL) to protect data:

  • While it travels between clients and brokers.
  • While it moves between Kafka brokers and controllers.

This ensures that sensitive data cannot be read or changed by attackers on the network. Kafka does not encrypt data stored on disk; that is covered in section 12.

4. Secure communication between brokers

Brokers and controllers also talk to each other to replicate data and share cluster metadata. Security settings make this internal traffic authenticated and encrypted as well, protecting the cluster from internal threats.

5. Audit and control

Kafka can log every authorization decision, so you can see who tried to access which topic and whether it was allowed. Together with quotas and metrics, this supports compliance and monitoring in enterprise systems. Full audit trails usually combine these logs with a central logging tool.

3. Listeners and Security Protocols

A broker accepts connections on one or more listeners. Each listener has a name, an address and a security protocol:

ProtocolEncryptionAuthenticationUse
PLAINTEXTNoNoLocal development only
SSLYes (TLS)Optional client certificates (mTLS)Encrypted traffic; certificate-based identity
SASL_PLAINTEXTNoSASLRarely; passwords or tokens travel unencrypted
SASL_SSLYes (TLS)SASLThe most common production choice
server.properties (listeners) 
# Client listener on 9093, controller listener on 9094
listeners=SASL_SSL://0.0.0.0:9093,CONTROLLER://0.0.0.0:9094
advertised.listeners=SASL_SSL://broker1.example.com:9093

# Map each listener name to its security protocol
listener.security.protocol.map=SASL_SSL:SASL_SSL,CONTROLLER:SSL

# Brokers talk to each other over the SASL_SSL listener
inter.broker.listener.name=SASL_SSL

You can run several listeners at once, for example a secured listener for applications and a separate one for internal traffic. This also makes it possible to switch an existing cluster to security step by step.

4. Encryption with TLS

TLS needs certificates. Each broker gets a keystore with its own certificate and private key. Clients get a truststore with the certificate authority (CA) that signed the broker certificates, so they can check they are talking to a real broker.

Step 1: Create the certificates

Create a CA and a broker certificate 
# 1. Create a private certificate authority (CA)
openssl req -new -x509 -days 365 -nodes \
  -keyout ca.key -out ca.crt -subj "/CN=Kafka-Demo-CA"

# 2. Create the broker key pair in a keystore
keytool -genkeypair -alias broker1 -keyalg RSA -keysize 2048 -validity 365 \
  -keystore broker1.keystore.p12 -storetype PKCS12 \
  -dname "CN=broker1.example.com" -ext SAN=DNS:broker1.example.com

# 3. Create a signing request and sign it with the CA
keytool -certreq -alias broker1 -keystore broker1.keystore.p12 -file broker1.csr
openssl x509 -req -in broker1.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -days 365 -out broker1.crt -copy_extensions copy

# 4. Import the CA and the signed certificate into the keystore
keytool -importcert -alias ca -file ca.crt -keystore broker1.keystore.p12 -noprompt
keytool -importcert -alias broker1 -file broker1.crt -keystore broker1.keystore.p12

# 5. Create a truststore containing the CA, for clients and brokers
keytool -importcert -alias ca -file ca.crt \
  -keystore client.truststore.p12 -storetype PKCS12 -noprompt

In production, certificates usually come from your company’s CA or a certificate management tool, and they must be renewed before they expire. The -copy_extensions option needs OpenSSL 3.0 or later.

Step 2: Configure the broker

server.properties (TLS) 
ssl.keystore.type=PKCS12
ssl.keystore.location=/etc/kafka/secrets/broker1.keystore.p12
ssl.keystore.password=change-me
ssl.truststore.type=PKCS12
ssl.truststore.location=/etc/kafka/secrets/client.truststore.p12
ssl.truststore.password=change-me

# Allow only modern TLS versions
ssl.enabled.protocols=TLSv1.3,TLSv1.2

Step 3: Configure the client

client.properties (TLS only) 
security.protocol=SSL
ssl.truststore.type=PKCS12
ssl.truststore.location=/etc/kafka/client.truststore.p12
ssl.truststore.password=change-me

Clients check that the broker’s certificate matches its host name by default (ssl.endpoint.identification.algorithm=https). Keep this check enabled; it prevents connections to fake brokers.

5. Choosing an Authentication Method

MethodHow the client proves identityGood forNotes
mTLSClient certificate signed by a trusted CAService-to-service traffic in companies with certificate toolingNo passwords; certificates must be issued and renewed
SASL/SCRAMUsername and password, checked with a challenge-responseMost teams; simple and secure with TLSUsers are managed inside Kafka
SASL/OAUTHBEARERShort-lived OAuth 2.0 tokenOrganizations with a central identity providerTokens expire automatically; needs a token endpoint
SASL/GSSAPIKerberos ticketEnterprises already using Kerberos or Active DirectoryPowerful but complex to operate
SASL/PLAINUsername and password sent as-is inside TLSSimple setups, or with a custom credential checkNever use without TLS; default setup stores passwords in a broker config file

Every authenticated client gets a principal, such as User:order-service. ACLs grant permissions to these principals.

6. Setting Up SASL/SCRAM Step by Step

SCRAM (Salted Challenge Response Authentication Mechanism) is a popular choice: users and passwords live inside Kafka, and passwords are never sent over the network.

Step 1: Enable SCRAM on the broker

server.properties (SASL/SCRAM) 
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512

# Login used by this broker for broker-to-broker connections
listener.name.sasl_ssl.scram-sha-512.sasl.jaas.config=\
  org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="broker" password="broker-secret";

In a KRaft cluster, the broker’s own SCRAM user must exist before the brokers start. Add it when formatting storage, for example kafka-storage.sh format ... --add-scram 'SCRAM-SHA-512=[name=broker,password=broker-secret]'.

Step 2: Create a user for each application

Create SCRAM users 
bin/kafka-configs.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --alter --entity-type users --entity-name order-service \
  --add-config 'SCRAM-SHA-512=[iterations=8192,password=order-secret]'

bin/kafka-configs.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --alter --entity-type users --entity-name billing-service \
  --add-config 'SCRAM-SHA-512=[iterations=8192,password=billing-secret]'

The file admin.properties holds the connection settings for an administrator user, in the same format as the client configuration below. Use long, random passwords in real systems.

Step 3: Configure the client

client.properties (SASL_SSL with SCRAM) 
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="order-service" password="order-secret";

ssl.truststore.type=PKCS12
ssl.truststore.location=/etc/kafka/client.truststore.p12
ssl.truststore.password=change-me

7. Authorization with ACLs

Authentication alone lets any valid user do anything. To limit permissions, enable an authorizer on the brokers and controllers:

server.properties (authorization) 
# Built-in authorizer for KRaft clusters
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer

# Users who can do everything (brokers and administrators)
super.users=User:broker;User:admin

# Deny access when no ACL matches (this is the default)
allow.everyone.if.no.acl.found=false

Each ACL says: principal is allowed or denied an operation on a resource, optionally from a host.

PartExamples
PrincipalUser:order-service
OperationRead, Write, Create, Delete, Describe, Alter, All
Resource typeTopic, Group, Cluster, TransactionalId
Patternliteral (exact name) or prefixed (all names starting with a value)
Grant producer and consumer permissions 
# Order Service may write to orders-topic
bin/kafka-acls.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --add --allow-principal User:order-service \
  --producer --topic orders-topic

# Billing Service may read orders-topic using the group billing-group
bin/kafka-acls.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --add --allow-principal User:billing-service \
  --consumer --topic orders-topic --group billing-group

# List the ACLs on the topic
bin/kafka-acls.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --list --topic orders-topic

The --producer and --consumer options add the usual set of permissions in one step. A consumer needs Read on both the topic and its consumer group. A deny ACL always wins over an allow ACL.

ACLs can also be managed from Java with the Admin client:

CreateAcl.java 
package com.example.kafka;

import java.util.List;
import java.util.Properties;

import org.apache.kafka.clients.admin.Admin;
import org.apache.kafka.common.acl.AccessControlEntry;
import org.apache.kafka.common.acl.AclBinding;
import org.apache.kafka.common.acl.AclOperation;
import org.apache.kafka.common.acl.AclPermissionType;
import org.apache.kafka.common.resource.PatternType;
import org.apache.kafka.common.resource.ResourcePattern;
import org.apache.kafka.common.resource.ResourceType;

public class CreateAcl {

    public static void main(String[] args) throws Exception {
        Properties props = SecureClientConfig.forUser("admin", System.getenv("KAFKA_ADMIN_PASSWORD"));

        AclBinding readOrders = new AclBinding(
                new ResourcePattern(ResourceType.TOPIC, "orders-topic", PatternType.LITERAL),
                new AccessControlEntry("User:billing-service", "*",
                        AclOperation.READ, AclPermissionType.ALLOW));

        try (Admin admin = Admin.create(props)) {
            admin.createAcls(List.of(readOrders)).all().get();
            System.out.println("ACL created");
        }
    }
}

8. Connecting a Secure Java Client

A small helper class keeps the security settings in one place. Passwords come from environment variables instead of being written in the code:

SecureClientConfig.java 
package com.example.kafka;

import java.util.Properties;

import org.apache.kafka.clients.CommonClientConfigs;
import org.apache.kafka.common.config.SaslConfigs;
import org.apache.kafka.common.config.SslConfigs;

public final class SecureClientConfig {

    private SecureClientConfig() {
    }

    public static Properties forUser(String username, String password) {
        Properties props = new Properties();
        props.put(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, "broker1.example.com:9093");

        // Encryption and authentication
        props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_SSL");
        props.put(SaslConfigs.SASL_MECHANISM, "SCRAM-SHA-512");
        props.put(SaslConfigs.SASL_JAAS_CONFIG, String.format(
                "org.apache.kafka.common.security.scram.ScramLoginModule required "
                        + "username=\"%s\" password=\"%s\";", username, password));

        // Trust the certificate authority that signed the broker certificates
        props.put(SslConfigs.SSL_TRUSTSTORE_TYPE_CONFIG, "PKCS12");
        props.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, "/etc/kafka/client.truststore.p12");
        props.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, System.getenv("KAFKA_TRUSTSTORE_PASSWORD"));
        return props;
    }
}
SecureOrderProducer.java 
package com.example.kafka;

import java.util.Properties;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

public class SecureOrderProducer {

    public static void main(String[] args) {
        Properties props = SecureClientConfig.forUser("order-service", System.getenv("KAFKA_PASSWORD"));
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);

        try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
            producer.send(new ProducerRecord<>("orders-topic", "ord-1001", "{\"amount\":250}"),
                    (metadata, exception) -> {
                        if (exception != null) {
                            System.err.println("Send failed: " + exception.getMessage());
                        } else {
                            System.out.println("Sent securely to partition " + metadata.partition());
                        }
                    });
        }
    }
}

The producer code itself does not change; only the connection settings do. A consumer uses the same helper with its own username, plus its deserializers and group.id.

9. Secure Kafka in Spring Boot

Spring Boot maps the same settings to application.yml. Secrets are read from environment variables:

application.yml 
spring:
  kafka:
    bootstrap-servers: broker1.example.com:9093
    security:
      protocol: SASL_SSL
    ssl:
      trust-store-location: file:/etc/kafka/client.truststore.p12
      trust-store-password: ${KAFKA_TRUSTSTORE_PASSWORD}
      trust-store-type: PKCS12
    properties:
      sasl.mechanism: SCRAM-SHA-512
      sasl.jaas.config: >-
        org.apache.kafka.common.security.scram.ScramLoginModule required
        username="billing-service" password="${KAFKA_PASSWORD}";
    consumer:
      group-id: billing-group

Producers, consumers and the admin client created by Spring all use these settings, so KafkaTemplate and @KafkaListener code stays exactly the same as without security.

10. Simple Example of Kafka Security

Imagine a company using Kafka for processing customer orders:

  • The Order Service produces messages to a topic called orders-topic.
  • The Billing Service consumes messages from orders-topic.

How Kafka security helps here

Security areaOrder ServiceBilling ServiceUnknown application
AuthenticationLogs in as User:order-serviceLogs in as User:billing-serviceRejected: no valid credentials
AuthorizationWrite to orders-topicRead from orders-topic with group billing-groupNo permissions
EncryptionOrder details encrypted on the networkOrder details encrypted on the networkCannot read captured traffic

If an unauthorized application tries to read or write data, Kafka blocks the request immediately. Even Billing Service cannot write fake orders, because it only has read permission.

11. Securing Broker and Controller Communication

Brokers copy data between each other, and in KRaft mode the controllers manage cluster metadata. Both kinds of traffic must be secured, because anyone who can join as a broker or controller can affect the whole cluster.

server.properties (internal traffic) 
# Brokers replicate over the secured listener
inter.broker.listener.name=SASL_SSL
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512

# Controllers use their own TLS listener
controller.listener.names=CONTROLLER
listener.security.protocol.map=SASL_SSL:SASL_SSL,CONTROLLER:SSL

Also limit network access: only brokers, controllers and approved clients should be able to reach the Kafka ports, using firewalls or network policies.

12. Protecting Data at Rest

Kafka encrypts data in transit, but it stores messages on disk unencrypted. Anyone with access to the broker’s disks or backups could read them. Common protections:

OptionWhat it protects
Disk or volume encryptionData on lost disks, stolen backups or decommissioned machines
Field-level encryption in the applicationSensitive fields, such as card numbers, stay encrypted even inside Kafka; only authorized consumers can decrypt them
Not sending sensitive dataSend a reference or masked value instead of the full personal data
Short retentionLimits how long sensitive data stays in Kafka

13. Client Quotas

Quotas stop one client from using so much network bandwidth or broker time that others slow down. They protect the cluster from both mistakes and misuse:

Set quotas for a user 
bin/kafka-configs.sh --bootstrap-server broker1.example.com:9093 \
  --command-config admin.properties \
  --alter --entity-type users --entity-name order-service \
  --add-config 'producer_byte_rate=1048576,consumer_byte_rate=2097152'

In this example, Order Service may produce up to 1 MB per second and consume up to 2 MB per second on each broker. Clients that exceed a quota are slowed down rather than disconnected.

14. Auditing and Monitoring

SourceWhat it tells you
Authorizer log (kafka.authorizer.logger)Denied requests are logged by default; allowed requests can be logged at debug level
Authentication metricsfailed-authentication-total and successful-authentication-total per listener
Broker logsTLS handshake failures, expired certificates and rejected logins
Certificate expiryTrack expiry dates and renew certificates well in advance

Send these logs and metrics to a central system and alert on sudden increases in failed logins or denied requests; they can point to misconfigured clients or attempted misuse.

15. Common Security Errors

ErrorLikely causeFix
SslAuthenticationExceptionClient does not trust the broker certificate, or the host name does not matchAdd the CA to the truststore; make sure the certificate includes the broker host name
SaslAuthenticationExceptionWrong username, password or mechanismCheck sasl.mechanism, the JAAS line and that the user exists
TopicAuthorizationExceptionMissing ACL on the topicGrant Write for producers or Read for consumers
GroupAuthorizationExceptionConsumer lacks permission on its groupGrant Read on the consumer group
Client hangs or disconnectsClient uses PLAINTEXT against a TLS listener, or the wrong portMatch security.protocol and port to the broker listener

16. Why Kafka Security Is Important

  • Protects sensitive business and customer data.
  • Prevents unauthorized access and misuse, including accidental deletion of topics.
  • Ensures compliance with security standards and data protection laws.
  • Limits damage if one application or password is compromised.
  • Builds trust in data-driven systems.

17. Best Practices

DoAvoid
Use SASL_SSL or mTLS on every listener in productionRunning PLAINTEXT outside local development
Give each application its own userSharing one user across many services
Grant only the permissions each application needsMaking applications super users
Keep allow.everyone.if.no.acl.found=falseLetting everyone in when no ACL matches
Keep host name verification enabledDisabling certificate checks to “fix” connection errors
Store passwords in a secrets manager or environment variablesHard-coding passwords in code or committing them to Git
Rotate passwords and renew certificates regularlyLetting certificates expire and break the cluster
Secure broker and controller traffic tooSecuring only the client listener
Monitor failed logins and denied requestsFinding out about problems only after an incident

18. Summary

Kafka security is essential for running Kafka safely in real-world applications. TLS encrypts traffic between clients, brokers and controllers; authentication with SCRAM, mTLS, OAuth or Kerberos proves who each client is; and ACLs limit what each authenticated user can do. Quotas, logs and metrics add control and visibility, and disk or field-level encryption protects stored data, which Kafka itself does not encrypt.

By using authentication, authorization and encryption together, Kafka provides a strong security foundation for modern distributed systems. Whether Kafka is used in small applications or large enterprises, proper security configuration keeps data safe and systems reliable.