[Kafka] Kafka producer reports an error after enabling idempotence: Cluster authorization failed._developdoc

[Kafka] Kafka producer reports an error after enabling idempotence: Cluster authorization failed.

2024-07-11

Table of Contents

background
solve
- Server Configuration
- ACL Add Authorization

background

To meet user business needs, the producer's idempotence needs to be enabled. The producer adds the configuration: enable.idempotence = true
The cluster used by the user has ACL authentication enabled: SASL_PLAINTEXT/SCRAM-SHA-512
When a user produces a message, an error message is displayed: org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.

solve

Server Configuration

Looking at the Kafka source code, I found that the producer configuration (enable.idempotence) has a description:

When set to 'true', the producer will ensure that exactly one copy of each message is written in the stream. If 'false', producer retries due to broker failures, etc., may write duplicates of the retried message in the stream. Note that enabling idempotence requires

Personal profile

I have devoted myself to the research of technology for more than 30 years. I am proficient in various languages such as Java, Linux, JavaScript, PHP, CSS, etc. I have made many contributions in the field of open source. I have established a developer documentation site to share some problems in technology development for everyone to read.

my contact information

Mail[email protected]