Technology sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Auctor Lewei Community (forum.lwops.cn) Xu Yuan
Cum aedificatio moderna systemata distribuit, securitatis transmissionis notitiarum pendet. Apache Kafka et Zookeeper, ut popularis nuntius distributus queues et officia coordinationis, praebent SSL (Secure Sockets Layer) mechanismum authenticas augendae securitatis in notitia tradenda.
Hic articulus singillatim totum processum introducet ex certificatorio SSL generando ad configurandum servitorem et clientem ut notitia in transmissione plene muniatur.

Kafka 1. Configure rationem password:
1. Primum, lima configurationem kafka mutare debes: vim /asop/kafka/kafka_2.11-2.1.0/config/server.properties

broker.id = 0
auditores=SASL_PLAINTEXT/:9092
advertised.listeners=SASL_PLAINTEXT/10.176.31.137:9092
num.network.threads = 3
num.io.threads = 8
socket.send.buffer.bytes = 102400
socket.receive.buffer.bytes = 102400
socket.request.max.bytes=104857600
log.dirs=/asop/kafka/logs
num.partitions = 1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor = 1
transaction.state.log.min.isr=1
log.retention.hours = 168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0

#Authentication protocol usus
security.inter.broker.protocol=SASL_PLAINTEXT
#SASLmechanism
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
#Class ad perficiendam authenticas
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
#Si nulla ACL (inscriptio ditionis accessus) figuratio invenitur, aliqua operatio admittitur.
allow.everyone.if.no.acl.found falsus
#Propositum super administratorem debes efficere ac visitator utentis eximius administrator.
super.users=User: visitor

2. Secundo, tabellam verificationis login pro servo crea. Potes tabellam nominare secundum optionis tuae, ut vim /asop/kafka/kafka_2.11-2.1.0/config/kafka_server_jaas.conf sequitur

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username = "visitator"
password="qaz@123"
user_visitor="qaz@123";
};

3. Dein modificare kafka directorii institutionis vim /asop/kafka/kafka_2.11-2.1.0/bin/kafka-server-start.sh et adde variabiles in summa tabella

exportare KAFKA_OPTS="-Djava.security.auth.login.config=/asop/kafka/kafka_2.11-2.1.0/config/kafka_server_jaas.conf"

4. Dein, creare verificationem login pro consumendi et producentis. Potes tabellam nominare secundum optionis tuae, ut kafka_client_jaas.conf. Contentum tabellae hoc modo est (si accessus programmatis est, ut vernacula accessus" , non felis eget.
vim / asop/kafka/kafka_2.11-2.1.0

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username = "visitator"
password="qaz@123";
};

5. Adde sequentes conformationes ad consumendi. proprietates et producer.properties respective:
vim
vim /asop/kafka/kafka_2.11-2.1.0/config/producer.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

6. Mutare kafka directoria institutionis bin/kafka-console-producer.sh et bin/kafka-console-consumer.sh, et variabiles in summa tabulariorum adde.
vim / asop/kafka/kafka_2.11-2.1.0/
vim / asop/kafka/kafka_2.11-2.1.0/

exportare KAFKA_OPTS="-Djava.security.auth.login.config=/asop/kafka/kafka_2.11-2.1.0/config/kafka_client_jaas.conf"
7. Satus zookeeper et kafka respectively. Hoc in loco, ministri lateris kafka verificationis usoris login perficitur (primum prope kafka et deinde zookeeper)

Munus inclusi sunt kafka
/asop/kafka/kafka_2.11-2.1.0/bin/kafka-server-stop.sh -daemon

Satus officium kafka
#/asop/kafka/kafka_2.11-2.1.0/bin/kafka-server-start.sh -daemon /asop/kafka/kafka_2.11-2.1.0/config/server.properties

Clausa servitium zookeeper-3.4.13
/asop/zk/zookeeper-3.4.13/bin/zkServer.sh stop /asop/zk/zookeeper-3.4.13/conf/zoo.cfg
Incipit officium zookeeper-3.4.13
/asop/zk/zookeeper-3.4.13/bin/zkServer.sh start /asop/zk/zookeeper-3.4.13/conf/zoo.cfg

8. crea ac visum themata

/asop/kafka/kafka_2.11-2.1.0/bin/kafka-console-producer.sh --broker-list 10.176.31.137:9092 res sasl.mechanism = PLAIN

accipere nuntiis

/asop/kafka/kafka_2.11-2.1.0/bin/kafka-console-consumer.sh --bootstrap-server 10.176.31.137:9092 --topic cmdb --ab initio SASL_PLAINTEXT -- consumptio proprietas sasl.mechanism=PLAIN

2. Configure ssal propter password pro zk et kafka;

1. Zookeeper configuration SASL
   1.1 Novam file zoo_jaas.conf
   Nullae speciales requisitiones sunt pro tabella nominis et tramite tabella zoo_jaas.conf. Communiter posita in ${ZOOKEEPER_HOME}/conf directorium vim /asop/zk/zookeeper-3.4.13/conf/zoo_jaas.conf.

Servo {
org.apache.kafka.common.security.plain.PlainLoginModule required
username = "admin"
password="admin@12"
user_kafka="kafka@123";
};

    Server.username、Server.password为 Zookeeper 内部通信的用户名和密码，因此保证每个 zk 节点该属性一致即可
    Server.user_xxx 中 xxx 为自定义用户名，用于 zkClient 连接所使用的用户名和密码，即为 kafka 创建的用户名
1
2

1.2 configurare fasciculum /asop/zk/zookeeper-3.4.13/conf/zoo.cfg
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme = sasl
jaasLoginRenew=3600000
zookeeper.sasl.client = true

Zookeeper.sasl.client verum positum est ut clientis authenticas enable.

Import dependentiam packages 1.3
Quia permissio verificationis usus classis est: org.apache.kafka.common.security.plain.PlainLoginModule, kafka-related fasciculi vasorum desiderantur. lib zookeeper et recenti directorio zk_sasl_lib creato:

kafka-clientes-2.4.1.jar
lz4-java-1.6.0.jar
slf4j-api-1.7.28.jar
slf4j-log4j12-1.7.28.jar
snappy-java-1.1.7.3.jar

mkdir /asop/zk/zookeeper-3.4.13/zk_sasl_lib
cp /asop/kafka/kafka_2.11-2.1.0/libs/kafka-clients-2.1.0.jar /asop/zk/zookeeper-3.4.13/lib/
cp /asop/kafka/kafka_2.11-2.1.0/libs/lz4-java-1.5.0.jar /asop/zk/zookeeper-3.4.13/lib/
cp /asop/kafka/kafka_2.11-2.1.0/libs/slf4j-api-1.7.25.jar /asop/zk/zookeeper-3.4.13/lib/
cp /asop/kafka/kafka_2.11-2.1.0/libs/slf4j-log4j12-1.7.25.jar /asop/zk/zookeeper-3.4.13/lib/
cp /asop/kafka/kafka_2.11-2.1.0/libs/snappy-java-1.1.7.2.jar /asop/zk/zookeeper-3.4.13/lib/
cp /asop/kafka/kafka_2.11-2.1.0/libs/kafka-clients-2.1.0.jar /asop/zk/zookeeper-3.4.13/zk_sasl_lib
cp /asop/kafka/kafka_2.11-2.1.0/libs/lz4-java-1.5.0.jar /asop/zk/zookeeper-3.4.13/zk_sasl_lib
cp /asop/kafka/kafka_2.11-2.1.0/libs/slf4j-api-1.7.25.jar /asop/zk/zookeeper-3.4.13/zk_sasl_lib
cp /asop/kafka/kafka_2.11-2.1.0/libs/slf4j-log4j12-1.7.25.jar /asop/zk/zookeeper-3.4.13/zk_sasl_lib
cp /asop/kafka/kafka_2.11-2.1.0/libs/snappy-java-1.1.7.2.jar /asop/zk/zookeeper-3.4.13/zk_sasl_lib

chmod 755 -R /asop/zk/zookeeper-3.4.13/zk_sasl_lib/
chmod 755 -R /asop/zk/zookeeper-3.4.13/zk_sasl_lib/

1.4 Mutare zkEnv.sh file/asop/zk/zookeeper-3.4.13/bin/zkEnv.sh
Ante modificationem: Si non, directe illud adde

export SERVER_JVMFLAGS="-Xmx${ZK_SERVER_HEAP}m $SERVER_JVMFLAGS"

Post modificationem:

for dolium in /asop/zk/zookeeper-3.4.13/zk_sasl_lib/*.jar;
do
CLASSPATH ="$jar*:$CLASSPATH"
factum

export SERVER_JVMFLAGS " -Djava.security.auth.login.config=/asop/zk/zookeeper-3.4.13/conf/zoo_jaas.conf"

Just sileo Zookeeper officium

Clausa servitium zookeeper-3.4.13
/asop/zk/zookeeper-3.4.13/bin/zkServer.sh stop /asop/zk/zookeeper-3.4.13/conf/zoo.cfg
Incipit officium zookeeper-3.4.13
/asop/zk/zookeeper-3.4.13/bin/zkServer.sh start /asop/zk/zookeeper-3.4.13/conf/zoo.cfg

2. Kakfa configurare SASL
   2.1 Novam kafka_server_jaas.conf file
   Nullae requiruntur ad kafka_server_jaas.conf nomen fasciculi et iter repositionis. Plerumque in ${KAFKA_HOME}/config directorium/asop/kafka/kafka_2.11-2.1.0/config/kafka_server_jaas.conf

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username = "visitator"
password="qaz@123"
user_visitor="qaz@123";
};
Client{
org.apache.kafka.common.security.plain.PlainLoginModule required
username = "kafka"
password="kafka@123";
};

    KafkaServer.username、KafkaServer.password 为 broker 内部通信的用户名密码，同上
1

KafkaServer.user_xxx ubi xxx consentaneum esse debet cum nomine usoris in KafkaServer.username conformatus, et tessera quoque constare debet
KafkaServer.user_producer et KafkaServer.user_consumer praeparant subsequenti ACL, ut consumentes et effectores diversis rationibus utantur, et rationes usarum solum notitias consumere possunt, et rationum effector notitias tantum producere possunt.
Client.username et Client.password imple tesserae rationis in Zookeeper descripti, quae pro communicatione inter sectorem et zookeeper adhibita est (si zookeeper cum SASL conformatus non est, dissimulari potest. Si zookeeper.sasl.client falsum est, potest. etiam praetermittendum est.

[2021-06-29 17:14:30,204] PRAEMONITIO SASL configurationem defecerunt: javax.security.auth.login.LoginException: Nulla JAAS sectionis configurationis nomine 'Client' inventa est in file configurationis JAAS definiti: '/Users/wjun/env /kafka/config/kafka_server_jaas.conf'. Vinculum perget servo Zookeeper sine SASL authenticas, si Servus Zookeeper id permittit. (org.apache.zookeeper.ClientCnxn)

2.2 Mutare file server.properties
broker.id = 0
auditores=SASL_PLAINTEXT/:9092
advertised.listeners=SASL_PLAINTEXT/192.168.157.198:9092
num.network.threads = 3
num.io.threads = 8
socket.send.buffer.bytes = 102400
socket.receive.buffer.bytes = 102400
socket.request.max.bytes=104857600
log.dirs=/asop/kafka/logs
num.partitions = 1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor = 1
transaction.state.log.min.isr=1
log.retention.hours = 168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=27.0.0.1:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0

#Authentication protocol usus
security.inter.broker.protocol=SASL_PLAINTEXT
#SASLmechanism
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
#Class ad perficiendam authenticas
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
#Si nulla ACL (inscriptio ditionis accessus) figuratio invenitur, aliqua operatio admittitur.
allow.everyone.if.no.acl.found falsus
#Propositum super administratorem debes efficere ac visitator utentis eximius administrator.
super.users=User: visitor

Ubi localhost mutari debet ad inscriptio IP

super.usores eximii usoris configurat qui non ACL configurationibus subsequentibus afficitur.

2.3 Modify startup scriptor
Mutare fasciculum kafka-server-start.sh ut eam load in kafka_server_jaas.conf file/asop/kafka/kafka_2.11-2.1.0/bin/kafka-server-start.sh

ante determinatio certae partis:

si ["x$KAFKA_HEAP_OPTS" = "x" ]; tunc "
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi**

Post modificationem:
(Adde hanc lineam primae lineae primae. Si iam habes, addere non debes.) KAFKA_OPTS " -Djava.security.auth.login.config=/asop/kafka/kafka_2.11- 2.1.0/config/kafka_server_jaas.conf "
si ["x$KAFKA_HEAP_OPTS" = "x" ]; tunc "
exportare KAFKA_HEAP_OPTS="-Xmx1G -Xms1G -Djava.security.auth.login.config=/asop/kafka/kafka_2.11-2.1.0/config/kafka_server_jaas.conf"
fi**

Set ACL praecepta pro zookeeper
/asop/zk/zookeeper-3.4.13/bin/zkCli.sh #Enter zk imperium modus

addauth digestum admin:admin@12 #Switch login user (eximius administrator est in zk configuratione file/asop/zk/zookeeper-3.4.13/conf/zoo_jaas.conf)

setAcl / ip:127.0.0.1: cdrwa,auth:kafka:kafka@123:cdrwa #(Pone IP tesseram rationis usoris quae initium fieri potest. Administrator est administrator in zk configurationem lima supra definitum et Kafka usor est /asop The kafka nexus zk definito usuario in /kafka/kafka_2.11-2.1.0/config/kafka_server_jaas.conf lima (sub Client))

addauth digestum kafka: kafka@123 #Switch to kafka user and set acl again
setAcl/ip:127.0.0.1:cdrwa,auth:kafka:kafka@123:cdrwa

Nota: Si album album IP vel usorem addere vis in fundamento originali debet addere, alioquin suprascriptum erit.
setAcl/ip:127.0.0.1:cdrwa,auth:kafka:kafka@123:cdrwa,auth:admin:admin@12:cdrwa,ip:1.1.1.1

Renovare permissiones necesse est, sine occasu acl currere
setAcl / world : any : cdrwa

Iustus sileo in kafka ministerium

Munus inclusi sunt kafka
.

Satus officium kafka
#/asop/kafka/kafka_2.11-2.1.0/bin/kafka-server-start.sh -daemon /asop/kafka/kafka_2.11-2.1.0/config/server.properties

Hoc loco SSL figura authenticas kafka et zookeeper completur. Plures operationes et artes sustentandae, quaeso operam in communitate Lewei.