Linux下kafka集群安装配置

大家好，又见面了，我是你们的朋友全栈君。

下载kafka

https://kafka.apache.org/downloads#2.3.1

解压

tar -zxvf kafka_2.12-2.3.1.tgz

配置server.xml

cd /usr/local/kafka/kafka_2.12-2.3.1/config

vi server.xml

本次设置三个节点，配置内容如下

192.168.0.1

broker.id=0

listeners=PLAINTEXT://192.168.0.1:9092

host.name=192.168.0.1

zookeeper.connect=192.168.0.1:2181,192.168.0.2:2181,192.168.0.3:2181

log.dirs=/tmp/kafka-logs1

192.168.0.2

broker.id=1

listeners=PLAINTEXT://192.168.0.2:9092

host.name=192.168.0.2

zookeeper.connect=192.168.0.1:2181,192.168.0.2:2181,192.168.0.3:2181

log.dirs=/tmp/kafka-logs1

192.168.0.3

broker.id=2

listeners=PLAINTEXT://192.168.0.3:9092

host.name=192.168.0.3

zookeeper.connect=192.168.0.1:2181,192.168.0.2:2181,192.168.0.3:2181

log.dirs=/tmp/kafka-logs1

配置myid文件

在kafka目录下新建zookeeper文件夹，用于放myid文件

cd /usr/local/kafka/kafka_2.12-2.3.1/

mkdir zookeeper

touch myid

vi myid

在新创建的zookeeper目录下新建myid文件，每个节点对应的myid内容见下。注意myid不能为0

192.168.0.1

1

192.168.0.2

2

192.168.0.3

3

配置zookeeper.properties文件

dataDir=/usr/local/kafka/kafka_2.12-2.3.1/zookeeper

tickTime=2000

initLimit=10

syncLimit=5

server.1=192.168.0.1:2888:3888

server.2=192.168.0.2:2888:3888

server.3=192.168.0.3:2888:3888

启动zookeeper

cd /usr/local/kafka/kafka_2.12-2.3.1/\rbin/zookeeper-server-start.sh -daemon config/zookeeper.properties &\r

启动kafka

cd /usr/local/kafka/kafka_2.12-2.3.1/bin\r ./kafka-server-start.sh -daemon ../config/server.properties &\r

常用报错处理：

启动报错

nohup: failed to run command `/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el6_8.x86_64/bin/java’: No such file or directory

配置jdk/jre路径

vi kafka-run-class.sh

# Which java to use

if [ -z “$JAVA_HOME” ]; then

JAVA=”java”

else

JAVA=”/usr/local/java/jdk1.8.0_161/bin/java”

fi

myid如果设值为0则报下面的错误

java.lang.RuntimeException: My id 0 not in the peer list

broke.id冲突

改了节点的broke.id或同一个节点部了两个kafka，可能会报这个错，可以将server.xml中的log.dirs重新指定一下即可，如改成：

log.dirs=/tmp/kafka-logs2

meta.properties的位置示例见

/tmp/kafka-logs/meta.properties

腾讯云节点配置集群需将各个节点加到安全组里面

Linux下kafka集群安装配置

发布者：全栈程序员栈长，转载请注明出处：https://javaforall.cn/141907.html原文链接：https://javaforall.cn