ubuntu下安装zookeeper和kafka伪集群

阿里云国内75折 回扣 微信号：monov8
阿里云国际，腾讯云国际，低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号：monov8 飞机：@monov6


1.创建目录
mkdir -p /zookeeper/zkp-1 /zookeeper/zkp-2 /zookeeper/zkp-3
2.下载zookeeper
链接地址：​​​http://mirror.bit.edu.cn/apache/zookeeper/​​​ 命令下载：
cd /zookeeper/zkp-1
wget ​​http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.5.5/apache-zookeeper-3.5.5-bin.tar.gz​​ 3.解压
tar -zxvf apache-zookeeper-3.5.5-bin.tar.gz
复制到zkp-2和zkp-3
cp -R /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin /zookeeper/zkp-2
cp -R /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin /zookeeper/zkp-3
4.修改配置文件
cd /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin/conf
vim zoo.cfg

tickTime=2000
dataDir= /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin/data
clientPort=2181
initLimit=5
syncLimit=2
server.1=192.168.0.79:2888:3888
server.2=192.168.0.79:4888:5888
server.3=192.168.0.79:6888:7888

cd /zookeeper/zkp-2/apache-zookeeper-3.5.5-bin/conf
vim zoo.cfg

tickTime=2000
dataDir=/zookeeper/zkp-2/apache-zookeeper-3.5.5-bin/data
clientPort=2182
initLimit=5
syncLimit=2
server.1=192.168.0.79:2888:3888
server.2=192.168.0.79:4888:5888
server.3=192.168.0.79:6888:7888

cd /zookeeper/zkp-3/apache-zookeeper-3.5.5-bin/conf
vim zoo.cfg

tickTime=2000
dataDir=/zookeeper/zkp-3/apache-zookeeper-3.5.5-bin/data
clientPort=2183
initLimit=5
syncLimit=2
server.1=192.168.0.79:2888:3888
server.2=192.168.0.79:4888:5888
server.3=192.168.0.79:6888:7888

5.创建数据目录
mkdir -p /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin/data
mkdir -p /zookeeper/zkp-2/apache-zookeeper-3.5.5-bin/data
mkdir -p /zookeeper/zkp-3/apache-zookeeper-3.5.5-bin/data
6.写入myid文件
cd /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin/data
echo ‘1’ > myid
cd /zookeeper/zkp-2/apache-zookeeper-3.5.5-bin/data
echo ‘2’ > myid
cd /zookeeper/zkp-3/apache-zookeeper-3.5.5-bin/data
echo ‘3’ > myid
7.启动，三个都执行
cd /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin/bin
./zkServer.sh start
cd /zookeeper/zkp-2/apache-zookeeper-3.5.5-bin/bin
./zkServer.sh start
cd /zookeeper/zkp-3/apache-zookeeper-3.5.5-bin/bin
./zkServer.sh start

打印日志启动：./zkServer.sh start-foreground
8.查看jps结果
jps
8099 Jps
6808 QuorumPeerMain
6746 QuorumPeerMain
8014 QuorumPeerMain

kafka伪集群安装

1.创建目录
mkdir -p /kafka/kafka-1 /kafka/kafka-2 /kafka/kafka-3
2.下载kafka
链接:​​​http://kafka.apache.org/downloads​​​ cd /kafka/kafka-1
wget ​​http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.2.1/kafka_2.12-2.2.1.tgz​​ 3.解压
tar -zxvf kafka_2.12-2.2.1.tgz
复制到kafka-2和kafka-3
cp -R /kafka/kafka-1/kafka_2.12-2.2.1 /kafka/kafka-2/kafka_2.12-2.2.1
cp -R /kafka/kafka-1/kafka_2.12-2.2.1 /kafka/kafka-3/kafka_2.12-2.2.1
4.修改配置文件
cd /kafka/kafka-1/kafka_2.12-2.2.1/config
vim server.properties
(listeners=PLAINTEXT://wzj-ThinkPad-Edge-E440:9092//这个参数非常重要，想要外网访问docker内的集群，只能配置主机名，不能配置IP地址，命令hostname, ​​配置文件时注意参数broker.id​​,log.dirs,zookeeper.connect)

broker.id=1
listeners=PLAINTEXT://192.168.0.79:9092   
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/kafka/kafka-1/kafka_2.12-2.2.1/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.0.79:2181,192.168.0.79:2182,192.168.0.79:2183
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
# 可删除topic
delete.topic.enable=true
# 每间隔1秒钟时间，刷数据到磁盘
log.flush.interval.ms=1000
# 保留三天，也可以更短 
log.retention.hours=72

cd /kafka/kafka-2/kafka_2.12-2.2.1/config
vim server.properties

broker.id=2
listeners=PLAINTEXT://192.168.0.79:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/kafka/kafka-2/kafka_2.12-2.2.1/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.0.79:2181,192.168.0.79:2182,192.168.0.79:2183
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
# 可删除topic
delete.topic.enable=true
# 每间隔1秒钟时间，刷数据到磁盘
log.flush.interval.ms=1000
# 保留三天，也可以更短 
log.retention.hours=72

cd /kafka/kafka-3/kafka_2.12-2.2.1/config
vim server.properties

broker.id=3
listeners=PLAINTEXT://192.168.0.79:9094
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/kafka/kafka-3/kafka_2.12-2.2.1/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.0.79:2181,192.168.0.79:2182,192.168.0.79:2183
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
# 可删除topic
delete.topic.enable=true
# 每间隔1秒钟时间，刷数据到磁盘
log.flush.interval.ms=1000
# 保留三天，也可以更短 
log.retention.hours=72

4.启动，三个都执行
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-server-start.sh /kafka/kafka-1/kafka_2.12-2.2.1/config/server.properties &

cd /kafka/kafka-2/kafka_2.12-2.2.1
bin/kafka-server-start.sh /kafka/kafka-2/kafka_2.12-2.2.1/config/server.properties &

cd /kafka/kafka-3/kafka_2.12-2.2.1
bin/kafka-server-start.sh /kafka/kafka-3/kafka_2.12-2.2.1/config/server.properties &

后台启动方式:
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-server-start.sh -daemon /kafka/kafka-1/kafka_2.12-2.2.1/config/server.properties

cd /kafka/kafka-2/kafka_2.12-2.2.1
bin/kafka-server-start.sh -daemon /kafka/kafka-2/kafka_2.12-2.2.1/config/server.properties

cd /kafka/kafka-3/kafka_2.12-2.2.1
bin/kafka-server-start.sh -daemon /kafka/kafka-3/kafka_2.12-2.2.1/config/server.properties

netstat lntp|grep 9092
5.测试kafka集群
#创建topic
cd /kafka/kafka-1/kafka_2.12-2.2.1

bin/kafka-topics.sh --create --zookeeper 192.168.0.79:2181,192.168.0.79:3181,192.168.0.79:4181 --replication-factor 3 --partitions 3 --topic test
#显示topic信息
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-topics.sh --describe --zookeeper 192.168.0.79:2181,192.168.0.79:3181,192.168.0.79:4181 --topic test
#列出topic
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-topics.sh --list --zookeeper 192.168.0.79:2181,192.168.0.79:3181,192.168.0.79:4181
#创建 producer
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-console-producer.sh --broker-list master:9092 -topic test
#创建 consumer
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-console-consumer.sh --zookeeper 192.168.0.79:2181,192.168.0.79:3181,192.168.0.79:4181 -topic test --from-beginning
#删除topic
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-topics.sh --delete --zookeeper 192.168.0.79:2181,192.168.0.79:3181,192.168.0.79:4181 --topic test
#查询topic内容
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topicName --from-beginning
6.关闭kafka
cd /kafka/kafka-1/kafka_2.12-2.2.1
bin/kafka-server-stop.sh /kafka/kafka-1/kafka_2.12-2.2.1/config/server.properties

cd /kafka/kafka-2/kafka_2.12-2.2.1
bin/kafka-server-stop.sh /kafka/kafka-2/kafka_2.12-2.2.1/config/server.properties

cd /kafka/kafka-3/kafka_2.12-2.2.1
bin/kafka-server-stop.sh /kafka/kafka-3/kafka_2.12-2.2.1/config/server.properties
7.关闭zookeeper
cd /zookeeper/zkp-1/apache-zookeeper-3.5.5-bin
bin/zkServer.sh stop conf/zoo.cfg

cd /zookeeper/zkp-2/apache-zookeeper-3.5.5-bin
bin/zkServer.sh stop conf/zoo.cfg

cd /zookeeper/zkp-3/apache-zookeeper-3.5.5-bin
bin/zkServer.sh stop conf/zoo.cfg


阿里云国内75折 回扣 微信号：monov8
阿里云国际，腾讯云国际，低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号：monov8 飞机：@monov6
标签: Ubuntu
返回列表

上一篇：Kettle 连接达梦数据库

下一篇：ubuntu下安装kafka集群connector

“ubuntu下安装zookeeper和kafka伪集群” 的相关文章

ubuntu 10/12 apt-get update失败1年前 (2023-02-02)
Ubuntu20.04+GLFW搭建OpenGL开发环境1年前 (2023-02-02)
ubuntu 安装微信（微信官方版本）_微信ubuntu版1年前 (2023-02-03)
ubuntu 下的opencv3的下载与实现简单功能1年前 (2023-02-03)
Ubuntu玩机记录，让我破电脑又飞起来了1年前 (2023-02-05)
【DIY小记】VMWare设置主机连接到的Ubuntu虚拟机的网络端口1年前 (2023-02-05)
ubuntu查看系统版本的命令1年前 (2023-02-05)
Ubuntu20.4下安装TensorFlow2.x1年前 (2023-02-05)
解决Ubuntu系统无法通过SSH连接和使用快捷键复制粘贴问题1年前 (2023-02-07)
BEGIN failed–compilation aborted at 报错、.pm 文件处理、yum、下载.pm模块【ubuntu】1年前 (2023-02-07)

阿里云国际版