Kafka installation and setup

 Kafka installation and setup

Once you unzipped Kafka tar create below data directory:

Datadir:
================
/u03/kafkadata1
/u03/kafkadata2
/u03/kafkadata3

Create PID 
================
echo 1 > /u03/kafkadata1/myid
echo 2 > /u03/kafkadata2/myid
echo 3 > /u03/kafkadata3/myid
Modify zookeeper1.properties,zookeeper2.properties and zookeeper3.properties
=================

[root@kafkaserv config]# pwd
/u03/kafka/config
[root@kafkaserv config]# vi zookeeper1.properties
[root@kafkaserv config]# vi zookeeper2.properties
[root@kafkaserv config]# vi zookeeper3.properties
add below lines
tickTime=2000
initLimit=5
syncLimit=2
server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889
maxClientCnxns=0

Then start zookeeper services:
==============================
[root@kafkaserv bin]# ./zookeeper-server-start.sh -daemon /u03/kafka/config/zookeeper1.properties
[root@kafkaserv bin]# ./zookeeper-server-start.sh -daemon /u03/kafka/config/zookeeper2.properties
[root@kafkaserv bin]# ./zookeeper-server-start.sh -daemon /u03/kafka/config/zookeeper3.properties

Check if services came up using
===============================
[root@kafkaserv bin]# nc -v localhost 2181
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to ::1:2181.
^C
[root@kafkaserv bin]#
[root@kafkaserv bin]# nc -v localhost 2182
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to ::1:2182.
^C
[root@kafkaserv bin]#
[root@kafkaserv bin]# nc -v localhost 2183
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to ::1:2183.
^C

Optional, if you wanted to start using OS service command then create below service:
============================
cd /etc/systemd/system
vi zookeeper1.service
[Unit]
Description=Zookeeper1 Service
[Service]
Type=simple
WorkingDirectory=/u03/kafkadata1
PIDFile=/u03/kafkadata1/myid
SyslogIdentifier=zookeeper1
User=root
Group=root
ExecStart=/u03/kafka/bin/zookeeper-server-start.sh /u03/kafka/config/zookeeper1.properties
ExecStop=/u03/kafka/bin/zookeeper-server-stop.sh
Restart=always
TimeoutSec=20
SuccessExitStatus=130 143
Restart=on-failure
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable zookeeper1.service
[root@kafkaserv system]# systemctl enable zookeeper1.service
ln -s '/etc/systemd/system/zookeeper1.service' '/etc/systemd/system/multi-user.target.wants/zookeeper1.service'
[root@kafkaserv system]#
[root@kafkaserv system]# systemctl enable zookeeper1.service
ln -s '/etc/systemd/system/zookeeper1.service' '/etc/systemd/system/multi-user.target.wants/zookeeper1.service'
[root@kafkaserv system]# systemctl status zookeeper1.service
zookeeper1.service - Zookeeper1 Service
   Loaded: loaded (/etc/systemd/system/zookeeper1.service; enabled)
   Active: inactive (dead)
[root@kafkaserv system]# systemctl start zookeeper1.service
[root@kafkaserv system]# systemctl status zookeeper1.service
zookeeper1.service - Zookeeper1 Service
   Loaded: loaded (/etc/systemd/system/zookeeper1.service; enabled)
   Active: active (running) since Sun 2022-12-04 12:29:22 IST; 3s ago
 Main PID: 24948 (java)
   CGroup: /system.slice/zookeeper1.service
           └─24948 java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+Ex...
Dec 04 12:29:22 kafkaserv.nsdr.com systemd[1]: Started Zookeeper1 Service.
Modify /u03/kafka/config/server.properties file 
===============================
Check if any error is there:
[root@kafkaserv bin]# ./kafka-server-start.sh /u03/kafka/config/server.properties
If above commands completed find then run it from backend
[root@kafkaserv bin]# ./kafka-server-start.sh -daemon /u03/kafka/config/server.properties
[root@kafkaserv bin]#

===============================
Create a TOPIC
=================================
[root@kafkaserv bin]# ./kafka-topics.sh --bootstrap-server 192.168.137.76:9092  --create --topic BroadcastTopic --replication-factor 1-partitions 1
Created topic BroadcastTopic.
[root@kafkaserv bin]#
[root@kafkaserv bin]# ./kafka-topics.sh --list --bootstrap-server 192.168.137.76:9092                                  BroadcastTopic
BroadcastTopic1

Produce and Consume Msg
===============================
[root@kafkaserv bin]# ./kafka-console-producer.sh --broker-list 192.168.137.76:9092 --topic BroadcastTopic
>Apple
>Oranage
>Lemon
>Grapes
[root@kafkaserv bin]# ./kafka-console-consumer.sh --bootstrap-server 192.168.137.76:9092 --topic BroadcastTopic --from-beginning
Apple
Oranage
Lemon
Grapes



Thanks,