Zookeeper Setup

  1. Create a file
    /var/zookeeper/myid
    on each cluster node with unique value (for example 1,2,3) on each node.
    mkdir -p /var/zookeeper echo "1" > /var/zookeeper/myid
  2. Edit
    zookeeper.properties
    file under
    <KAFKA_HOME>/config directory
    .
    dataDir=/var/zookeeper clientPort=2181 maxClientCnxns=0 tickTime=2000 initLimit=5 syncLimit=10 dataLogDir=/var/log/zookeeper server.1=<KAFKA_CLUSTER_HOST1_IPADDRESS>:3181:4181 server.2=<KAFKA_CLUSTER_HOST2_IPADDRESS>:3181:4181 server.3=<KAFKA_CLUSTER_HOST3_IPADDRESS>:3181:4181 authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider authProvider.2=org.apache.zookeeper.server.auth.SASLAuthenticationProvider authProvider.3=org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme=sasl
    2181 is the port on which Zookeeper will listen for connections. Instead of 3181 and 4181, any port number can be used, but ensure same ports are used across all 3 nodes.
  3. Create a new
    zookeeper_jaas.conf
    file under
    <KAFKA_HOME>/config
    directory with below content:
    Server { org.apache.zookeeper.server.auth.DigestLoginModule required user_super="zookeeper" user_kafka="zoo-pwd"; };
    The file zookeeper_jaas.conf is used for authentication.
    user_super: superuser will have automatically administrator privileges.
    user_kafka: kafka is the username and password is zoo-pwd. The user kafka and password can be anything.
  4. Start Zookeeper server by running below command:
    KAFKA_OPTS="-Djava.security.auth.login.config=<KAFKA_HOME>/config/zookeeper_jaas.conf" <KAFKA_HOME>/bin/zookeeper-server-start.sh -daemon <KAFKA_HOME>/config/zookeeper.properties
  5. Check the status of Zookeeper service by running any the below command:
    a. ps -aef | grep zookeeper or ps -aef | grep zookeeper.properties b. lsof -i:2181 c. netstat -tnlup | grep 2181