Apache ZooKeeper 3.9 on Ubuntu 24.04 on Azure User Guide

Overview

Apache ZooKeeper is the industry-standard distributed coordination service — service registry, leader election, distributed locks, configuration store, and the backing store for systems like Kafka (legacy mode), HBase, NiFi, Druid, and SolrCloud. The cloudimg image installs ZooKeeper 3.9.5 from the official Apache CDN with SHA-512 verification, alongside OpenJDK 17. The cluster runs as a single-node standalone ensemble. Per-VM cloudimg credentials are generated at first boot and used for both ZK's native digest ACL scheme AND the ZooNavigator 2.0.0 Web UI bundled in a Docker container behind an nginx HTTP basic auth wall on TCP 80.

What is included:

• Apache ZooKeeper 3.9.5 at /opt/zookeeper
• OpenJDK 17 JRE headless from Ubuntu 24.04 noble main
• Standalone single-node ensemble: zookeeper.service (one JVM, both server + client roles)
• ZK client port 2181 (TCP), admin server 127.0.0.1:8080 (loopback only)
• 4LW commands enabled (stat, ruok, conf, mntr, srvr, etc.)
• ZooNavigator 2.0.0 Docker container bound to 127.0.0.1:9000
• nginx reverse proxy on TCP 80 with HTTP basic auth (single per-VM cloudimg credential covers nginx auth + ZooNav container auth + ZK digest ACLs)
• Heap tuned for Standard_B2s (256m–512m JVM)
• Credentials at /stage/scripts/zookeeper-credentials.log
• 24/7 cloudimg support

Prerequisites

Active Azure subscription, SSH key, VNet + subnet. Standard_B2s (4 GB RAM) is suitable for dev, test, single-tenant production coordination, embedded service registry, and PoC workloads. NSG inbound: allow 22/tcp from your management CIDR, 2181/tcp from your ZK client CIDR (the application tier), and 80/tcp from your operator CIDR (ZooNavigator Web UI).

Security model: ZK's sessionRequireClientSASLAuth=true is NOT used because Java 17+ has deprecated DIGEST-MD5 SASL. Instead the cloudimg image relies on:

• NSG perimeter as the primary access control on :2181
• ZooKeeper digest ACLs for per-znode authorisation once a client has connected
• nginx HTTP basic auth on :80 for the ZooNavigator Web UI

Restrict NSG :2181 inbound to your application tier IPs.

Step 1-3: Deploy + SSH (standard pattern)

ssh azureuser@<vm-ip>

Step 4: Service Status + Version

sudo systemctl is-active zookeeper.service zoonavigator.service nginx.service
/opt/zookeeper/bin/zkServer.sh version 2>&1 | tail -1

Step 5: Read Per-VM Credentials

sudo cat /stage/scripts/zookeeper-credentials.log

Pick up ZK_CONNECT, ZK_ADMIN_USER, ZK_ADMIN_PASSWORD. The same password is used for the ZooNavigator Web UI basic auth.

Step 6: 4LW Health Check + Auth Roundtrip

echo "stat" | nc -q 2 127.0.0.1 2181 | head -10
PASS=$(sudo grep '^ZK_ADMIN_PASSWORD=' /stage/scripts/zookeeper-credentials.log | cut -d= -f2-)
printf "addauth digest cloudimg:%s\nls /\n" "$PASS" | /opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181 2>&1 | tail -5

The first call uses ZK's "stat" four-letter word command to confirm Mode: standalone. The second authenticates as cloudimg via digest ACL and lists the root znodes.

Step 7: ZooNavigator Web UI — Connect

Browse to http://<vm-ip>/ and authenticate as cloudimg with the password from Step 5. ZooNavigator auto-connects to the local ZK ensemble and lands on the znode browser.

Step 8: Create a znode from the UI

Click Create node in the ZooNavigator left pane, name it /myapp, mode Persistent. The new znode appears in the tree.

Step 9: Inspect a znode

Click any znode (e.g., the newly-created /myapp) to see its data, ACL list, stat (czxid, mzxid, version, etc.), and any children. The Edit tab lets operators set data inline.

Step 10: Create + protect a znode from the CLI

PASS=$(sudo grep '^ZK_ADMIN_PASSWORD=' /stage/scripts/zookeeper-credentials.log | cut -d= -f2-)
printf "addauth digest cloudimg:%s\ncreate /cloudimg/data hello digest:cloudimg:cdrwa\nls /cloudimg\nget /cloudimg/data\n" "$PASS" | /opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181 2>&1 | tail -10

This creates /cloudimg/data with an ACL that grants the cloudimg digest user full CRUD permissions, then reads it back. Replace the ACL with world:anyone:r for public-readable znodes.

Step 11: Tune Heap

Edit /etc/zookeeper/conf/java.env and restart:

sudo systemctl restart zookeeper.service

Defaults on Standard_B2s:

• JVMFLAGS: -Xms256m -Xmx512m
• Snapshot retention: 5 snapshots, autopurge every 24 hours
• Data dir: /var/lib/zookeeper/data
• Transaction log dir: /var/lib/zookeeper/log

For production with a busy ensemble, raise -Xmx to 1024m–2048m and place transaction logs on a separate fast disk (dataLogDir in /etc/zookeeper/conf/zoo.cfg).

Step 12: Add a Second ZooNavigator User

Replace <new-password> with the operator's chosen password:

sudo htpasswd -bB /etc/nginx/zoonavigator.htpasswd alice <new-password>
sudo systemctl reload nginx

Each operator authenticates with their own basic-auth credential against the same nginx vhost. The underlying ZK digest ACL is still cloudimg.

Step 13: Add a Second ZooKeeper digest user

Generate the digest hash + create a znode owned by the second user:

PASS=$(sudo grep '^ZK_ADMIN_PASSWORD=' /stage/scripts/zookeeper-credentials.log | cut -d= -f2-)
NEWUSER=alice
NEWPASS=<new-password>
HASH=$(echo -n "$NEWUSER:$NEWPASS" | openssl dgst -binary -sha1 | base64)
printf "addauth digest cloudimg:%s\nsetAcl / digest:cloudimg:cdrwa,digest:%s:%s:cdrwa\nls /\n" "$PASS" "$NEWUSER" "$HASH" | /opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181 2>&1 | tail -5

Step 14: Backups

ZooKeeper writes snapshots to /var/lib/zookeeper/data/version-2 and transaction logs to /var/lib/zookeeper/log/version-2. Take a consistent backup with the broker running by copying the latest snapshot + all transaction logs since:

sudo tar czf /var/backups/zookeeper-$(date +%F).tgz -C /var/lib zookeeper

Periodically copy /var/backups to Azure Blob Storage (az storage blob upload-batch) for off-VM retention. To restore, stop ZK, replace /var/lib/zookeeper, restart.

Step 15: Logs and Troubleshooting

sudo journalctl -u zookeeper.service --no-pager -n 80
sudo journalctl -u zoonavigator.service --no-pager -n 30
sudo journalctl -u nginx.service --no-pager -n 30
sudo ls /var/lib/zookeeper/log/

ZooKeeper writes its main log to /var/lib/zookeeper/log/zookeeper-zookeeper-server-*.out and its data log to /var/lib/zookeeper/log/version-2/. ZooNavigator's stdout goes to the systemd journal. nginx access log is at /var/log/nginx/access.log.

Security

• ZK client port :2181 is NOT auth-walled at the network layer (SASL DIGEST-MD5 deprecated in Java 17+) — restrict NSG inbound to your application tier IPs
• ZK digest ACLs gate per-znode access for connected clients — use them to isolate workloads
• Admin server bound to 127.0.0.1:8080 (loopback only) — not exposed to the network
• ZooNavigator Web UI bound to 127.0.0.1:9000 — not exposed directly
• nginx auth wall on :80 enforces HTTP basic authentication for the ZooNavigator Web UI
• Per-VM cloudimg password used for nginx, ZooNavigator container, and the default znode digest ACL — single rotation
• Terminate TLS at an upstream Application Gateway / Front Door, or enable ZK's own SSL listener (zoo.cfg secureClientPort=2281)

Licensing — ZooNavigator (AGPL-3.0)

Apache ZooKeeper itself is Apache License 2.0 (permissive). The ZooNavigator Web UI bundled in this image is AGPL-3.0-or-later, which has a network-copyleft clause: if you modify ZooNavigator and run the modified version on a network where users can interact with it, you must offer those users the source code of your modified version.

For unmodified deployments (the cloudimg default — we ship elkozmon/zoonavigator:2.0.0 from Docker Hub without modification), AGPL § 13 is satisfied because the upstream source is publicly available at:

• Source: https://github.com/elkozmon/zoonavigator
• Image: https://hub.docker.com/r/elkozmon/zoonavigator
• License: https://github.com/elkozmon/zoonavigator/blob/master/LICENSE.txt

If you modify the ZooNavigator container in your own deployment (e.g., custom theming, additional plugins), AGPL § 13 requires you to provide your modified source to your users. The cloudimg packaging, support, and CLI scripts around ZooNavigator are NOT modifications to ZooNavigator and remain under cloudimg's commercial terms.

If you'd prefer a strictly Apache-licensed image without ZooNavigator, contact support@cloudimg.co.uk and we'll spin a CLI-only ZooKeeper variant.

Support

cloudimg provides 24/7/365 expert technical support. Guaranteed response within 24 hours, one hour average for critical issues. Contact support@cloudimg.co.uk.