Apache Storm on Ubuntu 24.04 on Azure User Guide

Overview

Apache Storm is the open-source distributed real-time stream computation system. It processes unbounded streams of data with low latency, guaranteeing every record is processed, and scales horizontally across a cluster of worker nodes. The cloudimg image installs Apache Storm 2.7.1 on OpenJDK 17 at /opt/storm, runs a single-node cluster (Nimbus master, one Supervisor, and the Storm UI) coordinated by a local Apache ZooKeeper, fronts the Storm UI with an nginx reverse proxy on TCP 80 protected by HTTP Basic authentication, persists Storm and ZooKeeper state on a dedicated Azure data disk, and generates a unique UI password on the first boot of every VM. Backed by 24/7 cloudimg support.

What is included:

• Apache Storm 2.7.1 (Nimbus, Supervisor, UI) on OpenJDK 17 at /opt/storm
• A local single-node Apache ZooKeeper 3.9.3 providing the coordination service Storm requires
• nginx reverse proxy on :80 in front of the Storm UI (bound to loopback :8080) with HTTP Basic authentication, because the Storm UI ships with no authentication of its own
• A dedicated Azure data disk at /var/lib/storm holding storm.local.dir and the ZooKeeper data directories — separate from the OS disk and re-provisioned with every VM
• Per-VM Storm UI password (STORM_ADMIN_PASSWORD) generated at first boot, in a root-only file
• zookeeper.service, storm-nimbus.service, storm-supervisor.service, storm-ui.service and nginx.service as systemd units, enabled and active
• The bundled storm-starter WordCount topology submitted on first boot so the UI shows a running topology
• 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B4ms (4 vCPU / 16 GiB RAM) is a good starting point for the Storm + ZooKeeper JVMs and a single worker; scale up for higher throughput or more worker slots. NSG inbound: allow 22/tcp from your management network and 80/tcp for the Storm UI (front with TLS for public exposure — see Enabling HTTPS).

Step 1 — Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache Storm by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Review the dedicated data disk on the Disks tab, then Review + create → Create.

Step 2 — Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name storm \
  --image <marketplace-image-urn> \
  --size Standard_B4ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name storm --port 80 --priority 1010

Step 3 — Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 — Confirm the services are running

systemctl is-active zookeeper.service storm-nimbus.service storm-supervisor.service storm-ui.service nginx.service

All five services report active. Storm and ZooKeeper run on the JVM and can take a minute or two to fully start on first boot.

Step 5 — Retrieve the per-VM Storm UI password

On the first boot of every VM, apache-storm-firstboot.service generates a unique password for the Storm UI and writes it to a root-only file. Read it with:

sudo cat /root/storm-credentials.txt

The file contains the URL, the user name (admin) and the generated STORM_ADMIN_PASSWORD. Keep it safe — it is the credential nginx checks before proxying any request to the Storm UI.

Step 6 — Open the Storm UI

Browse to http://<vm-public-ip>/. nginx prompts for HTTP Basic authentication; sign in as admin with the password from Step 5. The Storm UI Cluster Summary shows the Storm version, Nimbus uptime, the number of supervisors, used and free worker slots, and total executors and tasks.

The Nimbus Summary and Supervisor Summary sections list the Nimbus host (leader) and the worker supervisor with its slot usage, so you can confirm the cluster is healthy at a glance.

Step 7 — Inspect the running topology

The image submits the bundled storm-starter WordCount topology on first boot, so the Topology Summary lists a running wordcount topology. Click it to open the topology view, which shows the spouts and bolts, the stream graph, and per-component statistics (emitted, transferred, acked and failed tuples).

Drill into a spout or bolt to see its executor and task counts and its tuple-level statistics over the selected time window.

Step 8 — Submit your own topology

Storm runs topologies packaged as a JAR. Copy your topology JAR to the VM and submit it with the storm CLI as the storm service user. For example, to resubmit the bundled WordCount example under a new name:

sudo -u storm JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 \
  STORM_CONF_DIR=/opt/storm/current/conf \
  /opt/storm/current/bin/storm jar \
  /opt/storm/current/examples/storm-starter/storm-starter-topologies-2.7.1.jar \
  org.apache.storm.starter.WordCountTopology demo

List the running topologies:

sudo -u storm JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 \
  STORM_CONF_DIR=/opt/storm/current/conf \
  /opt/storm/current/bin/storm list

To remove a topology, use storm kill <name> with the same command prefix.

Step 9 — Where data lives

storm.local.dir and the ZooKeeper data and transaction-log directories live on the dedicated Azure data disk mounted at /var/lib/storm:

findmnt /var/lib/storm

ls /var/lib/storm

Because the data disk is captured into the image and re-provisioned with every VM, the layout is identical on each deployment. You can resize this disk independently of the OS disk from the Azure Portal.

The release on disk, the Java runtime and the live cluster summary returned by the Storm UI REST API confirm the deployment at a glance:

Step 10 — Enabling HTTPS

The Storm UI is served over plain HTTP on port 80 behind nginx. For production, terminate TLS at nginx with your own certificate and domain. Install certbot, obtain a certificate, and let it configure nginx:

sudo snap install --classic certbot
sudo ln -sf /snap/bin/certbot /usr/bin/certbot
sudo certbot --nginx -d storm.example.com

certbot adds a listen 443 ssl server block and a redirect from port 80, and renews the certificate automatically. The HTTP Basic authentication in front of the Storm UI is preserved.

Step 11 — Maintenance

The OS receives unattended security updates. To update packages manually:

sudo apt-get update && sudo apt-get -y upgrade

To restart the Storm cluster (for example after changing storm.yaml):

sudo systemctl restart zookeeper.service storm-nimbus.service storm-supervisor.service storm-ui.service

Support

This image is maintained by cloudimg with 24/7 support. Contact support@cloudimg.co.uk for help with deployment, configuration or scaling Apache Storm on Azure.