Zipkin Distributed Tracing on Ubuntu 24.04 on Azure User Guide

Overview

Zipkin is the reference open source distributed tracing system. Instrumented microservices report spans (units of work with a trace ID, timing and tags) to Zipkin, which stitches them into traces so you can see end-to-end request latency, the exact call path across services, and a service dependency graph. It is the original tracing backend paired with Spring Cloud Sleuth/Micrometer Tracing, Brave, and any OpenTelemetry pipeline configured with the zipkin exporter. The cloudimg image installs the pinned official Zipkin 3.6.1 server jar (OpenJDK 17) running under systemd, then locks it down for a marketplace appliance: Zipkin itself is bound to loopback only, and nginx does all customer-facing work - port 80 fronts the web UI and full REST API behind a per-VM HTTP Basic Auth gate, and port 9411 (the well-known Zipkin collector port) accepts span uploads only, deliberately unauthenticated because tracing libraries cannot supply Basic Auth credentials, but hard-restricted so it can never be used to read a trace. Zipkin has no user accounts of its own, so the nginx Basic Auth credential (user admin, unique password generated on first boot) is the gate on the UI and query API. Backed by 24/7 cloudimg support.

What is included:

• Zipkin 3.6.1 installed as the official executable jar, running as the zipkin systemd service under OpenJDK 17
• The Zipkin web UI and full REST API on :80, fronted by nginx with Zipkin bound to loopback only
• Per-VM HTTP Basic Auth (user admin) protecting the UI and query API, with a unique password generated on first boot
• An unauthenticated span-ingest-only endpoint on the standard :9411 port (POST /api/v2/spans only - every other path returns 403), for instrumented apps and OpenTelemetry-zipkin exporters that cannot supply credentials
• A dedicated Azure data disk at /var/lib/zipkin holding Zipkin's own application log
• zipkin.service + nginx.service as systemd units, enabled and active
• An unauthenticated /healthz endpoint on :80 for Azure Load Balancer health probes
• In-memory trace storage (Zipkin's default STORAGE_TYPE=mem) - clearly documented below, including how to point at your own MySQL/Elasticsearch/Cassandra for production durability
• 24/7 cloudimg support

Important - trace storage is in-memory by default

This appliance ships Zipkin's default STORAGE_TYPE=mem. Traces live in the JVM's memory only and are lost whenever the zipkin service restarts or the VM reboots. This keeps the appliance simple and dependency-free out of the box, and is fine for development, demos and short-lived debugging sessions. For production tracing you care about across restarts, point Zipkin at your own persistent backend - MySQL, Elasticsearch or Cassandra - by editing /etc/systemd/system/zipkin.service (Environment=STORAGE_TYPE=mysql plus the corresponding MYSQL_HOST / MYSQL_TCP_PORT / MYSQL_DB / MYSQL_USER / MYSQL_PASS variables, per the upstream Zipkin server documentation), then sudo systemctl daemon-reload && sudo systemctl restart zipkin. The dedicated data disk at /var/lib/zipkin holds only Zipkin's own application log file, not trace data.

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B2s (2 vCPU / 4 GiB RAM) is a comfortable starting point for light-to-moderate tracing volume; size up the JVM heap (JAVA_OPTS in the systemd unit) and the VM for higher throughput. NSG inbound: allow 22/tcp from your management network, 80/tcp for the UI and query API, 9411/tcp if your instrumented apps need to send spans over the network (skip this if all reporting apps run on the same VM and can reach 127.0.0.1:9411... see Step 4 note), and 443/tcp if you add TLS. Zipkin serves plain HTTP; for production use, terminate TLS in front of it with your own domain and restrict :9411 to your application subnets.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Zipkin by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80), plus Custom (9411) if apps outside the VM will send spans. Review the dedicated data disk on the Disks tab, then Review + create -> Create.

Step 2 - Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name zipkin \
  --image <marketplace-image-urn> \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name zipkin --port 80 --priority 1010
az vm open-port --resource-group <your-rg> --name zipkin --port 9411 --priority 1020

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

systemctl is-active zipkin.service nginx.service

Both report active. Zipkin serves its web UI and full REST API on the loopback address 127.0.0.1:19411 only (an internal port - not the public 9411); nginx fronts port 80 with the per-VM HTTP Basic Auth gate for the UI and query API, and separately fronts the well-known port 9411 with an unauthenticated, ingest-only proxy that accepts POST /api/v2/spans and rejects everything else. If every service that reports traces runs on this same VM, you can point them at http://127.0.0.1:9411/api/v2/spans directly with no network exposure at all; only open 9411 in the NSG if reporting apps live elsewhere.

Step 5 - Retrieve your web UI password

nginx protects the Zipkin UI and query API with HTTP Basic Auth. The username is admin and a unique password is generated on the first boot of your VM and written to a root-only file:

sudo cat /root/zipkin-credentials.txt

This file contains ZIPKIN_USERNAME, ZIPKIN_PASSWORD, the ZIPKIN_URL to open in a browser, and ZIPKIN_SPAN_INGEST_URL for configuring your tracing exporters. The password is stored on disk only as a bcrypt hash in /etc/nginx/.zipkin.htpasswd, so no plaintext password ships in the image. Store the password somewhere safe.

Step 6 - Confirm the health endpoint

nginx serves an unauthenticated health endpoint for load balancers and probes:

curl -s http://localhost/healthz

It returns ok. This endpoint never requires authentication, so it is safe for an Azure Load Balancer health probe.

Step 7 - Confirm authentication on the UI/query path

Because a password is set on first boot, an unauthenticated request to the UI returns HTTP 401, so nobody reaches the trace data without the password. The following reads the per-VM password from the credentials file and proves the round-trip - unauthenticated is rejected, a wrong password is rejected, and the correct password authenticates:

PW=$(sudo grep '^ZIPKIN_PASSWORD=' /root/zipkin-credentials.txt | cut -d= -f2-)
echo "unauth  : $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1/)"
echo "wrongpw : $(curl -s -o /dev/null -w '%{http_code}' -u admin:wrong-pw http://127.0.0.1/)"
echo "authed  : $(curl -s -o /dev/null -w '%{http_code}' -u admin:$PW http://127.0.0.1/)"

It prints unauth : 401, wrongpw : 401, authed : 200. Only the per-VM password reaches the Zipkin UI and query API, because Zipkin itself is bound to loopback and nginx port 80 is the only way in.

Step 8 - Send a test span and query it back

Zipkin's collector accepts spans at POST /api/v2/spans. Instrumented apps normally send spans continuously; the following posts one by hand to confirm the pipeline end to end, through both entry points - the authenticated :80 API and the unauthenticated ingest-only :9411 path:

PW=$(sudo grep '^ZIPKIN_PASSWORD=' /root/zipkin-credentials.txt | cut -d= -f2-)
TID=$(openssl rand -hex 16)
TS=$(($(date +%s%N)/1000))

# authenticated path (port 80)
curl -s -o /dev/null -w 'POST :80/api/v2/spans  -> %{http_code}\n' -u admin:$PW \
  -H 'Content-Type: application/json' \
  -d "[{\"traceId\":\"$TID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"checkout\",\"timestamp\":$TS,\"duration\":45000,\"localEndpoint\":{\"serviceName\":\"web-frontend\"}}]" \
  http://127.0.0.1/api/v2/spans

curl -s -u admin:$PW "http://127.0.0.1/api/v2/trace/$TID" | python3 -m json.tool

The POST returns 200 (or 202), and the trace query returns the span you just sent - serviceName: web-frontend, name: checkout, and the duration in microseconds. Point real instrumentation at ZIPKIN_SPAN_INGEST_URL (http://<vm-ip>:9411/api/v2/spans, no credentials required) or, for apps running on this same VM, http://127.0.0.1:9411/api/v2/spans.

Step 9 - Sign in and search traces

Browse to http://<vm-public-ip>/. Your browser prompts for a username and password: enter admin and the password from Step 5. Zipkin opens on the trace search screen. Pick a service name, an optional span name and a lookback window, then Run Query to list matching traces sorted by duration.

Step 10 - Read a trace detail / span waterfall

Click any trace in the results to open its detail view. The waterfall shows every span in the trace nested by parent/child relationship with exact start offsets and durations, so you can see precisely which downstream call added the latency. Click any span to expand its tags and local/remote endpoint annotations.

Step 11 - View the service dependency graph

Open Dependencies from the top navigation and pick a date. Zipkin aggregates all traces from that day into a graph showing every service that called another, with call counts and error rates on each edge - a quick way to see your architecture as it actually behaves in production, not as documented.

Maintenance

• Password: the UI/API password is set on first boot and stored as a bcrypt entry in /etc/nginx/.zipkin.htpasswd. To change it, run sudo htpasswd -B /etc/nginx/.zipkin.htpasswd admin and then sudo systemctl reload nginx.
• Storage is in-memory by default: traces are lost on zipkin service restart or VM reboot. For durable storage, set STORAGE_TYPE (and the matching backend variables) in /etc/systemd/system/zipkin.service to mysql, elasticsearch or cassandra3 per the upstream storage docs, then sudo systemctl daemon-reload && sudo systemctl restart zipkin.
• Span ingest port: :9411 is intentionally unauthenticated (tracing libraries cannot supply Basic Auth) but restricted by nginx to POST /api/v2/spans only - every other path, including the query API, returns 403. Restrict :9411 in your NSG to your application subnets in production.
• Restrict access: Zipkin serves plain HTTP on port 80. For production, restrict access to trusted IP ranges in your Network Security Group, and front it with TLS (for example certbot with your own domain) terminating on :443.
• Loopback binding: Zipkin is bound to 127.0.0.1:19411 in /etc/systemd/system/zipkin.service (--armeria.ports[0].ip=127.0.0.1), so nginx is the only path in for both the UI/query API and the public ingest port. Keep it that way - do not change the bind address to a public interface.
• JVM sizing: the default heap is -Xms256m -Xmx512m (JAVA_OPTS in /etc/systemd/system/zipkin.service). Increase it and the VM size for higher trace volume, then sudo systemctl daemon-reload && sudo systemctl restart zipkin.
• Logs: Zipkin's own application log is written to /var/lib/zipkin/zipkin.log on the dedicated data disk; journalctl -u zipkin also shows recent output.
• Security patches: unattended-upgrades remains enabled so the OS continues to receive security updates automatically.

Support

cloudimg provides 24/7 expert support for this image. Contact support@cloudimg.co.uk.