Apache Pinot on Ubuntu 24.04 on Azure User Guide

Overview

Apache Pinot is the open-source real-time distributed OLAP datastore built for low-latency, high-concurrency analytics over streaming and batch data at scale. It serves sub-second queries through a columnar store, pluggable indexes and a distributed query engine, and ships a rich web console hosted by the Pinot Controller. This cloudimg image installs Apache Pinot 1.4.0 on OpenJDK 17 as a single-VM quickstart topology, brings up the bundled Apache ZooKeeper plus the Pinot Controller, Broker and Server under systemd in dependency order, fronts the no-auth Controller console with an nginx reverse proxy and HTTP basic auth on port 80, persists all data on a dedicated Azure data disk, and generates a unique console login on the first boot of every VM. Backed by 24/7 cloudimg support.

What is included:

• Apache Pinot 1.4.0 running on OpenJDK 17, installed at /opt/pinot
• A complete single-VM cluster: the bundled Apache ZooKeeper, the Pinot Controller (web console), the Pinot Broker and the Pinot Server, each as its own systemd service
• The Pinot Controller console served behind an nginx reverse proxy with HTTP basic auth on port 80
• A per-VM console password for user admin, generated at first boot and stored in a root-only file
• A dedicated Azure data disk at /var/lib/pinot holding ZooKeeper data, the Controller segment store and Server segment storage — separate from the OS disk and re-provisioned with every VM
• pinot-zookeeper, pinot-controller, pinot-broker, pinot-server and nginx as systemd units, enabled and active
• 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B4ms (4 vCPU / 16 GiB RAM) is the recommended minimum — Pinot runs several JVM services on one VM and is memory-hungry; scale up for heavier ingestion and query loads. NSG inbound: allow 22/tcp from your management network and 80/tcp from the networks that need the Pinot console (front the proxy with TLS for public exposure — see Enabling a trusted certificate).

Step 1 — Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache Pinot by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size (Standard_B4ms or larger); under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Review the dedicated data disk on the Disks tab, then Review + create then Create.

Step 2 — Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name pinot \
  --image <marketplace-image-urn> \
  --size Standard_B4ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name pinot --port 80 --priority 1010

Step 3 — Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 — Confirm the services are running

Pinot runs several JVM services and takes one to three minutes to fully start on first boot. Once it is up, all five systemd units report active, and the unauthenticated nginx health endpoint answers 200:

systemctl is-active pinot-zookeeper.service pinot-controller.service pinot-broker.service pinot-server.service nginx.service
curl -s -o /dev/null -w 'nginx health: %{http_code}\n' http://127.0.0.1/health

All services report active, and GET /health returns 200.

Step 5 — Retrieve your console login

The console username is admin; the password is generated uniquely on the first boot of your VM and written to a root-only file:

sudo cat /root/pinot-credentials.txt

The username is admin; the password is the PINOT_ADMIN_PASSWORD value, and PINOT_URL is the address to open in your browser.

Step 6 — Sign in to the Pinot Controller console

Browse to http://<vm-public-ip>/ and sign in with admin and the password from Step 5 when the browser prompts for HTTP basic authentication. Once authenticated you land on the Pinot Controller console, where you can browse tables, run Pinot SQL queries, inspect the cluster topology and manage tenants and segments.

The console has no native authentication, so nginx enforces it. A request with no credentials and a request with the wrong password are both rejected with 401. You can confirm this from the shell — the per-VM password is read from the credentials file and used inline (the wrong-password and no-credentials probes return 401, the correct password returns 200):

echo "no credentials -> $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1/)"
echo "wrong password -> $(curl -s -o /dev/null -w '%{http_code}' -u 'admin:wrong' http://127.0.0.1/)"

The first prints 401 and the second prints 401.

Step 7 — Query the Pinot cluster and health APIs

Pinot exposes a rich REST API through the same authenticated proxy. Read the Controller health and the live cluster instances (substitute the admin password from Step 5 where the example shows <PINOT_ADMIN_PASSWORD>):

curl -s -u "admin:<PINOT_ADMIN_PASSWORD>" http://127.0.0.1/health
curl -s -u "admin:<PINOT_ADMIN_PASSWORD>" http://127.0.0.1/instances

The health endpoint reports the Controller is healthy, and instances lists the registered Controller, Broker and Server instances in the cluster.

Step 8 — Load the quickstart data and run a query

Pinot ships quickstart datasets you can load to explore the console end to end. From the Query view in the console, select a table and run a Pinot SQL statement such as SELECT * FROM baseballStats LIMIT 10, then Run Query to see rows returned. You can also create your own schema and table, then ingest batch or streaming data through the Controller APIs.

Step 9 — Confirm persistence on the data disk

All of Pinot's persistent state lives on the dedicated Azure data disk at /var/lib/pinot, so your ZooKeeper metadata, Controller segment store and Server segments survive reboots and ride into any image you capture:

findmnt /var/lib/pinot
ls /var/lib/pinot

findmnt shows /var/lib/pinot mounted from the data disk, and the directory holds the zookeeper, controller and server subdirectories.

Enabling a trusted certificate

The image serves the console over plain HTTP behind nginx basic auth. For production, terminate TLS at the nginx proxy with a real domain pointed at the VM's public IP. Install certbot and request a certificate (replace the domain), which configures nginx to serve HTTPS on port 443:

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

After certbot completes, the Pinot console is reachable at https://your-domain.example.com/ with a browser-trusted certificate; the basic-auth prompt is unchanged.

Backup and maintenance

Your Pinot ZooKeeper metadata, Controller segment store and Server segments live on the data disk at /var/lib/pinot; snapshot that disk to back up your cluster state and ingested data. The Pinot binaries and single-VM configuration are at /opt/pinot/current. Keep the OS patched with sudo apt update && sudo apt upgrade. The cluster restarts cleanly in dependency order with sudo systemctl restart pinot-zookeeper pinot-controller pinot-broker pinot-server — allow one to three minutes for the cluster to come back up. To change the console password, regenerate the nginx basic-auth entry with sudo htpasswd -B /etc/nginx/pinot.htpasswd admin and reload nginx. The image already applies all available Ubuntu security updates at build time.

Support

This image is backed by 24/7 cloudimg support. Contact us by email and chat for help with schemas, table configs, ingestion jobs, Pinot SQL, scaling out to a clustered deployment and TLS.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.