Developer Tools Azure

Kestra on Ubuntu 24.04 on Azure User Guide

| Product: Kestra on Ubuntu 24.04 LTS on Azure

Overview

Kestra is an open source, event driven orchestration platform for building and running data and business workflows. You declare pipelines as YAML flows, so the logic is readable and version controllable, and a large plugin catalogue connects them to databases, cloud services, message queues and scripts in any language. The server schedules and executes flows, persists every execution so work survives restarts, and its web interface gives you a flow editor, a live executions view with a Gantt and topology breakdown, and a dashboard. The cloudimg image ships the Apache-2.0 licensed Kestra open source core in single node standalone mode, backed by a bundled PostgreSQL for a durable repository and queue, run the officially supported way as an upstream container under systemd and fronted by nginx. All images are pinned by digest and captured into the VM, so your instance starts in seconds.

Kestra executes arbitrary tasks and scripts, so the web UI and API are never left open. cloudimg fronts them with nginx HTTP Basic Auth using a unique password generated for each VM on first boot, before the port is reachable, and configures Kestra's own authentication with the same per VM credential. The Kestra server, its management port and the bundled PostgreSQL are bound to the loopback interface only. Backed by 24/7 cloudimg support.

Kestra is a trademark of Kestra Technologies Inc. This image is produced by cloudimg and is not affiliated with, endorsed by, or sponsored by Kestra Technologies Inc. It ships only the Apache-2.0 licensed open source Kestra core.

The Kestra web UI flows list showing declared flows and their last execution status

What is included:

  • Kestra 1.3.28 (the Apache-2.0 licensed open source orchestrator: scheduler, executor, worker and web server, run as server standalone), pinned by image digest with the full plugin catalogue
  • A bundled PostgreSQL 16 database, pinned by image digest, backing the repository and queue so flows and executions survive restarts, reachable only inside a private Docker network and never published to a host port
  • Docker Engine (Docker CE) with the Kestra UI and management port published to the loopback interface only, fronted by nginx on port 80
  • kestra.service, kestra-firstboot.service and nginx.service as systemd units, enabled and active on boot
  • A unique web UI password (nginx HTTP Basic Auth, bcrypt, plus Kestra's own authentication with the same credential), a unique PostgreSQL password and a unique secret encryption key generated per VM on first boot, never baked into the image
  • A clean, freshly initialised database on first boot: no default login, no shipped secret, no prior flows or executions
  • 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet plus subnet in the target region. Standard_B2ms (2 vCPU / 8 GiB RAM) is a sensible starting point; increase the size for heavier workloads and higher flow concurrency. NSG inbound: allow 22/tcp from your management network and 80/tcp (and 443/tcp once you add TLS) for the web UI. The web UI serves plain HTTP on port 80; for production, put it behind TLS with your own domain (see the final section).

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Kestra by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Then Review + create and Create.

Step 2 - Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name kestra \
  --image <marketplace-image-urn> \
  --size Standard_B2ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name kestra --port 80 --priority 1010

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

On first boot the image generates a unique web UI password, database password and encryption key, initialises a clean PostgreSQL database, applies the Kestra schema and starts the stack. Confirm the systemd services are active:

systemctl is-active docker kestra nginx

All three services report active. The Kestra server runs as a container alongside the bundled PostgreSQL under one Docker Compose project, and the public liveness endpoint returns 200 with no credentials:

curl -s -o /dev/null -w 'health: HTTP %{http_code}\n' http://127.0.0.1/healthz

The Kestra server and bundled PostgreSQL containers running under systemd, with the health endpoint returning 200

Step 5 - Secure by default: the password gates everything

Kestra executes arbitrary tasks, so an open UI or API is remote code execution. cloudimg fronts both with nginx HTTP Basic Auth and configures Kestra's own authentication with the same per VM credential. Any request to the UI or API without the exact credential is rejected with 401. Confirm it yourself - this request sends no credential:

curl -s -o /dev/null -w 'no credential: HTTP %{http_code}\n' http://127.0.0.1/api/v1/main/flows/search

It returns HTTP 401. Guessable defaults such as admin@kestra.io:admin are rejected too. Your unique credential authenticates and returns 200; the block below reads the password from the notes file (so it never appears on screen):

P=$(sudo grep '^KESTRA_UI_PASSWORD=' /root/kestra-notes.txt | cut -d= -f2-)
curl -s -o /dev/null -w 'authorized: HTTP %{http_code}\n' -u "admin@kestra.local:$P" http://127.0.0.1/api/v1/main/flows/search

The Kestra API rejects no credential and a guessable default (401) and accepts only the per-VM credential (200); health stays public

Your unique user and password are written to /root/kestra-notes.txt, readable only by root:

sudo stat -c '%a %U:%G' /root/kestra-notes.txt

It reports 600 root:root. View the notes (which include the web UI user, the web UI password and the URL) with:

sudo cat /root/kestra-notes.txt

The per-VM access notes, with the web UI password and database password masked

Step 6 - Open the Kestra web UI

Browse to http://<vm-public-ip>/. Enter the user admin@kestra.local and the password from /root/kestra-notes.txt at the login prompt. The UI opens on the Flows page, listing every flow on the server with its namespace, labels and last execution status.

The Kestra flows list showing declared flows and their last execution status

Step 7 - Build flows in the code editor

Kestra flows are declarative YAML. Select Create to open the built in code editor: write the flow source on the left with autocompletion and validation, read the live flow property documentation on the right, and switch between the code, a no-code form and a live topology view. A flow needs only an id, a namespace and a list of tasks; triggers, labels and inputs are optional.

The Kestra built in code editor showing a YAML flow alongside the live flow property documentation

Step 8 - Run and inspect executions

The server schedules and executes flows and persists every execution to PostgreSQL. You can trigger a flow from the UI or from the API. This lists the flows and their completed executions in a namespace using the API and your per VM credential:

P=$(sudo grep '^KESTRA_UI_PASSWORD=' /root/kestra-notes.txt | cut -d= -f2-)
curl -s -u "admin@kestra.local:$P" 'http://127.0.0.1/api/v1/main/flows/search?namespace=cloudimg.demo' | jq -r '.results[] | .namespace + "." + .id'

Declared flows and their completed executions listed from the Kestra API

Open any execution to see its Gantt view: each task rendered on a timeline with its duration and status, so you can see exactly how a pipeline ran. This history is what makes execution durable - it is persisted to PostgreSQL and can be replayed.

A completed execution's Gantt view showing the sequential tasks of a data pipeline on a timeline

The Executions page lists every run across your flows with a duration and count chart, so you can monitor throughput and drill into any execution.

The Kestra executions page showing a duration and count chart and the list of completed runs

Step 9 - Schedules and triggers

Flows can carry triggers so they run automatically. A Schedule trigger runs a flow on a cron spec, so you can drive nightly jobs and periodic pipelines without an external cron; flow triggers and webhook triggers let flows react to other flows or to external events. Add a trigger in the flow YAML and Kestra registers it the moment you save.

Step 10 - Rotate the password

The password is your access key. To set your own, rewrite the nginx Basic Auth file, update Kestra's own credential and restart:

sudo htpasswd -B /etc/nginx/kestra.htpasswd admin@kestra.local
sudo nano /etc/kestra-appliance/kestra.env   # update KESTRA_AUTH_PASSWORD to match
sudo systemctl restart kestra nginx

Step 11 - Production: your own domain with TLS

For production, front the web UI with your own domain and TLS. Point a DNS record at the VM public IP, then install a certificate with your preferred tool (for example Certbot with the nginx plugin) and reload nginx. Restrict inbound access in your NSG to 443/tcp (and 22/tcp from your management network) once TLS is in place, and keep the credential confidential - it grants full control of your flows, which can execute arbitrary tasks.

Support

This image is supported 24/7 by cloudimg. For help with deployment or configuration, contact support through cloudimg.co.uk.