Applications Azure

lakeFS on Ubuntu 24.04 on Azure User Guide

Last updated: 2026-07-11 | Product: lakeFS 1.83 on Ubuntu 24.04 LTS on Azure

Overview

This guide covers the deployment and configuration of lakeFS on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images. lakeFS is open source data version control, often described as Git for data lakes: it brings Git style branching, committing, merging and rollback to the data in your object storage, so data teams can build reproducible, isolated and safe data workflows over the exact same storage they already use.

This image installs the current stable lakeFS 1.83, the official prebuilt lakefs server and lakectl client Go binaries, and runs the server directly under systemd. Docker is not required. You interact with lakeFS through its web UI, the lakectl command line client, the REST API, or the S3 compatible gateway.

Security by design, no shipped secrets. lakeFS holds two secrets that must never be shared across machines: the admin access key and secret that authenticate API and UI access, and the auth.encrypt.secret_key that encrypts all stored credentials. This image ships neither. On the very first boot of every VM a fresh per instance encryption key is generated and a fresh per instance admin access key and secret are created by a headless lakefs setup. The admin credentials are written to /root/lakefs-credentials.txt, readable only by root. No two VMs ever share secret material, and nothing usable exists in the captured image.

Self contained storage on a dedicated disk. The image ships a local block adapter for the versioned data and an embedded key value store for the metadata, both on a dedicated 20 GiB Azure data disk mounted at /var/lib/lakefs. This makes the appliance fully self contained for evaluation and small teams. For production you point the block store at Azure Blob Storage or S3, as shown in the hardening step.

What is included:

lakeFS 1.83 (current stable release), the official prebuilt lakefs server binary, run under systemd as a dedicated unprivileged lakefs system user
The lakectl command line client bundled at /usr/local/bin/lakectl for branching, committing and merging your data from the shell
nginx on port 80 reverse proxying the lakeFS web UI, REST API and S3 gateway to the lakeFS server on loopback 127.0.0.1:8000, with an unauthenticated /healthz endpoint for Azure Load Balancer probes
A per instance admin access key and secret plus a per instance encryption key, both generated on first boot (no shipped secrets)
A local block store and embedded key value metadata store on a dedicated 20 GiB Azure data disk at /var/lib/lakefs
Ubuntu 24.04 LTS base with latest security patches applied at build time, and unattended security upgrades kept enabled
24/7 cloudimg support with a guaranteed 24 hour response SLA

Prerequisites

Active Azure subscription, SSH public key, VNet and subnet in target region
Subscription to the lakeFS listing on Azure Marketplace
Network Security Group rules allowing TCP 22 (admin), TCP 80 (web UI and API), and TCP 443 if you add your own TLS certificate

Recommended virtual machine size: Standard_B2s (2 vCPU, 4 GB RAM) for evaluation and small teams. Larger teams and busier metadata workloads should use Standard_D2s_v5 or larger, and back the metadata store with PostgreSQL and the block store with Azure Blob Storage.

Step 1: Deploy from the Azure Portal

Search lakeFS in Marketplace, select the cloudimg publisher, and click Create. Configure the Network Security Group to allow TCP 80 (web UI and API) from your team networks and TCP 22 for administration. Add TCP 443 if you plan to terminate TLS with your own certificate.

Step 2: Deploy from the Azure CLI

RG="lakefs-prod"; LOCATION="eastus"; VM_NAME="lakefs1"
GALLERY_IMAGE_ID="/subscriptions/<sub-id>/resourceGroups/azure-cloudimg/providers/Microsoft.Compute/galleries/cloudimgGallery/images/lakefs-ubuntu-24-04/versions/<version>"
SSH_KEY="$(cat ~/.ssh/id_rsa.pub)"
az group create --name "$RG" --location "$LOCATION"
az vm create \
  --resource-group "$RG" --name "$VM_NAME" \
  --image "$GALLERY_IMAGE_ID" \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values "$SSH_KEY" \
  --public-ip-sku Standard

# Open the web UI, API and admin ports on the VM NSG:
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 80 --priority 100
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 22 --priority 110

Step 3: First boot

On the first boot, a one time service generates the per instance encryption key, runs lakefs setup to create the per instance admin access key and secret, starts the lakeFS server and nginx, verifies that the admin authenticates while anonymous access is rejected, and writes the credentials to /root/lakefs-credentials.txt. This usually completes within a minute. Retrieve the credentials over SSH:

ssh azureuser@<vm-public-ip>
sudo cat /root/lakefs-credentials.txt

Step 4: Confirm the lakeFS server is running

systemctl is-active lakefs nginx
/opt/lakefs/lakefs --version
ss -tlnp | grep -E ':8000|:80'

The lakeFS server binds to loopback 127.0.0.1:8000 only, and nginx fronts it on port 80. This keeps the API server off the public interface so nginx is the single entry point.

The lakefs and nginx services both active under systemd, the systemctl status showing lakefs loaded and running, the lakefs and lakectl version 1.83.0, and the listener table showing the lakeFS API and web UI bound to loopback 127.0.0.1 on port 8000 while nginx serves HTTP on port 80

Step 5: Open the web UI and log in

Browse to http://<vm-public-ip>/ and sign in with the Access Key ID and Secret Access Key from /root/lakefs-credentials.txt. These credentials are unique to your VM.

The lakeFS sign in page in a browser, with the Access Key ID field populated with the per instance admin access key and the Secret Access Key field entered, and the blue Login button below

After signing in you land on the repositories list, where you can create repositories and open existing ones. Each repository has a default branch and a storage namespace on the local block store.

The lakeFS repositories list after logging in as admin, showing the sample data repo and validate demo repositories, each with its default branch main and its local block store namespace, plus the Create Repository button and the top navigation for Repositories, Datasets and Administration

Step 6: Configuration and storage on the data disk

The server configuration lives at /etc/lakefs/config.yaml. It selects the local block store for the versioned data and the embedded local key value store for the metadata, both under /var/lib/lakefs on the dedicated data disk. The auth.encrypt.secret_key shown in the file is only a placeholder: the real per instance key is injected from the data disk at first boot through an environment variable and is never written into this file or shipped in the image.

sudo grep -vE '^\s*#' /etc/lakefs/config.yaml | sed 's/secret_key:.*/secret_key: <redacted>/'
findmnt /var/lib/lakefs

Step 7: Version your data, branch, commit and merge

Configure lakectl once with your credentials and the server endpoint, then work with your data exactly like Git. Create a repository, upload objects, commit them, branch off to experiment in isolation, and merge back when ready.

# Configure lakectl (paste the access key, secret and http://<vm-public-ip>/ endpoint):
lakectl config

# Create a repository on the local block store:
lakectl repo create lakefs://sample-data-repo local://sample-data-repo --default-branch main

# Commit an object on main:
echo "id,name,amount" > transactions.csv
lakectl fs upload -s transactions.csv lakefs://sample-data-repo/main/datasets/transactions.csv
lakectl commit lakefs://sample-data-repo/main -m "Add transactions dataset"

# Branch off to experiment without touching main, then merge back:
lakectl branch create lakefs://sample-data-repo/experiment --source lakefs://sample-data-repo/main
lakectl merge lakefs://sample-data-repo/experiment lakefs://sample-data-repo/main

A terminal showing the lakectl data versioning round trip: creating a repository on the local block store, uploading a transactions dataset onto branch main, committing it, then listing the versioned objects back and showing the commit history with the admin author and the Add transactions dataset and Repository created commits

The same operations are available in the web UI. Open a repository to browse its objects on any branch, and open the Commits tab to see the full history.

The Objects view of the sample data repo in the lakeFS web UI on branch main, showing the README file and the datasets and models folders with their sizes, the tab bar for Data, Commits, Branches, Tags, Pull Requests, Compare, Actions and Settings, and the Upload and Import buttons

The Commits history view of the sample data repo in the lakeFS web UI on branch main, listing the Add transactions dataset model and README commit by admin and the Repository created commit, each with its commit identifier and timestamp

Step 8: Security, per instance credentials and enforced authentication

The admin credentials in /root/lakefs-credentials.txt are generated per VM on first boot and the file is readable only by root. Every API and UI request must authenticate: an unauthenticated call is rejected with HTTP 401. The health endpoints answer without authentication so a load balancer can probe them.

# Anonymous API access is rejected:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/api/v1/repositories   # 401

# Health endpoints answer 200:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1/healthz                     # 200
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/_health                # 200

A terminal showing the per instance admin credentials with the host, url and access key id visible while the secret access key is redacted because it is unique per VM and never shipped, then proof that an unauthenticated API request is rejected with HTTP 401 while the nginx healthz probe, the lakeFS health endpoint and the web UI all return HTTP 200

Step 9: Harden for production

Terminate TLS. Put your own certificate on nginx and serve the UI and API over HTTPS on port 443. lakeFS works over plain HTTP on a trusted network, but you should always terminate TLS before exposing it to the internet.
Point the block store at cloud storage. For real data lakes, set blockstore.type to azure (Azure Blob Storage) or s3 in /etc/lakefs/config.yaml so lakeFS versions the data already sitting in your object storage, rather than the local disk.
Back the metadata with PostgreSQL. For high availability and larger teams, switch database.type to postgres and point it at a managed PostgreSQL instance so several lakeFS nodes can share state.
Rotate and manage credentials. Create additional users and access keys from the Administration area of the web UI, and keep the root credentials file secured.
Keep the OS patched. Unattended security upgrades remain enabled on the image.

Support

cloudimg provides 24/7 support with a guaranteed 24 hour response SLA for this image. Contact support@cloudimg.co.uk for deployment help, configuration guidance, or production hardening advice.

lakeFS is a project of Treeverse Ltd. This image packages the Apache 2.0 licensed open source lakeFS and is not affiliated with, endorsed by, or sponsored by Treeverse or the lakeFS project. lakeFS is a trademark of its respective owner.