Machine Learning Azure

LiteLLM on Ubuntu 24.04 on Azure User Guide

Last updated: 2026-06-11 | Product: LiteLLM on Ubuntu 24.04 LTS on Azure

Overview

LiteLLM is the open-source proxy and gateway that puts a single OpenAI-compatible API in front of 100+ LLM providers (OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex AI, Mistral, Ollama and many more). Point your applications at one endpoint and switch or load-balance providers without changing client code, with virtual keys, per-key budgets, rate limits and request logging. The cloudimg image installs LiteLLM 1.88 in a Python virtual environment at /opt/litellm, fronts it with an nginx reverse proxy on TCP 80, and generates a unique master key on the first boot of every VM. The image ships no model weights and is CPU-only. Backed by 24/7 cloudimg support.

What is included:

LiteLLM 1.88 proxy in a Python venv at /opt/litellm
nginx reverse proxy on :80 in front of the proxy (bound to loopback :4000)
Per-VM master key sk-… (LITELLM_MASTER_KEY) generated at first boot, in a root-only file
litellm.service + nginx.service as systemd units, enabled and active
An empty model list ready for you to add your providers
24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B2s (2 vCPU / 4 GiB RAM) is a fine starting point — LiteLLM is a lightweight gateway, not a model server. NSG inbound: allow 22/tcp from your management network and 80/tcp from the clients that call the API (front port 80 with TLS for public exposure — see Enabling HTTPS).

Step 1 — Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for LiteLLM by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Then Review + create → Create.

Step 2 — Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name litellm \
  --image <marketplace-image-urn> \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name litellm --port 80 --priority 1010

Step 3 — Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 — Confirm the services are running

systemctl is-active litellm.service nginx.service
ss -tln | grep -E ':80 |:4000 '
curl -s http://127.0.0.1/health/liveliness

Both services report active, nginx listens on port 80, the LiteLLM proxy listens on loopback 4000, and the liveliness endpoint responds.

Step 5 — Retrieve your master key

The master key is generated uniquely on the first boot of your VM and written to a root-only file. Callers pass it as a Bearer token:

sudo cat /root/litellm-credentials.txt

The LITELLM_MASTER_KEY value (an sk-… string) is your admin/master key. Keep it secret; use it to create scoped virtual keys for your applications (Step 8).

Step 6 — Add a provider

LiteLLM ships with an empty model list. Add your providers and credentials to /etc/litellm/config.yaml, then restart the service. For example, to add an OpenAI model:

sudo tee -a /etc/litellm/config.yaml >/dev/null <<'YAML'
model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
YAML
echo 'OPENAI_API_KEY=sk-...' | sudo tee -a /etc/litellm/litellm.env >/dev/null
sudo systemctl restart litellm

LiteLLM supports 100+ providers; see the upstream docs for the parameters of each.

Step 7 — Call the OpenAI-compatible API

Any OpenAI-compatible client works — just point base_url at the VM and use your master (or a virtual) key. List the configured models:

KEY=$(sudo grep '^LITELLM_MASTER_KEY=' /root/litellm-credentials.txt | cut -d= -f2-)
curl -s http://127.0.0.1/models -H "Authorization: Bearer $KEY"

Chat completions, embeddings and the other OpenAI routes are served at the same base URL.

Step 8 — Virtual keys, budgets and the admin UI

Use the master key to mint scoped virtual keys for each application with their own budgets and rate limits via the /key/generate API, so you never share the master key. LiteLLM also serves an admin UI for keys, spend and logs.

Step 9 — Confirm the runtime

/opt/litellm/venv/bin/pip show litellm | grep ^Version

Enabling HTTPS

For production, terminate TLS at nginx with a real domain pointed at the VM's public IP. Install certbot and request a certificate (replace the domain):

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

Backup and maintenance

Your configuration lives in /etc/litellm/. Keep the OS patched with sudo apt update && sudo apt upgrade, and restart the proxy after config changes with sudo systemctl restart litellm. The proxy holds no model weights, so the image stays small and starts fast.

Support

This image is backed by 24/7 cloudimg support. Contact us by email and chat for help with provider integration, virtual keys and budgets, load balancing, TLS and scaling.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.