Bifrost on Ubuntu 24.04 on Azure User Guide

Overview

This guide covers the deployment and configuration of Bifrost on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images. Bifrost is a high performance LLM gateway from Maxim AI. It sits between your applications and the model providers you use, exposing one OpenAI compatible endpoint that fronts OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock, Vertex, Groq, Mistral, Ollama and many more.

Because the endpoint is OpenAI compatible, existing application code usually needs only a base URL change. Behind that single endpoint Bifrost gives you automatic failover between providers and keys, weighted load balancing, per key budgets and rate limits, semantic caching, MCP tool routing, and a full request log with cost and latency for every call.

Bifrost is a router, not a model host. It relays requests to providers using your API keys, so it needs no GPU and runs comfortably on a small CPU only VM. It is a single statically linked Go binary with the web dashboard compiled into it, which is why this appliance needs no container runtime, no language runtime and no database server.

The cloudimg image ships the free and open source, Apache-2.0 licensed Bifrost as the official upstream release binary, pinned by content hash and tied to an exact upstream commit. No model provider key is ever baked into the image — you add your own. A unique dashboard password and a unique virtual key are generated for each VM on first boot, and the gateway in front of Bifrost will not start until first boot has proved that an unauthenticated request is answered with 401. Backed by 24/7 cloudimg support.

Bifrost is an independent open source project. This image is produced by cloudimg and is not affiliated with, endorsed by, or sponsored by Maxim AI. It ships the free and open source Apache-2.0 licensed software, unmodified.

What is included:

• Bifrost v1.6.6 — the official upstream linux/amd64 release binary, pinned by sha256 content hash, not by a moving tag
• Verified provenance: Go stamps the source commit into the binary, and this appliance asserts it is a15edc24a8656950a10c6c3c60430b913cfb313b (upstream tag transports/v1.6.6) built from an unmodified tree
• The Bifrost web dashboard, compiled into the same binary
• An nginx access gateway on port 80, installed as a host package so unattended security upgrades keep the internet facing component patched
• Embedded SQLite configuration and log stores — no database server of any kind on the instance
• A unique dashboard password and a unique sk-bf-… virtual key generated per VM on first boot, never baked into the image
• A shipped dependency licence inventory at /usr/share/bifrost/dependency-licences.txt covering all 907 dependencies compiled into the binary

Prerequisites

• An Azure subscription with permission to create VMs
• An API key for at least one model provider (OpenAI, Anthropic, Google, Azure OpenAI, Bedrock, and so on) — you add this yourself after launch
• A Standard_B2s VM or larger. Bifrost is a single Go binary and is designed for sub 100 microsecond overhead at 5000 requests per second, so 2 vCPU and 4 GiB is genuinely sufficient for most deployments
• Inbound TCP 80 open to the networks your applications and administrators connect from
• An SSH key pair for administrative access

Step 1: Deploy from the Azure Portal

1. In the Azure Portal, search the Marketplace for Bifrost on Ubuntu 24.04 LTS by cloudimg and select Create.
2. Choose your subscription, resource group and region.
3. Set the VM size to Standard_B2s or larger.
4. Set the authentication type to SSH public key and supply your key.
5. Under Inbound port rules, allow SSH (22) and HTTP (80).
6. On the Disks tab, StandardSSD_LRS is a good balance of cost and performance.
7. Select Review + create, then Create.

First boot takes about a minute. During that minute the appliance generates this VM's credentials and proves its gateway rejects unauthenticated requests before opening port 80.

Step 2: Deploy from the Azure CLI

az vm create \
  --resource-group my-resource-group \
  --name bifrost-gateway \
  --image cloudimg:bifrost-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --storage-sku StandardSSD_LRS \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard

Then open the ports your applications need:

az vm open-port --resource-group my-resource-group --name bifrost-gateway --port 80 --priority 900

Restrict the source range to your own networks in production — this gateway relays requests that spend your provider credit.

Step 3: Connect to your VM

ssh azureuser@<vm-ip>

Step 4: Confirm the services are running

Three units make up the appliance. bifrost-firstboot.service runs once and generates this VM's secrets, bifrost.service is the gateway itself, and nginx.service is the access gate in front of it.

systemctl is-active bifrost.service nginx.service bifrost-firstboot.service

Expected output:

active
active
active

Confirm the gateway is bound to loopback only. This is what makes the nginx gate the sole way in — nothing can reach Bifrost directly from the network.

sudo ss -lntH | awk '{print $1, $4}' | grep -E ':(80|8080)$'

Expected output — Bifrost on 127.0.0.1:8080, nginx on every address:

LISTEN 127.0.0.1:8080
LISTEN 0.0.0.0:80

The gateway's own liveness literal is answered by nginx and reveals nothing about your configuration, so it is safe to point a load balancer probe at it:

curl -s http://127.0.0.1/nginx-health

Expected output:

OK

Step 5: Read the per instance credentials

Every secret this VM uses was generated on this VM at first boot. Nothing is shared with any other Bifrost image.

sudo cat /root/bifrost-credentials.txt

The file holds four values:

Value	What it is for
BIFROST_URL	The dashboard address for this VM
BIFROST_USERNAME	Dashboard username (always bifrost)
BIFROST_PASSWORD	Dashboard password, 32 characters, generated on this VM
BIFROST_VIRTUAL_KEY	The sk-bf-… virtual key your applications authenticate with

Confirm the file is readable only by root:

sudo stat -c '%a %U:%G' /root/bifrost-credentials.txt

Expected output:

600 root:root

Step 6: Open the dashboard

Browse to http://<vm-ip>/. Your browser will prompt for a username and password — use BIFROST_USERNAME and BIFROST_PASSWORD from the file above.

The dashboard is the full Bifrost control plane: observability, model providers, governance, guardrails, MCP gateway, adaptive routing and the prompt repository.

Step 7: Understand the access model

This appliance puts a default deny gate in front of Bifrost. That is deliberate, and it matters more here than for most products.

An LLM gateway relays requests to providers using your API keys. An unauthenticated gateway with a working provider key configured is, precisely, an open proxy for your inference spend — and unlike a chat interface it is machine callable by design. Bifrost's own authentication is off by default, is stored as a database toggle rather than a configuration file setting, ships an explicit switch to disable authentication on inference calls, and permanently exempts several routes regardless of settings. This appliance therefore does not rely on it.

Instead, nginx enforces the gate at server scope, so every route inherits it — the dashboard, the admin API, /metrics, and every OpenAI, Anthropic and Google compatible inference path. There is exactly one unauthenticated location and it is nginx's own /nginx-health literal, which never touches Bifrost.

Prove it. An anonymous request to the inference endpoint is refused:

curl -s -o /dev/null -w '%{http_code}\n' -X POST \
  -H 'Content-Type: application/json' -d '{}' \
  http://127.0.0.1/v1/chat/completions

Expected output:

401

So is an anonymous request to the metrics endpoint and to the governance API:

for R in /metrics /api/config /api/governance/virtual-keys /v1/models; do
  printf '%s -> %s\n' "$R" "$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1$R)"
done

Expected output:

/metrics -> 401
/api/config -> 401
/api/governance/virtual-keys -> 401
/v1/models -> 401

A credential that is wrong by a single character is refused too:

curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer <BIFROST_VIRTUAL_KEY>WRONG" \
  http://127.0.0.1/v1/models

Expected output:

401

The gate accepts six equivalent proofs, all compared exactly, all unique to this VM:

Proof	Who uses it
HTTP Basic with BIFROST_USERNAME / BIFROST_PASSWORD	Your browser, and curl -u
A gate cookie issued after the browser handshake	Your browser, on every dashboard API call
Authorization: Bearer <BIFROST_VIRTUAL_KEY>	The OpenAI SDK, LiteLLM, LangChain
x-api-key: <BIFROST_VIRTUAL_KEY>	The Anthropic SDK
x-goog-api-key: <BIFROST_VIRTUAL_KEY>	The Google GenAI SDK
x-bf-vk: <BIFROST_VIRTUAL_KEY>	Bifrost's native header

The last four are the header idioms Bifrost itself uses for virtual keys, so your application sends one credential that satisfies both the gate and Bifrost's own governance check. There is nothing extra to configure in your client.

Why the gate is not a plain password prompt

Two reasons, both of which break naive designs. A browser's stored HTTP Basic credential travels in the same Authorization header a single page application can overwrite, and an OpenAI or Anthropic SDK needs that same header for its API key. A plain auth_basic would therefore fight the SDK for the header and make legitimate programmatic use impossible. Accepting the virtual key in the header your SDK already uses solves both at once.

The gate cannot be started unconfigured

nginx.service carries two ConditionPathExists markers and an ExecStartPre guard. The second marker is created by first boot only after it has proved, against a throwaway loopback only listener running this VM's real gate configuration, that an unauthenticated request is answered with 401, that a one character mutation of every credential is refused, and that a 401 never leaks the gate cookie. If that proof fails, port 80 is never opened.

ls -l /var/lib/cloudimg/bifrost-gate.ready

Step 8: Add your own model provider

The image ships with zero providers configured. This is intentional: a baked provider key would hand every customer the same credential.

In the dashboard, go to Models → Model Providers → Add provider, choose your provider, and paste your own API key. Changes take effect immediately.

You can confirm from the shell that nothing is configured before you add it:

sudo python3 -c "import json;print('providers:', json.load(open('/var/lib/bifrost/data/config.json'))['providers'])"

Expected output on a freshly launched VM:

providers: {}

Step 9: Point your application at the gateway

Set the base URL to your VM and the API key to BIFROST_VIRTUAL_KEY. That single value authenticates through the gate and identifies the virtual key inside Bifrost.

With curl:

curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer <BIFROST_VIRTUAL_KEY>" \
  http://127.0.0.1/v1/models

Expected output — the gateway accepted the key:

200

With the OpenAI Python SDK, the only change to existing code is base_url and api_key:

from openai import OpenAI

client = OpenAI(
    base_url="http://<vm-ip>/v1",
    api_key="<BIFROST_VIRTUAL_KEY>",
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Once you have added a provider in Step 8, a chat completion through the gateway looks like this:

curl -X POST http://<vm-ip>/v1/chat/completions \
  -H "Authorization: Bearer <BIFROST_VIRTUAL_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

The model field is provider/model, so the same endpoint reaches every provider you have configured.

Step 10: Governance — budgets, rate limits and more keys

First boot created one virtual key for this VM. It is the credential in /root/bifrost-credentials.txt.

Confirm from the shell that exactly one key exists and that inference authentication is enforced:

curl -s -u "<BIFROST_USERNAME>:<BIFROST_PASSWORD>" http://127.0.0.1/api/governance/virtual-keys | jq '.virtual_keys | length'

Expected output:

1

curl -s -u "<BIFROST_USERNAME>:<BIFROST_PASSWORD>" http://127.0.0.1/api/config | jq '.client_config.enforce_auth_on_inference'

Expected output:

true

In Governance → Virtual Keys you can issue a key per team, application or customer, and attach a budget, a rate limit, an expiry and a restriction to specific providers or models. Give each application its own key rather than sharing the appliance key, so you can revoke and meter them independently.

Any key you issue in the dashboard is enforced by Bifrost. The nginx gate in front recognises only the appliance key from first boot, so if you want a per team key to reach the gateway from outside the VM, either put your applications on a network that reaches the VM through the appliance key, or terminate TLS and your own identity provider in front of the appliance and forward the appliance key from there.

Step 11: Where your data lives

sudo ls -la /var/lib/bifrost/data

The directory is 0700 root:root because it holds two sensitive things: the SQLite configuration store with your provider API keys, and the SQLite log store with the prompts and completions of every request that passes through.

sudo stat -c '%a %U:%G' /var/lib/bifrost/data

Expected output:

700 root:root

To back the appliance up, stop the gateway and copy that directory:

sudo systemctl stop bifrost
sudo tar czf /root/bifrost-backup-$(date +%F).tar.gz -C /var/lib/bifrost data
sudo systemctl start bifrost

If you do not want prompts and completions retained, turn the log store off in Observability → Logs Settings.

Step 12: What is in the image, and under what licence

Every dependency compiled into the shipped binary was enumerated and licence checked at build time, and the inventory ships with the image.

head -8 /usr/share/bifrost/dependency-licences.txt

awk -F'\t' '$1=="go"||$1=="npm"{print $4}' /usr/share/bifrost/dependency-licences.txt | sort | uniq -c | sort -rn | head -6

That inventory covers 225 Go modules read out of the binary's own build information and 682 npm packages from the pinned dashboard lockfile. Every one is permissive — MIT, Apache-2.0, BSD, ISC or MPL-2.0. There is no AGPL, SSPL, Business Source, Elastic Licence, Commons Clause or non commercial component.

Confirm the shipped binary really is the pinned upstream release:

LC_ALL=C grep -a -o 'vcs\.revision=[0-9a-f]\{40\}' /usr/local/bin/bifrost-http | head -1

Expected output:

vcs.revision=a15edc24a8656950a10c6c3c60430b913cfb313b

Day to day operation

systemctl status bifrost.service --no-pager --lines=0

sudo systemctl restart bifrost
sudo journalctl -u bifrost -f

Ubuntu's unattended security upgrades are enabled, so nginx and the rest of the OS keep receiving patches. Bifrost itself is a pinned binary and is upgraded by replacing it deliberately.

Security hardening

• Restrict port 80. This gateway spends your provider credit. Limit the network security group source range to the networks your applications and administrators use.
• Put TLS in front. The appliance serves plain HTTP so it works on a bare IP out of the box. For anything beyond a private network, terminate TLS at an Azure Application Gateway, a load balancer, or nginx with a certificate for your own hostname.
• Issue a virtual key per application in Governance, with a budget and a rate limit, rather than sharing the appliance key.
• Rotate the appliance key by editing it in Governance → Virtual Keys, then update /etc/nginx/conf.d/bifrost-gate-map.conf with the new value and sudo systemctl reload nginx.
• Review the request log retention in Observability → Logs Settings if prompts are sensitive.

Troubleshooting

The dashboard does not prompt for a password, or returns 401 forever. The gate is working; check you are using BIFROST_USERNAME and BIFROST_PASSWORD from /root/bifrost-credentials.txt exactly.

nginx will not start. It is held on two markers and a preflight guard by design. Check what it refused:

sudo /usr/local/sbin/bifrost-preflight.sh --nginx; echo "exit=$?"

Expected output on a healthy appliance:

exit=0

First boot did not complete. Read its log:

sudo journalctl -u bifrost-firstboot.service --no-pager | tail -20

Requests return a provider error. You have not added a provider yet — see Step 8. Until you do, the gateway authenticates and routes the request correctly but has nothing to relay it to.

Support

This image is supported 24/7 by cloudimg. Raise an issue at cloudimg.co.uk with the output of systemctl status bifrost and journalctl -u bifrost --no-pager | tail -50.