Bifrost 1.6.6 on AWS User Guide

Overview

This guide covers launching and verifying Bifrost 1.6.6 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other LLM servers, gateways and chat interfaces in the listing are described on the listing's guide hub.

Bifrost is an open-source, high-performance LLM gateway from Maxim AI. It puts one OpenAI-compatible endpoint in front of the model providers you use (OpenAI, Anthropic, Google, Amazon Bedrock, Azure OpenAI, Vertex AI, Groq, Mistral, Ollama and more), so existing application code usually needs only a base URL change. Behind that endpoint it adds automatic failover between providers and keys, weighted load balancing, per-key budgets and rate limits, semantic caching, MCP tool routing and a request log with cost and latency for every call.

Bifrost is a router, not a model host: it relays requests to providers using your own API keys, so it needs no GPU and runs on a small CPU instance. It is a single Go binary with the web dashboard compiled in, which is why this image needs no container runtime and no database server.

What is on the image:

• Bifrost 1.6.6, the official upstream linux/amd64 release binary pinned by sha256 and tied to upstream commit a15edc24a8656950a10c6c3c60430b913cfb313b (tag transports/v1.6.6), run by bifrost.service as the unprivileged bifrost user
• Bifrost bound to loopback only (127.0.0.1:8080), with nginx as the only network-facing service
• HTTPS on port 443 with a self-signed certificate created on your instance's first boot; port 80 redirects to HTTPS
• A per-instance dashboard password and virtual key (sk-bf-...), generated on first boot into a root-only file
• Embedded SQLite configuration and log stores in /var/lib/bifrost/data (mode 0700), and no provider configured: you add your own keys
• bifrost-set-hostname, a helper that serves Bifrost under your own DNS name and re-issues the certificate for it
• A dependency licence inventory at /usr/share/bifrost/dependency-licences.txt covering all 907 dependencies compiled into the binary

Default user: ubuntu · Architecture: x86_64 · Root volume: 8 GiB gp3

Security posture

• No default credential and no baked provider key. The dashboard password, the virtual key and the gate token are generated on each instance's first boot. None of them exists in the image, so no two instances share one, and no model provider key is ever shipped.
• Default-deny gate in front of every route. An LLM gateway spends your provider credit, and Bifrost's own authentication is off by default and exempts some routes. So nginx enforces the gate at server scope: the dashboard, the admin API, /metrics and every OpenAI, Anthropic and Google compatible inference path need this instance's credential. The only open location is nginx's own /nginx-health literal, which never reaches Bifrost.
• Fails closed. nginx will not start until first boot has proved, against this instance's real gate configuration, that an anonymous request gets 401 and a one-character mutation of every credential is refused. Both units carry a preflight guard that refuses an empty or plaintext gate file.
• Only nginx faces the network. It answers only for the configured hostname (any other Host header gets its connection closed), uses TLS 1.2 or 1.3, refuses cross-origin POST requests and WebSocket upgrades (403), strips Bifrost's CORS headers, and replaces any client-supplied X-Forwarded-For, X-Real-IP or Forwarded header with the real peer address.
• The hostname comes from configuration, never from a request. First boot reads it from EC2 instance metadata (IMDSv2) or from /etc/cloudimg/bifrost-hostname.
• No instance metadata access for the gateway. An administrator can point a provider at any base URL, so bifrost.service is blocked from the instance metadata service (169.254.169.254 and fd00:ec2::254) by systemd IPAddressDeny. Bifrost 1.6.6 also refuses link-local and private base URLs itself, both when a provider is saved and when it connects. A provider entry can never read the instance's role credentials. Amazon Time Sync (169.254.169.123) is unaffected.
• IMDSv2 required. The AMI is registered with ImdsSupport v2.0, so instances launched from it refuse token-less (IMDSv1) metadata requests.
• Sandboxed service and per-instance identity. bifrost.service runs with NoNewPrivileges, ProtectSystem=full, ProtectHome and PrivateTmp. SSH host keys and the machine ID are generated fresh on every instance.

Prerequisites

1. An AWS account and permission to launch EC2 instances
2. An EC2 key pair in the Region you launch in
3. An API key for at least one model provider (OpenAI, Anthropic, Bedrock and so on), or a self-hosted model server such as Ollama

Recommended instance type: m5.large. Bifrost is a gateway and holds no model, so its 2 vCPUs and 8 GiB of memory are ample; the model work happens at your providers.

Connecting to your instance

Version	Operating system	SSH user
Bifrost 1.6.6 on Ubuntu 24.04	Ubuntu 24.04 LTS	ubuntu

Step 1: Launch the instance

Option A: AWS Marketplace console

1. Open the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
2. Once the subscription is active, choose Launch your software.
3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose Bifrost 1.6.6 on Ubuntu 24.04, then choose your Region.
4. Pick an instance type (m5.large recommended; the software charge is $0.04 per vCPU per hour) and your key pair.
5. Security group: the listing's default group opens SSH (22), HTTP (80) and HTTPS (443). Restrict SSH to your own IP address, and restrict 80 and 443 to the networks your applications and administrators use. This gateway spends your provider credit.
6. Choose your VPC and subnet, leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and HTTPS from your application network, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDR ranges with your own:

SG=$(aws ec2 create-security-group --group-name bifrost-gateway \
  --description "Bifrost gateway" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr 10.0.0.0/16
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80 --cidr 10.0.0.0/16
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=bifrost-gateway}]'

You do not need --metadata-options: the AMI already makes IMDSv2 required.

Step 2: Connect and find your credentials

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot finishes within a minute of launch. It sets the hostname Bifrost is served as (the instance's public DNS name when it has one), creates the TLS certificate for it, generates the dashboard password, the virtual key and the gate token, proves the gate refuses anonymous and wrong credentials, starts Bifrost and nginx, and writes /root/bifrost-credentials.txt (mode 0600, root). These are the entries in that file (the secret values are not shown here):

sudo grep -E '^BIFROST_(URL|USERNAME)=' /root/bifrost-credentials.txt
sudo grep -oE '^[A-Z_]+=' /root/bifrost-credentials.txt
sudo stat -c '%a %U:%G' /root/bifrost-credentials.txt

BIFROST_URL=https://ec2-44-211-166-21.compute-1.amazonaws.com/
BIFROST_USERNAME=bifrost
BIFROST_URL=
BIFROST_USERNAME=
BIFROST_PASSWORD=
BIFROST_VIRTUAL_KEY=
600 root:root

Value	What it is for
BIFROST_URL	The dashboard and API address of this instance
BIFROST_USERNAME	Dashboard username (always bifrost)
BIFROST_PASSWORD	Dashboard password, generated on this instance
BIFROST_VIRTUAL_KEY	The sk-bf-... virtual key your applications authenticate with

To read the password and key, run sudo cat /root/bifrost-credentials.txt in your own session and store them in your secrets manager.

Step 3: Confirm the gateway is running

The first-boot unit, Bifrost and nginx are active, and only sshd and nginx listen beyond loopback. Bifrost itself is on 127.0.0.1:8080, so the nginx gate is the only way in.

systemctl is-active bifrost-firstboot.service bifrost.service nginx.service
sudo ss -Htlnp | awk '{print $4, $6}' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort -u
curl -sk https://127.0.0.1/nginx-health

active
active
active
0.0.0.0:22 users:(("sshd"),("systemd"))
0.0.0.0:443 users:(("nginx"),("nginx"),("nginx"))
0.0.0.0:80 users:(("nginx"),("nginx"),("nginx"))
127.0.0.1:8080 users:(("bifrost-http"))
127.0.0.53%lo:53 users:(("systemd-resolve"))
127.0.0.54:53 users:(("systemd-resolve"))
[::]:22 users:(("sshd"),("systemd"))
[::]:443 users:(("nginx"),("nginx"),("nginx"))
[::]:80 users:(("nginx"),("nginx"),("nginx"))
OK

/nginx-health is answered by nginx itself and reveals nothing about your configuration, so it is safe to use as a load balancer health check.

Step 4: Prove the gate

Anonymous requests to the inference endpoints, the metrics and the admin API are all refused, and so is a virtual key that is wrong by one character. This instance's own key is accepted:

for R in /v1/models /metrics /api/config /api/governance/virtual-keys /api/providers; do
  printf '%s -> %s\n' "$R" "$(curl -sk -o /dev/null -w '%{http_code}' https://127.0.0.1$R)"
done
curl -sk -o /dev/null -w 'anonymous POST /v1/chat/completions -> %{http_code}\n' \
  -X POST -H 'Content-Type: application/json' -d '{}' https://127.0.0.1/v1/chat/completions
K=$(sudo grep '^BIFROST_VIRTUAL_KEY=' /root/bifrost-credentials.txt | cut -d= -f2-)
curl -sk -o /dev/null -w 'wrong key -> %{http_code}\n' -K <(printf 'header = "Authorization: Bearer %sX"\n' "$K") https://127.0.0.1/v1/models
curl -sk -o /dev/null -w 'this instance key -> %{http_code}\n' -K <(printf 'header = "Authorization: Bearer %s"\n' "$K") https://127.0.0.1/v1/models

/v1/models -> 401
/metrics -> 401
/api/config -> 401
/api/governance/virtual-keys -> 401
/api/providers -> 401
anonymous POST /v1/chat/completions -> 401
wrong key -> 401
this instance key -> 200

The gate accepts six equivalent proofs, all compared exactly and all unique to this instance:

Proof	Who uses it
HTTP Basic with BIFROST_USERNAME / BIFROST_PASSWORD	Your browser, and curl -u
A gate cookie issued after the browser signs in (Secure, HttpOnly, SameSite=Lax)	Your browser, on every dashboard API call
Authorization: Bearer <virtual key>	The OpenAI SDK, LiteLLM, LangChain
x-api-key: <virtual key>	The Anthropic SDK
x-goog-api-key: <virtual key>	The Google GenAI SDK
x-bf-vk: <virtual key>	Bifrost's native header

The last four are the header idioms Bifrost itself uses for virtual keys, so your application sends one credential that satisfies both the gate and Bifrost's own governance check.

Step 5: Open the dashboard

Browse to BIFROST_URL (https://<public-dns-name>/). The certificate is self-signed, so your browser warns once; accept it for this host (or install your own certificate, see Step 8). The browser then asks for a username and password: use BIFROST_USERNAME and BIFROST_PASSWORD.

The dashboard is the full Bifrost control plane: observability, model providers, governance, guardrails, the MCP gateway, routing rules and the prompt repository. On a new instance Models → Model Providers is empty:

Step 6: Add your own model provider

The image ships with no provider configured, so no two customers ever share a key. You can confirm it from the shell:

U=$(sudo grep '^BIFROST_USERNAME=' /root/bifrost-credentials.txt | cut -d= -f2-)
P=$(sudo grep '^BIFROST_PASSWORD=' /root/bifrost-credentials.txt | cut -d= -f2-)
curl -sk -K <(printf 'user = "%s:%s"\n' "$U" "$P") https://127.0.0.1/api/providers; echo

{"providers":[],"total":0}

In the dashboard, go to Models → Model Providers → Add provider, choose your provider and paste your own API key. Changes take effect immediately. Provider keys are stored in the SQLite configuration store under /var/lib/bifrost/data, which only the bifrost user can read.

A self-hosted model server in your VPC (for example an Ollama instance from the same listing) is added the same way with its base URL. Bifrost refuses link-local and private-range base URLs by default, so check the provider's network settings in the Bifrost documentation if you route to a private address.

Step 7: Point your application at the gateway

Set your SDK's base URL to https://<BIFROST_HOST>/v1 and its API key to BIFROST_VIRTUAL_KEY. That single value authenticates through the gate and identifies the virtual key inside Bifrost. Copy the instance's certificate (/etc/ssl/bifrost/bifrost.crt) to your client and trust it, or put a certificate your clients already trust in front (Step 8).

With the OpenAI Python SDK, the only change to existing code is base_url and api_key:

from openai import OpenAI

client = OpenAI(
    base_url="https://<BIFROST_HOST>/v1",
    api_key="<virtual key>",
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Once you have added a provider, a chat completion through the gateway looks like this:

curl --cacert bifrost.crt -X POST https://<BIFROST_HOST>/v1/chat/completions \
  -H "Authorization: Bearer <virtual key>" -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

The model field is provider/model, so the same endpoint reaches every provider you configure.

Governance: budgets, rate limits and more keys

First boot created one virtual key, the cloudimg appliance key, and enforces authentication on inference. Exactly one key exists on a new instance:

U=$(sudo grep '^BIFROST_USERNAME=' /root/bifrost-credentials.txt | cut -d= -f2-)
P=$(sudo grep '^BIFROST_PASSWORD=' /root/bifrost-credentials.txt | cut -d= -f2-)
curl -sk -K <(printf 'user = "%s:%s"\n' "$U" "$P") https://127.0.0.1/api/governance/virtual-keys | jq -r '.virtual_keys[] | [.name, .is_active] | @tsv'
curl -sk -K <(printf 'user = "%s:%s"\n' "$U" "$P") https://127.0.0.1/api/config | jq '.client_config.enforce_auth_on_inference'

cloudimg appliance key  true
true

In Governance → Virtual Keys you can issue a key per team, application or customer, with a budget, a rate limit, an expiry and a restriction to specific providers or models. A new key starts with no provider allowed: add the providers it may use. Keys you issue in the dashboard are enforced by Bifrost, but the nginx gate in front recognises only the appliance key and the dashboard credential, so a request that carries only a new key is refused at the gate with 401. To give each application its own key from outside the instance, put your own identity-aware proxy or API gateway in front of this instance and have it forward the appliance key.

Step 8: Serve Bifrost under your own DNS name

Port 80 redirects to HTTPS on the configured hostname, and the certificate covers that name, the instance's public and private addresses, localhost and 127.0.0.1:

H=$(sudo grep '^BIFROST_URL=' /root/bifrost-credentials.txt | cut -d= -f2- | sed -E 's#https://([^/]+)/#\1#')
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' "http://$H/v1/models"
openssl x509 -in /etc/ssl/bifrost/bifrost.crt -noout -subject -enddate -ext subjectAltName

301 https://ec2-44-211-166-21.compute-1.amazonaws.com/v1/models
subject=CN = ec2-44-211-166-21.compute-1.amazonaws.com, O = cloudimg
notAfter=Jan 21 19:07:58 2029 GMT
X509v3 Subject Alternative Name: 
    DNS:ec2-44-211-166-21.compute-1.amazonaws.com, IP Address:44.211.166.21, IP Address:172.31.90.0, IP Address:127.0.0.1, DNS:localhost

To use your own name, point a DNS record at the instance (an Elastic IP keeps the address stable across stops), then run the helper. It records the name in /etc/cloudimg/bifrost-hostname, issues a new self-signed certificate for it, updates the allowed origins and the credentials file, and reloads nginx:

sudo bifrost-set-hostname <your-dns-name>

For a publicly trusted certificate, put the instance behind an Application Load Balancer with an AWS Certificate Manager certificate, or replace /etc/ssl/bifrost/bifrost.crt and bifrost.key with your own and run sudo systemctl reload nginx.

Step 9: Check the security controls

Bifrost's service carries the metadata block, nginx refuses cross-origin writes and requests for any hostname it was not configured with, and Bifrost itself refuses a provider pointed at the metadata address:

systemctl show -p IPAddressDeny bifrost.service
H=$(sudo grep '^BIFROST_URL=' /root/bifrost-credentials.txt | cut -d= -f2- | sed -E 's#https://([^/]+)/#\1#')
U=$(sudo grep '^BIFROST_USERNAME=' /root/bifrost-credentials.txt | cut -d= -f2-)
P=$(sudo grep '^BIFROST_PASSWORD=' /root/bifrost-credentials.txt | cut -d= -f2-)
curl -s -o /dev/null -w 'cross-origin POST: %{http_code}\n' --cacert /etc/ssl/bifrost/bifrost.crt -K <(printf 'user = "%s:%s"\n' "$U" "$P") \
  -X POST -H 'Origin: https://evil.example' -H 'Content-Type: application/json' -d '{}' "https://$H/api/governance/virtual-keys"
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Host: evil.example' http://127.0.0.1/ || true); echo "unknown Host: ${code} (connection closed by nginx)"
curl -s --cacert /etc/ssl/bifrost/bifrost.crt -K <(printf 'user = "%s:%s"\n' "$U" "$P") -H "Origin: https://$H" -H 'Content-Type: application/json' \
  -X POST -d '{"provider":"openai","network_config":{"base_url":"http://169.254.169.254/latest/meta-data"}}' "https://$H/api/providers"; echo

IPAddressDeny=168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32
cross-origin POST: 403
unknown Host: 000 (connection closed by nginx)
{"is_bifrost_error":false,"status_code":400,"error":{"message":"Invalid base URL: link-local IP addresses are not allowed"},"extra_fields":{"routing_info":{}}}

The cloudimg build also proves, before the image is captured, that a provider whose hostname is later re-pointed at 169.254.169.254 (DNS rebinding) is refused when Bifrost connects, and that a forged X-Forwarded-For never reaches Bifrost.

Step 10: Where your data lives

/var/lib/bifrost/data holds the SQLite configuration store (your provider API keys) and the SQLite log store (the prompts and completions of every request), so only the bifrost user can read it:

sudo stat -c '%a %U:%G' /var/lib/bifrost/data

700 bifrost:bifrost

If you do not want prompts and completions kept, turn content logging off (or shorten the retention) in Observability → Logs Settings:

To back the gateway up, stop it and copy the directory (or take an EBS snapshot of the instance):

sudo systemctl stop bifrost
sudo tar czf /root/bifrost-backup-$(date +%F).tar.gz -C /var/lib/bifrost data
sudo systemctl start bifrost

Step 11: What is in the image, and under what licence

Every dependency compiled into the binary was enumerated and licence-checked at build time, and the inventory ships with the image. They are all permissive (MIT, Apache-2.0, BSD, ISC, MPL-2.0); there is no AGPL, SSPL, Business Source, Elastic Licence, Commons Clause or non-commercial component. The binary carries the upstream commit it was built from:

head -8 /usr/share/bifrost/dependency-licences.txt
awk -F'\t' '$1=="go"||$1=="npm"{print $4}' /usr/share/bifrost/dependency-licences.txt | sort | uniq -c | sort -rn | head -6
LC_ALL=C grep -a -o 'vcs\.revision=[0-9a-f]\{40\}' /usr/local/bin/bifrost-http | head -1

# Bifrost appliance — dependency licence inventory (cloudimg)
#
# Every dependency compiled into /usr/local/bin/bifrost-http.
# Go modules  : 225 (read from the shipped binary's build info)
# npm packages: 682 (pinned ui/package-lock.json — the UI is
#               compiled into the same binary)
# TOTAL       : 907
#
    668 MIT
    106 Apache-2.0
     39 BSD-3-Clause
     31 ISC
     25 MPL-2.0
     16 Apache-2.0,BSD-3-Clause
vcs.revision=a15edc24a8656950a10c6c3c60430b913cfb313b

Maintenance

• Logs: sudo journalctl -u bifrost.service -f for the gateway, /var/log/nginx/ for HTTPS access.
• Restart: sudo systemctl restart bifrost.service.
• Operating system updates: Ubuntu's unattended security upgrades stay on, so nginx and the rest of the OS keep receiving patches. Apply the rest with sudo apt update && sudo apt upgrade in a maintenance window.
• Upgrading Bifrost: Bifrost is a pinned binary. Upgrade deliberately by replacing /usr/local/bin/bifrost-http with a newer official release, then restart the service.
• Rotating the appliance key: edit it in Governance → Virtual Keys, put the new value in /etc/nginx/conf.d/bifrost-gate-map.conf and /root/bifrost-credentials.txt, then run sudo systemctl reload nginx.

Troubleshooting

The browser keeps asking for a password. Use BIFROST_USERNAME and BIFROST_PASSWORD from /root/bifrost-credentials.txt exactly, over https://.

nginx will not start. It is held on the gate markers and a preflight guard by design. Check what it refused:

sudo /usr/local/sbin/bifrost-preflight.sh --nginx >/dev/null 2>&1; echo "exit=$?"

exit=0

First boot did not complete. Read its log with sudo journalctl -u bifrost-firstboot.service --no-pager | tail -20.

Requests return a provider error. No provider has been added yet (Step 6), or the virtual key you use is not allowed that provider (Governance → Virtual Keys).

Support

cloudimg provides 24/7 support for this image by email and live chat: launching and connecting, provider configuration, TLS and hostnames, governance and upgrades.

Trademarks

Bifrost is developed by Maxim AI. Ubuntu is a registered trademark of Canonical Ltd. All other product and company names are trademarks or registered trademarks of their respective holders. cloudimg is not affiliated with or endorsed by them.