Networking AWS

BFE 1.8.8.6 on AWS User Guide

| Product: BFE 1.8.8.6 on Ubuntu 24.04 on AWS

Overview

This guide covers launching and verifying BFE 1.8.8.6 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened WireGuard VPN & VPN, Proxy and Remote Access AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other VPN, proxy and remote access servers in the listing are described on the listing's guide hub.

BFE is an open-source layer-7 load balancer and reverse proxy, a CNCF sandbox project that started at Baidu. It routes HTTP, HTTPS and HTTP/2 traffic by host name, path, header, cookie or client address to clusters of backend servers, balances load across them with health checks, and is extended through modules (header rewriting, redirects, rate limiting, access logs and more).

On this image BFE itself terminates HTTPS on port 443 with a certificate generated on your instance's first boot, and port 80 only redirects to HTTPS. Out of the box BFE routes every request for the instance's own names to a small demo backend on 127.0.0.1:8081, so you can see the whole path working before you point it at your own servers.

cloudimg compiles BFE 1.8.8.6 from the upstream release source with Go 1.27.3 and patched dependencies: golang.org/x/net 0.60.0, google.golang.org/grpc 1.83.2 and go-jose 4.1.4, which clear the Go security advisories the upstream dependency set carries. mod_auth_jwt is not included (upstream depends on an unfixed JWT library): BFE's JWT authentication module uses github.com/golang-jwt/jwt v3, which has an advisory with no fix in that major version (GO-2025-3553), so cloudimg leaves the module out of the binary. govulncheck on the shipped binary reports no vulnerabilities. The build record, with the source, toolchain and dependency checksums, is in /usr/share/doc/bfe/BUILD-PROVENANCE.txt.

What is on the image:

  • BFE 1.8.8.6 (bfe.service) running as the unprivileged bfe user, allowed only to bind ports 80 and 443
  • HTTPS on 443 served by BFE with a self-signed certificate and a TLS session-ticket key generated on your instance's first boot; port 80 only redirects to HTTPS
  • BFE's monitor, metrics and reload API on 127.0.0.1:8421 only
  • A demo backend (nginx, Ubuntu 24.04 package) on 127.0.0.1:8081, the default cluster until you add your own servers
  • Helpers: bfe-selftest, bfe-reload (test and hot-reload the configuration) and bfe-tls-cert (certificate and served names)
  • No metadata access: neither BFE nor nginx can reach the instance metadata service

Default user: ubuntu · Architecture: x86_64 · Root volume: 8 GiB gp3

Security posture

  • No default credential and no shared secret. BFE has no login. The upstream sample configuration ships published private keys, a published session-ticket key and two demo accounts; none of them is on this image. First boot generates the TLS key, certificate and session-ticket key for this instance only (mode 0640, owner root, group bfe).
  • Patched dependencies, and no unfixed JWT library. mod_auth_jwt is not included (upstream depends on an unfixed JWT library). A configuration that names it is refused with no module for mod_auth_jwt, so it can never be switched on by mistake. The other compiled-in modules work as upstream documents them.
  • Only your names are served. BFE answers only the names first boot read from the instance metadata (public IPv4 address, public DNS name, private IPv4 address) and localhost. A request for any other host name is refused and never reaches a backend.
  • Client addresses come from the connection, never from headers. BFE overwrites X-Forwarded-For, X-Real-Ip, X-Forwarded-Port and X-Forwarded-Host with the real peer address and the configured host name before a request reaches your backend. (Upstream BFE appends to whatever the client sent; this image replaces it.)
  • Internal services stay local. The monitor and reload API (8421) and the demo backend (8081) listen on 127.0.0.1 only. The monitor binds there through BFE's MonitorAddr setting.
  • TLS 1.2 with AEAD ciphers only (ECDHE with AES-GCM or ChaCha20-Poly1305) on port 443. BFE's TLS stack does not offer TLS 1.3.
  • No SSRF to instance credentials. bfe.service and nginx.service carry an IPAddressDeny rule for the metadata addresses (169.254.169.254, fd00:ec2::254, 168.63.129.16), so the proxy can never fetch instance credentials, whatever backend it is pointed at. BFE's start-up check also refuses a configuration that names the metadata service. Nothing broader is blocked: your backends can be anywhere. The AMI is registered IMDSv2 only.
  • Least privilege. BFE runs as the bfe system user with one capability (binding ports below 1024), a read-only system and a private /tmp.
  • Fails closed. BFE starts only after first boot has written this instance's key material and a ready marker, and every start runs a check and BFE's own configuration test first. If first boot cannot finish, BFE stays stopped and first boot tries again on the next boot.
  • Per-instance identity. SSH host keys and the machine ID are generated fresh on every instance.

Prerequisites

  1. An AWS account and permission to launch EC2 instances
  2. An EC2 key pair in the Region you launch in
  3. The servers you want to balance, reachable from the instance (in the same VPC, or over the internet)

Recommended instance type: m5.large. BFE is light; size the instance for your traffic.

Connecting to your instance

Version Operating system SSH user
BFE 1.8.8.6 on Ubuntu 24.04 Ubuntu 24.04 LTS ubuntu

Step 1: Launch the instance

Option A: AWS Marketplace console

  1. Open the cloudimg Hardened WireGuard VPN & VPN, Proxy and Remote Access AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
  2. Once the subscription is active, choose Launch your software.
  3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose BFE 1.8.8.6 on Ubuntu 24.04, then choose your Region.
  4. Pick an instance type (m5.large recommended; the software charge is $0.04 per vCPU per hour) and your key pair.
  5. Security group: the listing's default group allows SSH (22), HTTP (80, redirect only) and HTTPS (443). Restrict SSH to your own IP address.
  6. Choose your VPC and a public subnet (or attach an Elastic IP later), leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and the web ports from everyone, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDR range with your own:

SG=$(aws ec2 create-security-group --group-name bfe \
  --description "BFE layer-7 load balancer" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22  --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80  --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr 0.0.0.0/0
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --associate-public-ip-address \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=bfe}]'

You do not need --metadata-options: the AMI already makes IMDSv2 required.

Step 2: Connect and read the connection details

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot finishes within a minute of launch. It issues the TLS certificate for the instance's addresses, generates the session-ticket key and starts the demo backend and BFE. The details are in /root/bfe-endpoints.txt, readable only by root. Nothing in it is a secret:

sudo cat /root/bfe-endpoints.txt

# BFE 1.8.8.6 - generated on first boot by bfe-firstboot.service. Nothing here is a secret.
# The TLS key and session-ticket key in /etc/bfe/tls_conf were generated on this instance; none was baked into the image.

BFE_HOST=3.93.246.80
BFE_URL=https://3.93.246.80/
BFE_NAMES=3.93.246.80 ec2-3-93-246-80.compute-1.amazonaws.com 172.31.82.170 localhost 127.0.0.1
TLS_CERT_SHA256=F1:A5:69:3E:51:3E:83:1F:BE:03:54:19:1A:F0:82:79:20:8B:38:37:3E:38:8E:57:FA:2B:70:B0:60:16:E0:A4
BFE_MONITOR=http://127.0.0.1:8421/monitor
BFE_CONFIG=/etc/bfe/bfe.conf
BFE_BACKENDS=/etc/bfe/cluster_conf/cluster_table.data

# Open TCP 443 (and 80, which only redirects) in the security group to the clients that should reach BFE.
# Your backends:          edit /etc/bfe/cluster_conf/cluster_table.data, then sudo bfe-reload
# New name or Elastic IP: sudo bfe-tls-cert <name-or-ip> ...
# Self-test:              sudo bfe-selftest
# Guide:                  https://www.cloudimg.co.uk/guides/bfe-1-8-8-6-on-ubuntu-24-04-aws/

Step 3: Confirm BFE is running

BFE and the demo backend are active and first boot has completed. BFE listens on ports 80 and 443 on every interface; the monitor (8421) and the demo backend (8081) listen on 127.0.0.1 only:

systemctl is-active bfe.service nginx.service bfe-firstboot.service
bfe -v 2>/dev/null
sudo ss -Htlnp '( sport = :80 or sport = :443 or sport = :8421 or sport = :8081 )' | awk '{print $4}' | sort -u

active
active
active
bfe version: 1.8.8.6
*:443
*:80
127.0.0.1:8081
127.0.0.1:8421

Step 4: Run the self-test

The shipped self-test checks the whole path from the instance itself: an HTTPS request for the instance's name traverses BFE to the demo backend, the certificate served is this instance's, port 80 redirects to HTTPS, a host name the instance does not serve is refused, BFE's request counter moves, the monitor and demo backend are refused on the private address, and the instance metadata service is unreachable from inside BFE and nginx:

sudo bfe-selftest

ok: bfe and nginx are active
ok: monitor answers on 127.0.0.1:8421
ok: an HTTPS request for 3.93.246.80 traverses BFE to the loopback-only demo backend
ok: the certificate served on 443 is this instance's (F1:A5:69:3E:51:3E:83:1F...)
ok: port 80 redirects to https://3.93.246.80/; an unknown Host is refused on 80 and 443
ok: BFE's request counter moved (10 -> 14)
ok: monitor 8421 and demo backend 8081 are refused on 172.31.82.170
ok: metadata: 169.254.169.254 is unreachable from inside bfe and nginx
SELFTEST_OK

Step 5: Send traffic through BFE

A request for the instance's name over HTTPS reaches the demo backend, and BFE adds its own X-Proxied-By: bfe header; plain HTTP is redirected to HTTPS; a request for a name the instance does not serve is refused with HTTP 500 and never reaches a backend:

H=$(sudo sed -n 's/^BFE_HOST=//p' /root/bfe-endpoints.txt)
curl -sk -D - -o /tmp/bfe-page.html --resolve "$H:443:127.0.0.1" "https://$H/" | grep -iE '^(HTTP|x-proxied-by)'
grep -o '<title>[^<]*</title>' /tmp/bfe-page.html; rm -f /tmp/bfe-page.html
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H "Host: $H" http://127.0.0.1/docs
curl -sk --http1.1 -o /dev/null -w '%{http_code}\n' -H 'Host: www.example.org' https://127.0.0.1/

HTTP/2 200
x-proxied-by: bfe
<title>cloudimg BFE demo backend</title>
301 https://3.93.246.80/docs
500

Open https://<public-ip>/ in your own browser: after accepting the self-signed certificate you see the demo backend page, served through BFE.

Step 6: Point BFE at your own servers

BFE's routing has three parts: server_data_conf/host_rule.data maps host names to a product (cloudimg_demo on this image), server_data_conf/route_rule.data sends a product's requests to a cluster, and cluster_conf/cluster_table.data lists the servers in each cluster. The shipped cluster has one server, the demo backend:

sudo cat /etc/bfe/cluster_conf/cluster_table.data

{
    "Version": "cloudimg-1",
    "Config": {
        "cluster_demo": {
            "demo.local": [
                {
                    "Addr": "127.0.0.1",
                    "Name": "cloudimg_demo_backend",
                    "Port": 8081,
                    "Weight": 10
                }
            ]
        }
    }
}

Replace the demo server with your own (add as many entries as you have servers; Weight balances between them), and change the health check in /etc/bfe/server_data_conf/cluster_conf.data (CheckConf: Uri, Host, StatusCode) to a URL your servers answer. Then test and hot-reload the configuration; nothing is reloaded if the test fails:

sudo nano /etc/bfe/cluster_conf/cluster_table.data
sudo nano /etc/bfe/server_data_conf/cluster_conf.data
sudo bfe-reload

BFE ignores SIGHUP; bfe-reload (also behind sudo systemctl reload bfe) runs bfe -t on the configuration and then calls BFE's reload API for routing, clusters, names, TLS and module rules. Allow traffic from this instance's security group to your servers, and stop exposing them directly. The self-test's traversal check looks for the demo page, so once you have replaced the demo server it reports that line as failed; everything else still applies.

To route different host names or paths to different clusters, add products to host_rule.data and conditions to route_rule.data (for example req_path_prefix_in("/api/", false)); see the BFE routing documentation.

Step 7: Use your own domain name

bfe-tls-cert re-issues the self-signed certificate, updates the names BFE serves and hot-reloads BFE. With no argument it reads the instance's current addresses from the instance metadata (useful after attaching an Elastic IP); with arguments it uses the names you give, for example the DNS name your record points at this instance:

sudo bfe-tls-cert <www.example.com>
sudo bfe-selftest | tail -1

bfe-tls-cert always adds localhost and 127.0.0.1. To use a certificate from a public CA instead, copy its full chain to /etc/bfe/tls_conf/certs/cloudimg.crt and its key to /etc/bfe/tls_conf/certs/cloudimg.key (mode 0640, owner root, group bfe), make sure every DNS name in /etc/bfe/tls_conf/tls_rule_conf.data (SniConf) is in the certificate, then run sudo bfe-reload.

Step 8: Read BFE's metrics

The monitor API listens on 127.0.0.1:8421 only. It has no password, so keep it there: it also serves the reload endpoints. Counters are JSON:

curl -s 'http://127.0.0.1:8421/monitor/proxy_state?format=json' | python3 -c 'import json,sys; c=json.load(sys.stdin)["CounterData"]; [print(k, c[k]) for k in ("CLIENT_REQ_SERVED","HTTPS_CLIENT_REQ_SERVED","ERR_BK_FIND_PRODUCT")]'

CLIENT_REQ_SERVED 17
HTTPS_CLIENT_REQ_SERVED 5
ERR_BK_FIND_PRODUCT 9

ERR_BK_FIND_PRODUCT counts requests for host names BFE does not serve. The full list of monitor pages is at http://127.0.0.1:8421/monitor. To collect metrics from a server elsewhere, use an SSH tunnel or a local agent rather than opening port 8421.

Step 9: mod_auth_jwt is not included

mod_auth_jwt is not included (upstream depends on an unfixed JWT library). The binary does not contain the module or the JWT library at all, and the build record says why:

grep -ac mod_auth_jwt /usr/local/bin/bfe || true
grep -A2 'mod_auth_jwt is not included' /usr/share/doc/bfe/BUILD-PROVENANCE.txt

0
modules were raised in go.mod to clear Go security advisories, and mod_auth_jwt is not included (upstream depends on
an unfixed JWT library: github.com/golang-jwt/jwt v3.2.2, GO-2025-3553 / CVE-2025-30204, which has no v3 fix).
Licence: Apache-2.0 (LICENSE and NOTICE in this directory). Copyright The BFE Authors.

If Modules = mod_auth_jwt is added to /etc/bfe/bfe.conf, BFE's configuration test fails with no module for mod_auth_jwt and BFE does not start with it. To authenticate clients, put the authentication in your backend application, or use BFE's mod_auth_request module to ask an authentication service you run. cloudimg will include mod_auth_jwt again once BFE moves to a JWT library with the fix.

Step 10: Metadata access is blocked

Both services carry the metadata deny rule, and the self-test proved in Step 4 that the metadata service does not answer from inside either of them:

systemctl show -p IPAddressDeny bfe.service nginx.service | grep .

IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32
IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32

Even if a cluster were pointed at 169.254.169.254, BFE's start-up check would refuse the configuration, and the deny rule would block the connection anyway.

Step 11: How the fail-closed start works

BFE starts only once first boot has written /var/lib/cloudimg/bfe-ready. Every start runs a check (as root) that refuses to start BFE without this instance's key material, with a cluster that names the metadata service or with mod_auth_jwt, then BFE's own configuration test, and only then BFE as the bfe user:

systemctl cat bfe.service | grep -E '^(ConditionPathExists|ExecStartPre|User|AmbientCapabilities)='
sudo stat -c '%a %U:%G %n' /etc/bfe/tls_conf/certs/cloudimg.key /etc/bfe/tls_conf/session_ticket_key.data /root/bfe-endpoints.txt
sudo grep -E '^(MonitorAddr|MinTlsVersion)' /etc/bfe/bfe.conf

ConditionPathExists=/var/lib/cloudimg/bfe-ready
User=bfe
ExecStartPre=+/usr/local/sbin/bfe-preflight /etc/bfe
ExecStartPre=/usr/local/bin/bfe -t -c /etc/bfe -l /var/log/bfe
AmbientCapabilities=CAP_NET_BIND_SERVICE
640 root:bfe /etc/bfe/tls_conf/certs/cloudimg.key
640 root:bfe /etc/bfe/tls_conf/session_ticket_key.data
600 root:root /root/bfe-endpoints.txt
MonitorAddr = 127.0.0.1
MinTlsVersion = VersionTLS12

Step 12: Security recommendations

  • Keep SSH open only to your management addresses. Open 80 and 443 to the clients BFE serves.
  • Put your backend servers where only this instance can reach them, so clients cannot bypass BFE.
  • Use your own DNS name and a CA certificate (Step 7) for a public service.
  • Keep the monitor API on 127.0.0.1: it has no authentication and can reload the configuration.
  • Leave unattended security upgrades on. BFE is a single binary in /usr/local/bin/bfe; cloudimg ships updated versions as new versions of this listing.

Support and licensing

BFE is open-source software by the BFE Authors, distributed under the Apache License 2.0; the licence, the NOTICE file and cloudimg's build record (source, toolchain, the dependency changes and their checksums, plus a source offer) are in /usr/share/doc/bfe/. cloudimg provides the packaging, the patched build, the secure defaults, the per-instance certificate and session-ticket key, the metadata block, the fail-closed start, the self-test and helpers, and 24/7 support.

cloudimg is not affiliated with or endorsed by the BFE project, the CNCF or Baidu. BFE is used here only to identify the software. nginx is a trademark of F5, Inc.

For help with deployment or configuration, email support@cloudimg.co.uk or visit cloudimg support.