Apache BifroMQ MQTT Broker on AWS User Guide

Overview

This guide covers launching and using Apache BifroMQ MQTT Broker on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Apache Kafka, RabbitMQ & Message Broker AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other brokers and streaming platforms in the listing are described on the listing's guide hub.

Apache BifroMQ (Incubating) is a high-performance MQTT broker written in Java and built around native multi-tenancy. It implements MQTT 3.1, 3.1.1 and 5.0, and every connection belongs to a tenant whose topics, subscriptions and sessions are isolated from every other tenant on the same broker.

Stock BifroMQ is not safe to expose: with no authentication plugin configured it accepts every connection and grants every action, its authorisation check grants actions when the auth service errors, its administrative HTTP API has no authentication, and it ships no TLS certificate. The cloudimg image closes each of those.

What is on the image:

• Apache BifroMQ 4.0.0-incubating, the official Apache Software Foundation binary release, verified against the published SHA-512 checksum and the ASF release signature, under /opt/bifromq/current (bifromq.service)
• OpenJDK 21 (headless) from Ubuntu's main archive, patched by unattended upgrades
• MQTT over TLS on port 8883, with an ECDSA key and self-signed certificate created on your instance's first boot (TLS 1.2 or newer)
• One MQTT credential, user cloudimg/admin, with a password generated on your instance's first boot, checked by bifromq-authd.service, a small authentication service on 127.0.0.1:8099
• Plain MQTT on 127.0.0.1:1883 only, for programs running on the instance itself
• A self-test, sudo bifromq-selftest, a test client, sudo bifromq-mqtt-probe, and a certificate helper, sudo bifromq-tls-cert
• The Apache-2.0 LICENSE, NOTICE and incubation DISCLAIMER in /usr/share/doc/bifromq/, with the verified download recorded in BUILD-PROVENANCE.txt

Default user: ubuntu · Architecture: x86_64 · Root volume: 8 GiB gp3

Security posture

• No default login. The MQTT password is created on each instance's first boot and exists nowhere else. The image ships a credential store with no entry, so until first boot every connection is refused.
• Authentication required. BifroMQ is configured with its webhook authentication plugin, backed by bifromq-authd. A client with no credentials, an unknown user name or a wrong password is refused at connect (MQTT 3.1.1 return code 4, MQTT 5 reason code 0x86).
• Authorisation stays closed. Upstream BifroMQ allows a publish or subscribe when its authorisation check fails. This image runs the broker with -DByPassPermCheckError=false, so if bifromq-authd is down, connected clients cannot publish until it is back (it restarts itself).
• TLS by default. The only MQTT port open to the network is 8883, MQTT over TLS, with a certificate made for this instance's addresses. Plain MQTT 1883, the administrative API (8091) and the authentication service (8099) listen on the loopback interface only, and BifroMQ's internal RPC and cluster ports are bound to loopback too (this is a single-node broker).
• A host firewall for the loopback services. BifroMQ's bundled plugin opens a Prometheus metrics endpoint on port 9090 on every interface, and that address cannot be configured. bifromq-firewall.service loads an nftables table that drops any packet to 1883, 8091, 8099 or 9090 that does not come from the instance itself. The broker does not start without it.
• The listing security group opens SSH (22) and MQTT over TLS (8883). Restrict both to the networks that need them.
• No metadata access for the broker. bifromq.service and bifromq-authd.service are blocked from reaching the instance metadata service (169.254.169.254 and fd00:ec2::254). Only cloudimg's first-boot unit and the certificate helper read the instance's addresses, over IMDSv2.
• IMDSv2 required. The AMI is registered with ImdsSupport v2.0, so instances launched from it refuse token-less (IMDSv1) metadata requests.
• Fails closed. The broker and the authentication service start only after first boot has created the credential and the TLS key. If first boot cannot finish, both stay stopped and first boot runs again on the next boot.
• Sandboxed services. Both services run as the unprivileged bifromq user with no capabilities and systemd sandboxing (read-only system directories, private /tmp, no new privileges). The BifroMQ files and configuration are owned by root.
• Per-instance identity. SSH host keys and the machine ID are generated fresh on every instance. SSH accepts keys only, and root cannot log in.

Prerequisites

1. An AWS account and permission to launch EC2 instances
2. An EC2 key pair in the Region you launch in
3. A VPC subnet, and the address ranges of the devices or services that will connect to the broker

Recommended instance type: m5.large (2 vCPU, 8 GiB). The broker is sized to run in 4 GiB, so t3.medium also works for small fleets; see Step 8 to give it more memory on a larger instance.

Connecting to your instance

Step 1: Launch the instance

Option A: AWS Marketplace console

1. Open the cloudimg Hardened Apache Kafka, RabbitMQ & Message Broker AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
2. Once the subscription is active, choose Launch your software.
3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose Apache BifroMQ MQTT Broker on Ubuntu 24.04, then choose your Region.
4. Pick an instance type (m5.large recommended; the software charge is $0.04 per vCPU per hour) and your key pair.
5. Security group: the listing's default group allows SSH (22) and MQTT over TLS (8883). Restrict 22 to your own IP address and 8883 to the networks your clients use.
6. Choose your VPC and subnet, leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and MQTT over TLS from your client network, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and address ranges with your own:

SG=$(aws ec2 create-security-group --group-name bifromq-mqtt \
  --description "Apache BifroMQ MQTT broker" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 8883 --cidr 10.0.0.0/16
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=bifromq}]'

You do not need --metadata-options: the AMI already makes IMDSv2 required.

Step 2: Connect and read your credentials

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot runs within a minute of launch. It creates the TLS certificate and the MQTT credential and writes /root/bifromq-credentials.txt, which only root can read; the authentication service and the broker start straight after. The Java broker then takes about a minute to open its stores before it serves its first message.

sudo cat /root/bifromq-credentials.txt

# Apache BifroMQ (Incubating) 4.0.0-incubating - generated on first boot by bifromq-firstboot.service.
# Unique to this instance; nothing here was baked into the image. Store it somewhere safe.

BIFROMQ_HOST=54.210.10.20
BIFROMQ_PRIVATE_IP=172.31.20.15
BIFROMQ_URL=mqtts://54.210.10.20:8883
BIFROMQ_MQTT_USERNAME=cloudimg/admin
BIFROMQ_MQTT_PASSWORD=<MQTT_PASSWORD>
BIFROMQ_TLS_CA_FILE=/etc/bifromq/tls/server.crt
BIFROMQ_TLS_CERT_SHA256=3A:9C:41:D2:7E:05:B8:6F:...
BIFROMQ_LOCAL_PLAINTEXT=mqtt://127.0.0.1:1883
...

Your file shows your own addresses, password and certificate fingerprint. Keep the password safe: it is the only credential the broker accepts.

Step 3: Confirm the broker is running

Four units make up the broker: the first-boot unit, the loopback firewall, the authentication service and BifroMQ itself. All four should be active:

systemctl is-active bifromq-firstboot bifromq-firewall bifromq-authd bifromq
curl -s http://127.0.0.1:8091/cluster | head -c 300; echo

Then run the self-test. It waits for the broker to finish warming up, publishes and receives a message over TLS and over local plain MQTT, checks that anonymous, unknown-user and wrong-password clients are refused on both (MQTT 3.1.1 and 5), that a client which does not trust the instance certificate is refused, that TLS 1.1 is refused, that the only network ports are 22 and 8883 (plus 9090, which the firewall drops), and that the broker cannot reach the instance metadata service:

sudo bifromq-selftest

READY round-trips served on tls and plain after 37s
ROUNDTRIP_OK tls (QoS 1, instance credential, delivered after 1 publish(es))
ROUNDTRIP_OK plain (QoS 1, instance credential, delivered after 2 publish(es))
ACCEPTED tls mqtt4 CONNACK 0
REFUSED tls mqtt4 anonymous: CONNACK 0x04
REFUSED tls mqtt4 wrong-password: CONNACK 0x04
REFUSED tls mqtt4 unknown-user: CONNACK 0x04
ACCEPTED tls mqtt5 CONNACK 0
REFUSED tls mqtt5 anonymous: CONNACK 0x86
...
UNTRUSTED_REJECTED system CA bundle and foreign hostname do not verify
TLS 1.1 refused; TLS 1.2 verifies against this instance's certificate
listeners: off-box TCP {22,8883,9090} (9090 dropped off-box), 1883/8091/8099 on 127.0.0.1
metadata: 169.254.169.254 is unreachable from inside bifromq and bifromq-authd (shell control 200)
BIFROMQ_SELFTEST_OK

Step 4: Publish and subscribe on the instance

The image includes a small test client, bifromq-mqtt-probe, that reads the credential from the credentials file itself, so you never type the password. A round trip over TLS (port 8883) and over local plain MQTT (127.0.0.1:1883):

sudo bifromq-mqtt-probe roundtrip --transport tls
sudo bifromq-mqtt-probe roundtrip --transport plain

ROUNDTRIP_OK tls (QoS 1, instance credential, delivered after 2 publish(es))
ROUNDTRIP_OK plain (QoS 1, instance credential, delivered after 1 publish(es))

Two BifroMQ behaviours are worth knowing before you connect your own clients:

• Use QoS 1 where delivery matters. BifroMQ adds a new subscription to its routing table asynchronously, about a second after it acknowledges the SUBSCRIBE. A QoS 0 message published in that window has no subscriber yet and is dropped, which is correct MQTT behaviour for best-effort delivery. The probe republishes QoS 1 messages until one arrives, which is why it reports how many publishes it took.
• Publisher and subscriber must share a tenant. The tenant is the part of the user name before the slash (cloudimg in cloudimg/admin). Clients in different tenants never see each other's messages and get no error, just silence. With the one credential this image ships, every client is in the same tenant.

Step 5: See what the broker refuses

Each of these connects and prints the return code the broker sent. Anonymous, unknown-user and wrong-password connections are refused, on the TLS listener and on local plain MQTT, in MQTT 3.1.1 (--mqtt 4) and MQTT 5 (--mqtt 5):

sudo bifromq-mqtt-probe refuse --transport tls --who anonymous
sudo bifromq-mqtt-probe refuse --transport tls --who wrong-password
sudo bifromq-mqtt-probe refuse --transport tls --who unknown-user --mqtt 5
sudo bifromq-mqtt-probe accept --transport tls --mqtt 5
sudo bifromq-mqtt-probe untrusted

REFUSED tls mqtt4 anonymous: CONNACK 0x04
REFUSED tls mqtt4 wrong-password: CONNACK 0x04
REFUSED tls mqtt5 unknown-user: CONNACK 0x86
ACCEPTED tls mqtt5 CONNACK 0
UNTRUSTED_REJECTED system CA bundle and foreign hostname do not verify

The loopback-only services are firewalled as well as bound to loopback. The nftables table that enforces it:

sudo nft list table inet cloudimg_bifromq
sudo ss -tlnp | grep -E ':(22|1883|8091|8099|8883|9090) '

Step 6: Connect a remote client

Copy the instance certificate to the machine that will connect. It is public, not secret:

scp -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>:/etc/bifromq/tls/server.crt ./bifromq-ca.crt

Then connect over TLS to the instance's public or private IP (both are in the certificate), with the user name and password from Step 2. From a workstation with mosquitto-clients installed (sudo apt install mosquitto-clients on Ubuntu or Debian):

mosquitto_sub -h <public-ip> -p 8883 --cafile ./bifromq-ca.crt -u 'cloudimg/admin' -P '<MQTT_PASSWORD>' -t 'sensors/#' -q 1 -v
mosquitto_pub -h <public-ip> -p 8883 --cafile ./bifromq-ca.crt -u 'cloudimg/admin' -P '<MQTT_PASSWORD>' -t sensors/room1 -q 1 -m '21.5'

Any MQTT 3.1.1 or 5.0 client library works the same way: host, port 8883, TLS on, server.crt as the trusted CA, user cloudimg/admin and your password. You can check the certificate fingerprint against BIFROMQ_TLS_CERT_SHA256 from Step 2, and see the names it covers:

openssl x509 -in /etc/bifromq/tls/server.crt -noout -fingerprint -sha256 -ext subjectAltName

sha256 Fingerprint=3A:9C:41:D2:7E:05:B8:6F:...
X509v3 Subject Alternative Name:
    DNS:localhost, IP Address:127.0.0.1, IP Address:172.31.20.15, DNS:ip-172-31-20-15.ec2.internal, IP Address:54.210.10.20, DNS:ec2-54-210-10-20.compute-1.amazonaws.com

Step 7: Metadata access is blocked for the broker

Neither the broker nor the authentication service needs the instance metadata service, so both carry an IPAddressDeny drop-in for it. The self-test in Step 3 proves the block from inside each service's own cgroup, with a reachable control request from your shell:

systemctl show -p IPAddressDeny bifromq.service bifromq-authd.service
cat /etc/systemd/system/bifromq.service.d/50-cloudimg-imds-deny.conf

IPAddressDeny=fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32
IPAddressDeny=fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32
[Service]
IPAddressDeny=169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128

Step 8: Configuration

The listener section of the broker configuration:

grep -A22 '^mqttServiceConfig:' /opt/bifromq/current/conf/standalone.yml
ls -l /etc/bifromq/ /etc/bifromq/tls/

After editing standalone.yml or /etc/default/bifromq, restart the broker with sudo systemctl restart bifromq and allow about a minute for it to warm up.

Change the MQTT password. The store holds one line per user, username:salt:sha256(salt+password):tenant:user, and is re-read on every connection, so no restart is needed. This reads the new password without echoing it or saving it in your shell history:

read -rs -p 'New MQTT password: ' NEWPW; echo
printf '%s' "$NEWPW" | sudo python3 -c '
import hashlib, secrets, sys
pw = sys.stdin.buffer.read(); salt = secrets.token_hex(16)
digest = hashlib.sha256(bytes.fromhex(salt) + pw).hexdigest()
sys.stdout.write("cloudimg/admin:%s:%s:cloudimg:admin\n" % (salt, digest))' | sudo tee /etc/bifromq/mqtt-users.new >/dev/null
sudo install -o root -g bifromq -m 0640 /etc/bifromq/mqtt-users.new /etc/bifromq/mqtt-users && sudo rm -f /etc/bifromq/mqtt-users.new
unset NEWPW

Update BIFROMQ_MQTT_PASSWORD in /root/bifromq-credentials.txt too, so the self-test and probe keep working. To add a user in another tenant, append a line such as acme/sensor:<salt>:<digest>:acme:sensor.

New address or your own DNS name. The certificate names the addresses the instance had at first boot. After you attach an Elastic IP or point a DNS name at the broker, issue a new certificate (extra DNS names are arguments); the helper restarts the broker and updates the fingerprint in the credentials file. Give clients the new server.crt:

sudo bifromq-tls-cert mqtt.example.com

Use a certificate from your own CA. Replace /etc/bifromq/tls/server.crt (your certificate, followed by any intermediates) and /etc/bifromq/tls/server.key (an unencrypted PEM private key, owned root:bifromq, mode 0640), then restart the broker:

sudo install -o root -g root -m 0644 ./fullchain.pem /etc/bifromq/tls/server.crt
sudo install -o root -g bifromq -m 0640 ./privkey.pem /etc/bifromq/tls/server.key
sudo systemctl restart bifromq

Give the broker more memory. The JVM is pinned at a 1.25 GiB heap plus 384 MiB of direct memory so it fits a 4 GiB instance with room for RocksDB, which allocates outside the heap. On a larger instance, raise -Xmx (and -Xms) in JVM_HEAP_OPTS in /etc/default/bifromq, keeping -XX:MaxDirectMemorySize set explicitly, and restart the broker. Without it, direct memory defaults to the heap size and doubles the footprint.

Administrative API. BifroMQ has no web console; it is managed through an HTTP API on 127.0.0.1:8091 that has no authentication of its own, so never open 8091 (or 9090) in a security group. Reach it from your workstation over an SSH tunnel, then browse to http://127.0.0.1:8091/cluster:

ssh -i ~/.ssh/my-key-pair.pem -L 8091:127.0.0.1:8091 ubuntu@<public-ip>

Step 9: Maintenance

• Operating system updates. Unattended security upgrades are on, including the OpenJDK runtime. Apply the rest with sudo apt update && sudo apt upgrade in a maintenance window, then sudo systemctl restart bifromq to load a new Java runtime.
• Logs. BifroMQ writes info.log, warn.log, error.log and plugin.log under /var/log/bifromq/; sudo journalctl -u bifromq -u bifromq-authd -u bifromq-firstboot shows the service logs.
• Persistence. Sessions, subscriptions and retained messages are kept in the RocksDB stores under /var/lib/bifromq/ and survive a restart.
• Licences. Apache BifroMQ is licensed under the Apache License 2.0. /usr/share/doc/bifromq/ holds the LICENSE, NOTICE and DISCLAIMER, and /opt/bifromq/current/licenses/ the licences of the bundled libraries. BifroMQ is undergoing incubation at the Apache Software Foundation; incubation status does not necessarily reflect the completeness or stability of the code.

Support

cloudimg provides 24/7 technical support by email and live chat for every version in this listing: support@cloudimg.co.uk. The other versions in the listing are on the guide hub.

Apache BifroMQ, BifroMQ and Apache are trademarks of the Apache Software Foundation. cloudimg is not affiliated with or endorsed by the Apache Software Foundation.