Apache BifroMQ MQTT Broker on AWS User Guide

Overview

Apache BifroMQ (Incubating) is a distributed MQTT broker written in Java, built around native multi tenancy. It implements MQTT 3.1, 3.1.1 and 5.0, and every connection is resolved to a tenant whose topics, subscriptions and sessions are isolated from every other tenant on the same broker.

This cloudimg image deploys BifroMQ 4.0.0-incubating as a hardened single node broker. An authenticated MQTT endpoint is answering on TCP 1883 by the time first boot completes, with no configuration required.

What is included:

• Apache BifroMQ 4.0.0-incubating, installed from the official Apache distribution and SHA-512 verified
• OpenJDK 21 (headless) runtime
• mosquitto-clients (mosquitto_pub, mosquitto_sub) for testing on the box
• MQTT over TCP on port 1883, supporting MQTT 3.1, 3.1.1 and 5.0
• Username and password authentication enforced on every connection
• A unique MQTT password generated on each instance's first boot
• Administrative HTTP API and metrics bound to loopback and firewalled
• JVM heap sized for the instance rather than left at a default that would exhaust it
• 24/7 cloudimg support

A note on security, and why it matters for this product

Left at its own defaults, BifroMQ loads a development authentication provider that accepts every connection without checking the password, and grants every publish and subscribe. A stock deployment is an open broker. This image replaces that with real credential checking, and closes a related upstream behaviour where a broker whose authentication service is unreachable would silently grant every action rather than deny it. If you build your own BifroMQ deployment from upstream, these are the two things to get right first.

Connecting to your instance

Connect over SSH on port 22 as the login user for the operating system variant you launched:

ssh ubuntu@<vm-ip>

Prerequisites

The recommended launch security group opens only SSH. MQTT on 1883 is authenticated, but you should still open it only to the client networks that actually need it.

Step 1: Launch the instance

Subscribe to the listing and launch the AMI from the AWS console, or from the CLI. Scope the MQTT rule to the narrowest source range that works for you:

aws ec2 create-security-group --group-name bifromq-sg \
  --description "Apache BifroMQ" --vpc-id <your-vpc-id>

# SSH from your admin network only
aws ec2 authorize-security-group-ingress --group-name bifromq-sg \
  --protocol tcp --port 22 --cidr <your-mgmt-cidr>

# MQTT from your device networks only
aws ec2 authorize-security-group-ingress --group-name bifromq-sg \
  --protocol tcp --port 1883 --cidr <your-mgmt-cidr>

aws ec2 run-instances --image-id ami-xxxxxxxxxxxxxxxxx \
  --instance-type m5.large --key-name <your-key> \
  --security-groups bifromq-sg

There is no reason to expose 1883 to the whole internet, and 8091/9090 must never be opened at all (Step 6).

Step 2: Connect and confirm the broker is running

Three services make up the appliance: bifromq-firstboot.service generates this instance's credential, bifromq-authd.service is the local authentication service, and bifromq.service is the broker itself.

sudo systemctl is-active bifromq-firstboot.service bifromq-authd.service bifromq.service
sudo ss -tln | grep -E ':1883|:8091'

Give the broker a moment on first boot. BifroMQ is a JVM application with embedded RocksDB storage. Port 1883 accepts connections within a few seconds, but the broker needs roughly one to two minutes after boot before it will serve traffic. During that window a client is refused with Connection Refused: broker unavailable, and subscriptions can be refused for slightly longer than publishes. This is warm up, not a fault - retry and it will connect.

Step 3: Read this instance's MQTT credentials

There is no default password in the image. A unique one is generated on each instance's first boot and written to a root only file:

sudo cat /root/bifromq-credentials.txt

The username is cloudimg/admin. The part before the slash is the tenant, and it matters - see the note in Step 4.

Step 4: Publish and subscribe

This runs a complete round trip on the instance itself. It reads the credentials, subscribes in the background, publishes one message, and prints what arrived. Because the broker may still be warming up, it retries the publish for a short while:

U=$(sudo grep '^BIFROMQ_MQTT_USERNAME=' /root/bifromq-credentials.txt | cut -d= -f2-)
P=$(sudo grep '^BIFROMQ_MQTT_PASSWORD=' /root/bifromq-credentials.txt | cut -d= -f2-)

timeout 90 mosquitto_sub -h 127.0.0.1 -p 1883 -i demo-sub -u "$U" -P "$P" \
  -t 'demo/#' -q 1 -v > /tmp/mqtt-demo.txt 2>&1 &
for i in $(seq 1 20); do
  mosquitto_pub -h 127.0.0.1 -p 1883 -i demo-pub -u "$U" -P "$P" \
    -t demo/sensors/temp -q 1 -m '22.5' 2>/dev/null
  grep -q '22.5' /tmp/mqtt-demo.txt && break
  sleep 3
done
cat /tmp/mqtt-demo.txt

Expected output:

demo/sensors/temp 22.5

To connect from your own machine instead, use the instance's public IP (open 1883 to your client network first):

mosquitto_sub -h <vm-ip> -p 1883 -u 'cloudimg/admin' -P '<your-password>' -t 'demo/#' -q 1 -v
mosquitto_pub -h <vm-ip> -p 1883 -u 'cloudimg/admin' -P '<your-password>' -t demo/hello -q 1 -m 'hi'

Two things that will save you an afternoon

Use QoS 1 where delivery matters. BifroMQ routes messages through an asynchronous subscription table. A QoS 0 publish that arrives before a subscription has fully propagated is dropped silently. That is correct MQTT behaviour - QoS 0 is explicitly best effort with no delivery guarantee - but on a freshly created subscription it is very noticeable. Every example in this guide uses -q 1.

Publisher and subscriber must share a tenant. The tenant is the part of the username before the slash. Two clients in different tenants cannot see each other's messages, and neither one gets an error - you simply get silence, which is easy to mistake for a broken broker. With the single credential this image ships, both ends use cloudimg/admin and this does not arise; it becomes relevant as soon as you add tenants of your own.

Step 5: Confirm authentication is actually enforced

Worth doing once, because an MQTT broker that silently accepts everything looks identical to a working one until it matters. Both of these must be refused:

U=$(sudo grep '^BIFROMQ_MQTT_USERNAME=' /root/bifromq-credentials.txt | cut -d= -f2-)

if mosquitto_pub -h 127.0.0.1 -p 1883 -u "$U" -P 'wrong-password' \
     -t demo/hello -q 1 -m nope 2>/dev/null; then
  echo 'PROBLEM: a wrong password was accepted'
else
  echo 'Wrong password correctly rejected'
fi

if mosquitto_pub -h 127.0.0.1 -p 1883 -t demo/hello -q 1 -m nope 2>/dev/null; then
  echo 'PROBLEM: an anonymous connection was accepted'
else
  echo 'Anonymous connection correctly rejected'
fi

Expected output:

Wrong password correctly rejected
Anonymous connection correctly rejected

Step 6: The administrative API

BifroMQ has no web console or CLI - management is through an HTTP API. That API performs no authentication of its own, and it can publish to any topic and terminate any session, so this image binds it to loopback and enforces that with a firewall rule. The plugin metrics endpoint on 9090 is treated the same way.

Check broker health locally:

curl -s http://127.0.0.1:8091/cluster

To reach the API from your own machine, tunnel over SSH rather than opening the port:

ssh -L 8091:127.0.0.1:8091 ubuntu@<vm-ip>

Then browse to http://127.0.0.1:8091/cluster locally. Do not add a security group rule for 8091 or 9090.

Step 7: Where everything lives

Logs are verbose and split by level. Start with:

sudo tail -n 20 /var/log/bifromq/info.log

Step 8: Changing the MQTT password

Credentials live in /etc/bifromq/mqtt-users, one record per line, as username:salt:hash:tenant:user. The password is stored as a salted SHA-256 digest, never in plain text. To rotate it, generate a new salt and digest and replace the record:

NEWPW='<new-password>'
SALT=$(openssl rand -hex 16)
DIGEST=$(SALT_HEX="$SALT" PW="$NEWPW" python3 -c "
import binascii, hashlib, os
print(hashlib.sha256(binascii.unhexlify(os.environ['SALT_HEX']) + os.environ['PW'].encode()).hexdigest())
")
printf 'cloudimg/admin:%s:%s:cloudimg:admin\n' "$SALT" "$DIGEST" | \
  sudo tee /etc/bifromq/mqtt-users > /dev/null
sudo chown root:bifromq /etc/bifromq/mqtt-users
sudo chmod 640 /etc/bifromq/mqtt-users
echo 'Password rotated'

The change takes effect on the next connection; the file is re-read per request, so no restart is needed. Update /root/bifromq-credentials.txt too if you want it to stay accurate.

To add a second tenant, append another record with a different username and tenant, for example acme/sensor:...:acme:sensor.

Step 9: Scaling up

The JVM is deliberately sized to fit a small instance. BifroMQ's own start script would otherwise claim about 70% of system memory as heap plus a further 20% as direct memory, which does not fit on a small instance once RocksDB's off heap usage is included.

If you move to a larger instance type, raise the heap to match by editing JVM_HEAP_OPTS in /etc/default/bifromq. Keep -XX:MaxDirectMemorySize set explicitly - if you remove it, direct memory silently defaults to the maximum heap size and doubles the footprint. Leave headroom for RocksDB, which allocates outside the heap. Restart with sudo systemctl restart bifromq.service.

Step 10: Enabling TLS

The image ships without a certificate, so MQTT over TLS is disabled. To enable it, place your certificate and key on the instance, then set tlsListener.enable: true with the certFile and keyFile paths under mqttServiceConfig.server.tlsListener in /opt/bifromq/current/conf/standalone.yml, restart the broker, and open TCP 8883 in the security group to your client networks. Clients then connect with --cafile against port 8883. Prefer TLS for anything crossing an untrusted network - plain MQTT sends credentials in the clear.

Security notes

• Keep 1883 scoped in the security group to the networks your devices actually use.
• Never open 8091 or 9090. Both are unauthenticated. Use the SSH tunnel in Step 6.
• Rotate the shipped password if the instance will be long lived (Step 8).
• Use TLS (Step 10) for anything crossing an untrusted network.
• Patch regularly. Unattended security upgrades are enabled; reboot periodically to pick up kernel updates.

Licensing

Apache BifroMQ is licensed under Apache-2.0 and is free to use. BifroMQ is currently undergoing incubation at the Apache Software Foundation; incubation status does not necessarily reflect the completeness or stability of the code, and indicates that the project has yet to be fully endorsed by the ASF. cloudimg provides the packaging, hardening and commercial support separately - support@cloudimg.co.uk.