Fluent Bit on Ubuntu 24.04 on Azure User Guide

Overview

Fluent Bit is the fast, lightweight, vendor-neutral log and metrics processor and forwarder from the Fluent project (a CNCF graduated project). It collects data from many sources, parses and filters it, and routes it to dozens of destinations - all from a tiny memory footprint. The cloudimg image installs Fluent Bit from the official Fluent Bit apt repository as /opt/fluent-bit/bin/fluent-bit, ships a clear ready-to-run sample pipeline, and enables the built-in HTTP monitoring server so you can check health and metrics from the moment the VM boots. Backed by 24/7 cloudimg support.

What is included:

• Fluent Bit 5.x (Apache-2.0) from the official Fluent Bit apt repository
• The package fluent-bit.service systemd unit, enabled and running on boot
• A sample pipeline at /etc/fluent-bit/fluent-bit.conf: a cpu input, a record_modifier filter, and two outputs (a rotating file under /var/log/fluent-bit/ and stdout)
• The built-in HTTP monitoring server on 127.0.0.1:2020 (/api/v1/health, /api/v1/metrics, /api/v1/uptime)
• 24/7 cloudimg support

This is a headless service product: the HTTP monitoring server binds to 127.0.0.1 only, so no inbound application port is needed beyond SSH.

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B2s (2 vCPU / 4 GiB RAM) is a good starting point. NSG inbound: allow 22/tcp from your management network. No inbound application ports are required.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Fluent Bit by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22). Then Review + create -> Create.

Step 2 - Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name fluent-bit \
  --image <marketplace-image-urn> \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm Fluent Bit is installed and running

Check the version and that the service is active:

fluent-bit --version
systemctl is-active fluent-bit

fluent-bit --version reports Fluent Bit v5.x, and systemctl is-active fluent-bit prints active.

Step 5 - Review the sample pipeline

The ready-to-run pipeline lives at /etc/fluent-bit/fluent-bit.conf. It enables the HTTP monitoring server, reads CPU metrics every second (a guaranteed source of records), stamps each record with the hostname, and fans the stream out to a file and to stdout:

cat /etc/fluent-bit/fluent-bit.conf

Step 6 - Check the HTTP monitoring server health

Fluent Bit's built-in HTTP server exposes a liveness endpoint that returns 200 ok when the engine is healthy:

curl -s http://127.0.0.1:2020/api/v1/health

It prints ok.

Step 7 - Inspect live pipeline metrics

The /api/v1/metrics endpoint returns JSON with per-plugin record and byte counters. Watch the input, filter and output counters climb as the pipeline processes records:

curl -s http://127.0.0.1:2020/api/v1/metrics | head -c 600

The input object shows the cpu plugin's records count, and the output object shows the file and stdout plugins each processing those records.

Step 8 - Watch records flow through the pipeline

The file output writes every record to /var/log/fluent-bit/records.log. Tail it to see the live stream - each line is a CPU sample stamped with your hostname and the cloudimg-sample pipeline tag:

tail -n 5 /var/log/fluent-bit/records.log

Step 9 - Customise the pipeline

Edit /etc/fluent-bit/fluent-bit.conf to point Fluent Bit at your own sources and destinations, then restart the service. A few common building blocks:

# Tail application log files
[INPUT]
    Name   tail
    Path   /var/log/myapp/*.log
    Tag    myapp

# Forward to Elasticsearch / OpenSearch
[OUTPUT]
    Name   es
    Match  *
    Host   my-search-host
    Port   9200

# Forward to Loki, Kafka, S3, Azure Blob, HTTP, and many more

After editing, apply your changes:

sudo systemctl restart fluent-bit

Fluent Bit ships dozens of input, parser, filter and output plugins (tail, systemd, syslog, HTTP, Prometheus, OpenTelemetry, Elasticsearch, Loki, Kafka, S3, Azure, and more). See the Fluent Bit documentation for the full catalogue.

Maintenance

• Service: managed by systemd - systemctl status fluent-bit, journalctl -u fluent-bit.
• Configuration: /etc/fluent-bit/fluent-bit.conf and /etc/fluent-bit/parsers.conf; restart the service after edits.
• Monitoring: the HTTP server on 127.0.0.1:2020 exposes /api/v1/health, /api/v1/metrics and /api/v1/uptime; scrape /api/v1/metrics/prometheus for Prometheus.
• Upgrades: the package is held at the shipped version; to upgrade, run sudo apt-mark unhold fluent-bit && sudo apt-get update && sudo apt-get install --only-upgrade fluent-bit.
• Security patches: unattended-upgrades remains enabled so the OS continues to receive security updates automatically.

Support

cloudimg provides 24/7 expert support for this image. Contact support@cloudimg.co.uk.

Fluent Bit is a trademark of the Fluent project, hosted by the Cloud Native Computing Foundation. cloudimg is not affiliated with or endorsed by the Fluent project or the CNCF.