pmacct NetFlow, sFlow and IPFIX Traffic Accounting on Ubuntu 24.04 on Azure User Guide

Overview

This guide covers the deployment and use of pmacct on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images.

pmacct is a mature, widely deployed suite of passive network monitoring tools. Its collector daemon nfacctd receives flow exports, NetFlow v5 and v9, and IPFIX, from the routers, switches and firewalls on your network; sfacctd does the same for sFlow; and pmacctd can account traffic directly from a captured interface. Every flow is aggregated in memory by the primitives you choose, such as source and destination host, source and destination port, IP protocol and autonomous system, and the running totals of packets and bytes are periodically written to a backend. That turns a firehose of raw flow records into compact, queryable accounting: who talked to whom, over which ports and protocols, and how much traffic each conversation carried.

A local PostgreSQL database is bundled on this same VM as the accounting backend. pmacct on its own only aggregates flow in memory; to keep and query the results it needs somewhere to write them. This image ships PostgreSQL alongside it, bound to the loopback interface, with the collector wired up to write aggregated records into a table called acct_v9. You are not required to keep that backend: pmacct can equally write to files, and the configuration is a couple of lines to change, covered later in this guide.

A flow generator is bundled so the appliance proves itself out of the box. A collector with nothing exporting to it is idle, so this image also runs softflowd, which reads this VM's own network interface and exports NetFlow to the collector on the loopback address. From the first boot, nfacctd is genuinely accounting real traffic. A second, cloudimg authored tool, pmacct-flowgen, injects a synthetic flow with a known signature so you can prove the whole pipeline end to end before you point a single router at it.

pmacct has no web interface. It is a collector: flow records in, aggregated accounting out. Everything here is done on the command line, with psql to query the accounting table.

Security by design. A flow collector with a database behind it is only as safe as its exposure, so this image is locked down by default:

• Exactly one port is open to the network, and it is the collector's legitimate ingress. UDP 2055 is where nfacctd receives NetFlow, sFlow and IPFIX. NetFlow has no application level authentication, by protocol design, so you restrict access to it at the network layer: open 2055 in your Network Security Group only to the subnet your routers and switches export from.

• The database is bound to the loopback interface. PostgreSQL listens on 127.0.0.1 only and refuses connections on the VM's routable address, so the accounting data and the database credential are never reachable from the network. You query it from the VM itself or over an SSH tunnel.

• Nothing is baked into the image. The captured image contains no database credential and no collector configuration at all. The per VM database password is generated on first boot, written into the collector config and into a root only credentials file, and the collector physically cannot start until first boot has created it, so there is no shared password between customers.

What is included:

• pmacct 1.7.9 built from the official upstream release, verified against a pinned SHA 256 checksum at build time, providing nfacctd, sfacctd, pmacctd and the pmacct client, running under systemd as nfacctd.service

• PostgreSQL from Ubuntu, bound to loopback, as the accounting backend holding the acct_v9 table

• softflowd 1.1.0 running as softflowd.service, exporting this VM's own interface traffic to the collector so it accounts real flow from first boot

• pmacct-flowgen, a bundled synthetic NetFlow generator, and pmacct-selfcheck, a command that proves the whole pipeline end to end

• A per VM PostgreSQL credential generated on first boot and written to a root only credentials file

Prerequisites

• Active Azure subscription, an SSH public key, and a VNet and subnet in the target region

• Subscription to this listing on Azure Marketplace

• A Network Security Group allowing TCP 22 for administration and UDP 2055 for the flow collector. In production, restrict 2055 to the subnet your routers, switches and exporters send from. No other port needs to be opened, because the database is bound to the loopback interface.

Recommended virtual machine size: Standard_B2s with 2 vCPU and 4 GB RAM suits development and moderate flow volumes. For high export rates or many exporters, choose Standard_D2s_v5 or above.

Deploy the virtual machine

Deploy from the Azure Portal by selecting the image from Azure Marketplace, choosing your VM size, and supplying your SSH public key for the azureuser account. Or deploy from the Azure CLI.

These commands run on your own workstation, not on the VM:

az vm create \
  --resource-group my-resource-group \
  --name my-pmacct-vm \
  --image <this-marketplace-image> \
  --size Standard_B2s \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard

az vm open-port --resource-group my-resource-group --name my-pmacct-vm --port 2055 --protocol Udp

Connect over SSH once the VM is running:

ssh azureuser@<vm-ip>

Confirm the collector is healthy

The database, the collector and the flow source all start automatically on boot. Check them and confirm the version:

systemctl is-active postgresql nfacctd softflowd
nfacctd -V | head -1

Expected output

active
active
active
NetFlow Accounting Daemon, nfacctd 1.7.9 [RELEASE]

Now confirm the network posture. Exactly one port is bound to all interfaces, the flow collector, and the database is bound to the loopback address only:

ss -tuln | grep -E ':(2055|5432) '

Expected output

udp   UNCONN 0      0             0.0.0.0:2055      0.0.0.0:*
tcp   LISTEN 0      200         127.0.0.1:5432      0.0.0.0:*

That is the shape you want: UDP 2055 reachable from your routers so they can export flow, and PostgreSQL on 5432 bound to loopback and unreachable from outside the VM.

Retrieve your per VM credentials

The database password is generated uniquely on your VM at first boot and written to a root only file, along with the connection details for the collector and the backend:

sudo cat /root/pmacct-credentials.txt

This file lists the collector endpoint your routers export to, and the loopback PostgreSQL connection, db_host, db_port, db_name, db_user, db_password and db_table. Keep the password safe; it is the credential for the accounting database.

Prove the accounting pipeline end to end

Before pointing any router at the collector, prove the whole pipeline works on this one VM. The bundled pmacct-flowgen injects a synthetic NetFlow v5 flow with a recognisable signature, source 198.51.100.10, destination 203.0.113.20, into the collector on the loopback address:

sudo pmacct-flowgen

Expected output

pmacct-flowgen: sent NetFlow v5 flow 198.51.100.10:40001 -> 203.0.113.20:443 proto=6 packets=7 bytes=4200 to 127.0.0.1:2055

Give the collector a few seconds to aggregate and flush to the database, then query the accounting table for that flow. This reads the per VM database password from the credentials file, so it never appears on the command line:

sleep 16
PW=$(sudo grep -E '^pmacct.db_password=' /root/pmacct-credentials.txt | cut -d= -f2-)
PGPASSWORD="$PW" psql -h 127.0.0.1 -U pmacct -d pmacct \
  -c "SELECT ip_src, ip_dst, port_src, port_dst, ip_proto, packets, bytes
      FROM acct_v9 WHERE ip_src='198.51.100.10' AND ip_dst='203.0.113.20';"

Expected output

    ip_src     |    ip_dst    | port_src | port_dst | ip_proto | packets | bytes
---------------+--------------+----------+----------+----------+---------+-------
 198.51.100.10 | 203.0.113.20 |    40001 |      443 |        6 |       7 |  4200
(1 row)

There it is: the flow you injected, received by nfacctd, aggregated by source, destination, port and protocol, and written to the database with its packet and byte counts. That is pmacct doing its job. Running pmacct-flowgen again and re querying shows the counters accumulate for the same flow key, because nfacctd aggregates repeated flows into a single running total.

Run the bundled self check

pmacct-selfcheck re proves the whole appliance at any time: that the collector is listening, that the per VM database credential authenticates, that an injected flow genuinely lands aggregated in the table, and that the database refuses connections off the loopback interface:

sudo pmacct-selfcheck

Expected output

pmacct appliance self-check
  [ok] nfacctd is listening on UDP 2055 (NetFlow ingress)
  [ok] per-VM PostgreSQL credential authenticates on 127.0.0.1:5432
  [ok] injected synthetic NetFlow v5 flow 198.51.100.10 -> 203.0.113.20
  ... waiting for nfacctd to aggregate + flush to PostgreSQL
  [ok] acct_v9 shows the injected flow aggregated: 198.51.100.10 -> 203.0.113.20 = 7/4200 (packets/bytes)
  [ok] acct_v9 holds 1 aggregated flow row(s) in total (incl. real interface traffic)
  [ok] PostgreSQL refuses connections on <vm-ip> — SQL surface is loopback only
SELFCHECK OK — pmacct is collecting NetFlow and producing aggregated accounting output.

Point your own exporters at the collector

To account real traffic from your network, configure your routers, switches or firewalls to export NetFlow, sFlow or IPFIX to this VM's IP on UDP 2055, and make sure your Network Security Group allows 2055/udp from those devices. On Cisco IOS, for example, that is a flow exporter with destination <vm-ip> and transport udp 2055. nfacctd auto detects the flow version per packet, so v5, v9 and IPFIX exporters can all send to the same port.

The appliance also accounts its own traffic out of the box: softflowd reads this VM's primary interface and exports NetFlow to the collector, so the acct_v9 table fills with the VM's real conversations even before you add external exporters.

Query the aggregated accounting table

The accounting table is ordinary PostgreSQL, so you analyse it with SQL. The password is read from the credentials file so it stays off the command line:

PW=$(sudo grep -E '^pmacct.db_password=' /root/pmacct-credentials.txt | cut -d= -f2-)
PGPASSWORD="$PW" psql -h 127.0.0.1 -U pmacct -d pmacct \
  -c "SELECT ip_src, ip_dst, port_src, port_dst, ip_proto, packets, bytes
      FROM acct_v9 ORDER BY bytes DESC LIMIT 10;"

That gives you the top talkers by bytes. Because the data is a normal SQL table you can group by host, by port or by protocol, join it to your own reference tables, or point Grafana's PostgreSQL data source at it over an SSH tunnel to build dashboards.

Reach the database over an SSH tunnel

The database is bound to loopback, so to reach it from your workstation, for a GUI client or Grafana, forward the port over SSH:

ssh -L 5432:127.0.0.1:5432 azureuser@<vm-ip>

Then connect your client to 127.0.0.1:5432, database pmacct, user pmacct, with the password from the credentials file.

Change what is aggregated

pmacct aggregates by the primitives named on the aggregate line of its configuration. The shipped default is src_host, dst_host, src_port, dst_port, proto. To account by autonomous system, or to add more primitives, edit the collector config and reload:

sudo sed -n '/^aggregate:/p' /etc/pmacct/nfacctd.conf

The upstream documentation under /usr/share/doc/pmacct/ lists every primitive. After changing the aggregate line you reload the collector with sudo systemctl restart nfacctd. Note that the SQL table schema must contain the columns for the primitives you aggregate by; the shipped acct_v9 schema, from /usr/share/pmacct/sql/pmacct-create-table_v9.pgsql, already covers the common set.

Server components

• pmacct 1.7.9 (/usr/local/sbin/nfacctd, sfacctd, pmacctd, and the /usr/local/bin/pmacct client), GPL 2.0, built from source with the PostgreSQL and Jansson plugins

• PostgreSQL (Ubuntu package), the accounting backend, bound to 127.0.0.1

• softflowd 1.1.0 (/usr/sbin/softflowd), BSD licensed, the bundled flow source

• pmacct-flowgen and pmacct-selfcheck (/usr/local/sbin/), cloudimg authored tooling

Filesystem layout

• /etc/pmacct/nfacctd.conf — the collector configuration, written at first boot, 0640 root:pmacct

• /usr/share/pmacct/sql/ — pmacct's own PostgreSQL schema scripts

• /root/pmacct-credentials.txt — the per VM database credential, 0600 root:root

• /var/lib/cloudimg/pmacct-firstboot.done — the first boot sentinel

• /var/log/cloudimg-firstboot.log — the first boot log

Network ports

• UDP 2055 — the NetFlow, sFlow and IPFIX collector, bound to all interfaces. This is the only port open to the network. Restrict it in your NSG to your exporters.

• TCP 5432 — PostgreSQL, bound to 127.0.0.1 only. Reachable over an SSH tunnel.

• TCP 22 — SSH administration.

Managing the services

sudo systemctl status nfacctd
sudo systemctl restart nfacctd
sudo systemctl status softflowd
sudo systemctl status postgresql

Scripts and log files

• sudo pmacct-selfcheck — prove the whole pipeline end to end

• sudo pmacct-flowgen — inject a synthetic test flow

• journalctl -u nfacctd — the collector log

• /var/log/cloudimg-firstboot.log — the first boot log

On startup

On the first boot, pmacct-firstboot.service waits for PostgreSQL, generates the per VM database password, creates the pmacct role and database and loads the acct_v9 schema, writes the collector configuration, and starts nfacctd and softflowd. It writes the credentials file and then disables itself, so it runs exactly once. On every subsequent boot the collector, the database and the flow source start normally.

Troubleshooting

• The collector is not listening on 2055. Check sudo systemctl status nfacctd and journalctl -u nfacctd. On a brand new VM, confirm first boot completed: sudo cat /var/log/cloudimg-firstboot.log and check that /var/lib/cloudimg/pmacct-firstboot.done exists.

• No rows appear in the table. Confirm something is exporting flow: sudo systemctl status softflowd for the built in source, or verify your router is exporting to this VM's IP on 2055/udp and that the NSG allows it. Inject a known flow with sudo pmacct-flowgen to isolate the collector from the exporters.

• psql cannot connect. The database is loopback only; connect from the VM itself with -h 127.0.0.1, or forward the port over SSH. The password is in /root/pmacct-credentials.txt.

• Run the self check. sudo pmacct-selfcheck exercises every part of the pipeline and reports exactly which step fails.

Security recommendations

• Restrict UDP 2055 in your Network Security Group to only the subnet your routers, switches and firewalls export from. NetFlow is unauthenticated at the application layer, so network level restriction is your control.

• Keep the database on the loopback interface. If you must reach it remotely, use an SSH tunnel rather than opening 5432.

• Keep the credentials file root only, and rotate the database password if you ever expose the VM more widely.

• Apply operating system updates regularly; the image ships with unattended security upgrades enabled.

Support

This image is provided by cloudimg with 24/7 support. For assistance, contact support@cloudimg.co.uk or visit https://cloudimg.co.uk.