Apache StreamPipes on Ubuntu 24.04 on Azure User Guide

Overview

Apache StreamPipes is a self-service (Industrial) IoT toolbox that enables users to connect, analyse and explore IoT data streams without writing code. It pairs a rich web UI with a visual pipeline editor, a library of connectors (StreamPipes Connect) for industrial and IoT protocols, a catalogue of data processors and sinks, and live dashboards. The cloudimg image installs the official StreamPipes installer/compose Docker stack (the StreamPipes core backend, the Angular web UI, a CouchDB user and metadata store, an InfluxDB time-series sink, a NATS message broker, and the full IIoT extensions library), runs it under Docker behind an nginx reverse proxy on port 80, persists every container volume on a dedicated Azure data disk, and seeds a fresh per-VM administrator on the first boot of every VM. Backed by 24/7 cloudimg support.

What is included:

• The official Apache StreamPipes 0.98.0 self-hosted stack running under Docker (backend core, web UI, CouchDB, InfluxDB, NATS, the all-IIoT extensions library)
• nginx on :80 as a reverse proxy to the StreamPipes UI, with an unauthenticated static health endpoint for load balancers
• A per-VM administrator generated on first boot - a unique password recorded in a root-only file - so every VM is secured independently
• Docker data-root relocated onto a dedicated 60 GiB Azure data disk at /var/lib/streampipes, so all container images and volumes (the CouchDB store, the InfluxDB time-series data, the backend assets) survive OS changes and can be resized independently
• The whole multi-container stack managed as one streampipes.service systemd unit, plus docker.service and nginx.service, all enabled
• The message broker and databases bound to the internal Docker network only - just 22/tcp and 80/tcp are exposed
• 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B8ms (8 vCPU / 32 GiB RAM) is the recommended starting size - the multi-container StreamPipes stack needs the memory. NSG inbound: allow 22/tcp from your management network and 80/tcp. StreamPipes serves plain HTTP on port 80; for production, terminate TLS in front of it with your own domain.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache StreamPipes by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Review the dedicated data disk on the Disks tab, then Review + create -> Create.

Step 2 - Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name streampipes \
  --image <marketplace-image-urn> \
  --size Standard_B8ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name streampipes --port 80 --priority 1010

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the stack is running

The StreamPipes containers run under Docker, fronted by nginx. On first boot the stack seeds the per-VM administrator and starts; the multi-container stack can take a few minutes to become fully healthy after the VM is created.

systemctl is-active docker.service nginx.service

Both report active. You can list the running containers with:

sudo docker compose -f /opt/streampipes/compose/docker-compose.yml ps

The whole stack is also managed as a single systemd unit - use sudo systemctl status streampipes and sudo systemctl restart streampipes to control every container together.

Step 5 - Retrieve your per-VM admin credentials

The administrator password is generated uniquely on the first boot of your VM and written to a root-only file:

sudo cat /root/streampipes-credentials.txt

This file contains STREAMPIPES_ADMIN_EMAIL (admin@cloudimg.local) and STREAMPIPES_ADMIN_PASSWORD - the credentials for the web UI and the REST API. Store them somewhere safe.

Step 6 - Confirm the health endpoint

nginx exposes an unauthenticated static health endpoint for load balancers and probes, and proxies everything else to the StreamPipes web UI:

curl -s http://localhost/health

It returns ok. The web UI is reachable through the same nginx on port 80:

curl -s -o /dev/null -w 'UI -> HTTP %{http_code}\n' http://localhost/

It returns HTTP 200.

Step 7 - Confirm the admin login API

The StreamPipes backend exposes a login endpoint that returns a JWT token for valid credentials and rejects invalid ones. Substitute the password from Step 5 for <STREAMPIPES_ADMIN_PASSWORD> and confirm a correct password is accepted while a wrong one is rejected:

curl -s -o /dev/null -w 'login -> HTTP %{http_code}\n' \
  -X POST -H 'Content-Type: application/json' \
  -d '{"username":"admin@cloudimg.local","password":"<STREAMPIPES_ADMIN_PASSWORD>"}' \
  http://localhost/streampipes-backend/api/v2/auth/login

A correct password returns HTTP 200; a wrong password returns HTTP 401. This proves the per-VM administrator was seeded and authentication is enforced.

Step 8 - Sign in to the web UI

Browse to http://<vm-public-ip>/ and sign in with the email admin@cloudimg.local and the password from Step 5. The StreamPipes web UI is where you connect data, build pipelines and explore your streams.

The Pipelines view is where you create, start, stop and monitor your data pipelines and reusable functions.

StreamPipes Connect is where you add data sources - choose a built-in adapter (MQTT, OPC-UA, Kafka, HTTP, files and many more from the IIoT extensions library) to ingest a live stream.

Dashboards let you visualise the output of your running pipelines with live widgets - charts, gauges, tables and maps.

Step 9 - Confirm data lives on the dedicated disk

Docker's data-root is relocated onto the dedicated Azure data disk, so every container image and volume - the CouchDB user and metadata store, the InfluxDB time-series data and the backend assets - lives there and survives OS changes:

findmnt /var/lib/streampipes

The mount is backed by a separate Azure data disk captured into the image and re-provisioned on every VM. You can confirm Docker is using it:

sudo docker info -f '{{.DockerRootDir}}'

It prints /var/lib/streampipes/docker.

Building your first pipeline

StreamPipes turns raw streams into actionable insight without code. In the web UI: open Connect and add an adapter for your data source (for example an MQTT topic or an OPC-UA server); open Pipelines and drag your new stream onto the canvas, then chain data processors (filters, aggregations, enrichments) and a sink (store to the built-in time-series database, forward to a broker, or trigger a notification); start the pipeline and open a Dashboard to watch the live output. See the Apache StreamPipes documentation for the full catalogue of adapters, processors and sinks.

Maintenance

• Admin credentials: the per-VM administrator is in /root/streampipes-credentials.txt; change the password from the user menu in the web UI after first sign-in.
• Data: all container volumes live on the /var/lib/streampipes data disk (Docker data-root) - snapshot or resize that disk for backups.
• Stack: manage the whole stack with sudo systemctl <status|restart|stop> streampipes, or per-container with sudo docker compose -f /opt/streampipes/compose/docker-compose.yml <ps|logs|restart>.
• TLS: StreamPipes serves plain HTTP on port 80; front it with TLS (e.g. certbot) and your own domain before production use.
• Security patches: unattended-upgrades remains enabled so the OS continues to receive security updates automatically.

Support

cloudimg provides 24/7 expert support for this image. Contact support@cloudimg.co.uk.