Observability Azure

OpenObserve with Vector on Ubuntu 24.04 on Azure User Guide

| Product: OpenObserve with Vector on Ubuntu 24.04 LTS on Azure

Overview

This image is a complete, pre-wired observability appliance. Two well known open source projects are installed, and — more importantly — they are already connected to each other:

  • Vector 0.58.0 is the collection and ingest tier. It reads this host's journald logs and host metrics, and it accepts telemetry from your other machines.
  • OpenObserve 0.92.2 is the backend. It stores, searches, dashboards and alerts on everything Vector sends it.

cloudimg already publishes each of these separately. The reason this image exists is the wiring between them. On a plain install you would still have to write a Vector configuration, create an ingest credential, point one at the other, and build a dashboard. Here that is done before the VM is handed to you: by the time you first sign in, this host's own logs and metrics are already flowing into named streams and a Host Overview (Vector) dashboard is already charting them.

Everything is served over HTTPS on a certificate generated uniquely on your VM's first boot. OpenObserve itself is bound to loopback and is never exposed directly — nginx is the only network gate, and the only ports reachable from off the box are SSH, HTTP (which redirects to HTTPS) and HTTPS.

What is included:

  • OpenObserve 0.92.2 (the open source AGPL-3.0 build) as the openobserve systemd service, bound to 127.0.0.1:5080
  • Vector 0.58.0 as the vector systemd service, pre-configured as OpenObserve's collection tier
  • Host logs from journald and host metrics (CPU, memory, load, disk, filesystem, network) flowing on first boot, with no configuration required
  • A Host Overview (Vector) dashboard, provisioned at first boot, charting log volume, memory, load average and filesystem usage
  • Two authenticated endpoints for your other machines: /ingest/ for JSON over HTTPS and /otlp/ for OTLP/HTTP
  • nginx terminating TLS on a certificate generated on this VM, with HTTP permanently redirecting to HTTPS
  • Four independent secrets generated on first boot — the OpenObserve login, the ingest credential, Vector's own ingest token, and the TLS key — none of which ship inside the image
  • Vector authenticates to OpenObserve with a dedicated, revocable ingest-only token, never the administrator account
  • A dedicated Azure data disk mounted at /var/lib/openobserve holding the telemetry store
  • An unauthenticated /healthz endpoint for Azure Load Balancer health probes
  • 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet and subnet in the target region. Standard_B2s (2 vCPU / 4 GiB RAM) is a comfortable starting point — on a freshly deployed VM the whole stack uses well under 1 GiB — and you can scale the VM and the data disk up as your ingest volume grows.

NSG inbound rules: allow 22/tcp from your management network, and 443/tcp from wherever you will browse the UI and ship telemetry from. 80/tcp is optional and only exists to redirect callers to HTTPS.

The TLS certificate generated on first boot is self-signed, so browsers will show a warning until you replace it with your own certificate (see Maintenance). The curl examples below therefore use -k.

Step 1 - Deploy from the Azure Marketplace

In the Azure portal, choose Create a resource, search for OpenObserve with Vector on Ubuntu 24.04 LTS by cloudimg, and select Create. Pick your subscription, resource group and region, choose Standard_B2s or larger, and select SSH public key authentication with the username azureuser. On the Networking tab allow inbound 22 and 443. Review and create.

Step 2 - Deploy from the Azure CLI

Replace the resource group, region and image reference to match your subscription.

az group create --name observability-rg --location eastus

az vm create \
  --resource-group observability-rg \
  --name openobserve-vector-01 \
  --image <cloudimg-openobserve-vector-image-urn> \
  --size Standard_B2s \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard

az vm open-port --resource-group observability-rg --name openobserve-vector-01 --port 443 --priority 1001

Step 3 - Connect to your VM

ssh azureuser@<your-vm-ip>

Step 4 - Confirm the services are running

Three services make up the appliance: OpenObserve (the backend), Vector (the collector) and nginx (the TLS front door).

systemctl is-active openobserve vector nginx

Expected output:

active
active
active

Confirm the versions and the network posture. Only SSH, HTTP and HTTPS are reachable from off the box — OpenObserve, Vector's management API and Vector's ingest listeners are all bound to loopback.

sudo /usr/local/bin/openobserve --version
vector --version | awk '{print $1, $2}'
ss -tlnH | awk '{print $4}' | grep -vE '^(127\.|\[::1\])' | sed -E 's/.*:([0-9]+)$/\1/' | sort -un | tr '\n' ' '

Expected output:

openobserve v0.92.2
vector 0.58.0
22 80 443

Terminal output showing openobserve v0.92.2, vector 0.58.0 and nginx 1.24.0 installed, all three services reporting active, the only off box TCP listeners being ports 22, 80 and 443, and the six loopback only listeners for the OpenObserve HTTP and gRPC ports, the Vector management API and the Vector HTTP and OTLP ingest sources

Step 5 - Retrieve your credentials

Every secret on this VM was generated during its first boot. Nothing here shipped inside the image, and no two deployments share a value.

sudo cat /root/openobserve-vector-credentials.txt

The file contains your OpenObserve sign-in, the credential your other machines will use to ship telemetry in, and the ingest token Vector itself uses:

OPENOBSERVE_URL=https://<your-vm>/
OPENOBSERVE_USERNAME=admin@openobserve.local
OPENOBSERVE_PASSWORD=<generated on this VM>
INGEST_URL=https://<your-vm>/ingest/
OTLP_URL=https://<your-vm>/otlp/
INGEST_USERNAME=ingest
INGEST_PASSWORD=<generated on this VM>
OPENOBSERVE_INGEST_USER=default
OPENOBSERVE_INGEST_TOKEN=<generated on this VM>

Terminal output showing the credentials file is mode 0600 owned by root, its contents listing the OpenObserve URL and username with the password redacted, the ingest and OTLP URLs with the ingest username and redacted password, and the redacted OpenObserve ingest token, followed by the per VM TLS certificate subject and validity dates and the o2oi token prefix confirming Vector uses an ingest only token

Confirm the TLS front door is answering and that plain HTTP redirects to it:

curl -sk -o /dev/null -w 'healthz: HTTP %{http_code}\n' https://127.0.0.1/healthz
curl -sk -o /dev/null -w 'port 80: HTTP %{http_code}\n' http://127.0.0.1/

Expected output:

healthz: HTTP 200
port 80: HTTP 301

/healthz is deliberately unauthenticated so an Azure Load Balancer health probe can use it. Everything else requires a login.

Step 6 - Sign in to the web UI

Open https://<your-vm-ip>/ in a browser. Accept the certificate warning (the certificate was generated on your VM and is not signed by a public authority — see Maintenance to replace it). Sign in with the OPENOBSERVE_USERNAME and OPENOBSERVE_PASSWORD from Step 5.

The OpenObserve sign in page served over HTTPS, showing the OpenObserve logo above a Login heading with User Email and Password fields and a Login button

Step 7 - Your host logs are already there

This is the part that is already done for you. Vector has been reading this host's journal since first boot and writing it into the host_logs stream.

In the UI choose Logs in the left sidebar and select the host_logs stream. To make the records readable, hover a field in the left-hand field list and use its interesting field control to promote service and message into table columns.

The OpenObserve logs explorer with the host_logs stream selected, a query filtering on service equals init.scope, a histogram of matching events over time, the narrow field list showing cloudimg_stream, command, host, message, pid, service, severity, timestamp and transport, and a message column of real systemd journal lines including nginx, vector and openobserve services starting and the cloudimg first boot units finishing

You can confirm the same thing from the shell. This asks OpenObserve for the streams Vector is feeding:

PW='<OPENOBSERVE_PASSWORD>'
RESP=$(curl -sk -u "admin@openobserve.local:${PW}" https://127.0.0.1/api/default/streams)
echo "${RESP}" | jq -r '(.list // [])[].name' 2>/dev/null \
  || echo "not authenticated - check OPENOBSERVE_PASSWORD from Step 5"

Expected output:

host_logs
host_metrics
remote_logs

otlp_logs appears as soon as the first OTLP event arrives (Step 10).

Step 8 - Read the shipped dashboard

Choose Dashboards in the sidebar and open Host Overview (Vector). It was created during first boot and charts the telemetry Vector is collecting from this host: log volume, memory used, load average and filesystem usage.

The Host Overview Vector dashboard in OpenObserve showing four panels drawing real data - a pink bar chart of host log volume in events, an orange line chart of memory used in megabytes, a blue line chart of the one minute load average, and a purple line chart of filesystem used in gigabytes, each with a time axis and a legend

Host metrics are collected by Vector's host_metrics source and stored in the host_metrics stream as structured records, one per metric sample, carrying the metric name and its value. The dashboard panels query that stream directly, so you can build your own panels the same way.

Step 9 - Ship logs from your other machines

The appliance exposes an authenticated JSON endpoint at https://<your-vm-ip>/ingest/. Anything you POST there is processed by Vector and lands in the remote_logs stream. Use the INGEST_USERNAME and INGEST_PASSWORD from Step 5.

INGEST_USER='<INGEST_USERNAME>'
INGEST_PASS='<INGEST_PASSWORD>'
curl -sk -u "${INGEST_USER}:${INGEST_PASS}" \
  -H 'Content-Type: application/json' \
  -d '[{"message":"order-service timeout id=8821","level":"warn","app":"order-service"}]' \
  -o /dev/null -w 'authenticated ingest: HTTP %{http_code}\n' \
  https://127.0.0.1/ingest/

Expected output:

authenticated ingest: HTTP 200

Without credentials the endpoint refuses the request:

curl -sk -X POST -H 'Content-Type: application/json' \
  -d '[{"message":"no credentials"}]' \
  -o /dev/null -w 'anonymous ingest: HTTP %{http_code}\n' \
  https://127.0.0.1/ingest/

Expected output:

anonymous ingest: HTTP 401

Read the event back out of the remote_logs stream:

PW='<OPENOBSERVE_PASSWORD>'
S=$(( ($(date -u +%s) - 600) * 1000000 )); E=$(( $(date -u +%s) * 1000000 ))
curl -sk -u "admin@openobserve.local:${PW}" -H 'Content-Type: application/json' \
  -d "{\"query\":{\"sql\":\"SELECT app, level, message FROM \\\"remote_logs\\\" ORDER BY _timestamp DESC\",\"start_time\":${S},\"end_time\":${E},\"from\":0,\"size\":3}}" \
  https://127.0.0.1/api/default/_search > /tmp/remote-logs.json
jq -c '(.hits // [])[]' /tmp/remote-logs.json 2>/dev/null \
  || echo "not authenticated - check OPENOBSERVE_PASSWORD from Step 5"

Terminal output showing an authenticated POST to the ingest endpoint returning HTTP 200, the same endpoint without credentials returning HTTP 401 rejected, the ingested order-service event read back out of the remote_logs stream with its app, level and message fields, and the three streams Vector feeds - host_logs, host_metrics and remote_logs

From another machine, point any HTTP-capable log shipper at the same endpoint. For example, a Vector instance elsewhere in your estate:

sinks:
  cloudimg_appliance:
    type: http
    inputs: [your_source]
    uri: "https://<your-vm-ip>/ingest/"
    method: post
    auth:
      strategy: basic
      user: "ingest"
      password: "<your INGEST_PASSWORD>"
    encoding:
      codec: json

Step 10 - Ship OpenTelemetry (OTLP) telemetry

The appliance also accepts OTLP over HTTP at https://<your-vm-ip>/otlp/, protected by the same ingest credential. Point any OpenTelemetry SDK or Collector at it:

OTEL_EXPORTER_OTLP_ENDPOINT=https://<your-vm-ip>/otlp
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic <base64 of ingest:INGEST_PASSWORD>

Logs arriving this way land in the otlp_logs stream. Like /ingest/, the endpoint rejects unauthenticated callers:

curl -sk -X POST -H 'Content-Type: application/json' -d '{}' \
  -o /dev/null -w 'anonymous OTLP: HTTP %{http_code}\n' \
  https://127.0.0.1/otlp/v1/logs

Expected output:

anonymous OTLP: HTTP 401

The OTLP gRPC listener is deliberately bound to loopback only and is not exposed off the box, because gRPC cannot be protected by the front door's HTTP authentication. Use the OTLP/HTTP endpoint above from remote machines.

Step 11 - Verify the whole pipeline end to end

The image ships a probe that proves the pipeline really is joined up: it writes a uniquely tagged line into this host's journal, then asks OpenObserve's search API for that exact line back.

sudo /usr/local/sbin/openobserve-vector-e2e.sh

Expected output:

E2E_PIPELINE_OK tag=cloudimg-e2e-103075c9cee57be6 path=journald->vector->openobserve/host_logs

A second script additionally checks the authentication posture — that anonymous and wrong-password calls are refused, that your per-VM password works, and that Vector's ingest token is genuinely ingest-only and cannot read your data back:

sudo /usr/local/sbin/openobserve-vector-roundtrip.sh

Terminal output showing the end to end probe returning E2E_PIPELINE_OK with a unique tag and the path journald to vector to openobserve host_logs, followed by the round trip script reporting that the per VM root login authenticates while anonymous and wrong password calls are rejected, and that Vector uses a dedicated ingest scoped token whose read attempt is refused with 401

Step 12 - Where your data lives

The telemetry store sits on a dedicated 20 GiB Azure data disk, separate from the OS disk, so you can grow it independently.

findmnt -no SOURCE,TARGET,FSTYPE /var/lib/openobserve
df -h /var/lib/openobserve | tail -1

Expected output (the device name varies with how Azure attaches the disk — /dev/sda and /dev/sdc are both normal; the mount point is what matters):

/dev/sdc /var/lib/openobserve ext4
/dev/sdc         20G   80M   19G   1% /var/lib/openobserve

To grow it, resize the disk in the Azure portal, then extend the filesystem with sudo resize2fs "$(findmnt -no SOURCE /var/lib/openobserve)".

Step 13 - Add your own sources and destinations

Vector's pipeline lives at /etc/vector/vector.yaml. It is generated on first boot from the template at /usr/share/cloudimg/vector.yaml.tmpl with this VM's credentials substituted in.

sudo grep -E '^\s{2}[a-z_]+:$' /etc/vector/vector.yaml | tr -d ' :' | tr '\n' ' '

Expected output:

host_journal host_metrics remote_http remote_otlp drop_self_noise journal_tagged metrics_as_logs metrics_tagged remote_tagged otlp_tagged oo_host_logs oo_host_metrics oo_remote_logs oo_otlp_logs

To add a source — a file, a syslog listener, a cloud provider's log stream — edit that file, add your source and wire it into one of the existing OpenObserve sinks, then validate and reload:

sudo vector validate /etc/vector/vector.yaml
sudo systemctl reload vector

Two things worth knowing when you edit it:

  • Vector's journald source emits field names in upper case (_SYSTEMD_UNIT, PRIORITY). OpenObserve lower-cases them when it stores them, so the name you see in the search UI is not the name to use in a VRL expression.
  • The shipped pipeline drops OpenObserve's own routine INFO log lines. Without that filter OpenObserve logs every request Vector makes to it, Vector ships that line back, and the two feed each other — on a test build that loop accounted for around 90% of all collected log volume. Warnings and errors from OpenObserve are still collected.

The OpenObserve Data sources page showing the ingestion catalogue with Recommended, Custom, Web Servers, Databases, Security, DevOps, Networking, Message Queues, Languages and Frameworks, AI Integrations and Other tabs, a source list including Kubernetes, Windows, Linux, AWS, GCP, Azure and OpenTelemetry, a Manage Tokens button, and a quick install panel with a ready made command

Maintenance

Replace the TLS certificate. The certificate generated at first boot is self-signed. To use your own, replace the two files and reload nginx:

sudo cp your-certificate.crt /etc/ssl/cloudimg/openobserve-vector.crt
sudo cp your-private.key    /etc/ssl/cloudimg/openobserve-vector.key
sudo chown root:www-data /etc/ssl/cloudimg/openobserve-vector.key
sudo chmod 0640 /etc/ssl/cloudimg/openobserve-vector.key
sudo nginx -t && sudo systemctl reload nginx

Rotate the ingest credential. The credential your remote machines use is an nginx basic-auth entry. To change it:

sudo htpasswd -B /etc/nginx/.htpasswd-ingest ingest

Rotate Vector's OpenObserve token. Vector authenticates with an ingest-only token, visible in the UI under Data sources → Manage Tokens, where it can be disabled or rotated. If you rotate it, update /etc/openobserve/ingest-user.env and the sink credentials in /etc/vector/vector.yaml, then sudo systemctl restart vector.

Change the OpenObserve password. Use Settings → Users in the web UI. Additional users can be created there too.

Retention. OpenObserve's retention settings are under Settings; per-stream retention is configurable from the Streams page.

Operating system updates are delivered by Ubuntu's unattended-upgrades, which remains enabled on the image.

Restart the stack:

sudo systemctl restart openobserve vector nginx

Licensing and trademarks

OpenObserve is distributed under the GNU Affero General Public License v3.0. This image ships the unmodified open source build of OpenObserve 0.92.2; the corresponding source is available from github.com/openobserve/openobserve. Note that OpenObserve also publishes a separately licensed enterprise build — this image does not contain it.

Vector is distributed under the Mozilla Public License 2.0. This image ships the unmodified official release of Vector 0.58.0; the corresponding source is available from github.com/vectordotdev/vector.

"OpenObserve" is a trademark of OpenObserve Inc. and "Vector" is a project of Datadog, Inc. Both names are used here only to identify the software this image contains. cloudimg is not affiliated with, endorsed by, or sponsored by OpenObserve Inc. or Datadog, Inc.

Support

cloudimg provides 24/7 support for this image. Contact support@cloudimg.co.uk with your Azure subscription ID and the VM name.