Observability AWS

Grafana Tempo 3 Tracing Stack on AWS User Guide

| Product: Grafana Tempo 3 Tracing Stack on Ubuntu 24.04 on AWS

Overview

This guide covers launching and using the Grafana Tempo 3 Tracing Stack on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg tracing and APM listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

Distributed tracing needs more than a trace store. Grafana Tempo stores traces but has no user interface and no authentication of its own, so this image ships the whole path, wired together and tested end to end:

  • Grafana Alloy 1.21.3 receives spans from your applications over OpenTelemetry (OTLP), batches them and forwards them to Tempo.
  • Grafana Tempo 3.1.1 stores them on the instance's disk, in single-binary mode, and answers trace-by-ID and TraceQL queries.
  • Grafana OSS 13.3.0 is the user interface, with a Tempo data source already set up, so trace search and the trace view work the first time you sign in.
  • nginx is the only way in: Grafana over HTTPS, and an authenticated OTLP/HTTP endpoint at https://<host>/otlp/v1/traces for your applications.

What the cloudimg image adds:

  • Logins made on your instance. On first boot the instance generates its own Grafana administrator password, Grafana secret_key, OTLP ingest password and HTTPS certificate. The image ships no Grafana database at all, so no account, password hash or salt is shared with any other instance. Grafana's default admin/admin never exists, and sign-up and anonymous access are off.
  • Authenticated trace ingest. Tempo and Alloy have no authentication, so nginx guards the ingest path with its own login (otlp) that is separate from the Grafana administrator. Browsers cannot post to it from other web sites.
  • Everything on localhost. Tempo (3200, 9096, 4327), Alloy (4317, 4318, 12345) and Grafana (3000) listen on 127.0.0.1 only, and a firewall rule stops other local accounts from reaching them. Tempo runs as one process with no cluster gossip, so nothing listens on port 7946.
  • HTTPS from the first boot, with a certificate for the instance's own names. Port 80 redirects to HTTPS and requests for a host name the instance does not serve are dropped. Sign-in attempts are rate limited and cross-site requests to Grafana are refused.
  • Built from source with patched dependencies. Tempo and Alloy are compiled from their release tags with a current Go toolchain, after raising the dependencies that had published security advisories. Grafana is the open source edition from Grafana's signed package repository.
  • Nothing sent home. Tempo's usage report, Grafana's usage statistics, update checks, news feed and plugin installs, and Alloy's reporting are all off.
  • No metadata or private network access for the services. tempo, alloy, grafana-server and nginx are blocked from the instance metadata service, the AMI requires IMDSv2, and the Tempo, Alloy and Grafana accounts cannot open connections to private addresses unless you allow them (Step 11).
  • Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: Grafana Tempo 3.1.1, Grafana Alloy 1.21.3 (both compiled from source), Grafana OSS 13.3.0 (the grafana package from apt.grafana.com), nginx 1.24 and OpenSSL 3.0 from Ubuntu 24.04, cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

A trace in Grafana: the waterfall of a checkout request across three services, read from Tempo

Prerequisites

  • An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (or a private route you use for management).
  • A security group that allows TCP 22 (SSH) from your admin network and TCP 443 (Grafana and trace ingest) only from the people and the applications that need it. Port 80 only redirects to HTTPS, so you can leave it closed. Never open 3000, 3200, 4317, 4318, 4327, 9096 or 12345: they listen on the loopback address only.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick Grafana Tempo 3 Tracing Stack on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large (or larger for heavy trace volumes), your key pair and a security group as above. Traces are stored on the root volume under /var/lib/tempo, so give it room for your traffic and retention.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=50,VolumeType=gp3}' \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=tempo}]'

First boot finishes a few seconds after the instance is running; Grafana is ready about half a minute later.

Step 2: Connect to your instance

Version SSH user
Grafana Tempo 3 Tracing Stack on Ubuntu 24.04 ubuntu 
ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

systemctl is-active tempo alloy grafana-server nginx tempo-stack-firstboot
sudo test -f /var/lib/cloudimg/tempo-stack-firstboot.done && echo FIRSTBOOT_DONE

active
active
active
active
active
FIRSTBOOT_DONE

Only SSH and nginx listen on network ports; Tempo, Alloy and Grafana are on the loopback address:

sudo ss -lntpH | awk '{match($6, /"[^"]+"/); print $4, substr($6, RSTART + 1, RLENGTH - 2)}' | sort -u

0.0.0.0:22 sshd
0.0.0.0:443 nginx
0.0.0.0:80 nginx
127.0.0.1:12345 alloy
127.0.0.1:3000 grafana
127.0.0.1:3200 tempo
127.0.0.1:4317 alloy
127.0.0.1:4318 alloy
127.0.0.1:4327 tempo
127.0.0.1:9096 tempo
127.0.0.53%lo:53 systemd-resolve
127.0.0.54:53 systemd-resolve
[::]:22 sshd
[::]:443 nginx
[::]:80 nginx

Tempo's own readiness check, on the loopback address:

curl -s http://127.0.0.1:3200/ready; echo

ready

Step 4: Get your URL and passwords

sudo cat /var/lib/cloudimg/tempo-stack-credentials.txt

The file is readable by root only. It looks like this, with this instance's own passwords where the <...> placeholders are:

# Grafana Tempo 3 Tracing Stack (Tempo 3.1.1, Grafana OSS 13.3.0, Alloy 1.21.3) - generated on first boot by
# tempo-stack-firstboot.service. Every password here is unique to this instance; none was baked into the image.
# Keep this file secret.

GRAFANA_URL=https://ec2-203-0-113-10.compute-1.amazonaws.com/
GRAFANA_ADMIN_USER=admin
GRAFANA_ADMIN_PASSWORD=<GRAFANA_ADMIN_PASSWORD>
#
# Send traces (OTLP/HTTP, protobuf or JSON) from other machines to this URL with HTTP Basic auth:
OTLP_HTTP_TRACES_URL=https://ec2-203-0-113-10.compute-1.amazonaws.com/otlp/v1/traces
OTLP_USER=otlp
OTLP_PASSWORD=<OTLP_PASSWORD>
OTLP_AUTHORIZATION_HEADER=Basic <base64 of otlp:OTLP_PASSWORD>
# On this instance (or through an SSH tunnel) Alloy also takes OTLP/gRPC on 127.0.0.1:4317 and OTLP/HTTP on
# 127.0.0.1:4318 without a login.
#
# Sign in to Grafana as admin with the password above (change it under your profile). The Tempo datasource is ready in
# Explore. New ingest password: sudo tempo-stack-otlp-password   Self-test: sudo tempo-stack-selftest
# Change the served name: sudo tempo-stack-set-hostname <your-domain>
# User guide: https://www.cloudimg.co.uk/guides/grafana-tempo-3-on-ubuntu-24-04-aws/

The URL uses the instance's public DNS name from the EC2 instance metadata (or its public IP address when it has no DNS name). The certificate also names the public and private IP addresses; requests to an IP address are redirected to the name. To serve Grafana under your own DNS name, see Step 10.

Without a login, Grafana's API and the ingest endpoint refuse the request, and the health check answers without one:

H=$(sudo sed -n 's#^GRAFANA_URL=https://\([^/]*\)/.*#\1#p' /var/lib/cloudimg/tempo-stack-credentials.txt)
curl -sk --resolve "$H:443:127.0.0.1" -o /dev/null -w '%{http_code}\n' "https://$H/api/user"
curl -sk --resolve "$H:443:127.0.0.1" -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Type: application/json' --data '{"resourceSpans":[]}' "https://$H/otlp/v1/traces"
curl -sk --resolve "$H:443:127.0.0.1" "https://$H/healthz"; echo

401
401
ok

Step 5: Run the self-test

The shipped self-test sends a test span through the public ingest path (nginx, Alloy, Tempo) and reads it back through Grafana's Tempo data source by trace ID and with TraceQL. It also checks the sign-in and ingest logins, the cross-site guards, the loopback-only listeners and the metadata block. Passwords reach curl in a root-only temporary file, never on a command line:

sudo tempo-stack-selftest

SELFTEST-OK https 301 + healthz + Grafana 13.3.0 OSS healthy + /api/user 401 anonymous and admin/admin, 200 per-instance admin + Tempo datasource provisioned + OTLP ingest 401 none/wrong/admin login, 405 GET, 403 browser, 200 ingest login + span baeedf65 read back by trace ID and TraceQL through Grafana + 403 cross-origin login + 403 cross-site datasource proxy + 444 unknown host + loopback-only grafana/alloy/tempo + metadata 000 inside 4 services (control 200)

Step 6: Send your first trace

Applications send traces with OTLP/HTTP to https://<host>/otlp/v1/traces with the otlp login from the credentials file. To see it work on the instance, send an example trace for an online shop: a POST /checkout request in a service called shop-frontend that calls cart.total and payment.charge in a shop-payments service, which runs a database query in shop-db.

The shell function otlp passes the Authorization header from the credentials file to curl on standard input, so it never appears on a command line:

otlp() { sudo sed -n 's/^OTLP_AUTHORIZATION_HEADER=\(.*\)/header = "Authorization: \1"/p' /var/lib/cloudimg/tempo-stack-credentials.txt | curl -sk -K - "$@"; }
H=$(sudo sed -n 's#^GRAFANA_URL=https://\([^/]*\)/.*#\1#p' /var/lib/cloudimg/tempo-stack-credentials.txt)
TID=$(openssl rand -hex 16); echo "$TID" > /tmp/demo-trace-id
python3 - "$TID" > /tmp/demo-trace.json <<'EOF'
import json, os, sys, time
tid = sys.argv[1]
now = time.time_ns()
def span(name, sid, parent, start_ms, dur_ms, kind=2, attrs=()):
    s = {"traceId": tid, "spanId": sid, "name": name, "kind": kind,
         "startTimeUnixNano": str(now - 900_000_000 + start_ms * 1_000_000),
         "endTimeUnixNano": str(now - 900_000_000 + (start_ms + dur_ms) * 1_000_000),
         "attributes": [{"key": k, "value": {"stringValue": v}} for k, v in attrs]}
    if parent:
        s["parentSpanId"] = parent
    return s
ids = [os.urandom(8).hex() for _ in range(4)]
def res(svc, spans):
    return {"resource": {"attributes": [{"key": "service.name", "value": {"stringValue": svc}}]},
            "scopeSpans": [{"scope": {"name": "guide-example"}, "spans": spans}]}
print(json.dumps({"resourceSpans": [
    res("shop-frontend", [span("POST /checkout", ids[0], None, 0, 420, 2, [("http.request.method", "POST"), ("http.route", "/checkout")]),
                          span("cart.total", ids[1], ids[0], 15, 60, 1)]),
    res("shop-payments", [span("payment.charge", ids[2], ids[0], 90, 300, 2, [("payment.provider", "example")])]),
    res("shop-db", [span("SELECT orders", ids[3], ids[2], 120, 180, 3, [("db.system", "postgresql")])]),
]}))
EOF
otlp --resolve "$H:443:127.0.0.1" -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Type: application/json' --data-binary @/tmp/demo-trace.json "https://$H/otlp/v1/traces"

200

Read it back from Tempo by its trace ID, on the loopback address (spans are readable within a few seconds):

sleep 5
curl -s "http://127.0.0.1:3200/api/v2/traces/$(cat /tmp/demo-trace-id)" | python3 -c '
import json, sys
d = json.load(sys.stdin)
for b in d["trace"]["resourceSpans"]:
    svc = [a["value"]["stringValue"] for a in b["resource"]["attributes"] if a["key"] == "service.name"][0]
    for ss in b["scopeSpans"]:
        for s in ss["spans"]:
            print(svc, "|", s["name"])' | sort

shop-db | SELECT orders
shop-frontend | POST /checkout
shop-frontend | cart.total
shop-payments | payment.charge

TraceQL search finds a trace once Tempo has written it to a block, usually within a minute. This asks Tempo for traces from shop-frontend that took longer than 100 ms:

for i in $(seq 1 20); do
  R=$(curl -sG http://127.0.0.1:3200/api/search --data-urlencode 'q={ resource.service.name = "shop-frontend" && duration > 100ms }' --data-urlencode limit=5)
  echo "$R" | grep -q "$(cat /tmp/demo-trace-id | sed 's/^0*//')" && break; sleep 5
done
echo "$R" | python3 -c 'import json,sys; [print(t["rootServiceName"], "|", t["rootTraceName"], "|", t["durationMs"], "ms") for t in json.load(sys.stdin)["traces"]]' | sort -u

shop-frontend | POST /checkout | 420 ms

Step 7: Explore traces in Grafana

Open the GRAFANA_URL from the credentials file in your browser. Your browser warns once about the self-signed certificate; accept it (or install your own certificate, Step 10), then sign in as admin with the GRAFANA_ADMIN_PASSWORD:

The Grafana sign-in page served over HTTPS by the instance

Grafana's brute-force protection is on: after five failed sign-ins the account is locked for a few minutes, and nginx also limits how fast sign-ins can be tried.

Open Explore (the compass icon), keep the Tempo data source, choose the Search or TraceQL query type and run a query such as { resource.service.name = "shop-frontend" }. Each row is one trace with its root service, root span and duration:

TraceQL search in Explore, listing the example checkout trace from shop-frontend

Click a trace ID to open the trace view. The waterfall shows every span with its service and timing: here the checkout spends most of its time in payment.charge, and most of that in the SELECT orders database query:

The trace view: POST /checkout in shop-frontend, cart.total, payment.charge in shop-payments and SELECT orders in shop-db

The Tempo data source is provisioned from /etc/grafana/provisioning/datasources/cloudimg-tempo.yaml and points at Tempo on http://127.0.0.1:3200. Because it is provisioned from a file it is read-only in the UI, Grafana re-applies it on every start, and it is the default data source:

The provisioned Tempo data source, read-only, pointing at Tempo on the loopback address

Streaming search is switched off in the data source: Grafana streams TraceQL results over a WebSocket, which fails silently when you reach Grafana through a different name or a proxy. Searches return the same results without it.

Step 8: Send traces from your applications

Point your OpenTelemetry SDKs, collectors or agents at the ingest URL with the otlp login. Both OTLP/HTTP encodings (protobuf and JSON) are accepted. Copy the instance's certificate (/etc/ssl/tempo-stack/tempo-stack.crt) to the senders so they can verify it, or install a certificate your systems already trust (Step 10).

Any OpenTelemetry SDK (environment variables; the header value is OTLP_AUTHORIZATION_HEADER from the credentials file, with the space written as %20):

OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<host>/otlp/v1/traces
OTEL_EXPORTER_OTLP_TRACES_HEADERS=Authorization=Basic%20<base64 of otlp:password>
OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE=/etc/otel/tempo-stack.crt
OTEL_SERVICE_NAME=my-service

An OpenTelemetry Collector or Grafana Alloy on another machine:

otelcol.auth.basic "tempo_stack" {
  username = "otlp"
  password = sys.env("TEMPO_STACK_OTLP_PASSWORD")
}

otelcol.exporter.otlphttp "tempo_stack" {
  client {
    endpoint = "https://<host>/otlp"
    auth     = otelcol.auth.basic.tempo_stack.handler
    tls {
      ca_file = "/etc/alloy/tempo-stack.crt"
    }
  }
}

OTLP/gRPC. Alloy's gRPC receiver has no login, so it stays on the loopback address. From your workstation, forward a local port over SSH and point the exporter at localhost:4317 (plaintext, inside the SSH tunnel):

ssh -i <your-key.pem> -N -L 4317:127.0.0.1:4317 ubuntu@<public-ip>

Applications on the instance itself can send to 127.0.0.1:4317 (gRPC) or http://127.0.0.1:4318/v1/traces (HTTP) without a login.

Keep the ingest password in a secrets store, never in source code. To issue a new one (the old one stops working at once), run sudo tempo-stack-otlp-password and update your senders.

Step 9: Storage and retention

Tempo runs as one process (-target=all) with local storage: the write-ahead log and the blocks are under /var/lib/tempo. Its configuration is /etc/tempo/tempo.yaml:

sudo grep -A6 '^storage:' /etc/tempo/tempo.yaml
sudo du -sh /var/lib/tempo

storage:
  trace:
    backend: local
    wal:
      path: /var/lib/tempo/wal
    local:
      path: /var/lib/tempo/blocks
3.7M    /var/lib/tempo

Blocks are kept for 14 days (336h, Tempo's default) by default. To change it, add a compaction block under backend_worker in /etc/tempo/tempo.yaml, for example seven days:

backend_worker:
  compaction:
    block_retention: 168h

then sudo systemctl restart tempo. Keep every address in the file on 127.0.0.1.

For long retention or high volumes, Tempo can store blocks in Amazon S3 instead (backend: s3 with an s3: block giving bucket, endpoint and region); see the Tempo storage documentation. tempo.service is blocked from the instance metadata service, so it cannot use an instance role: give it an access key for a bucket-only IAM user through a root-only environment file (sudo systemctl edit tempo, then EnvironmentFile= with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY). If you reach S3 through a VPC interface endpoint with private addresses, add the endpoint's subnet to the egress allow-list (Step 11).

Step 10: Your own DNS name and certificate

Point a DNS record at the instance (an Elastic IP keeps the address stable), then set the name Grafana is served as. This mints a new self-signed certificate for the name, updates nginx, the cross-site guards, Grafana's root_url and the URLs in the credentials file, and keeps the name across reboots:

sudo tempo-stack-set-hostname <your-domain>

To use a certificate from a public or internal certificate authority instead (for example one issued by Let's Encrypt for <your-domain>), copy it over the generated pair and reload nginx (running tempo-stack-set-hostname again replaces it with a self-signed one):

sudo install -m 0644 <your-domain>.crt /etc/ssl/tempo-stack/tempo-stack.crt
sudo install -m 0600 <your-domain>.key /etc/ssl/tempo-stack/tempo-stack.key
sudo nginx -t && sudo systemctl reload nginx

Step 11: Outbound connections and the egress allow-list

The tempo, alloy and grafana accounts may connect to public internet addresses, but not to the instance metadata service, link-local, private (RFC 1918), carrier-grade NAT or reserved addresses, or your VPC's own address ranges, and not over IPv6. This stops a compromised or misconfigured service from probing your private network. nginx and your own shell are not affected. Check the rules:

sudo iptables -S CLOUDIMG-TEMPO-STACK-EGRESS | head -5
sudo iptables -S OUTPUT | grep -c CLOUDIMG

-N CLOUDIMG-TEMPO-STACK-EGRESS
-A CLOUDIMG-TEMPO-STACK-EGRESS -d 127.0.0.0/8 -o lo -j RETURN
-A CLOUDIMG-TEMPO-STACK-EGRESS -d 169.254.0.0/16 -p tcp -j REJECT --reject-with tcp-reset
-A CLOUDIMG-TEMPO-STACK-EGRESS -d 169.254.0.0/16 -j REJECT --reject-with icmp-admin-prohibited
-A CLOUDIMG-TEMPO-STACK-EGRESS -d 100.64.0.0/10 -p tcp -j REJECT --reject-with tcp-reset
4

If Tempo or Grafana must reach something on a private address (an S3 VPC endpoint, a Prometheus or alert receiver in your VPC), add one CIDR per line to /etc/cloudimg/tempo-stack-egress-allow and reload the rules (sudo systemctl reload cloudimg-tempo-stack-egress). Metadata and link-local addresses can never be allowed.

Step 12: Security checks

The metadata block is a systemd setting on each service:

systemctl show -p IPAddressDeny tempo alloy grafana-server nginx

IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128
IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128
IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128
IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128

A request for a host name the instance does not serve gets no answer at all (nginx closes the connection, so curl exits with code 52):

curl -sk -o /dev/null -w '%{http_code}\n' -H 'Host: attacker.example' https://127.0.0.1/healthz; echo "curl exit $?"

000
curl exit 52

The certificate was generated on this instance for its own names:

sudo openssl x509 -in /etc/ssl/tempo-stack/tempo-stack.crt -noout -subject -ext subjectAltName

subject=CN = ec2-203-0-113-10.compute-1.amazonaws.com, O = cloudimg
X509v3 Subject Alternative Name: 
    DNS:ec2-203-0-113-10.compute-1.amazonaws.com, IP Address:203.0.113.10, IP Address:172.31.20.15, DNS:ip-172-31-20-15.ec2.internal, IP Address:127.0.0.1, DNS:localhost

Grafana runs the open source edition and loads its bundled plugins, the Tempo data source among them, from the root-owned package directory; the grafana account cannot change plugin code and no other plugins are installed:

sudo journalctl -u grafana-server --no-pager -o cat | grep -F 'msg="Path Plugins"' | tail -1 | grep -o 'path=.*'

path="[/var/lib/grafana/plugins /usr/share/grafana/data/plugins-bundled]"

Step 13: Versions, files and licences

/usr/local/bin/tempo --version 2>&1 | head -1
/usr/local/bin/alloy --version 2>&1 | head -1
dpkg-query -W -f='grafana ${Version}\n' grafana
nginx -v 2>&1

tempo, version v3.1.1 (branch: HEAD, revision: 9db7df7)
alloy, version v1.21.3 (branch: HEAD, revision: f2108cd)
grafana 13.3.0
nginx version: nginx/1.24.0 (Ubuntu)
Path What it is
/etc/tempo/tempo.yaml Tempo's configuration: loopback listeners, local storage, no usage report
/var/lib/tempo Trace storage (write-ahead log, live store and blocks)
/etc/alloy/config.alloy Alloy's pipeline: OTLP receiver, batch, export to Tempo
/etc/grafana/grafana.ini Grafana's settings (127.0.0.1:3000, no sign-up, no anonymous access, outbound features off)
/etc/grafana/provisioning/datasources/cloudimg-tempo.yaml The provisioned Tempo data source
/etc/nginx/tempo-stack-otlp.htpasswd The ingest login (otlp, bcrypt hash)
/etc/ssl/tempo-stack/ This instance's certificate and private key
/var/lib/cloudimg/tempo-stack-credentials.txt URLs, logins and passwords (root only)
/etc/cloudimg/tempo-stack-egress-allow Private networks the services may reach (empty by default)
/usr/share/doc/cloudimg-tempo-stack/ The licences, SOURCE-OFFER.txt and the go.mod/go.sum the binaries were built from

Step 14: Changing passwords

Grafana administrator: sign in, open your profile (the avatar at the top right), choose Change password, and update or delete GRAFANA_ADMIN_PASSWORD in the credentials file (sudo nano /var/lib/cloudimg/tempo-stack-credentials.txt). Add other users under Administration > Users and access.

Trace ingest: sudo tempo-stack-otlp-password replaces the otlp password and writes the new password and Authorization header to the credentials file. nginx picks it up at once; update every sender.

Step 15: Security recommendations

  • Keep port 443 open only to the people who use Grafana and the applications that send traces, and port 22 to your admin network.
  • Use your own DNS name and a certificate your senders trust (Step 10).
  • Traces can contain URLs, user IDs and query text. Treat them like application logs, and keep the Grafana and ingest passwords in a secrets store.
  • The instance applies Ubuntu security updates automatically, including Grafana updates from Grafana's repository within the 13.3 line. A Grafana package upgrade copies its bundled plugins to /var/lib/grafana/plugins-bundled again; this image does not use that copy (it reads the packaged one), and you can delete it. To update Tempo or Alloy, replace /usr/local/bin/tempo or /usr/local/bin/alloy with a newer build after reading its release notes, then restart the service; back up /var/lib/tempo first.

Support and licensing

Grafana Tempo and Grafana are open source software licensed under the GNU Affero General Public License v3.0 (AGPL-3.0-only); Grafana Alloy is licensed under the Apache License 2.0; nginx under the 2-clause BSD license. This image runs Grafana's open source edition, never Grafana Enterprise. The source code of the shipped versions is published at github.com/grafana/tempo (v3.1.1), github.com/grafana/grafana (v13.3.0) and github.com/grafana/alloy (v1.21.3). Tempo and Alloy are built from those tags with only their Go module dependencies raised to patched versions; the exact go.mod and go.sum files and a written source offer are in /usr/share/doc/cloudimg-tempo-stack/ on every instance.

cloudimg provides 24/7 support for this image by email at support@cloudimg.co.uk and through the cloudimg website: we help with launching, sending traces from your applications and collectors, TraceQL, storage and retention, TLS with your own domain, upgrades and troubleshooting.

Grafana, Tempo and Alloy are trademarks of Grafana Labs. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.