Trickster 2.1.0 on AWS User Guide

Overview

This guide covers launching and using the Trickster 2.1.0 on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg metrics and dashboards listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

Trickster is an open source HTTP reverse proxy cache and time series dashboard accelerator. It sits between your dashboards (Grafana, for example) and your time series database, understands the query APIs of Prometheus, InfluxDB, ClickHouse, Graphite, Druid and MySQL, and caches the results of the range queries a dashboard issues on every refresh. When a panel reloads, Trickster serves the part of the time range it already holds from cache and asks the database only for the newly elapsed slice. It also collapses identical concurrent requests into one origin fetch, so a wall of dashboards opening at once does not stampede the database.

Trickster needs something to accelerate, so this image bundles Prometheus 3.13.4 on the same instance as the default origin. It listens on the loopback address, scrapes itself and Trickster, and gives the appliance real data from the moment first boot finishes. Pointing Trickster at your own Prometheus is a one-line change (Step 8).

What the cloudimg image adds:

• No default login. A 32-character basic-auth password is generated on your instance at first boot, stored as a bcrypt hash for Trickster and written in plain text only to /root/trickster-credentials.txt (root only). The image ships no password, no hash, no configuration and no TLS key.
• HTTPS from the first boot. nginx serves Trickster on port 443 with a certificate generated on your instance, port 80 redirects to HTTPS, and requests for a host name the instance does not serve are dropped. Trickster, its metrics and management listeners and Prometheus listen on 127.0.0.1 only.
• Never an open proxy. Trickster has one backend with a fixed origin from its configuration file. nginx refuses CONNECT, protocol upgrades and requests for other hosts, and only /trickster/ping is reachable without the credential; every other Trickster path (configuration, health, purge) stays on the instance.
• Fail closed. Trickster and nginx start only after the boot service has written your credential and configuration. If it cannot finish, nothing is served.
• Hardened defaults. Trickster, Prometheus, nginx and the boot service are blocked from the instance metadata service (Trickster fetches from origins you configure, so a misconfigured origin can never reach the instance's credentials), the AMI requires IMDSv2, nginx hands Trickster the real client address from the connection and the configured host name (never values a client sends), and every service runs under systemd sandboxing.
• Built from source with the patched Go toolchain. Trickster 2.1.0 is compiled on the build instance from the upstream v2.1.0 source with Go 1.27.2 and golang.org/x/net v0.60.0, which fix the Go security advisories present in the upstream 2.1.0 binary. The provenance, every checksum and the Apache 2.0 licence and notice ship in /usr/share/doc/trickster.
• Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: Trickster 2.1.0 (compiled from the checksum-verified upstream source), Prometheus 3.13.4 (the upstream release, checksum verified), nginx 1.24 (from Ubuntu 24.04), cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

Trickster has no web interface of its own: it is an HTTP API you query with curl or add to Grafana as a Prometheus data source. Prometheus has its own web UI, which you reach over an SSH tunnel (Step 7).

Prerequisites

• An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (or a private route you use for management).
• A security group that allows TCP 22 (SSH) from your admin network and TCP 443 (the accelerated API) from the networks your dashboards run in, plus TCP 80 if you want plain HTTP requests redirected to HTTPS. Never open 8480, 8481, 8484 or 9090: Trickster and Prometheus listen on the loopback address only.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick Trickster 2.1.0 on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large, your key pair and a security group as above.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=trickster}]'

First boot takes a few seconds after the instance is running.

Step 2: Connect to your instance

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

The boot service generates the credential, the configuration and the certificate, then Trickster and nginx start:

systemctl is-active prometheus trickster nginx trickster-firstboot
sudo test -f /var/lib/cloudimg/trickster-firstboot.done && echo FIRSTBOOT_DONE

All four units are active and the first boot marker is present:

active
active
active
active
FIRSTBOOT_DONE

Only SSH and nginx listen on network ports; Trickster (proxy 8480, metrics 8481, management 8484) and Prometheus (9090) listen on the loopback address:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v '127.0.0.5' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort

0.0.0.0:22 users:(("sshd"),("systemd"))
0.0.0.0:443 users:(("nginx"),("nginx"),("nginx"),("nginx"),("nginx"))
0.0.0.0:80 users:(("nginx"),("nginx"),("nginx"),("nginx"),("nginx"))
127.0.0.1:8480 users:(("trickster"))
127.0.0.1:8481 users:(("trickster"))
127.0.0.1:8484 users:(("trickster"))
127.0.0.1:9090 users:(("prometheus"))
[::]:22 users:(("sshd"),("systemd"))
[::]:443 users:(("nginx"),("nginx"),("nginx"),("nginx"),("nginx"))
[::]:80 users:(("nginx"),("nginx"),("nginx"),("nginx"),("nginx"))

Step 4: Get your URL and credential

sudo cat /root/trickster-credentials.txt

The file is readable by root only:

# Trickster 2.1.0 (with Prometheus 3.13.4 as the local origin) - generated on first boot by trickster-firstboot.service.
# This credential is unique to this instance; none was baked into the image. Keep this file secret.

TRICKSTER_URL=https://<public-ip>/
trickster.username=cloudimg
trickster.password=<TRICKSTER_PASSWORD>

# The accelerated Prometheus API (HTTP Basic auth; the certificate is self signed, minted on this instance):
#   https://<public-ip>/api/v1/query?query=up
# Grafana: add a Prometheus data source with URL https://<public-ip>/ , Basic auth with the user and password
# above, and "Skip TLS certificate verification" (or install your own certificate in /etc/nginx/ssl).
# Liveness (no auth): https://<public-ip>/trickster/ping   Re-verify the appliance: sudo trickster-selfcheck
# Change the served name:  sudo trickster-set-hostname <your-dns-name>
# Loopback only (SSH tunnel): Prometheus UI 127.0.0.1:9090, Trickster metrics 127.0.0.1:8481/metrics,
# running config 127.0.0.1:8484/trickster/config. User guide: https://www.cloudimg.co.uk/guides/trickster-2-1-0-on-ubuntu-24-04-aws/

The URL uses the instance's public IP address from the EC2 instance metadata (the private address when the instance has no public one). The certificate also names the public DNS name and the private IP address. If you stop and start the instance without an Elastic IP, the boot service picks up the new address and issues a new certificate for it. To serve Trickster under your own DNS name, see Step 9.

Step 5: Query through the accelerator

The commands below run on the instance and read the URL and password from the credentials file. From your workstation, use the same requests against https://<public-ip>/ with the user cloudimg and the password from Step 4 (curl -k accepts the self signed certificate).

The liveness endpoint is the only path that answers without the credential, and every query without it is refused with HTTP 401:

TK_URL=$(sudo sed -n 's/^TRICKSTER_URL=//p' /root/trickster-credentials.txt)
curl -sk "${TK_URL}trickster/ping"; echo
curl -sk -o /dev/null -w '%{http_code}\n' "${TK_URL}api/v1/query?query=up"

pong
401

With the credential, the Prometheus HTTP API answers through Trickster:

TK_URL=$(sudo sed -n 's/^TRICKSTER_URL=//p' /root/trickster-credentials.txt)
TK_PASS=$(sudo sed -n 's/^trickster.password=//p' /root/trickster-credentials.txt)
curl -sk -u "cloudimg:${TK_PASS}" "${TK_URL}api/v1/query?query=up" | jq -c '.data.result[] | {job: .metric.job, value: .value[1]}'

{"job":"trickster","value":"1"}
{"job":"prometheus","value":"1"}

Step 6: Watch the cache accelerate

Run the same range query twice. Trickster reports how it served each request in the X-Trickster-Result response header: the first request is a cache miss (kmiss) fetched from Prometheus, the repeat is a cache hit:

TK_URL=$(sudo sed -n 's/^TRICKSTER_URL=//p' /root/trickster-credentials.txt)
TK_PASS=$(sudo sed -n 's/^trickster.password=//p' /root/trickster-credentials.txt)
END=$(date -u +%s); START=$((END-1800))
Q="${TK_URL}api/v1/query_range?query=rate(prometheus_http_requests_total%5B1m%5D)&start=${START}&end=${END}&step=30"
for i in 1 2; do curl -sk -u "cloudimg:${TK_PASS}" -o /dev/null -D- "$Q" | tr -d '\r' | grep -i '^X-Trickster-Result'; done

X-Trickster-Result: engine=DeltaProxyCache; status=kmiss; ffstatus=kmiss
X-Trickster-Result: engine=DeltaProxyCache; status=hit; ffstatus=hit

When a dashboard refreshes later, Trickster answers with phit (partial hit): the cached part of the window comes from memory and only the new slice is fetched from Prometheus. Trickster's own Prometheus metrics, on the loopback metrics listener, count every request by cache status:

curl -s http://127.0.0.1:8481/metrics | grep '^trickster_proxy_requests_total' | head -5

trickster_proxy_requests_total{backend_name="default",cache_status="hit",http_status="200",method="GET",path="query",provider="prometheus"} 2
trickster_proxy_requests_total{backend_name="default",cache_status="hit",http_status="200",method="GET",path="query_range",provider="prometheus"} 2
trickster_proxy_requests_total{backend_name="default",cache_status="kmiss",http_status="200",method="GET",path="query",provider="prometheus"} 2
trickster_proxy_requests_total{backend_name="default",cache_status="kmiss",http_status="200",method="GET",path="query_range",provider="prometheus"} 2

Connect Grafana

In Grafana add a Prometheus data source with:

• URL: https://<public-ip>/ (or your DNS name from Step 9)
• Authentication: Basic authentication, user cloudimg, the password from Step 4
• TLS settings: Skip TLS certificate verification while the instance uses its self signed certificate, or add the certificate from /etc/nginx/ssl/trickster.crt as the CA certificate

Save and test; your dashboards then query Prometheus through Trickster's cache.

Step 7: The Prometheus web UI and the loopback endpoints

Prometheus, Trickster's metrics listener and Trickster's management listener (which prints the running configuration) are reachable only from the instance. Forward them over SSH from your workstation:

ssh -i <your-key.pem> -L 9090:127.0.0.1:9090 -L 8484:127.0.0.1:8484 ubuntu@<public-ip>

Then browse to http://127.0.0.1:9090/ for Prometheus and http://127.0.0.1:8484/trickster/config for Trickster's running configuration. On the instance itself:

curl -s http://127.0.0.1:8484/trickster/config | grep -E 'origin_url|users_file:'

    origin_url: http://127.0.0.1:9090
    users_file: /etc/trickster/users.htpasswd

Step 8: Accelerate your own Prometheus

Trickster's configuration is /etc/trickster/trickster.yaml. Change origin_url under backends: default: to your own Prometheus (any address the instance can reach) and save; Trickster checks the file every 10 seconds and reloads it without dropping connections. For example:

sudo sed -i 's#origin_url: http://127.0.0.1:9090#origin_url: http://prometheus.<your-domain>:9090#' /etc/trickster/trickster.yaml
sleep 15; sudo journalctl -u trickster --since '-1 min' --no-pager | grep -i reload

You can also put extra settings in /etc/trickster/conf.d/*.yaml; Trickster loads them after the main file and merges them. To accelerate InfluxDB, ClickHouse or another provider instead, change provider as well; the upstream documentation and example configurations are on the instance in /usr/share/doc/trickster/docs and /usr/share/doc/trickster/conf. Leave the listeners on 127.0.0.1 so all access keeps going through nginx and the credential. If you no longer need the bundled Prometheus, stop it with sudo systemctl disable --now prometheus.

Trickster is denied the instance metadata address, so an origin can never be pointed at the instance's own credentials; any other address your network routes still works.

Step 9: Your own DNS name and certificate

Point a DNS record at the instance, then tell Trickster to serve that name. This updates nginx, issues a new self signed certificate that names it and updates the URL in the credentials file:

sudo trickster-set-hostname trickster.<your-domain>

The name is saved in /etc/cloudimg/trickster-hostname and survives reboots. To use a certificate from your own CA or from Let's Encrypt, replace /etc/nginx/ssl/trickster.crt and /etc/nginx/ssl/trickster.key (keep the key mode 0600) and run sudo systemctl reload nginx.

Step 10: Changing the password

Trickster reads its users from /etc/trickster/users.htpasswd (bcrypt). Set a new password (it is read from standard input, so it does not appear in the process list), update the credentials file and reload Trickster:

printf '%s\n' '<new-password>' | sudo htpasswd -iB -C 12 /etc/trickster/users.htpasswd cloudimg
sudo sed -i 's/^trickster.password=.*/trickster.password=<new-password>/' /root/trickster-credentials.txt
sudo systemctl reload trickster

Add more users the same way with a different user name (without -c, the file keeps the existing entries).

Step 11: Security checks

trickster-selfcheck re-proves the whole appliance with this instance's credential: HTTPS, refusal of anonymous and wrong-password requests, real data, a cache hit, the reserved paths kept off the network, no forward proxying, the redirect and host checks, the loopback-only ports and the metadata block inside every service:

sudo trickster-selfcheck

Trickster appliance self-check (https://<public-ip>/)
  [ok] HTTPS liveness: /trickster/ping -> pong
  [ok] unauthenticated query -> 401
  [ok] wrong password -> 401
  [ok] per-instance credential -> 200, 2 live series from the local Prometheus
  [ok] range query -> 27 data points
  [ok] repeat range query served from cache: X-Trickster-Result: engine=DeltaProxyCache; status=hit; ffstatus=off
  [ok] Trickster-reserved paths (config, health, ready, certificates) -> 404 off the box
  [ok] not a forward proxy: CONNECT refused, upgrades 403, unknown Host closed (444), URL-in-path 4xx
  [ok] port 80 -> 301 https://<public-ip>/; unknown Host -> 444
  [ok] 8480, 8481, 8484, 9090 answer on 127.0.0.1 only
  [ok] instance metadata refused inside the trickster, prometheus and nginx cgroups (loopback and <private-ip>:443 reachable)
SELFCHECK OK - authenticated over HTTPS, locked down, metadata blocked and accelerating.

The metadata block is a systemd IPAddressDeny rule on each service; you can see it in effect:

for u in trickster prometheus nginx trickster-firstboot; do printf '%s: ' $u; systemctl show -p IPAddressDeny --value $u.service; done

trickster: 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32
prometheus: 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32
nginx: 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32
trickster-firstboot: 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32

Step 12: Versions, build provenance and the command line

trickster --version
prometheus --version | head -1
sed -n '4,9p' /usr/share/doc/trickster/BUILD-PROVENANCE.txt

Trickster version: v2.1.0 (linux/amd64), buildInfo: 2026-09-21T17:05:54Z 61809fbb3f4affcb1e916efc755dc857a21baf21, goVersion: go1.27.2, copyright: © 2018 The Trickster Authors
prometheus, version 3.13.4 (branch: HEAD, revision: 007a9e9ba628f815ce83e1ba9f29b97981996c3c)
source        https://github.com/trickstercache/trickster/archive/refs/tags/v2.1.0.tar.gz
source sha256 fe8814abaefd9ba7f9d882329ed184303ed52a20a50f15d8165d09c7d3c91137
tag commit    61809fbb3f4affcb1e916efc755dc857a21baf21
toolchain     go1.27.2 linux/amd64, official tarball https://go.dev/dl/go1.27.2.linux-amd64.tar.gz
toolchain sha256 ecbadb99091a3f46e31f5f934b068b1864eafa7995211b39eaddf76996045fe5
dependency    golang.org/x/net v0.59.0 -> v0.60.0 (go get; module proxy + sum.golang.org)

Step 13: Backup and maintenance

• Trickster's cache is in memory and needs no backup; back up /etc/trickster, /etc/nginx/ssl and, if you keep using it, /var/lib/prometheus.
• Ubuntu security updates install automatically (unattended-upgrades). Trickster and Prometheus are standalone binaries in /usr/local/bin; to upgrade, launch the newer version from the listing and move your configuration across.
• The memory cache is 512 MiB by default (max_size_bytes under caches:); raise it on larger instance types.

Step 14: Security recommendations

• Limit TCP 443 in the security group to the networks your dashboards run in, and TCP 22 to your admin network.
• Serve Trickster under your own DNS name with a certificate from a trusted CA (Step 9) so clients can verify it.
• Rotate the password if it has been shared (Step 10), and give each dashboard tool its own user.
• Keep Trickster's listeners on 127.0.0.1; publish nothing but nginx.

Support and licensing

cloudimg provides 24/7 technical support for this image by email and live chat: support@cloudimg.co.uk. Trickster and Prometheus are licensed under the Apache License 2.0; their LICENSE and NOTICE files are on the instance in /usr/share/doc/trickster and /usr/share/doc/prometheus. Trickster and Prometheus are trademarks of their respective owners; cloudimg is not affiliated with or endorsed by them.