Gigapipe 5.5.3 on AWS User Guide

Overview

This guide covers launching and using the Gigapipe 5.5.3 on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg log management listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

Gigapipe, formerly known as qryn, is an open source polyglot observability backend. One lightweight service ingests logs, metrics and traces through the APIs your tooling already speaks, including Loki push, Prometheus remote write, Tempo and Zipkin, OpenTelemetry (OTLP) over HTTP, InfluxDB line protocol, Datadog and Elastic bulk formats, and stores everything in ClickHouse. You query it with LogQL, PromQL and TraceQL from the built in explorer or from any Grafana compatible client.

A stock Gigapipe install listens on every interface and accepts pushes and queries from anyone who can reach it. The cloudimg image turns it into a locked down appliance:

• No anonymous access. Every ingest and query API and the explorer sit behind HTTP basic authentication, enforced twice: by nginx at the front door and by Gigapipe itself on loopback. Anonymous and wrong password pushes and queries are refused with 401, and the bundled self test proves it on every Loki, Prometheus, Tempo, OTLP, Influx, Datadog and Elastic path each time it runs.
• Every secret is per instance. The basic authentication password, both ClickHouse passwords and the TLS certificate are generated on your instance's first boot and written to a root only file. The ClickHouse default user is never without a password, and Gigapipe is not allowed to start until first boot has produced its secrets.
• HTTPS from the first boot. nginx serves the explorer and every API on port 443 with a certificate made for your instance's own hostname; port 80 only redirects to HTTPS. Requests for any other hostname are dropped.
• Exactly three public ports. SSH on 22 and nginx on 80 and 443. Gigapipe (3100) and ClickHouse (8123, 9000, 9009) are bound to loopback, and the bundled OpenTelemetry Collector opens no port at all.
• Built from source with the patched toolchain. The Gigapipe binary is compiled by cloudimg from the unmodified 5.5.3 source with Go 1.27.3 and an updated golang.org/x/net, which clears the Go security advisories the upstream release binary carries. govulncheck reports no reachable vulnerability in the shipped binary.
• Live data from the first minute. The OpenTelemetry Collector ships the instance's own systemd journal and host metrics into Gigapipe, so the explorer already has something to show when you sign in. The image itself ships with an empty database.
• Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: Gigapipe 5.5.3 (AGPL-3.0) with the qryn-view 3.3.2 explorer, ClickHouse 26.8 LTS (Apache-2.0) from the official ClickHouse repository, OpenTelemetry Collector Contrib 0.163.1 (Apache-2.0), nginx, a self test command, Ubuntu 24.04 LTS, cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large (ClickHouse needs at least 8 GiB of memory)

Gigapipe and qryn are names of their respective owners. This image is produced by cloudimg and is not affiliated with, endorsed by, or sponsored by Gigapipe, HEPVEST BV, Grafana Labs, ClickHouse Inc. or The Linux Foundation. Loki, Tempo and Grafana are trademarks of Grafana Labs; Prometheus and OpenTelemetry of The Linux Foundation; ClickHouse of ClickHouse Inc.; each is named only to identify a compatible API.

Prerequisites

• An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a public IP (or a route to your network).
• A security group that allows TCP 22 (SSH) and TCP 443 (HTTPS) from your own networks. TCP 80 is optional: it only redirects to HTTPS. Never open Gigapipe's or ClickHouse's ports; they are not reachable from off the instance anyway.
• An instance type with at least 8 GiB of memory (m5.large or larger) and a root volume sized for the telemetry you plan to keep.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick Gigapipe 5.5.3 on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large or larger, your key pair and a security group as above, and size the root volume for your retention. Under Advanced details, set Metadata version to V2 only (token required).

Or from the AWS CLI, with the AMI ID shown for your Region on the listing's launch page:

aws ec2 run-instances --image-id <ami-id-for-your-region> --instance-type m5.large \
  --key-name <your-key-pair> --security-group-ids <your-security-group> \
  --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=50,VolumeType=gp3}' \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled

First boot takes about a minute: it generates the secrets and the certificate, then nginx starts.

Step 2: Connect to your instance

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

Gigapipe runs as a systemd service on loopback behind nginx, with ClickHouse on the instance and the OpenTelemetry Collector feeding it. All four are active:

systemctl is-active clickhouse-server nginx gigapipe otelcol-contrib

active
active
active
active

The first boot unit reports active (exited) after its one successful run and then disables itself, so it never runs again:

systemctl --no-pager --plain --no-legend list-units 'qryn*' 'gigapipe*'

gigapipe.service       loaded active running Gigapipe (qryn) polyglot observability backend (loopback :3100) (cloudimg)
qryn-firstboot.service loaded active exited  Gigapipe first-boot setup (per-VM secrets, certificate, schema) (cloudimg)

Rather than checking the ports you expect, list every socket bound to something other than loopback, so anything unexpected has nowhere to hide:

sudo ss -ltn | awk 'NR>1{print $4}' | grep -vE '^(127\.|\[::1\])' | sort -u

0.0.0.0:22
0.0.0.0:443
0.0.0.0:80
[::]:22
[::]:443
[::]:80

The only entries are SSH on 22 and nginx on 80 and 443. Gigapipe on 127.0.0.1:3100 and ClickHouse on 127.0.0.1:8123, 9000 and 9009 stay on loopback, and the collector listens nowhere. Re run this after you install anything else on the instance.

Step 4: Get your credentials

Every instance generates its own secrets on first boot and writes them to the root only file /root/qryn-credentials.txt. List the entries it holds (names only) and read the two you need to sign in:

sudo grep -oE '^[A-Z_]+=' /root/qryn-credentials.txt
sudo grep -E '^(QRYN_USER|QRYN_URL)=' /root/qryn-credentials.txt

QRYN_USER=
QRYN_PASSWORD=
QRYN_URL=
CLICKHOUSE_ADMIN_USER=
CLICKHOUSE_ADMIN_PASSWORD=
CLICKHOUSE_APP_USER=
CLICKHOUSE_APP_PASSWORD=
CLICKHOUSE_DB=
QRYN_USER=admin
QRYN_URL=https://ec2-203-0-113-25.compute-1.amazonaws.com/

Print the password itself only when you need it, for example to paste into the browser's sign in prompt:

sudo grep '^QRYN_PASSWORD=' /root/qryn-credentials.txt | cut -d= -f2-

The file holds:

• QRYN_USER and QRYN_PASSWORD: the HTTP basic authentication credential for the explorer and every API (the user is admin)
• QRYN_URL: the HTTPS address of your instance
• CLICKHOUSE_ADMIN_USER and CLICKHOUSE_ADMIN_PASSWORD: the ClickHouse default user, for administering the database from the instance
• CLICKHOUSE_APP_USER and CLICKHOUSE_APP_PASSWORD: the gigapipe ClickHouse user the service connects with, limited to the gigapipe database
• CLICKHOUSE_DB: the database that holds every log, metric and trace

The address in QRYN_URL is your instance's public DNS name, read from the EC2 instance metadata service on first boot. nginx answers only to that name, the instance's own IP addresses and localhost; a request for any other hostname is dropped.

Step 5: Prove nothing is open without the credential

The front door answers over HTTPS, and only with the credential. This reads the per instance password from the credentials file, probes the readiness endpoint with and without it, and checks that plain HTTP redirects to your instance's HTTPS address:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
QH=$(sudo awk -F'[/=]' '/^QRYN_URL=/{print $4}' /root/qryn-credentials.txt)
curl -sk -o /dev/null -w 'https /ready with credential -> %{http_code}\n' -u "admin:${QP}" https://127.0.0.1/ready
curl -sk -o /dev/null -w 'https /ready without credential -> %{http_code}\n' https://127.0.0.1/ready
curl -s -o /dev/null -w 'http / -> %{http_code} %{redirect_url}\n' -H "Host: ${QH}" http://127.0.0.1/

https /ready with credential -> 200
https /ready without credential -> 401
http / -> 301 https://ec2-203-0-113-25.compute-1.amazonaws.com/

Pushing or querying without the credential, or with a wrong password, is refused, and Gigapipe refuses an anonymous push even on loopback, behind nginx:

curl -sk -o /dev/null -w 'anonymous Loki push -> %{http_code}\n' -H 'Content-Type: application/json' -d '{"streams":[]}' https://127.0.0.1/loki/api/v1/push
curl -sk -o /dev/null -w 'anonymous PromQL query -> %{http_code}\n' -G --data-urlencode 'query=up' https://127.0.0.1/api/v1/query
curl -sk -o /dev/null -w 'wrong password OTLP push -> %{http_code}\n' -u 'admin:not-the-password' -H 'Content-Type: application/json' -d '{"resourceLogs":[]}' https://127.0.0.1/v1/logs
curl -s -o /dev/null -w 'anonymous push straight to gigapipe on loopback -> %{http_code}\n' -H 'Content-Type: application/json' -d '{"streams":[]}' http://127.0.0.1:3100/loki/api/v1/push
curl -sk -o /dev/null -w 'unknown Host -> %{http_code}\n' -H 'Host: evil.example' https://127.0.0.1/healthz || true

anonymous Loki push -> 401
anonymous PromQL query -> 401
wrong password OTLP push -> 401
anonymous push straight to gigapipe on loopback -> 401
unknown Host -> 000

000 means nginx closed the connection without answering (curl reports an empty reply, hence the || true), which is what it does for a hostname it does not serve.

The bundled self test proves the whole appliance end to end. It checks the TLS front door and the redirect, sends anonymous and wrong password requests to 18 ingest and query paths (Loki, Prometheus, Tempo, Zipkin, OTLP, Influx, Datadog and Elastic) and confirms nginx refuses each one, confirms Gigapipe refuses them on loopback too and that an anonymous push stored nothing, pushes a log line through the Loki API and finds it again with LogQL, pushes a metric through OTLP and finds it again with PromQL, checks the collector's journal and host metrics are queryable, that ClickHouse refuses a login without a password or with a wrong one and accepts the per instance one, and that the exposed port set is exactly 22, 80 and 443:

sudo /usr/local/bin/qryn-selftest

OK qryn self-test passed: healthz 200, :80 301, anonymous + wrong-password push/query refused 401 by the nginx gate on 18 ingest/query paths and by gigapipe on loopback, anonymous push stored nothing, /ready 200 + explorer UI served, Loki push -> LogQL round-trip OK (cloudimg-qryn-selftest-...), OTLP metric -> PromQL round-trip OK (...), collector journald + host metrics queryable, ClickHouse default refuses no-password + per-VM creds OK + schema present (29 tables), exposed ports exactly {22,80,443}, gigapipe/ClickHouse loopback-only, collector listens nowhere, no swap

ClickHouse refuses a login without the password, even on the instance itself, and accepts the per instance one. Pass the password in the CLICKHOUSE_PASSWORD environment variable rather than on the command line, where other users could see it in the process list:

clickhouse-client --user default -q 'SELECT 1' 2>&1 | grep -m1 -o 'Authentication failed[^,]*'
CP=$(sudo awk -F= '/^CLICKHOUSE_ADMIN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
CLICKHOUSE_PASSWORD="${CP}" clickhouse-client --user default -q "SELECT concat('per instance password -> ClickHouse ', version())"

Authentication failed: password is incorrect
per instance password -> ClickHouse 26.8.31.2

Step 6: Open the explorer

Open your instance's address from QRYN_URL in a browser:

https://<public-dns-name>/

The certificate is self signed for your instance, so your browser warns once; accept it to continue, or install a CA issued certificate as described below. The browser then asks for the basic authentication credential: sign in as admin with QRYN_PASSWORD from your credentials file. After a successful sign in nginx sets an HttpOnly, Secure session cookie for the explorer, so its own queries are authenticated for the rest of the browser session.

You land on the explorer's search view with the Logs data source selected. The data sources point at the address you signed in to, so nothing needs configuring, and the collector has been shipping the instance's journal and host metrics since first boot.

Enter a LogQL stream selector and press Show Results. {service_name="journald"} returns the instance's own systemd journal: every line carries the unit, syslog_identifier, level, host_name and service_name labels the collector attached, so {service_name="journald", unit="ssh.service"} shows SSH logins and {service_name="journald", level=~"WARN|ERROR"} shows anything that went wrong.

Switch the data source to Metrics and enter a PromQL expression. The collector scrapes host metrics every 15 seconds, so system_memory_usage_bytes{state="used"}, avg(system_cpu_utilization{state!="idle"}) and system_filesystem_usage_bytes{mountpoint="/", state="used"} chart immediately.

The Data Sources settings page shows the Logs, Metrics and Traces sources all pointing at the address you signed in on. This is where you would point the explorer at another Gigapipe instead.

Step 7: Query from the command line

Every API is available on the instance at https://127.0.0.1/ and from your network at your QRYN_URL, always with the basic authentication credential and, until you install a CA issued certificate, curl -k. The Loki label API lists what the collector has produced so far:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" https://127.0.0.1/loki/api/v1/labels | jq -c .data

["host_name","job","level","os_type","service_name","source","syslog_identifier","unit"]

A LogQL range query returns the instance's own SSH log lines. Gigapipe needs both start and end (in nanoseconds) on range queries:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" -G https://127.0.0.1/loki/api/v1/query_range \
  --data-urlencode 'query={service_name="journald", unit="ssh.service"}' \
  --data-urlencode "start=$(( $(date +%s) - 3600 ))000000000" \
  --data-urlencode "end=$(( $(date +%s) + 60 ))000000000" \
  --data-urlencode 'limit=2' | jq -r '.data.result[].values[][1]'

pam_unix(sshd:session): session closed for user ubuntu
pam_unix(sshd:session): session opened for user ubuntu(uid=1000) by ubuntu(uid=0)

A PromQL instant query returns the latest host metric sample. The job label is hostmetrics for everything the collector scrapes:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" -G https://127.0.0.1/api/v1/query \
  --data-urlencode 'query=system_memory_usage_bytes{state="used"}' \
  | jq -c '.data.result[0] | {name: .metric.__name__, job: .metric.job, state: .metric.state, bytes: .value[1]}'

{"name":"system_memory_usage_bytes","job":"hostmetrics","state":"used","bytes":"1557647360"}

Step 8: Send your own data

Anything that speaks Loki, Prometheus remote write, OTLP, InfluxDB line protocol, Datadog or Elastic can send to this instance. This pushes one log line through the Loki API and reads it back with LogQL:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
NOW=$(date +%s%N)
curl -sk -o /dev/null -w 'push -> %{http_code}\n' -u "admin:${QP}" -H 'Content-Type: application/json' \
  -d "{\"streams\":[{\"stream\":{\"job\":\"guide-example\",\"app\":\"hello\"},\"values\":[[\"${NOW}\",\"hello from the deployment guide\"]]}]}" \
  https://127.0.0.1/loki/api/v1/push
sleep 3
curl -sk -u "admin:${QP}" -G https://127.0.0.1/loki/api/v1/query_range \
  --data-urlencode 'query={job="guide-example"}' \
  --data-urlencode "start=$(( $(date +%s) - 600 ))000000000" \
  --data-urlencode "end=$(( $(date +%s) + 60 ))000000000" \
  --data-urlencode 'limit=5' | jq -r '.data.result[].values[][1]'

push -> 204
hello from the deployment guide

Metrics work the same way. This pushes a gauge as OTLP JSON to /v1/metrics and queries it back with PromQL; service.name becomes the job label:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
NOW=$(date +%s%N)
curl -sk -o /dev/null -w 'push -> %{http_code}\n' -u "admin:${QP}" -H 'Content-Type: application/json' \
  -d "{\"resourceMetrics\":[{\"resource\":{\"attributes\":[{\"key\":\"service.name\",\"value\":{\"stringValue\":\"guide-example\"}}]},\"scopeMetrics\":[{\"metrics\":[{\"name\":\"guide_example_gauge\",\"gauge\":{\"dataPoints\":[{\"asDouble\":42,\"timeUnixNano\":\"${NOW}\"}]}}]}]}]}" \
  https://127.0.0.1/v1/metrics
sleep 3
curl -sk -u "admin:${QP}" -G https://127.0.0.1/api/v1/query --data-urlencode 'query=guide_example_gauge' \
  | jq -c '.data.result[0] | {metric: .metric, value: .value[1]}'

push -> 200
{"metric":{"__name__":"guide_example_gauge","job":"guide-example","service_name":"guide-example"},"value":"42"}

The endpoints your agents and applications use, all under your QRYN_URL with the basic authentication credential:

Data	Send to	Compatible with
Logs	/loki/api/v1/push	Promtail, Grafana Alloy, Fluent Bit, Vector and any Loki client
Logs, metrics, traces	/v1/logs, /v1/metrics, /v1/traces	OpenTelemetry Collector and SDKs (OTLP over HTTP, protobuf or JSON)
Metrics	/api/v1/prom/remote/write	Prometheus remote_write, the OpenTelemetry prometheusremotewrite exporter
Metrics	/influx/api/v2/write	Telegraf and InfluxDB line protocol clients
Traces	/tempo/api/push and /api/v2/spans	Tempo and Zipkin clients
Logs and metrics	/api/v2/logs, /api/v2/series	Datadog agent formats
Logs	/_bulk	Elastic bulk clients

Point an OpenTelemetry Collector at the instance with an otlphttp exporter and basic authentication:

extensions:
  basicauth/gigapipe:
    client_auth:
      username: admin
      password: <QRYN_PASSWORD>
exporters:
  otlphttp/gigapipe:
    endpoint: https://<public-dns-name>
    auth:
      authenticator: basicauth/gigapipe
    tls:
      insecure_skip_verify: true   # until you install a CA issued certificate
service:
  extensions: [basicauth/gigapipe]
  pipelines:
    logs:    { receivers: [otlp], exporters: [otlphttp/gigapipe] }
    metrics: { receivers: [otlp], exporters: [otlphttp/gigapipe] }
    traces:  { receivers: [otlp], exporters: [otlphttp/gigapipe] }

For Prometheus, add a remote_write block:

remote_write:
  - url: https://<public-dns-name>/api/v1/prom/remote/write
    basic_auth:
      username: admin
      password: <QRYN_PASSWORD>
    tls_config:
      insecure_skip_verify: true   # until you install a CA issued certificate

Allow your agents to reach TCP 443 in the instance's security group, from your own VPC or VPN ranges only. OTLP over gRPC is not exposed by this image; use OTLP over HTTP, which every OpenTelemetry exporter supports.

Step 9: Use Grafana, if you want to

Grafana is not included in this image. If you run Grafana elsewhere, add this instance as a Loki, a Prometheus and a Tempo data source, each with the URL https://<public-dns-name>, Basic auth enabled with the admin / QRYN_PASSWORD credential, and Skip TLS verify until you install a CA issued certificate. No plugin is needed: Gigapipe implements the native Loki, Prometheus and Tempo query APIs.

Step 10: Configuration, retention and key paths

Gigapipe reads its configuration from /etc/gigapipe/gigapipe.env, written on first boot. Shown here with the secrets redacted:

sudo sed -E 's/^(CLICKHOUSE_AUTH|GIGAPIPE_PASSWORD)=.*/\1=********/' /etc/gigapipe/gigapipe.env | grep -vE '^#|^$'

CLICKHOUSE_SERVER=127.0.0.1
CLICKHOUSE_PORT=9000
CLICKHOUSE_DB=gigapipe
CLICKHOUSE_AUTH=********
HOST=127.0.0.1
PORT=3100
GIGAPIPE_LOGIN=admin
SAMPLES_DAYS=7
LOG_LEVEL=info
GIGAPIPE_PASSWORD=********

• SAMPLES_DAYS: retention in days for logs, metrics and traces (7 as shipped, applied as a ClickHouse TTL). Raise it once you have sized the volume.
• LOG_LEVEL: debug, info, warn or error.
• BULK_MAX_AGE_MS and BULK_MAX_SIZE_BYTES: how long and how much Gigapipe batches before writing to ClickHouse.
• Keep HOST=127.0.0.1: nginx is the only public listener.

After editing, restart the service:

sudo systemctl restart gigapipe

ClickHouse keeps everything under /var/lib/clickhouse. Its settings live in /etc/clickhouse-server/config.d/cloudimg.xml (loopback only, telemetry off, memory capped at 60 percent of RAM) and the two users in /etc/clickhouse-server/users.d/cloudimg-users.xml (SHA-256 password hashes, loopback only). This shows how much space the largest tables use:

CP=$(sudo awk -F= '/^CLICKHOUSE_ADMIN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
CLICKHOUSE_PASSWORD="${CP}" clickhouse-client --user default -q "SELECT count() AS tables FROM system.tables WHERE database='gigapipe'"
CLICKHOUSE_PASSWORD="${CP}" clickhouse-client --user default -q "SELECT table, formatReadableSize(sum(bytes_on_disk)) AS size, sum(rows) AS rows FROM system.parts WHERE database='gigapipe' AND active GROUP BY table ORDER BY sum(bytes_on_disk) DESC LIMIT 3 FORMAT PrettyCompactNoEscapes"

29
   ┌─table───────┬─size──────┬─rows─┐
1. │ metrics_15s │ 25.24 KiB │ 1427 │
2. │ samples_v3  │ 23.14 KiB │ 2192 │
3. │ time_series │ 18.28 KiB │  237 │
   └─────────────┴───────────┴──────┘

To grow storage, increase the root EBS volume and extend the filesystem, or move /var/lib/clickhouse to a dedicated EBS volume.

The collector's pipeline is /etc/otelcol-contrib/config.yaml: the instance's journal and host metrics into Gigapipe on loopback. To receive telemetry from other hosts, send it to Gigapipe's endpoints above rather than through the local collector.

Path	Purpose
/etc/gigapipe/gigapipe.env	Gigapipe configuration: ClickHouse connection, basic authentication credential, retention
/usr/local/bin/gigapipe	The Gigapipe 5.5.3 binary, compiled by cloudimg from the upstream source
/usr/share/doc/gigapipe/	The AGPL-3.0 licence and BUILD-PROVENANCE.txt (source, toolchain, checksums and the source offer)
/opt/qryn-view/dist	The explorer UI bundle served by nginx
/root/qryn-credentials.txt	Per instance credentials and URL, root only
/var/lib/clickhouse	ClickHouse data (every log, metric and trace)
/etc/clickhouse-server/config.d/cloudimg.xml	ClickHouse server settings: loopback only, no telemetry, memory cap
/etc/clickhouse-server/users.d/cloudimg-users.xml	The default and gigapipe ClickHouse users
/etc/apt/preferences.d/clickhouse-pin	Pins ClickHouse to the verified version; edit it to upgrade deliberately
/etc/otelcol-contrib/config.yaml	The collector pipeline
/etc/nginx/sites-available/cloudimg-qryn	The TLS and basic authentication front door
/etc/ssl/gigapipe/gigapipe.crt and .key	The per instance TLS certificate and key
/usr/local/sbin/qryn-set-password	Rotates the basic authentication credential everywhere at once
/usr/local/sbin/gigapipe-set-hostname	Changes the hostname Gigapipe is served as and re issues the certificate
/usr/local/bin/qryn-selftest	End to end appliance self test

Confirm the shipped versions and how the binary was built:

sha256sum /usr/local/bin/gigapipe
dpkg-query -W -f='${Package} ${Version}\n' clickhouse-server otelcol-contrib
grep -E '^(toolchain  |govulncheck)' /usr/share/doc/gigapipe/BUILD-PROVENANCE.txt

6f180dd357db429f7fd9cfb5d4942c973fc0a4e43668ad3bb9006bf1e035d2dd  /usr/local/bin/gigapipe
clickhouse-server 26.8.31.2
otelcol-contrib 0.163.1
toolchain         go1.27.3 linux/amd64, official tarball https://go.dev/dl/go1.27.3.linux-amd64.tar.gz
govulncheck       v1.9.0 -mode=binary on this binary: No vulnerabilities found. Your code is affected by 0 vulnerabilities.

Step 11: Rotate the credential, change the hostname, replace the certificate

The basic authentication credential is enforced by nginx and by Gigapipe, used by the collector, recorded in the credentials file and backed by the explorer session token. One helper updates all of them and restarts the services. It reads the new password from standard input, so it never appears on a command line (at least 12 characters, no colon, quotes, backslashes or spaces):

read -rs PW && printf '%s' "$PW" | sudo qryn-set-password

Existing explorer sessions are signed out, because the session token is re minted with the password.

If you point your own DNS name at the instance, serve Gigapipe under that name. This updates nginx and issues a new self signed certificate for it:

sudo gigapipe-set-hostname gigapipe.example.com

For production, obtain a CA issued certificate for that name, install it over /etc/ssl/gigapipe/gigapipe.crt and /etc/ssl/gigapipe/gigapipe.key (key mode 0600, owner root), and reload nginx:

sudo systemctl reload nginx

Backups

All state is in ClickHouse under /var/lib/clickhouse and in the configuration files listed above. The simplest consistent backup is an EBS snapshot of the root volume (or of a dedicated data volume if you move /var/lib/clickhouse onto one), for example with Amazon Data Lifecycle Manager. For logical backups, ClickHouse's own BACKUP DATABASE gigapipe TO ... statement works once you configure a backup destination in /etc/clickhouse-server/config.d/. Keep a copy of /etc/gigapipe/gigapipe.env, /etc/clickhouse-server/users.d/cloudimg-users.xml and /root/qryn-credentials.txt somewhere safe off the instance.

Security notes

• Nothing answers without the credential. Gigapipe and ClickHouse bind to loopback and the collector opens no port; nginx on 443 is the only network facing surface and it demands basic authentication on every path except the /healthz liveness probe. Gigapipe enforces the same credential again on loopback. The explorer's session cookie is issued only after a successful basic authentication page load and is as secret as the password: rotate the password to invalidate it.
• Every secret is per instance. Both ClickHouse passwords, the basic authentication password and the TLS certificate are generated on first boot; the ClickHouse default user is never without a password. Two instances launched from this AMI share no secret.
• The hostname comes from configuration, never from a request. First boot reads the public DNS name from the EC2 instance metadata service (IMDSv2) and nginx serves only that name and the instance's own addresses. nginx passes Gigapipe a fixed host and sets the client address from the TCP connection, never from X-Forwarded-For. Gigapipe itself cannot reach the instance metadata service.
• The image ships empty. The gigapipe database is created on your instance at first boot. The collector ships this instance's own journal, which includes the source addresses of SSH connection attempts; restrict port 22 to your management network.
• Use TLS end to end. Replace the self signed certificate with a CA issued one before sending production telemetry over untrusted networks, and drop insecure_skip_verify from your exporters once you have.
• Updates. Ubuntu security updates install automatically. Gigapipe, the collector and ClickHouse are pinned, so none is upgraded automatically; take a newer release deliberately after reading its release notes.

Support

cloudimg provides 24/7 support with a guaranteed 24 hour response SLA for this image. Gigapipe and its explorer are open source under the GNU Affero General Public License v3; the Gigapipe binary is compiled from the unmodified upstream source with one dependency updated for security, and the complete corresponding source is described in /usr/share/doc/gigapipe/BUILD-PROVENANCE.txt and is available from cloudimg on request. ClickHouse and the OpenTelemetry Collector are Apache 2.0. For questions about the image, its configuration or its security posture, contact support@cloudimg.co.uk.