Streaming & Messaging AWS

Apache Pulsar 5.0.1 on AWS User Guide

Last updated: 2026-10-29 | Product: Apache Pulsar 5.0.1 on Ubuntu 24.04 on AWS

Overview

This guide covers launching and using the Apache Pulsar 5.0.1 on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg streaming and messaging listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

Apache Pulsar is a distributed messaging and event streaming platform with multi-tenancy (tenants, namespaces, topics), persistent storage in Apache BookKeeper, and both queueing and streaming subscriptions. The image runs Pulsar in standalone mode: the broker, one local bookie and the metadata store in one process on one instance, ready for development, testing, single-node workloads and proof-of-concept work.

What the cloudimg image adds:

Authentication and authorization on for every client. Pulsar's JWT token authentication (RS256) and its built-in authorization are switched on, and there is no anonymous role: a client without a valid token is refused, and a client with a valid token can only do what its role has been granted. Pulsar's default authorization provider is used (never the multi-roles provider).
No default credentials, no shared secret. The token signing key pair, your superuser token and the broker's own internal token are generated on your instance at first boot. The image ships none of them, and Pulsar's data (cluster metadata, bookie identity, topics) is created on your instance when it first starts.
TLS only off the instance. Clients on other hosts connect over pulsar+ssl:// on port 6651 and use the admin REST API over https:// on port 8443, both with TLS 1.3/1.2 and a certificate generated on your instance. It is an end-entity certificate: it can only vouch for this instance's own names and can never sign other certificates. The plaintext listeners (pulsar:// on 6650, http:// on 8080) and the bookie (3181) listen on 127.0.0.1 only.
Fail closed. Pulsar starts only after the boot service has written your keys and tokens, and a configuration check refuses to start it with authentication or authorization off, an anonymous role, a plaintext listener on a network address, a placeholder host name or a missing key, token or certificate.
Least-privilege tokens without shell history. pulsar-token mints a token for an application role straight into a root-only file; the token is never printed and never on a command line. You grant each role only the namespaces and actions it needs.
Smaller attack surface. No web UI (the end-of-life Pulsar Manager is not included), no functions worker, no connectors, no WebSocket service, no proxy, no package management and no transaction coordinator. The broker is blocked from the instance metadata service, the AMI requires IMDSv2, and Pulsar runs under systemd sandboxing as an unprivileged user.
Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: Apache Pulsar 5.0.1 (the official binary release, verified against its published SHA-512 and its Apache release signature) on OpenJDK 21 (from Ubuntu 24.04), cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

Prerequisites

An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (or a private route you use for management).
A security group that allows TCP 22 (SSH) from your admin network and TCP 6651 (Pulsar over TLS) and TCP 8443 (admin REST over HTTPS) from the networks your clients run in. Never open 6650, 8080 or 3181: they listen on the loopback address only.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick Apache Pulsar 5.0.1 on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large, your key pair and a security group as above.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=pulsar}]'

The AMI requires IMDSv2 (session tokens) for the instance metadata service. First boot takes about a minute after the instance is running: the boot service reads the instance's own DNS names and addresses from the metadata service, generates the token key pair, the tokens and the certificate, and then Pulsar starts and creates its cluster.

Step 2: Connect to your instance

Version	SSH user
Apache Pulsar 5.0.1 on Ubuntu 24.04	`ubuntu`

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

systemctl is-active pulsar-metadata-names pulsar-firstboot pulsar
sudo test -f /var/lib/cloudimg/pulsar-firstboot.done && echo FIRSTBOOT_DONE
sudo pulsar-health

All three units are active, the first boot marker is present and the broker answers an authenticated health check:

active
active
active
FIRSTBOOT_DONE
PRIMARY_HEALTH_OK broker healthy (authenticated health check on the loopback admin listener)

Only SSH and Pulsar's TLS listeners listen on network addresses; the plaintext listeners and the bookie are on the loopback address:

sudo ss -Htlnp | awk '{print $4, $6}' | grep -v '127.0.0.5' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort

0.0.0.0:22 users:(("sshd"),("systemd"))
0.0.0.0:6651 users:(("java"))
0.0.0.0:8443 users:(("java"))
127.0.0.1:3181 users:(("java"))
127.0.0.1:6650 users:(("java"))
127.0.0.1:8080 users:(("java"))
[::]:22 users:(("sshd"),("systemd"))

Step 4: Your URLs, superuser token and certificate fingerprint

sudo cat /root/pulsar-credentials.txt

The file is readable by root only:

# Apache Pulsar 5.0.1 on Ubuntu 24.04 - generated on first boot by pulsar-firstboot.service.
# Every key and token below is unique to this instance; none was baked into the image. Keep this file secret.
# ...
PULSAR_SERVICE_URL_TLS=pulsar+ssl://<public-dns>:6651
PULSAR_WEB_URL_TLS=https://<public-dns>:8443/
PULSAR_SERVICE_URL_LOCAL=pulsar://127.0.0.1:6650
PULSAR_WEB_URL_LOCAL=http://127.0.0.1:8080/
PULSAR_TLS_CERT_SHA256=<sha256-fingerprint>
PULSAR_TLS_CERT_FILE=/etc/cloudimg/pulsar-broker.pem
#
# Superuser (administration only; do not hand it to applications):
PULSAR_ADMIN_ROLE=cloudimg-admin
PULSAR_ADMIN_TOKEN_FILE=/root/pulsar-admin.token
PULSAR_ADMIN_TOKEN=<PULSAR_ADMIN_TOKEN>
# Change the advertised name: sudo pulsar-set-hostname <dns-name>    Re-verify the appliance: sudo pulsar-selfcheck

Pulsar advertises itself to clients under the instance's public DNS name (its public IP address when the VPC gives no DNS name, the private address when there is no public one), read from the EC2 instance metadata at boot, never from a client request. The certificate also names the public IP address, the private DNS name and address, 127.0.0.1 and localhost. If you stop and start the instance without an Elastic IP, the boot service picks up the new names and issues a new certificate for them. To use your own DNS name, see Step 9.

The superuser token is also in /root/pulsar-admin.token (root only). Use it for administration only; give every application its own least-privilege token (Step 6).

Print the certificate's SHA-256 fingerprint:

sudo openssl x509 -in /etc/cloudimg/pulsar-broker.pem -noout -fingerprint -sha256

sha256 Fingerprint=<sha256-fingerprint>

The certificate is self signed and unique to this instance. Pin this broker's certificate for your Pulsar client only: copy /etc/cloudimg/pulsar-broker.pem to the client host, check that openssl x509 -in pulsar-broker.pem -noout -fingerprint -sha256 prints the PULSAR_TLS_CERT_SHA256 value from /root/pulsar-credentials.txt, and point that client's own tlsTrustCertsFilePath at the copy. Never add it to an operating-system, browser or JVM-wide trust store. Or install a CA-issued certificate: sudo pulsar-install-cert <chain.pem> <key.pem> (Step 9).

Step 5: Administer Pulsar on the instance

pulsar-local runs Pulsar's own pulsar-admin, pulsar-client and pulsar-perf tools against the loopback listener as the superuser. It reads the token from a file through /root/pulsar-client.conf (authParams=file:///root/pulsar-admin.token), so the token never appears on a command line or in your shell history:

sudo pulsar-local admin brokers version
sudo pulsar-local admin clusters list
sudo pulsar-local admin tenants list

5.0.1
standalone
public
pulsar

Create a tenant, a namespace and a partitioned topic for an application:

sudo pulsar-local admin tenants create acme --allowed-clusters standalone
sudo pulsar-local admin namespaces create acme/orders
sudo pulsar-local admin topics create-partitioned-topic persistent://acme/orders/events --partitions 3
sudo pulsar-local admin topics list-partitioned-topics acme/orders

persistent://acme/orders/events

Set a retention policy so messages are kept for a day (or up to 1 GB) after they are acknowledged:

sudo pulsar-local admin namespaces set-retention acme/orders --time 1d --size 1G
sudo pulsar-local admin namespaces get-retention acme/orders

{
  "retentionTimeInMinutes" : 1440,
  "retentionSizeInMB" : 1024
}

Step 6: Least-privilege tokens for your applications

Each application gets its own role and token. pulsar-token signs the token with this instance's key (/etc/pulsar/token-signing, root only) and writes it to /root/pulsar-tokens/<role>.token (0600); it prints only the file path. An optional second argument sets an expiry such as 90d, 12h or 1y:

sudo pulsar-token create orders-app 90d

pulsar-token: token for role 'orders-app' (expires in 90d) written to /root/pulsar-tokens/orders-app.token (0600; not printed)
pulsar-token: grant it what it needs, e.g. sudo pulsar-local admin namespaces grant-permission public/default --role orders-app --actions produce,consume

A new role can do nothing until you grant it permissions. Grant this one produce and consume on the acme/orders namespace only:

sudo pulsar-local admin namespaces grant-permission acme/orders --role orders-app --actions produce,consume
sudo pulsar-local admin namespaces permissions acme/orders

orders-app    [consume, produce]

Try it on the instance with a client configuration that uses the role's token file (the loopback listener, so no certificate is needed here):

sudo tee /root/orders-app-local.conf >/dev/null <<'EOF'
webServiceUrl=http://127.0.0.1:8080/
brokerServiceUrl=pulsar://127.0.0.1:6650/
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
authParams=file:///root/pulsar-tokens/orders-app.token
EOF
sudo chmod 0600 /root/orders-app-local.conf
sudo env PULSAR_CLIENT_CONF=/root/orders-app-local.conf /opt/pulsar/bin/pulsar-client produce persistent://acme/orders/events -m "order 1001 created" -n 3 2>&1 | grep -o '[0-9]* messages successfully produced'
sudo env PULSAR_CLIENT_CONF=/root/orders-app-local.conf /opt/pulsar/bin/pulsar-client consume persistent://acme/orders/events -s billing -p Earliest -n 3 2>&1 | grep -o 'content:order 1001 created' | sort | uniq -c

3 messages successfully produced
      3 content:order 1001 created

The same token is refused outside its grant, for example on the public/default namespace:

OUT=$(sudo env PULSAR_CLIENT_CONF=/root/orders-app-local.conf /opt/pulsar/bin/pulsar-client produce persistent://public/default/other -m x 2>&1 || true)
grep -o 'AuthorizationException[^,]*' <<<"$OUT" | head -1

AuthorizationException: {"errorMsg":"Client is not authorized to Get Partition Metadata"

To cut a role off, revoke its permissions (a token stays valid until it expires, so revoking is what takes effect at once), then delete its file:

sudo pulsar-local admin namespaces revoke-permission acme/orders --role orders-app
sudo pulsar-token delete orders-app

sudo pulsar-token list shows the role tokens you have minted.

Step 7: Connect a client from another host (pulsar+ssl, port 6651)

On your client host you need three things from the instance: the broker certificate, the application's token file and the TLS URLs. Copy the certificate and the token over SSH (the token file is root only on the instance, so copy it through a root-readable pipe):

# on your client host
scp -i <your-key.pem> ubuntu@<public-ip>:/etc/cloudimg/pulsar-broker.pem .
ssh -i <your-key.pem> ubuntu@<public-ip> 'sudo cat /root/pulsar-tokens/orders-app.token' > orders-app.token
chmod 0600 orders-app.token

Check the certificate is the one your instance generated before you trust it: the fingerprint must equal PULSAR_TLS_CERT_SHA256 in /root/pulsar-credentials.txt (Step 4):

# on your client host
openssl x509 -in pulsar-broker.pem -noout -fingerprint -sha256

Then point the client's own trust file at that copy. For Pulsar's command line tools (pulsar-client, pulsar-admin) and the Java client, a client.conf like this one (the instance has a template in /root/pulsar-client-tls.conf):

webServiceUrl=https://<public-dns>:8443/
brokerServiceUrl=pulsar+ssl://<public-dns>:6651
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
authParams=file:///home/you/orders-app.token
tlsTrustCertsFilePath=/home/you/pulsar-broker.pem
tlsAllowInsecureConnection=false
tlsEnableHostnameVerification=true

# on your client host, from an Apache Pulsar 5.0.1 binary distribution
PULSAR_CLIENT_CONF=./client.conf bin/pulsar-client produce persistent://acme/orders/events -m "hello over TLS"
PULSAR_CLIENT_CONF=./client.conf bin/pulsar-client consume persistent://acme/orders/events -s billing -n 1

The Python client (pip install pulsar-client) takes the same three settings:

import pulsar

token = open("orders-app.token").read().strip()
client = pulsar.Client(
    "pulsar+ssl://<public-dns>:6651",
    authentication=pulsar.AuthenticationToken(token),
    tls_trust_certs_file_path="pulsar-broker.pem",
    tls_validate_hostname=True,
)
producer = client.create_producer("persistent://acme/orders/events")
producer.send(b"order 1002 created")
client.close()

A client without a token, with a token signed by another key, or with an expired token is refused with AuthenticationException: Failed to authenticate; a client that trusts only the system certificate authorities, or connects under a name the certificate does not carry, refuses the connection during the TLS handshake. A plaintext pulsar:// client gets no answer on 6651, and 6650 is not reachable from other hosts at all.

Step 8: The admin REST API over HTTPS (port 8443)

pulsar-admin on another host uses the webServiceUrl from the same client.conf (with the superuser token file for administration). You can also call the REST API directly. Keep the token out of the command line by putting the header in a root-only curl config file:

sudo bash -c 'umask 077; printf "header = \"Authorization: Bearer %s\"\n" "$(cat /root/pulsar-admin.token)" > /root/pulsar-admin.curl'
H=$(sudo sed -n 's#^PULSAR_WEB_URL_TLS=https://\([^:]*\):8443/$#\1#p' /root/pulsar-credentials.txt)
sudo curl -s --cacert /etc/cloudimg/pulsar-broker.pem --resolve "$H:8443:127.0.0.1" -K /root/pulsar-admin.curl "https://$H:8443/admin/v2/tenants"; echo
curl -s -o /dev/null -w '%{http_code}\n' --cacert /etc/cloudimg/pulsar-broker.pem --resolve "$H:8443:127.0.0.1" "https://$H:8443/admin/v2/tenants"

["acme","public","pulsar"]
401

The second request has no token and is refused with HTTP 401. Prometheus metrics (/metrics/) also need a token. Delete the curl config when you no longer need it:

sudo shred -u /root/pulsar-admin.curl

Step 9: Your own DNS name and certificate

To advertise the broker under your own DNS name (for example a Route 53 record pointing at an Elastic IP), run:

sudo pulsar-set-hostname pulsar.example.com

It writes the name to /etc/cloudimg/pulsar-hostname (so it survives reboots), issues a new self-signed certificate for it, updates the URLs and the fingerprint in /root/pulsar-credentials.txt and restarts Pulsar. Clients then use pulsar+ssl://pulsar.example.com:6651; re-check and re-pin the new certificate.

To use a certificate from a certificate authority (for example Let's Encrypt), put the certificate chain (server certificate first) and its private key on the instance and run:

sudo pulsar-install-cert fullchain.pem privkey.pem pulsar.example.com

The boot service never replaces a certificate you installed. Clients then trust your CA as usual (tlsTrustCertsFilePath pointing at your CA bundle) instead of pinning. sudo pulsar-set-hostname <name> goes back to a self-signed certificate.

Step 10: Security checks

pulsar-selfcheck re-verifies the whole appliance in about a minute: the listening set, the certificate on both TLS ports, the refusal matrix on the wire (no token, a forged token, an expired token and a role without permissions are each refused, the superuser is accepted), the REST API, a real produce and consume over TLS with the pinned certificate, and the instance metadata block:

sudo pulsar-selfcheck

  ok  listening: off-host only 22, 6651 (pulsar+ssl), 8443 (https); 6650, 8080, 3181 on 127.0.0.1 only
  ok  certificate: end-entity (CA:FALSE), names <public-dns>, served on 6651 and 8443, fingerprint <sha256-fingerprint>
  ok  pulsar+ssl :6651 superuser token connects + looks up; anonymous, forged and expired tokens refused by authentication; a role without permissions refused by authorization
  ok  no Pulsar answer to plaintext on 6651; a system-CA-only client and a client expecting another name both reject the self-signed certificate
  ok  pulsar :6650 (loopback) also requires a token
  ok  broker log reasons: anonymous = no anonymous role; forged = JWT signature mismatch; expired = JWT expired
  ok  admin REST: https :8443 superuser 200; anonymous / forged / expired 401; no-permission role 401; /metrics needs a token; loopback :8080 needs a token
  ok  pulsar-client: 3 messages produced and consumed over pulsar+ssl with the pinned certificate (topic deleted again)
  ok  instance metadata refused inside pulsar.service (loopback control answered)
SELFCHECK OK

The broker is blocked from the instance metadata service (169.254.169.254) and the Azure/WireServer address by its systemd unit, so a request the broker makes on a client's behalf can never reach instance credentials. Only pulsar-metadata-names.service, a one-shot unit that runs at boot and can reach nothing but the metadata service, reads the instance's names:

systemctl show pulsar -p IPAddressDeny --value
systemctl show pulsar-metadata-names -p IPAddressAllow --value

169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128
169.254.169.254/32 fd00:ec2::254/128

Step 11: Configuration and tuning

Configuration: /etc/pulsar/standalone.conf (root:pulsar 0640). After a change, restart with sudo systemctl restart pulsar; the configuration check refuses to start Pulsar if a change switches off authentication or authorization, adds an anonymous role, opens a plaintext listener on a network address or turns on a feature this image keeps off.
Memory: the broker runs with a 1 GB initial and 2 GB maximum heap and 2 GB of direct memory, sized for m5.large (8 GiB). On a larger instance raise them with a drop-in: sudo systemctl edit pulsar, add [Service] and Environment="PULSAR_MEM=-Xms4g -Xmx4g -XX:MaxDirectMemorySize=4g", then restart.
Data: /var/lib/pulsar (bookie ledgers and journal, metadata store). Grow the instance's root EBS volume when you need more space. Logs: /var/log/pulsar/pulsar-standalone.log.
Not included: a web UI, the Pulsar functions worker, built-in connectors, tiered-storage offloaders, the WebSocket service and the Pulsar proxy. Standalone mode is a single node; for a multi-node production cluster, use separate brokers, bookies and a metadata store.

Step 12: Versions, provenance and file locations

cat /usr/share/doc/apache-pulsar/BUILD-PROVENANCE.txt
java -version 2>&1 | head -1

Apache Pulsar 5.0.1 (Apache-2.0) as shipped by cloudimg
source:     https://archive.apache.org/dist/pulsar/pulsar-5.0.1/apache-pulsar-5.0.1-bin.tar.gz
sha512:     97baef9c5d04d906b1a5fb7b9a8905fd8f9db5e639c400bcedf0e5092dcde56d5b036bef4d9213fbe52fcc0030708689db1d7c8bffe13e4781d5659ca2637f27
signature:  apache-pulsar-5.0.1-bin.tar.gz.asc verified against the Apache Pulsar KEYS file, signer C663D2F64DA2CA09DB28D9ABD3FA67D522C55256
license:    LICENSE (Apache License 2.0) sha256 2224d108489b90a5298407bad70fe14948cef147f46c343d5778e584cc07291c
runtime:    openjdk version "21.0.12.1" 2026-08-18 (Ubuntu openjdk-21-jre-headless)
removed:    examples/, instances/ (functions runtime), bin/*.cmd
not shipped: Apache Pulsar Manager (end of life, CVE-2026-105944), functions worker, connectors, stream storage,
            Pulsar proxy, WebSocket service, package management, transaction coordinator
openjdk version "21.0.12.1" 2026-08-18

Path	What
`/opt/pulsar`	Apache Pulsar 5.0.1 (root owned, read only to the service)
`/etc/pulsar/standalone.conf`	Broker configuration
`/etc/pulsar/auth/token-public.key`	Token verification key (the broker reads only this)
`/etc/pulsar/token-signing/token-private.key`	Token signing key (root only)
`/etc/pulsar/tls/`	TLS certificate and key
`/etc/cloudimg/pulsar-broker.pem`	The broker certificate to copy to clients
`/root/pulsar-credentials.txt`, `/root/pulsar-admin.token`	URLs, fingerprint and the superuser token
`/root/pulsar-tokens/`	Role tokens minted with `pulsar-token`
`/var/lib/pulsar`, `/var/log/pulsar`	Data and logs

Step 13: Backup and maintenance

Back up the token signing key pair and the superuser token (/etc/pulsar/token-signing/, /etc/pulsar/auth/, /root/pulsar-admin.token) together with /var/lib/pulsar: every token you issued was signed with that key. Take EBS snapshots of the root volume with Pulsar stopped for a consistent copy.
Ubuntu security updates install automatically. Check for a newer cloudimg version of the listing for Pulsar upgrades.

Step 14: Security recommendations

Open 6651 and 8443 only to the networks your clients run in, and 22 only to your admin network.
Give every application its own role, the narrowest grant it needs and an expiry; keep the superuser token for administration.
Rotate a leaked role token by revoking the role's permissions and minting a new role; keep backups of the signing key pair private.
Prefer a certificate from a certificate authority for production clients (Step 9).

Support and licensing

Apache Pulsar is licensed under the Apache License 2.0 (/usr/share/doc/apache-pulsar/LICENSE). cloudimg provides the image, the secure-by-default configuration and support for launching and using it. Apache Pulsar, Pulsar and Apache are trademarks of the Apache Software Foundation; this image is not affiliated with or endorsed by the Apache Software Foundation.