Apache Ozone on AWS User Guide

Overview

Apache Ozone is the scalable, distributed object store for Hadoop and cloud-native workloads. It exposes both an S3-compatible API and an HDFS-compatible (ofs) API over a single namespace that scales to billions of objects. The cloudimg image installs Ozone 1.4.1 as a single-instance all-roles cluster: the Ozone Manager (OM), the Storage Container Manager (SCM), one Datanode, the Recon management UI and the S3 Gateway all run on one EC2 instance, so a single instance gives you a fully functional Ozone cluster for development, testing, single-tenant object storage and proof-of-concept workloads with no multi-node cluster to stand up. All Ozone services bind to loopback; the Recon management console is fronted by an nginx reverse proxy on TCP 80 with HTTP Basic auth, and a unique administrator password plus per-instance S3 credentials are generated on the first boot of every instance. Backed by 24/7 cloudimg support.

What is included:

• Apache Ozone 1.4.1 (Apache-2.0) as a single-instance all-roles cluster (OM + SCM + Datanode + Recon + S3 Gateway)
• OpenJDK 17
• A dedicated 50 GiB EBS data volume at /var/lib/ozone holding the OM and SCM metadata, the HDDS datanode container data and the Recon database — separate from the OS disk, mounted by filesystem UUID and re-provisioned with every instance
• All Ozone services bound to 127.0.0.1 (OM 9862/9874, SCM 9860/9863/9876, Datanode 9882, Recon 9888, S3 Gateway 9878)
• nginx reverse proxy on :80 with HTTP Basic auth in front of the Recon management console, plus an unauthenticated /health endpoint
• Per-instance admin password and per-instance S3 access key and secret generated at first boot, in a root-only file
• ozone-scm, ozone-om, ozone-datanode, ozone-recon, ozone-s3g and nginx as systemd units, enabled and active
• 24/7 cloudimg support

Prerequisites

An active AWS account with permission to launch EC2 instances, create security groups and subscribe to AWS Marketplace products, and an EC2 key pair in the target region. m5.large (2 vCPU / 8 GiB RAM) is the recommended instance type — Ozone runs five JVMs and is memory hungry. Security group inbound: allow 22/tcp from your management network and 80/tcp from the users who reach the Recon console (front port 80 with TLS for public exposure — see Enabling HTTPS). The S3 Gateway is reached over an SSH tunnel and needs no inbound port.

Connecting to your instance

Connect over SSH as the default login user for your AMI's operating system. The password and S3 credentials referenced throughout this guide live in a root-only file on the instance.

Step 1 — Deploy from the AWS Marketplace

Sign in to the AWS Marketplace, search for Apache Ozone by cloudimg, choose Continue to Subscribe, accept the terms, then Continue to Configuration and Continue to Launch. Pick the m5.large instance type, select your key pair, and in the security group allow SSH (22) from your management network and HTTP (80) from the users who reach the Recon console. Review the dedicated 50 GiB data volume, then launch.

Step 2 — Deploy from the AWS CLI

aws ec2 run-instances \
  --image-id <ami-id-from-the-listing> \
  --instance-type m5.large \
  --key-name <your-key-pair> \
  --security-group-ids <your-sg> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=ozone}]'

Step 3 — Connect to your instance

ssh ubuntu@<instance-public-ip>

Step 4 — Confirm the services are running

systemctl is-active ozone-scm.service ozone-om.service ozone-datanode.service ozone-recon.service ozone-s3g.service nginx.service

All six services report active.

Step 5 — Confirm the gateway answers

The nginx reverse proxy serves an unauthenticated /health endpoint for liveness checks and fronts the Ozone Recon management console behind HTTP Basic auth:

curl -s -o /dev/null -w 'health endpoint -> HTTP %{http_code}\n' http://127.0.0.1/health
curl -s -o /dev/null -w 'Recon UI (no auth) -> HTTP %{http_code}\n' http://127.0.0.1/

The /health endpoint returns HTTP 200; the Recon console returns HTTP 401 until you supply the per-instance credentials.

Step 6 — Retrieve your admin password and S3 credentials

The administrator password and the per-instance S3 access key and secret are generated uniquely on the first boot of your instance and written to a root-only file:

sudo grep -E '^OZONE_(ADMIN|S3)_' /root/ozone-credentials.txt

OZONE_ADMIN_USER is admin and OZONE_ADMIN_PASSWORD is the password that protects the Recon management console. OZONE_S3_ACCESS_KEY and OZONE_S3_SECRET_KEY are the credentials for the loopback S3 gateway.

Step 7 — Open the Recon management console

The Recon console sits behind nginx HTTP Basic auth. Confirm a wrong password is rejected while the real one returns the console. Run the password-bearing request with your own password substituted in — for example, to fetch the console with the credentials from Step 6:

curl -s -o /dev/null -w '%{http_code}\n' -u admin:<OZONE_ADMIN_PASSWORD> http://127.0.0.1/

A wrong password returns HTTP 401; the correct one returns HTTP 200 (the Recon overview page). In a browser, open http://<instance-public-ip>/ and sign in as admin with the password from Step 6. The overview shows datanode health, pipelines and the cluster storage capacity backed by the dedicated data volume.

Step 8 — Create a volume, bucket and key with the ozone shell

Ozone organises objects as volume / bucket / key. The ozone sh command-line creates them and puts and gets keys. Run it as the ozone service user:

echo "hello ozone" | sudo -u ozone tee /tmp/demo.txt
sudo -u ozone /opt/ozone/bin/ozone sh volume create /demovol
sudo -u ozone /opt/ozone/bin/ozone sh bucket create /demovol/demobucket
sudo -u ozone /opt/ozone/bin/ozone sh key put /demovol/demobucket/demokey /tmp/demo.txt
sudo -u ozone /opt/ozone/bin/ozone sh key list /demovol/demobucket
sudo -u ozone /opt/ozone/bin/ozone sh key get /demovol/demobucket/demokey /tmp/demo-out.txt

key list shows demokey and key get writes the object back to /tmp/demo-out.txt, confirming the data plane round-trips end to end.

Step 9 — Use the S3 gateway

Ozone exposes the same namespace over an S3-compatible API on the gateway (loopback 127.0.0.1:9878). From your workstation, open an SSH tunnel to the gateway and point the AWS CLI at it with the per-instance S3 credentials from Step 6:

ssh -L 9878:127.0.0.1:9878 ubuntu@<instance-public-ip>

Then, in another terminal on your workstation, with OZONE_S3_ACCESS_KEY and OZONE_S3_SECRET_KEY exported as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY:

aws --endpoint-url http://localhost:9878 s3api create-bucket --bucket demo
echo "hello s3" > hello.txt
aws --endpoint-url http://localhost:9878 s3 cp hello.txt s3://demo/hello.txt
aws --endpoint-url http://localhost:9878 s3 ls s3://demo/

The S3 gateway accepts and lists your objects, so existing S3 tooling and SDKs work unchanged against Ozone.

Step 10 — Review datanodes and pipelines in Recon

The Recon console is the management console for the cluster. The Datanodes view lists the datanode, its state and capacity; the Pipelines view shows the single write pipeline (RATIS factor ONE for a single datanode); and the Buckets / keys views list every volume, bucket and object with its size and replication.

Confirm the Ozone version

/opt/ozone/bin/ozone version 2>/dev/null | head -3

Enabling HTTPS

For production, terminate TLS at nginx with a real domain pointed at the instance's public IP. Install certbot and request a certificate (replace the domain):

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

Backup and maintenance

All Ozone data — the OM and SCM metadata, the HDDS datanode containers and the Recon database — lives on the dedicated EBS data volume at /var/lib/ozone. Take an EBS snapshot of that volume to back up your store, and keep the OS patched with sudo apt update && sudo apt upgrade. Ozone restarts cleanly with sudo systemctl restart ozone-scm ozone-om ozone-datanode ozone-recon ozone-s3g nginx. For larger datasets, resize the EBS volume in the AWS console and grow the filesystem.

Support

This image is backed by 24/7 cloudimg support. Contact us by email and chat for help with volume and bucket design, the ozone shell, the S3 gateway, replication tuning, scaling, TLS and backups.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.