Storage AWS

S3Proxy 3.3.0 on AWS User Guide

Last updated: 2026-10-19 | Product: S3Proxy 3.3.0 on Ubuntu 24.04 on AWS

Overview

This guide covers launching and using the S3Proxy 3.3.0 on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg object storage listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

S3Proxy is an open source (Apache License 2.0) Java application that implements the Amazon S3 API in front of a storage backend. This image uses the local filesystem backend: buckets are directories and objects are files on a dedicated EBS data volume attached to your instance. Point the AWS CLI, an AWS SDK, rclone, restic or any other S3 client at the instance and create buckets and read and write objects the way you would against Amazon S3, with the data staying on a volume you control.

Upstream S3Proxy can run as an open, anonymous endpoint. The cloudimg image does not:

AWS Signature Version 4 on every request. S3Proxy runs with s3proxy.authorization=aws-v4. An access key and secret key unique to your instance are generated on first boot and written to /root/s3proxy-info.txt, readable by root only. No key pair is baked into the image.
No anonymous access at all. nginx refuses any request that carries no signature with S3's own AccessDenied error, so even an object marked public-read is not served to anonymous clients. Share objects with presigned URLs instead.
Fail closed. A start guard refuses to run S3Proxy unless its configuration enforces SigV4 with a real key pair on a loopback endpoint, so the open authorization=none mode cannot be switched on by accident. nginx refuses to start until first boot has finished.
HTTPS from the first boot. nginx serves the S3 API on port 443 with a certificate generated on your instance's first boot. There is no plain HTTP listener, so keys and data never cross the network unencrypted. Requests for any host name the instance does not serve are dropped.
Hardened defaults. S3Proxy listens on loopback 127.0.0.1:8080 only, runs as its own unprivileged user in a systemd sandbox and reads its configuration through a private systemd credential. S3Proxy and nginx are both blocked from the instance metadata service, the AMI requires IMDSv2, and no AWS S3 backend or instance role is ever used.
Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: S3Proxy 3.3.0 (/opt/s3proxy/s3proxy, the official release jar, checksum verified), OpenJDK 17, nginx, the AWS CLI v2 (2.38.1) as an on-instance S3 client, Ubuntu 24.04 LTS, cloud-init and the AWS Systems Manager agent. Object data lives on a separate 30 GiB gp3 volume mounted at /var/lib/s3proxy.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

Prerequisites

An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet.
A security group that allows TCP 22 (SSH) and TCP 443 (the S3 API over HTTPS) from your own networks only. Nothing else needs to be open; the image has no port 80 listener.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick S3Proxy 3.3.0 on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large (size up for many concurrent clients or very large objects), your key pair and a security group as above. The AMI carries two volumes: the root volume and the 30 GiB data volume for your buckets. Raise the data volume's size at launch if you need more space, and turn on EBS encryption for it if your data requires encryption at rest.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled,HttpPutResponseHopLimit=1 \
  --block-device-mappings 'DeviceName=/dev/sdf,Ebs={VolumeSize=100,VolumeType=gp3,Encrypted=true}' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=s3proxy}]'

Step 2: Connect to your instance

Version	SSH user
S3Proxy 3.3.0 on Ubuntu 24.04	`ubuntu`

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

First boot generates the key pair and the certificate, then starts S3Proxy and nginx. s3proxy-names re-reads the instance's addresses on every boot:

systemctl is-active s3proxy nginx s3proxy-firstboot s3proxy-names
sudo test -f /var/lib/cloudimg/s3proxy-firstboot.done && echo FIRSTBOOT_DONE

active
active
active
active
FIRSTBOOT_DONE

Only SSH and nginx listen on public addresses; S3Proxy listens on loopback:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v '127.0.0.5' | sort

0.0.0.0:22 users:(("sshd",...))
0.0.0.0:443 users:(("nginx",...))
[::]:22 users:(("sshd",...))
[::]:443 users:(("nginx",...))
[::ffff:127.0.0.1]:8080 users:(("java",...))

Step 4: Get your endpoint and keys

sudo cat /root/s3proxy-info.txt

The file is readable by root only and holds the endpoint URL and this instance's key pair:

S3PROXY_ENDPOINT_URL=https://<public-ip>/
S3PROXY_ACCESS_KEY=<your-access-key>
S3PROXY_SECRET_KEY=<your-secret-key>
S3PROXY_REGION=us-east-1

The endpoint uses the instance's public IPv4 address from EC2 instance metadata. The certificate also names the public DNS name, the private IP address and the private DNS name, so clients inside your VPC can use https://<private-ip>/. Use path-style addressing (https://<endpoint>/<bucket>/<key>). Any region name works for signing; us-east-1 is the convention.

Step 5: Store and read objects on the instance

The AWS CLI is installed on the image. This round trip creates a bucket, uploads a file, lists it, downloads it again and compares the bytes, all over HTTPS with your instance's keys. AWS_CA_BUNDLE points at the instance's own certificate, so TLS is verified rather than skipped:

F=/root/s3proxy-info.txt
export AWS_ACCESS_KEY_ID=$(sudo grep '^S3PROXY_ACCESS_KEY=' $F | cut -d= -f2-)
export AWS_SECRET_ACCESS_KEY=$(sudo grep '^S3PROXY_SECRET_KEY=' $F | cut -d= -f2-)
export AWS_DEFAULT_REGION=us-east-1 AWS_EC2_METADATA_DISABLED=true AWS_CA_BUNDLE=/etc/nginx/tls/s3proxy.crt
cd "$(mktemp -d)"
echo "hello from cloudimg S3Proxy $(date -u +%FT%TZ)" > hello.txt
aws --endpoint-url https://127.0.0.1 s3 mb s3://demo-bucket
aws --endpoint-url https://127.0.0.1 s3 cp hello.txt s3://demo-bucket/docs/hello.txt
aws --endpoint-url https://127.0.0.1 s3 ls s3://demo-bucket --recursive
aws --endpoint-url https://127.0.0.1 s3 cp s3://demo-bucket/docs/hello.txt hello-back.txt
cmp hello.txt hello-back.txt && echo "BYTES MATCH: $(cat hello-back.txt)"

make_bucket: demo-bucket
upload: ./hello.txt to s3://demo-bucket/docs/hello.txt
2026-10-19 18:31:38         49 docs/hello.txt
download: s3://demo-bucket/docs/hello.txt to ./hello-back.txt
BYTES MATCH: hello from cloudimg S3Proxy 2026-10-19T18:31:36Z

The object is an ordinary file on the data volume:

sudo ls -l /var/lib/s3proxy/data/demo-bucket/docs/
df -h /var/lib/s3proxy

total 8
-rw-r----- 1 s3proxy s3proxy 49 Oct 19 18:31 hello.txt
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme1n1     30G   44K   28G   1% /var/lib/s3proxy

Large files are uploaded in parts automatically (multipart upload), and current AWS CLI and SDK versions work with their default checksum settings.

To share one object without handing out your keys, create a presigned URL. Anyone with the URL can fetch that object until it expires, without an Authorization header:

F=/root/s3proxy-info.txt
export AWS_ACCESS_KEY_ID=$(sudo grep '^S3PROXY_ACCESS_KEY=' $F | cut -d= -f2-)
export AWS_SECRET_ACCESS_KEY=$(sudo grep '^S3PROXY_SECRET_KEY=' $F | cut -d= -f2-)
export AWS_DEFAULT_REGION=us-east-1 AWS_EC2_METADATA_DISABLED=true AWS_CA_BUNDLE=/etc/nginx/tls/s3proxy.crt
URL=$(aws --endpoint-url https://127.0.0.1 s3 presign s3://demo-bucket/docs/hello.txt --expires-in 600)
curl -s --cacert /etc/nginx/tls/s3proxy.crt "$URL" | grep -q 'hello from cloudimg S3Proxy' && echo "PRESIGNED URL OK"

PRESIGNED URL OK

Step 6: See the authentication refuse bad requests

A request with no signature, a request signed with the wrong secret key and a request with an unknown access key are all refused with HTTP 403 and S3's own XML error codes. The key pair is handed to curl on standard input, so it never appears in the process list or your shell history:

AK=$(sudo grep '^S3PROXY_ACCESS_KEY=' /root/s3proxy-info.txt | cut -d= -f2-)
C=/etc/nginx/tls/s3proxy.crt
U=$(curl -s --cacert $C -w ' HTTP %{http_code}' https://127.0.0.1/demo-bucket/)
W=$(printf 'user = "%s:wrong-secret"\naws-sigv4 = "aws:amz:us-east-1:s3"\n' "$AK" | curl -K - -s --cacert $C -w ' HTTP %{http_code}' https://127.0.0.1/demo-bucket/)
K=$(printf 'user = "AKNOTAREALKEY0000000:x"\naws-sigv4 = "aws:amz:us-east-1:s3"\n' | curl -K - -s --cacert $C -w ' HTTP %{http_code}' https://127.0.0.1/demo-bucket/)
for r in "unsigned:$U" "wrong secret:$W" "unknown key:$K"; do echo "${r%%:*}: $(grep -o '<Code>[A-Za-z]*</Code>' <<<"$r") $(grep -o 'HTTP [0-9]*' <<<"$r")"; done
[[ $U == *'<Code>AccessDenied</Code>'*' HTTP 403' && $W == *'<Code>SignatureDoesNotMatch</Code>'*' HTTP 403' && $K == *'<Code>InvalidAccessKeyId</Code>'*' HTTP 403' ]] && echo "ALL THREE REFUSED"

unsigned: <Code>AccessDenied</Code> HTTP 403
wrong secret: <Code>SignatureDoesNotMatch</Code> HTTP 403
unknown key: <Code>InvalidAccessKeyId</Code> HTTP 403
ALL THREE REFUSED

Clean up the demo bucket when you are done:

F=/root/s3proxy-info.txt
export AWS_ACCESS_KEY_ID=$(sudo grep '^S3PROXY_ACCESS_KEY=' $F | cut -d= -f2-)
export AWS_SECRET_ACCESS_KEY=$(sudo grep '^S3PROXY_SECRET_KEY=' $F | cut -d= -f2-)
export AWS_DEFAULT_REGION=us-east-1 AWS_EC2_METADATA_DISABLED=true AWS_CA_BUNDLE=/etc/nginx/tls/s3proxy.crt
aws --endpoint-url https://127.0.0.1 s3 rb s3://demo-bucket --force

delete: s3://demo-bucket/docs/hello.txt
remove_bucket: demo-bucket

Step 7: Connect from your own machine

Copy the instance's certificate to your machine so clients can verify TLS, then use the endpoint and keys from Step 4. With the AWS CLI:

scp -i <your-key.pem> ubuntu@<public-ip>:/etc/nginx/tls/s3proxy.crt ./s3proxy.crt
aws configure set aws_access_key_id <your-access-key> --profile s3proxy
aws configure set aws_secret_access_key <your-secret-key> --profile s3proxy
aws configure set region us-east-1 --profile s3proxy
aws --profile s3proxy --endpoint-url https://<public-ip> --ca-bundle ./s3proxy.crt s3 ls

With Python and boto3:

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://<public-ip>",
    aws_access_key_id="<your-access-key>",
    aws_secret_access_key="<your-secret-key>",
    region_name="us-east-1",
    verify="./s3proxy.crt",
    config=Config(s3={"addressing_style": "path"}),
)
s3.create_bucket(Bucket="backups")
s3.put_object(Bucket="backups", Key="notes.txt", Body=b"hello")
print(s3.get_object(Bucket="backups", Key="notes.txt")["Body"].read())

rclone, restic, s3cmd and backup tools work the same way: set the endpoint to https://<public-ip> (or your DNS name), path-style addressing, and the instance's access key and secret key.

Step 8: Use your own DNS name

If you point a DNS name at the instance, add it so nginx serves it and the certificate names it. The name you add becomes the endpoint URL in /root/s3proxy-info.txt:

sudo s3proxy-set-hostname s3.<your-domain>

The names are kept in /etc/cloudimg/s3proxy-hostname (one per line). The instance's own public and private addresses are re-read from instance metadata on every boot, so a new public IP after a stop and start is picked up automatically; the certificate is only re-issued when the set of names changes. For a certificate from a public CA, replace /etc/nginx/tls/s3proxy.crt and /etc/nginx/tls/s3proxy.key (key mode 0600, owner root) and run sudo systemctl reload nginx.

Step 9: Rotate the key pair

Generate a new access key and secret key; the old pair stops working at once and /root/s3proxy-info.txt is updated:

sudo s3proxy-rotate-keys

s3proxy-rotate-keys: new key pair in /root/s3proxy-info.txt; the old pair no longer works

The configuration is /etc/s3proxy/s3proxy.conf (root only). If you edit it, keep exactly one s3proxy.authorization=aws-v4 line and the loopback endpoint, or the start guard keeps S3Proxy stopped; sudo /usr/local/sbin/s3proxy-auth-guard tells you why.

Security notes

Both network-facing services are blocked from the instance metadata service, so S3Proxy can never pick up instance-role credentials:

systemctl show -p IPAddressDeny s3proxy nginx

IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128 168.63.129.16/32

IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128 168.63.129.16/32

nginx passes S3Proxy the client address it sees on the socket and ignores any X-Forwarded-For header a client sends.
S3 server side encryption headers are accepted but not acted on; encrypt the EBS data volume to protect data at rest.
S3Proxy implements most of the S3 API on the filesystem backend. Bucket policies, versioning, object lock and ACLs other than private are not supported; see the S3Proxy compatibility notes.
Keep TCP 443 limited to the networks that need the endpoint.

Logs and troubleshooting

sudo journalctl -u s3proxy -n 20 --no-pager
sudo tail -n 5 /var/log/nginx/access.log

SignatureDoesNotMatch from a client that worked before: check the host name in the client's endpoint is one the instance serves (cat /etc/nginx/s3proxy-host.conf); add a new DNS name with s3proxy-set-hostname.
The connection closes without a response: the request used a host name the instance does not serve. Use the endpoint from Step 4 or add your name in Step 8.
Bucket names in the host (virtual-hosted style) fail: use path-style addressing.

Support

cloudimg provides 24/7 technical support for this image by email and live chat: deployment, client configuration, HTTPS setup, upgrades and troubleshooting. Contact support@cloudimg.co.uk or visit cloudimg.co.uk/support.

S3Proxy is a trademark of its respective owner. cloudimg is not affiliated with or endorsed by the S3Proxy project. Amazon S3 and AWS are trademarks of Amazon.com, Inc. or its affiliates.