New API on AWS User Guide

Overview

This guide covers launching and using the New API on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg MLOps and Speech AI listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

New API is an open source, AGPL-3.0 licensed AI gateway. You connect upstream model services as channels (OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, Vertex AI, DeepSeek or any OpenAI compatible endpoint) and New API exposes them to your applications through one OpenAI compatible API, with an API key per application, quotas, model routing and a usage log of every call. New API hosts no model: it relays requests to the providers you connect, using your own provider accounts, so it needs no GPU. It is a single Go program with its web console built in and an embedded SQLite database, so the instance runs no database server and no container runtime.

Out of the box the upstream release lets whoever reaches it first create the administrator through its setup wizard, and allows open self-registration. On this image both are closed before anything is reachable, and the gateway is fenced off from your network:

• No first caller takeover. First boot runs New API on loopback only, completes the setup wizard with an administrator password generated for your instance, turns self-registration off, and proves the wizard and registration refuse anonymous callers. Only then is nginx allowed to start.
• HTTPS from the first boot. nginx serves the console and the API on port 443 with a certificate generated on your instance's first boot. Port 80 redirects to HTTPS. Requests for a host name the instance does not serve are dropped.
• Secrets unique to your instance. The administrator password, the session secret, the encryption secret New API uses for stored provider keys, and the TLS key are all generated on first boot. None is baked into the image.
• Port 3000 is fenced. New API listens on port 3000 on every interface (it has no loopback only option), so an nftables rule drops port 3000 on every interface except loopback, and on loopback only root and nginx may connect. nginx is the only way in.
• An egress fence against server side request forgery. A channel's base URL is a URL New API fetches, so the new-api service account may reach public addresses only (your model providers). The instance metadata service, link local addresses, loopback, private and reserved IPv4 ranges, your VPC's own address blocks and IPv6 are refused. You can allow a private model server (for example Ollama or vLLM in your VPC) in one file (Step 9).
• Request hygiene at nginx. nginx passes the real client address and a fixed host name, never a forwarding header a client sends, refuses any POST to the setup wizard, rejects encoded or doubled slashes and dot segments in the raw request line, refuses cross site requests to the console, and rate limits sign in.
• Built from source, patched. New API v1.0.0-rc.48 is compiled by cloudimg from the unmodified upstream source with the current Go toolchain and six Go dependencies raised to clear Go security advisories; govulncheck finds no vulnerability that affects the binary. Ubuntu 24.04 LTS is fully patched at build time and unattended security upgrades are on. No compiler or build tool is left on the image.

New API is an independent open source project by QuantumNous and its contributors. This image is produced by cloudimg and is not affiliated with, endorsed by, or sponsored by the New API project. It ships the free and open source AGPL-3.0 licensed software.

What is on the image: New API v1.0.0-rc.48 (/opt/new-api/new-api, with its web console built in), nginx, nftables, SQLite storage in /var/lib/new-api, the licence, notices and build provenance in /usr/share/doc/new-api, Ubuntu 24.04 LTS, cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

Prerequisites

• An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (New API must reach your model providers).
• A security group that allows TCP 22 (SSH), TCP 443 (the console and API) and TCP 80 (redirects to HTTPS) from your own networks only. A gateway holds your provider keys and spends on your provider accounts, so the fewer networks that can reach it, the better.
• An account and API key for at least one model provider. You add these after launch.

You are responsible for complying with the terms of every provider you connect, and with any legal or regulatory obligations that apply to the service you run with this gateway.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick New API on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large, your key pair and a security group as above.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=new-api}]'

First boot takes well under a minute.

Step 2: Connect to your instance

Version	SSH user
New API on Ubuntu 24.04	ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check that the gateway answers

Check the gateway itself, not just the service state. The public status endpoint, through nginx on HTTPS, reports the version, that setup is complete and that registration is off:

S=$(curl -sk https://127.0.0.1/api/status)
echo "$S" | jq '.data | {version, setup, register_enabled, self_use_mode_enabled}'
[ "$(echo "$S" | jq -r '.data.setup')" = true ] && [ "$(echo "$S" | jq -r '.data.register_enabled')" = false ] && echo GATEWAY_OK

{
  "version": "v1.0.0-rc.48",
  "setup": true,
  "register_enabled": false,
  "self_use_mode_enabled": true
}
GATEWAY_OK

Port 80 redirects to HTTPS. New API's port 3000 answers on loopback for root and nginx only: any other user on the instance is refused, and from other machines port 3000 is dropped on every interface except loopback:

H=$(sudo sed -n 's#^NEWAPI_URL=https://\([^/]*\)/.*#\1#p' /root/new-api-credentials.txt)
curl -s -o /dev/null -w "http://$H/ -> HTTP %{http_code} %{redirect_url}\n" --resolve "$H:80:127.0.0.1" "http://$H/"
sudo curl -s -o /dev/null -w 'root to 127.0.0.1:3000 -> HTTP %{http_code}\n' http://127.0.0.1:3000/api/status
P=$(hostname -I | awk '{print $1}')
C=$(sudo -u nobody curl -s -o /dev/null -m 4 -w '%{http_code}' "http://$P:3000/api/status" || true)
echo "another user to the instance address :3000 -> HTTP $C"
[ "$C" = 000 ] && echo FENCED_OK

http://<your-instance-public-dns>/ -> HTTP 301 https://<your-instance-public-dns>/
root to 127.0.0.1:3000 -> HTTP 200
another user to the instance address :3000 -> HTTP 000
FENCED_OK

HTTP 000 means the connection was refused: nothing reaches New API except through nginx.

Step 4: Get your URL and password

sudo cat /root/new-api-credentials.txt

The file is readable by root only. Your values differ:

# New API v1.0.0-rc.48 - generated on first boot by new-api-firstboot.service
# These credentials are unique to this instance; none was baked into the image. Keep this file secret.

NEWAPI_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/
NEWAPI_ADMIN_USERNAME=admin
NEWAPI_ADMIN_PASSWORD=<24 characters generated on your instance>
NEWAPI_TLS_SHA256_FINGERPRINT=<unique to your instance>
#
# Open NEWAPI_URL in a browser (self-signed certificate minted on this instance's first boot; compare its SHA-256
# fingerprint with the line above; port 80 redirects to https) and sign in as admin with the password above.
# Self-registration is OFF. Add your upstream providers under Channels and create keys under API Keys; clients use
# NEWAPI_URL + v1 (for example https://<host>/v1/chat/completions) with a token as the Bearer key.
# Change the name it serves as:   sudo new-api-set-hostname <your-dns-name>
# Reach a model server inside your VPC:  /etc/cloudimg/new-api-egress-allow, then sudo systemctl reload new-api-egress-guard
# User guide: https://www.cloudimg.co.uk/guides/new-api-on-ubuntu-24-04-aws/

NEWAPI_URL uses the instance's public DNS name from EC2 instance metadata, and the certificate also names the public and private IP addresses. The session secret and the encryption secret are in /etc/new-api/.env, readable only by root and the new-api service account; you never need to type them. To serve your own DNS name, see Step 11.

Step 5: Sign in to the console

Open NEWAPI_URL in your browser. The certificate is self signed for your instance, so the browser warns once; compare the fingerprint it shows with NEWAPI_TLS_SHA256_FINGERPRINT before you accept it. Sign in as admin with NEWAPI_ADMIN_PASSWORD. There is no sign up link, because registration is off.

After signing in you land on the console. The admin sections (Channels, Models, Users and System Settings) are in the sidebar.

Change the administrator password under Security & Access if you want one of your own; the credentials file keeps the original.

Step 6: The access model

Anonymous callers are refused by the setup wizard, by registration and by the API, both by nginx and by New API itself:

B=https://127.0.0.1
H=$(sudo sed -n 's#^NEWAPI_URL=https://\([^/]*\)/.*#\1#p' /root/new-api-credentials.txt)
R="--resolve $H:443:127.0.0.1"
SETUP=$(curl -sk $R -o /dev/null -w '%{http_code}' -H 'Content-Type: application/json' -d '{"username":"someone","password":"Some-password-1","confirmPassword":"Some-password-1"}' "https://$H/api/setup")
REG=$(curl -sk $R -H 'Content-Type: application/json' -d '{"username":"someone","password":"Some-password-1"}' "https://$H/api/user/register" | jq -r .message)
MODELS=$(curl -sk $R -o /dev/null -w '%{http_code}' "https://$H/v1/models")
echo "POST /api/setup -> HTTP $SETUP"; echo "POST /api/user/register -> $REG"; echo "anonymous /v1/models -> HTTP $MODELS"
[ "$SETUP" = 403 ] && [ "$MODELS" = 401 ] && case "$REG" in *disabled*) echo ACCESS_MODEL_OK ;; *) exit 1 ;; esac

POST /api/setup -> HTTP 403
POST /api/user/register -> New user registration has been disabled by administrator
anonymous /v1/models -> HTTP 401
ACCESS_MODEL_OK

• The setup wizard is closed. It was completed on first boot with your instance's administrator, so nobody else can claim it. nginx refuses any POST to it, and New API answers that it is already initialised.
• Self-registration is off. Only the administrator creates accounts (Users in the sidebar). Turn registration on under System Settings only if you intend to run a service people sign up to.
• Self-use mode is on. Models can be called without first entering a price for each one, which suits a private gateway for your own team. If you will bill users by usage, turn it off and configure model prices.
• Every API call needs a key. The OpenAI compatible endpoints under /v1 refuse requests without a valid key.

Step 7: Connect a model provider (a channel)

A channel is one upstream provider account: its type, its base URL if it is not the default, your API key for it, and the models it serves.

1. In the console, open Channels and add a channel.
2. Choose the provider. Built in types include OpenAI, Anthropic, Gemini, Azure OpenAI, AWS Bedrock, Vertex AI and DeepSeek; choose OpenAI for any OpenAI compatible service and set its base URL.
3. Paste your provider API key, select the models this channel serves (or fetch them from the provider), and save.
4. Use Test on the channel to send a real request through it.

Add several channels for the same model to spread load or fail over between accounts; channel priority and weight control the routing. Provider keys are stored encrypted in the database on your instance, which is readable only by the new-api service account.

Step 8: Create an API key for each application

Give every application its own key, so you can see its usage and revoke it on its own. In the console open API Keys, create a key with a name, and optionally a quota, an expiry date, an allowed model list or an IP allow list.

You can do the same from the command line on the instance. This signs in with the administrator password, creates a key named cli-app, uses it, and deletes it again:

H=$(sudo sed -n 's#^NEWAPI_URL=https://\([^/]*\)/.*#\1#p' /root/new-api-credentials.txt)
B="https://$H"; R="--resolve $H:443:127.0.0.1"
AT=$(sudo sed -n 's/^NEWAPI_ADMIN_PASSWORD=//p' /root/new-api-credentials.txt \
  | jq -Rc '{username:"admin", password:.}' \
  | curl -sk $R -H 'Content-Type: application/json' --data-binary @- "$B/api/user/login" | jq -r .data.access_token)
A=(-H "Authorization: Bearer $AT" -H 'New-Api-User: 1')
curl -sk $R "${A[@]}" -H 'Content-Type: application/json' \
  -d '{"name":"cli-app","remain_quota":0,"expired_time":-1,"unlimited_quota":true}' "$B/api/token/" | jq -r '"created: \(.success)"'
ID=$(curl -sk $R "${A[@]}" "$B/api/token/?p=1&size=100" | jq -r '[(.data.items // .data)[] | select(.name=="cli-app")][-1].id')
KEY=$(curl -sk $R "${A[@]}" -X POST "$B/api/token/$ID/key" | jq -r .data.key)
echo "API key for cli-app: sk-${KEY:0:4}...(${#KEY} characters)"
M=$(curl -sk $R "$B/v1/models" -H "Authorization: Bearer sk-$KEY")
echo "$M"
E=$(curl -sk $R "$B/v1/chat/completions" -H "Authorization: Bearer sk-$KEY" -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}' | jq -r .error.code)
echo "chat completion before any channel: $E"
curl -sk $R "${A[@]}" -X DELETE "$B/api/token/$ID" | jq -r '"deleted: \(.success)"'
[ "$(echo "$M" | jq -r .success)" = true ] && [ "$E" = model_not_found ] && echo API_KEY_OK

created: true
API key for cli-app: sk-xxxx...(48 characters)
{"data":[],"object":"list","success":true}
chat completion before any channel: model_not_found
deleted: true
API_KEY_OK

The key is accepted and the request reaches the router; it answers model_not_found because no channel serves that model yet. Once you add a channel (Step 7), /v1/models lists its models and the same request returns the provider's answer.

Step 9: Point your applications at the gateway

Any OpenAI compatible client works. Set the base URL to NEWAPI_URL followed by v1, and the API key to the key you created. For example, with the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.example.com/v1",
    api_key="sk-your-new-api-key",
)
reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

Or with curl from your own machine, with the key in the NEWAPI_KEY environment variable:

curl https://<public-ip>/v1/chat/completions \
  -H "Authorization: Bearer $NEWAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}],"stream":true}'

Streaming responses and WebSocket upgrades pass straight through nginx. Until you install a certificate from a public authority (Step 11), clients must trust this instance's certificate: copy /etc/nginx/ssl/new-api.crt to the client and point it at that file (for the Python SDK, set SSL_CERT_FILE).

A model server inside your VPC. The egress fence lets New API reach public addresses only. To connect a private model server, for example an Ollama or vLLM host at 10.0.5.25, list its address in /etc/cloudimg/new-api-egress-allow and reload the fence:

echo '10.0.5.25/32' | sudo tee -a /etc/cloudimg/new-api-egress-allow
sudo systemctl reload new-api-egress-guard

The metadata service, link local, loopback and multicast ranges are always refused and cannot be listed.

Step 10: Security checks

The egress fence in force for the new-api service account, with each rule's purpose:

sudo nft list table inet cloudimg_new_api_egress | grep -oE 'comment "cloudimg-[a-z0-9]+"' | sort -u

comment "cloudimg-allow"
comment "cloudimg-allow6"
comment "cloudimg-fence3000"
comment "cloudimg-ipv6"
comment "cloudimg-local3000"
comment "cloudimg-loopback"
comment "cloudimg-metadata"
comment "cloudimg-metadata6"
comment "cloudimg-multicast"
comment "cloudimg-private"
comment "cloudimg-reply"
comment "cloudimg-vpc"

The metadata service is refused to New API, while root on the same instance reaches it (the control). The same holds for the instance's own SSH port, which stands in for any service in your VPC:

I=$(sudo curl -s -m 4 -o /dev/null -w '%{http_code}' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token)
N=$(sudo setpriv --reuid=new-api --regid=new-api --clear-groups curl -s -m 4 -o /dev/null -w '%{http_code}' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token || true)
P=$(hostname -I | awk '{print $1}')
VR=$(sudo timeout 4 bash -c "exec 3<>/dev/tcp/$P/22" 2>/dev/null && echo OPEN || echo REFUSED)
VN=$(sudo setpriv --reuid=new-api --regid=new-api --clear-groups timeout 4 bash -c "exec 3<>/dev/tcp/$P/22" 2>/dev/null && echo OPEN || echo REFUSED)
echo "metadata service: root HTTP $I, new-api HTTP $N"
echo "own address port 22: root $VR, new-api $VN"
[ "$I" = 200 ] && [ "$N" = 000 ] && [ "$VR" = OPEN ] && [ "$VN" = REFUSED ] && echo EGRESS_FENCE_OK

metadata service: root HTTP 200, new-api HTTP 000
own address port 22: root OPEN, new-api REFUSED
EGRESS_FENCE_OK

nginx judges the raw request line, before it decodes or normalises the path, so encoded slashes, dot segments and doubled slashes are refused (--path-as-is sends the path exactly as written):

H=$(sudo sed -n 's#^NEWAPI_URL=https://\([^/]*\)/.*#\1#p' /root/new-api-credentials.txt)
R="--resolve $H:443:127.0.0.1"
OK=1
for p in '/api/%2e%2e/status' '/api//setup' '/api/../api/status' '/api/%2fsetup' '/API/SETUP'; do
  c=$(curl -sk $R --path-as-is -o /dev/null -w '%{http_code}' -X POST "https://$H$p")
  echo "POST $p -> HTTP $c"
  case "$c" in 400|403|404) ;; *) OK=0 ;; esac
done
c=$(curl -sk $R -o /dev/null -w '%{http_code}' "https://$H/api/status"); echo "GET /api/status -> HTTP $c"
[ "$OK" = 1 ] && [ "$c" = 200 ] && echo RAW_LINE_OK

POST /api/%2e%2e/status -> HTTP 400
POST /api//setup -> HTTP 400
POST /api/../api/status -> HTTP 400
POST /api/%2fsetup -> HTTP 400
POST /API/SETUP -> HTTP 403
GET /api/status -> HTTP 200
RAW_LINE_OK

New API runs as an unprivileged service account in a systemd sandbox, blocked from the metadata service, and its database is readable only by that account:

systemctl show new-api -p User -p NoNewPrivileges -p ProtectSystem -p IPAddressDeny
sudo stat -c '%U:%G %a %n' /var/lib/new-api/new-api.db /etc/new-api/.env /root/new-api-credentials.txt

IPAddressDeny=192.0.0.0/24 fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 240.0.0.0/4 198.18.0.0/15 224.0.0.0/4
User=new-api
ProtectSystem=strict
NoNewPrivileges=yes
new-api:new-api 640 /var/lib/new-api/new-api.db
root:new-api 640 /etc/new-api/.env
root:root 600 /root/new-api-credentials.txt

There is no database server, so there is no database port to protect.

Step 11: Your own DNS name and certificate

Point a DNS record at the instance, then set the name the gateway is served as. This mints a new self signed certificate for the name, updates nginx, New API's trusted origins and the URL and fingerprint in /root/new-api-credentials.txt, and keeps the name across reboots:

sudo new-api-set-hostname <your-domain>

To use a certificate from your own or a public certificate authority instead, install it over the generated pair and reload nginx (running new-api-set-hostname again replaces it with a self signed one):

sudo install -m 0644 <your-domain>.crt /etc/nginx/ssl/new-api.crt
sudo install -m 0600 <your-domain>.key /etc/nginx/ssl/new-api.key
sudo nginx -t && sudo systemctl reload nginx

Step 12: What is in the image, and under what licence

/opt/new-api/new-api --version
sha256sum /opt/new-api/new-api
ls /usr/share/doc/new-api
sed -n '/^SOURCE OFFER/,/^$/p' /usr/share/doc/new-api/BUILD-PROVENANCE.txt

v1.0.0-rc.48
0161240846b4e125d34d82a7c47908ffbf30ea18412c8768872922a2b8578057  /opt/new-api/new-api
BUILD-PROVENANCE.txt
LICENSE
NOTICE
THIRD-PARTY-LICENSES.md
go.mod
go.sum
SOURCE OFFER (AGPLv3 sections 6 and 13): the corresponding source is the upstream tag below
(https://github.com/QuantumNous/new-api/tree/v1.0.0-rc.48, commit 706144769e5bed5a47b78721e3bb11da3cf4c0b9) with go.mod and go.sum replaced by
the two files in this directory. cloudimg also provides it on request: support@cloudimg.co.uk.

New API is licensed under the GNU Affero General Public License v3.0, with attribution terms under section 7 in its NOTICE file, which the console honours. BUILD-PROVENANCE.txt records exactly how the binary was built (source archive and checksum, Go toolchain, the raised dependencies with the resulting go.mod and go.sum, which ship beside it, and the build flags) and is the AGPL source offer. If you modify New API and let others use it over a network, the AGPL requires you to offer them the source of your modified version.

Day to day operation

systemctl status new-api --no-pager | head -3
sudo journalctl -u new-api -n 20 --no-pager | tail -3

• Restart the gateway with sudo systemctl restart new-api, and reload nginx with sudo systemctl reload nginx.
• Data lives in /var/lib/new-api/new-api.db (SQLite). Application logs are in /var/log/new-api.
• Back up by stopping the service briefly and copying the database. Keep /etc/new-api/.env with any backup you restore elsewhere: the encryption secret in it is needed to read the provider keys New API stored.

sudo systemctl stop new-api
sudo tar czf /root/new-api-backup-$(date +%F).tgz -C /var/lib/new-api . -C /etc/new-api .env
sudo systemctl start new-api

• Upgrades. Unattended upgrades keep Ubuntu and nginx patched. New API itself is a single binary; to move to a newer release, back up first, read the upstream release notes, replace /opt/new-api/new-api and restart the service. New API migrates its database on start.

Troubleshooting

• Nothing answers on port 443. nginx starts only after first boot has completed: sudo systemctl status new-api-firstboot --no-pager and sudo journalctl -u new-api-firstboot --no-pager | tail -30.
• The browser warns about the certificate. Expected until you install your own certificate; compare the fingerprint with NEWAPI_TLS_SHA256_FINGERPRINT.
• A request returns model_not_found. No enabled channel serves that model for the key's group. Add the model to a channel (Step 7), or check the channel is enabled and its test passes.
• A channel test to a private address fails quickly. The egress fence refuses private and VPC addresses by default; allow the server in /etc/cloudimg/new-api-egress-allow (Step 9).
• A channel test to a public provider fails. The error comes from the provider: check the key, the base URL and that your account can use the model.
• HTTP 429 on sign in. nginx allows 20 sign in attempts a minute per address; wait a minute.

Security recommendations

• Allow ports 22, 80 and 443 in the security group from your own networks only.
• Give each application its own key, with a quota and an expiry where it makes sense, and revoke keys you no longer use.
• Keep registration off unless you intend to run a public service.
• Use your own DNS name and a certificate your clients trust (Step 11).
• List only the private model servers you need in the egress allow-list.
• Keep the operating system patched; unattended security upgrades are on.

Support and licensing

New API is open source under the AGPL-3.0; there is no per user licence fee. The AMI price covers cloudimg's hardening, first boot setup and support.

• Email: support@cloudimg.co.uk
• Website: https://www.cloudimg.co.uk
• Support: 24/7, with a 24 hour response SLA