whisper.cpp Server 1.9.1 on AWS User Guide

Overview

This guide covers launching and using the whisper.cpp Server 1.9.1 on Ubuntu 24.04 AMI from cloudimg on Amazon EC2. It is one version of the cloudimg MLOps and Speech AI listing on AWS Marketplace; you pick it as the software version when you subscribe and launch.

whisper.cpp is a high performance C and C++ implementation of OpenAI's Whisper automatic speech recognition model. It runs entirely on the CPU, with no GPU, no Python runtime and no external service. This image ships the project's bundled HTTP server with a speech recognition model already installed, so the instance is a private speech to text service from its first boot: post an audio file and get the transcript back. Audio never leaves the instance you control.

Upstream's server has no authentication and an endpoint that loads any file on the server as a model. The cloudimg image closes that down:

• HTTPS from the first boot. nginx serves the web interface and the API on port 443 with a certificate generated on your instance's first boot. Port 80 redirects to HTTPS.
• A password unique to your instance. HTTP Basic Auth is required for the user whisper. The password is generated on first boot and written to /root/whisper-cpp-server-credentials.txt; nginx stores only its bcrypt hash. No password is baked into the image.
• Fail closed. The image ships with an empty password store, and nginx refuses to start until first boot has finished, so the API is never reachable without a login.
• Only what you need is reachable. The inference engine listens on loopback 127.0.0.1:8080. nginx passes on exactly three things: the web page at /, the transcription API at /inference and a status check at /health. The model loading endpoint /load is refused outright.
• One request cannot exhaust the instance. Uploads are limited to 25 MB, a transcription may run for at most 15 minutes, and one client can run at most 2 transcriptions at a time (4 in total). A client that disconnects stops its transcription.
• Hardened defaults. The engine runs as its own unprivileged user in a systemd sandbox with a memory ceiling, the engine and nginx are both blocked from the instance metadata service, the AMI requires IMDSv2, and requests for any host name the instance does not serve are dropped.
• Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.

What is on the image: whisper.cpp 1.9.1 (/usr/local/bin/whisper-server, built from the pinned upstream release), the ggml-base.en English speech recognition model, nginx, ffmpeg (automatic conversion of MP3, M4A, FLAC, OGG and video audio), two short test clips, Ubuntu 24.04 LTS, cloud-init and the AWS Systems Manager agent.

Default user: ubuntu · Architecture: x86_64 · Recommended instance type: m5.large

About the bundled model

whisper.cpp is a program, not a model; it needs a separate model file with its own licence. This image ships ggml-base.en, a format conversion of OpenAI's original Whisper base.en checkpoint into the GGML format whisper.cpp reads. It is not a fine tune and not a distillation. OpenAI released Whisper's code and model weights under the MIT License, and whisper.cpp is MIT licensed too. Both licences and a written provenance record ship in /opt/whisper/. The model is pinned by file name, source revision and SHA-256 checksum at build time, so the image cannot ship a different model.

Prerequisites

• An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (or a private route you use for management).
• A security group that allows TCP 22 (SSH), TCP 443 (the web interface and API) and TCP 80 (redirects to HTTPS) from your own networks only.

Step 1: Launch the instance

In the AWS Marketplace listing choose Continue to Subscribe, then Continue to Configuration, pick whisper.cpp Server 1.9.1 on Ubuntu 24.04 as the software version and your Region, then Launch through EC2. Choose m5.large (see Performance for larger sizes), your key pair, and a security group as above.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=whisper-cpp}]'

Step 2: Connect to your instance

Version	SSH user
whisper.cpp Server 1.9.1 on Ubuntu 24.04	ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

Step 3: Check the services

First boot generates the certificate and the password, then starts the inference engine and nginx:

systemctl is-active whisper-server nginx whisper-cpp-server-firstboot
sudo test -f /var/lib/cloudimg/whisper-cpp-server-firstboot.done && echo FIRSTBOOT_DONE

All three units are active and the first boot marker is present:

active
active
active
FIRSTBOOT_DONE

Only SSH and nginx listen on public addresses; the inference engine listens on loopback:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v '127.0.0.5' | sort

0.0.0.0:22 users:(("sshd",...))
0.0.0.0:443 users:(("nginx",...))
0.0.0.0:80 users:(("nginx",...))
127.0.0.1:8080 users:(("whisper-server",...))
[::]:22 users:(("sshd",...))
[::]:443 users:(("nginx",...))
[::]:80 users:(("nginx",...))

Step 4: Get your URL and password

sudo cat /root/whisper-cpp-server-credentials.txt

The file is readable by root only and holds the URLs, the user name and this instance's password:

WHISPER_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/
WHISPER_INFERENCE_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/inference
WHISPER_USER=whisper
WHISPER_PASSWORD=<WHISPER_PASSWORD>

The URL uses the instance's public DNS name from EC2 instance metadata. The certificate also names the public and private IP addresses, so https://<public-ip>/ works too. To use your own DNS name, see Step 9.

Step 5: Transcribe a test clip over HTTPS

The image ships two short clips in /usr/local/share/whisper-cpp-server/samples/: jfk.wav, 11 seconds of President Kennedy's 1961 inaugural address (public domain, from the whisper.cpp source tree), and cloudimg-test.wav, a sentence synthesised on the build machine. Transcribe the recording through nginx with your password, and check the transcript itself, not only the status code. The password is passed to curl on standard input, so it never appears in the process list or your shell history:

sudo bash -c '
  F=/root/whisper-cpp-server-credentials.txt
  P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-)
  H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#")
  auth() { printf "user = \"whisper:%s\"\n" "$P"; }
  T=$(auth | curl -K - -sk --resolve "$H:443:127.0.0.1" \
        -F file=@/usr/local/share/whisper-cpp-server/samples/jfk.wav -F response_format=json \
        "https://$H/inference")
  echo "$T"
  case "$T" in
    *"ask not what your country can"*) echo TRANSCRIPT_OK ;;
    *) echo "unexpected transcript"; exit 1 ;;
  esac
'

{"text":" And so my fellow Americans, ask not what your country can\n do for you, ask what you can do for your country.\n"}
TRANSCRIPT_OK

Without the password, or with a wrong one, nginx refuses the request before it reaches the engine:

H=$(sudo grep '^WHISPER_URL=' /root/whisper-cpp-server-credentials.txt | sed -E 's#^WHISPER_URL=https://([^/]+)/.*#\1#')
S=/usr/local/share/whisper-cpp-server/samples/jfk.wav
A=$(curl -sk -o /dev/null -w '%{http_code}' --resolve "$H:443:127.0.0.1" -F file=@$S "https://$H/inference")
W=$(printf 'user = "whisper:not-the-password"\n' | curl -K - -sk -o /dev/null -w '%{http_code}' --resolve "$H:443:127.0.0.1" -F file=@$S "https://$H/inference")
echo "no password: HTTP $A"; echo "wrong password: HTTP $W"
[ "$A" = 401 ] && [ "$W" = 401 ] && echo REFUSED_OK

no password: HTTP 401
wrong password: HTTP 401
REFUSED_OK

An HTTP 200 with an empty text would mean the model failed to load, which is why the check above looks at the words.

Step 6: Use the browser interface

Open the WHISPER_URL in your browser. The certificate is self signed, so your browser asks you to confirm it the first time (or install your own certificate, Step 9). Without credentials nginx answers 401:

Sign in as whisper with the password from Step 4. The page documents the API and has an upload form:

Choose an audio file, pick a response format and select Submit:

The transcript comes back in the browser:

The page also documents a /load endpoint. It is refused on this image (HTTP 403); see Step 8.

Step 7: Transcribe your own audio

From your own machine, use the URL and password from Step 4 (-k accepts the self signed certificate). ffmpeg on the instance converts the input, so MP3, M4A, FLAC, OGG, WAV and the audio track of most video files are accepted:

curl -k -u whisper:<WHISPER_PASSWORD> -F file=@recording.mp3 -F response_format=text https://<public-ip>/inference

Pass response_format to choose the output:

Value	Output
text	Plain transcript text
json	{"text": "..."}
verbose_json	Transcript, language, duration, segments and per word timings and probabilities
srt	SubRip subtitles
vtt	WebVTT subtitles

Subtitles for the test recording, made on the instance:

sudo bash -c '
  F=/root/whisper-cpp-server-credentials.txt
  P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-)
  H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#")
  OUT=$(printf "user = \"whisper:%s\"\n" "$P" | curl -K - -sk --resolve "$H:443:127.0.0.1" \
        -F file=@/usr/local/share/whisper-cpp-server/samples/jfk.wav -F response_format=srt "https://$H/inference")
  echo "$OUT"
  case "$OUT" in *"-->"*"fellow Americans"*) echo SUBTITLES_OK ;; *) echo "no subtitles"; exit 1 ;; esac
'

1
00:00:00,000 --> 00:00:06,740
 And so my fellow Americans, ask not what your country can

2
00:00:06,740 --> 00:00:11,000
 do for you, ask what you can do for your country.

SUBTITLES_OK

Uploads are limited to 25 MB each, the common limit for speech to text APIs. For a longer recording, either compress it (ffmpeg -i long.wav -ac 1 -b:a 32k long.mp3 fits about 1 hour 45 minutes into 25 MB) or raise client_max_body_size (Step 10). Audio and transcripts are not stored: each upload is converted in the engine's private temporary directory and deleted when the request finishes.

Step 8: Security checks

The model loading endpoint is refused even with the password, any other path is not found, and the status check works without a login (for load balancers; it returns only {"status":"ok"}):

sudo bash -c '
  F=/root/whisper-cpp-server-credentials.txt
  P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-)
  H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#")
  R="--resolve $H:443:127.0.0.1"
  auth() { printf "user = \"whisper:%s\"\n" "$P"; }
  L=$(auth | curl -K - -sk $R -o /dev/null -w "%{http_code}" -F model=/etc/shadow "https://$H/load")
  N=$(auth | curl -K - -sk $R -o /dev/null -w "%{http_code}" "https://$H/anything-else")
  echo "load endpoint: HTTP $L"; echo "other path: HTTP $N"
  echo "health: $(curl -sk $R "https://$H/health")"
  [ "$L" = 403 ] && [ "$N" = 404 ] && echo SURFACE_OK
'

load endpoint: HTTP 403
other path: HTTP 404
health: {"status":"ok"}
SURFACE_OK

The engine is not reachable on the instance's own network address, only through nginx:

C=$(curl -s -o /dev/null -m 5 -w '%{http_code}' "http://$(hostname -I | awk '{print $1}'):8080/health" || true)
echo "engine on the network address: HTTP $C"
[ "$C" = 000 ] && echo LOOPBACK_ONLY

engine on the network address: HTTP 000
LOOPBACK_ONLY

The engine runs unprivileged in a sandbox with a memory ceiling, and neither it nor nginx can reach the instance metadata service:

systemctl show whisper-server -p User -p ProtectSystem -p NoNewPrivileges -p MemoryMax -p IPAddressDeny
systemctl show nginx -p IPAddressDeny

MemoryMax=<75% of the instance memory, in bytes>
IPAddressDeny=168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32
User=whisper
ProtectSystem=strict
NoNewPrivileges=yes
IPAddressDeny=168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32

nginx passes the engine a fixed host name and the real client address from the connection, never values a client sends, and enforces the limits described above:

grep -E 'proxy_set_header (Host|X-Forwarded-For|X-Real-IP)|client_max_body_size|proxy_read_timeout|limit_conn whisper' /etc/nginx/sites-available/cloudimg-whisper-cpp-server

    client_max_body_size 25m;
    proxy_set_header Host 127.0.0.1:8080;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_read_timeout 900s;
        limit_conn whisper_perip 2;
        limit_conn whisper_total 4;

nginx also refuses an upload whose Origin is another web site (HTTP 403), because browsers resend Basic Auth credentials on cross site requests, and it removes the engine's permissive cross origin header from every response.

Step 9: Your own DNS name and certificate

Point a DNS record at the instance, then set the name the service is served as. This mints a new self signed certificate for the name, updates nginx and the URLs in /root/whisper-cpp-server-credentials.txt, and keeps the name across reboots:

sudo whisper-set-hostname <your-domain>

To use a certificate from your own certificate authority instead, copy it over the generated pair and reload nginx (running whisper-set-hostname again replaces it with a self signed one):

sudo install -m 0644 <your-domain>.crt /etc/ssl/whisper-cpp-server/whisper.crt
sudo install -m 0600 <your-domain>.key /etc/ssl/whisper-cpp-server/whisper.key
sudo nginx -t && sudo systemctl reload nginx

Step 10: Performance and sizing

whisper.cpp runs on the CPU, so speed scales with vCPUs. Measured on this image on a 2 vCPU instance (the same vCPU count as m5.large) with the bundled ggml-base.en model:

Input	Wall clock
3 second synthesised sentence	about 4.7 seconds (mostly fixed per request cost)
11 second recording (jfk.wav)	about 5.0 seconds
88 seconds of speech	about 24.5 seconds, roughly 3.5 times faster than real time

A 10 minute recording therefore takes about 3 minutes on m5.large. The engine uses about 470 MB of memory with this model. For sustained or batch work choose a compute optimised type with more vCPUs (c5.xlarge, c5.2xlarge); the engine uses every vCPU (WHISPER_THREADS=auto, up to 16). Requests are processed one at a time and the rest wait in turn; for more throughput run several instances behind a load balancer.

Step 11: Configuration, models and password rotation

The engine reads /etc/whisper-cpp-server/server.env:

Variable	Meaning	Image default
WHISPER_MODEL	GGML model file	/var/lib/whisper-cpp-server/models/ggml-base.en.bin
WHISPER_THREADS	CPU threads, or auto (vCPU count, at most 16)	auto
WHISPER_EXTRA_ARGS	Extra whisper-server options, e.g. --language auto with a multilingual model	empty

The bind address is fixed to 127.0.0.1:8080 by /usr/local/sbin/whisper-server-launch; keep it on loopback, because the engine has no authentication of its own. After editing the file run sudo systemctl restart whisper-server.

ggml-base.en balances accuracy and speed and is English only. Larger models are more accurate and slower; multilingual models drop the .en suffix. To use another model from the same repository (here ggml-small.en.bin):

sudo curl -fL -o /var/lib/whisper-cpp-server/models/ggml-small.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en.bin
sudo sed -i 's#^WHISPER_MODEL=.*#WHISPER_MODEL=/var/lib/whisper-cpp-server/models/ggml-small.en.bin#' /etc/whisper-cpp-server/server.env
sudo systemctl restart whisper-server

Check the licence of any model you add. The models in that repository are conversions of OpenAI's Whisper checkpoints (MIT), but community fine tunes elsewhere can carry terms that forbid commercial use.

The upload limit, the per request time limit and the concurrency limits are in /etc/nginx/sites-available/cloudimg-whisper-cpp-server (client_max_body_size, proxy_read_timeout, limit_conn); after a change run sudo nginx -t && sudo systemctl reload nginx.

To set a new password, give it to htpasswd on standard input (start the line with a space so most shells keep it out of your history); nginx uses it at once:

 printf '%s' '<new-password>' | sudo htpasswd -i -B -C 10 /etc/nginx/whisper-cpp-server.htpasswd whisper

Then update WHISPER_PASSWORD in /root/whisper-cpp-server-credentials.txt so it does not hold an old password.

Step 12: Verify the bundled model

The model matches the checksum recorded at build time:

sha256sum /var/lib/whisper-cpp-server/models/ggml-base.en.bin

a03779c86df3323075f5e796cb2ce5029f00ec8869eee3fdfb897afe36c6d002  /var/lib/whisper-cpp-server/models/ggml-base.en.bin

The licence notices and the provenance record:

ls /opt/whisper/
head -12 /opt/whisper/MODEL-PROVENANCE.txt

LICENSE.openai-whisper
LICENSE.whisper-cpp
MODEL-PROVENANCE.txt
whisper.cpp Server - bundled speech recognition model provenance
================================================================

Model file : ggml-base.en.bin
Location   : /var/lib/whisper-cpp-server/models/ggml-base.en.bin
Size       : 147964211 bytes
SHA-256    : a03779c86df3323075f5e796cb2ce5029f00ec8869eee3fdfb897afe36c6d002
SHA-1      : 137c40403d78fd54d454da0f9bd998f78703390c (as published in the source repository's model card)
Source     : https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base.en.bin

Provenance chain:

Step 13: Server components

Component	Path
whisper.cpp server 1.9.1	/usr/local/bin/whisper-server
Launcher (loopback bind)	/usr/local/sbin/whisper-server-launch
Engine settings	/etc/whisper-cpp-server/server.env
Model	/var/lib/whisper-cpp-server/models/ggml-base.en.bin
Test clips	/usr/local/share/whisper-cpp-server/samples/
Licences and provenance	/opt/whisper/
Service unit	/etc/systemd/system/whisper-server.service
nginx site	/etc/nginx/sites-available/cloudimg-whisper-cpp-server
Password store (bcrypt)	/etc/nginx/whisper-cpp-server.htpasswd
Certificate and key	/etc/ssl/whisper-cpp-server/
First boot script	/usr/local/sbin/whisper-cpp-server-firstboot.sh
Host name helper	/usr/local/sbin/whisper-set-hostname
Credentials	/root/whisper-cpp-server-credentials.txt (mode 0600)

/usr/local/bin/whisper-server --help 2>&1 | head -3
systemctl status whisper-server --no-pager | head -3

usage: /usr/local/bin/whisper-server [options] 

● whisper-server.service - whisper.cpp HTTP server (speech to text, loopback behind nginx; cloudimg)
     Loaded: loaded (/etc/systemd/system/whisper-server.service; enabled; preset: enabled)
    Drop-In: /etc/systemd/system/whisper-server.service.d

Troubleshooting

• HTTP 502 in the first minute after launch. The engine is still loading the model from the new EBS volume (about 20 to 40 seconds on a fresh instance); curl -sk https://127.0.0.1/health returns {"status":"ok"} once it is ready.
• Every request returns 401. Use this instance's own password from Step 4; passwords are never shared between instances. If the credentials file is missing, first boot did not finish: sudo journalctl -u whisper-cpp-server-firstboot --no-pager.
• HTTP 200 but an empty transcript. Silent audio correctly returns no text. Otherwise check the model against its checksum (Step 12).
• HTTP 413. The upload is over 25 MB; compress it or raise the limit (Steps 7 and 11).
• HTTP 429. You already have 2 transcriptions running from the same address (or 4 in total); wait for them to finish.
• HTTP 504 on a very long file. The transcription took longer than 15 minutes; split the audio, use a larger instance or raise proxy_read_timeout.
• The browser does not ask for the password again. Browsers cache Basic Auth for the session; open a private window.

Step 14: Security recommendations

• Allow ports 22, 80 and 443 in the security group from your own networks only.
• Keep the per instance password secret, and rotate it if it may have been shared (Step 11).
• Use your own DNS name and a certificate your clients trust (Step 9).
• Keep the engine on loopback, and check the licence of any model you add.
• Keep the operating system patched; unattended security upgrades are on, and sudo apt-get update && sudo apt-get upgrade applies everything else.

Support and licensing

whisper.cpp is open source under the MIT licence, and the bundled ggml-base.en model is MIT licensed by OpenAI; there is no per user or per CPU licence fee. The AMI price covers cloudimg's hardening, first boot setup and support.

• Email: support@cloudimg.co.uk
• Website: https://www.cloudimg.co.uk
• Support: 24/7, with a 24 hour response SLA