whisper.cpp Server on AWS User Guide

Overview

This guide covers deploying and using whisper.cpp Server on Amazon Web Services with cloudimg's preconfigured Amazon Machine Image.

whisper.cpp is a high-performance implementation of OpenAI's Whisper automatic speech recognition model, written in plain C and C++. It runs entirely on the CPU, with no GPU, no Python runtime and no external service dependencies. This image ships the project's bundled HTTP server, so the instance becomes a private speech-to-text service the moment it boots: post an audio file and receive a transcript back.

Because everything runs locally, audio never leaves the instance you control, which suits recordings that cannot be sent to a third-party transcription service.

What's included:

• whisper.cpp 1.9.1, built from the pinned upstream source tag
• The bundled whisper.cpp HTTP server, listening on the loopback interface only
• The ggml-base.en speech recognition model, preinstalled and verified by checksum
• nginx as an authenticating reverse proxy on port 80
• ffmpeg, for automatic conversion of common audio and video formats
• A short test clip, so you can prove the service works before using your own audio
• 24/7 cloudimg support

Key facts:

Item	Value
Platform	Amazon Web Services (EC2)
Operating system	Ubuntu 24.04 LTS
Default SSH user	ubuntu
Web interface and API	Port 80, HTTP Basic authentication
Inference engine	127.0.0.1:8080, loopback only, not reachable from the network
Model file	/var/lib/whisper-cpp-server/models/ggml-base.en.bin
Credentials file	/root/whisper-cpp-server-credentials.txt
Licence notices	/opt/whisper/

About the bundled model

whisper.cpp is a program, not a model. It needs a separate model file to do anything, and that file carries its own licence. This image ships ggml-base.en, and its provenance was traced to its root rather than assumed from a repository badge:

This image includes the ggml-base.en speech recognition model, a format conversion of OpenAI's original Whisper base.en checkpoint into the GGML format used by whisper.cpp. It is not a fine tune and not a distillation of any third party model. Both OpenAI's Whisper weights and whisper.cpp itself are distributed under the MIT License, and copies of both licences ship in the image at /opt/whisper/, together with a written provenance record.

The model file is pinned by name and by SHA-256 checksum at build time, so the image cannot silently ship a different model.

Prerequisites

Before deploying, ensure you have:

1. An active AWS account
2. A subscription to this product in AWS Marketplace
3. An EC2 key pair for SSH access in your target region
4. Permission to launch EC2 instances and manage security groups

Recommended instance type: m5.large (2 vCPU, 8 GB RAM). See Performance for what that delivers and when to choose something larger.

Connecting to your instance

Connect over SSH as the default login user for the operating system variant you launched:

OS variant	SSH login user
Ubuntu 24.04	ubuntu

Security group configuration

Protocol	Port	Description
TCP	22	SSH access
TCP	80	whisper.cpp web interface and transcription API, authenticated

Port 8080 is deliberately not in this table. The inference engine binds to the loopback interface only and is never reachable from the network. See Security model.

Step 1: Launch the instance

Option A: AWS Marketplace console

1. In AWS Marketplace, search for whisper.cpp Speech to Text Server and subscribe.
2. Click Continue to Configuration, choose your region and the software version, then Continue to Launch.
3. Choose the instance type (m5.large or larger), your VPC and subnet, and your EC2 key pair.
4. Under the security group, allow SSH (22) and HTTP (80).
5. Launch the instance.

Option B: AWS CLI

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name my-key \
  --security-group-ids sg-xxxxxxxx \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=my-whisper}]'

Replace <ami-id> with the AMI ID shown on the Marketplace launch page for your region, and use a security group that allows TCP 22 and 80.

Step 2: Connect via SSH

ssh ubuntu@<public-ip>

Step 3: Retrieve your credentials

On the first boot of every instance, this image generates a password that is unique to that instance. Nothing is shared between deployments and no password is baked into the image. Read it with:

sudo cat /root/whisper-cpp-server-credentials.txt

The file is readable by root only, and contains your username, your password and the URL of your service:

WHISPER_USER=whisper
WHISPER_PASSWORD=...
WHISPER_URL=http://<public-ip>/
WHISPER_API_URL=http://<public-ip>/inference

Confirm the file exists with the correct ownership and permissions without printing the password itself:

sudo ls -l /root/whisper-cpp-server-credentials.txt

Step 4: Confirm the service is running

sudo systemctl status whisper-server.service --no-pager

sudo systemctl status nginx.service --no-pager

Both units should report active (running).

Check that the inference engine is answering on the loopback interface:

curl -s -o /dev/null -w 'health: HTTP %{http_code}\n' http://127.0.0.1:8080/health

Expected output:

health: HTTP 200

Step 5: Transcribe the bundled test clip

The image ships a short test clip so you can prove the service works end to end before introducing your own audio. It was synthesised on the build machine with eSpeak NG, so no third-party audio is bundled with this image.

Transcribe it, substituting the password from Step 3:

sudo curl -s -u "whisper:<your-password>" \
  -F file=@/usr/local/share/whisper-cpp-server/samples/cloudimg-test.wav \
  -F response_format=json \
  http://127.0.0.1/inference

Expected output:

{"text":" A quick brown fox jumps over the lazy dog.\n"}

If you get a transcript back containing those words, the model loaded correctly and the whole path works. An empty text value with an HTTP 200 would mean the model failed to load, so check the transcript content rather than the status code.

Step 6: Use the browser interface

Open http://<public-ip>/ in a browser. You will be prompted for the username and password from Step 3. Once authenticated, the interface documents the API and provides an upload form.

Choose an audio file, pick a response format and click Submit.

The transcript is returned directly in the browser.

Note that the interface page documents a /load endpoint. That endpoint is deliberately blocked on this image and returns HTTP 403. See Security model for why.

Step 7: Transcribe your own audio

Copy a file to the instance, then transcribe it. ffmpeg is installed and the server converts input automatically, so MP3, M4A, FLAC, OGG, WAV and the audio track of most video files are all accepted. Run these from your workstation, replacing the placeholders:

scp recording.mp3 ubuntu@<public-ip>:/tmp/
curl -s -u "whisper:<your-password>" \
  -F file=@/tmp/recording.mp3 \
  -F response_format=text \
  http://<public-ip>/inference

Response formats

Pass response_format to choose the output shape:

Value	Output
text	Plain transcript text
json	{"text": "..."}
verbose_json	Transcript plus per segment and per word timings and probabilities
srt	SubRip subtitles
vtt	WebVTT subtitles

Generate subtitles for a recording:

sudo curl -s -u "whisper:<your-password>" \
  -F file=@/usr/local/share/whisper-cpp-server/samples/cloudimg-test.wav \
  -F response_format=srt \
  http://127.0.0.1/inference

Expected output:

1
00:00:00,000 --> 00:00:03,600
 A quick brown fox jumps over the lazy dog.

Performance

whisper.cpp runs on the CPU, so transcription speed scales with the number of vCPUs. With the bundled ggml-base.en model on m5.large (2 vCPU), a few-second clip returns in roughly two to three seconds, dominated by a fixed per-request startup cost, and sustained speech transcribes several times faster than real time.

Guidance for choosing a size:

• m5.large is a good fit for occasional transcription, evaluation and low-volume API use.
• For sustained or batch transcription, choose a compute-optimised size such as c5.2xlarge or larger. Throughput scales roughly with vCPU count.
• Memory is not the constraint with this model. The ggml-base.en model uses a few hundred megabytes, and 8 GB of RAM is ample.

Requests are processed one at a time. If you need concurrency, place a queue in front of the API or run several instances behind a load balancer.

Using a different model

ggml-base.en balances accuracy and speed, and is English only. Larger models are more accurate and slower; multilingual models drop the .en suffix. To use a different one, download it, point the service unit at it and restart. On the instance:

sudo curl -fL -o /var/lib/whisper-cpp-server/models/ggml-small.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en.bin
sudo chown whisper:whisper /var/lib/whisper-cpp-server/models/ggml-small.en.bin
# edit ExecStart in /etc/systemd/system/whisper-server.service to reference the new file
sudo systemctl daemon-reload
sudo systemctl restart whisper-server.service

Check the licence of any model you add. The models in the repository above are conversions of OpenAI's original Whisper checkpoints and are MIT licensed, but models elsewhere, including community fine tunes, may carry more restrictive terms that prohibit commercial use.

Security model

whisper.cpp's bundled server has no authentication of any kind, and the upstream project's own documentation warns against exposing it. It accepts user file uploads and shells out to ffmpeg for conversion, and its /load endpoint allows a caller to load an arbitrary server-side file path as a model. Left open on a public IP address, that is an abuse and cost surface, and transcription endpoints handle audio that is often sensitive.

This image does not expose it. The protections below are applied in the image and verified on a freshly launched instance before the image is published:

1. The inference engine binds to the loopback interface only. It listens on 127.0.0.1:8080 and is not reachable from the network, whatever your security group allows.
2. Every path requires authentication. nginx fronts the engine on port 80 and demands HTTP Basic credentials. There is no anonymous route to any endpoint.
3. The /load endpoint is blocked outright. It returns HTTP 403 even with valid credentials, so the remote model-swap primitive is unreachable rather than merely protected.
4. No default credential exists. The password is generated on each instance's first boot. Until that happens the credential store is empty, which fails closed: every request is refused.
5. The service runs unprivileged as the whisper system user, under systemd hardening including NoNewPrivileges, ProtectSystem=strict and PrivateTmp.
6. Audio and transcripts are not persisted. Uploads are handled per request and nothing but the model file remains on disk afterwards.

You can confirm points 1 to 3 yourself. The engine must refuse a connection on the instance's own routable address:

curl -s -o /dev/null -m 5 -w '%{http_code}\n' http://$(hostname -I | awk '{print $1}'):8080/health || echo "connection refused, as expected"

An anonymous request must be refused:

curl -s -o /dev/null -w 'anonymous: HTTP %{http_code}\n' http://127.0.0.1/inference

A wrong password must be refused:

curl -s -o /dev/null -w 'wrong password: HTTP %{http_code}\n' -u whisper:definitely-not-the-password http://127.0.0.1/

And the model-swap endpoint must be blocked even with the correct credentials:

sudo curl -s -o /dev/null -w 'load endpoint: HTTP %{http_code}\n' -u "whisper:<your-password>" http://127.0.0.1/load

Expected output from those four commands:

connection refused, as expected
anonymous: HTTP 401
wrong password: HTTP 401
load endpoint: HTTP 403

Recommended additional hardening

• Restrict port 80 by source address in your security group so only your own networks can reach the service.
• Add TLS. Basic authentication sends credentials in a reversible encoding, so on any untrusted network you should terminate TLS in front of it. The nginx configuration at /etc/nginx/sites-available/cloudimg-whisper-cpp-server is ready for a certificate.
• Change the password if you wish, using sudo htpasswd /etc/nginx/whisper-cpp-server.htpasswd whisper, and record the new value safely. nginx picks the change up immediately with no reload.
• Keep the engine on loopback. Do not change --host 127.0.0.1 in the service unit.

Verifying the bundled model

The model shipped with this image can be checked against the checksum recorded at build time:

sha256sum /var/lib/whisper-cpp-server/models/ggml-base.en.bin

Expected output:

a03779c86df3323075f5e796cb2ce5029f00ec8869eee3fdfb897afe36c6d002  /var/lib/whisper-cpp-server/models/ggml-base.en.bin

The licence notices and the written provenance record are here:

ls -l /opt/whisper/

cat /opt/whisper/MODEL-PROVENANCE.txt

Server components

Component	Version	Install path
whisper.cpp server	1.9.1	/usr/local/bin/whisper-server
Speech recognition model	ggml-base.en	/var/lib/whisper-cpp-server/models/ggml-base.en.bin
nginx	1.24	/usr/sbin/nginx
ffmpeg	6.1.1	/usr/bin/ffmpeg
eSpeak NG	1.51	/usr/bin/espeak-ng

Filesystem layout

Key directories:

Path	Purpose
/var/lib/whisper-cpp-server/models/	Speech recognition model files
/usr/local/share/whisper-cpp-server/samples/	The bundled test clip
/opt/whisper/	MIT licence notices and the model provenance record
/etc/nginx/whisper-cpp-server.htpasswd	Credential store for the reverse proxy
/root/whisper-cpp-server-credentials.txt	Per-instance credentials, root only

Managing the service

sudo systemctl status whisper-server.service --no-pager

Restart the inference engine:

sudo systemctl restart whisper-server.service

View the engine log:

sudo journalctl -u whisper-server.service --no-pager -n 20

On startup

On the first boot only, whisper-cpp-server-firstboot.service runs before the application and nginx start. It generates the password unique to this instance, writes it into the reverse proxy credential store and into the root-only credentials file, resolves the instance's public address (via EC2 IMDSv2) for the login banner, and then writes its completion marker. Subsequent boots skip it, so your password is never regenerated behind your back.

Read the first boot log with:

sudo cat /var/log/cloudimg-firstboot.log

Troubleshooting

Every request returns HTTP 401

Confirm you are using the password from this instance. Passwords are per-instance and are not shared between deployments:

sudo grep -c WHISPER_PASSWORD /root/whisper-cpp-server-credentials.txt

If the credentials file is missing, first boot did not complete. Check the log:

sudo journalctl -u whisper-cpp-server-firstboot.service --no-pager -n 20

A request returns HTTP 200 but the transcript is empty

An empty transcript with a healthy status usually means the model file is missing or corrupt. Verify it against its checksum as shown in Verifying the bundled model. Genuinely silent audio also returns an empty transcript, which is correct behaviour, so confirm your input actually contains speech.

The /load endpoint returns HTTP 403

This is intentional. The endpoint is blocked on this image because it allows an arbitrary server-side file to be loaded as a model.

Transcription is slower than expected

Consider a compute-optimised instance type. See Performance.

The browser will not prompt for credentials again

Browsers cache Basic authentication credentials for the session. Open a private window to be prompted again.

Support

For assistance, contact cloudimg support:

• Email: support@cloudimg.co.uk
• Response time: 24/7 with a guaranteed 24 hour response SLA
• Website: https://www.cloudimg.co.uk