Artificial Intelligence (AI) AWS

Ollama 0.40.6 on AWS User Guide

| Product: Ollama 0.40.6 on Ubuntu 24.04 on AWS

Overview

This guide covers launching and verifying Ollama 0.40.6 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other LLM servers, gateways and chat interfaces in the listing are described on the listing's guide hub.

Ollama is an open-source (MIT) server for running open-weights large language models. It has its own REST API (/api/generate, /api/chat, /api/embed, /api/tags, /api/pull and more) and an OpenAI-compatible API (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models), so code written for the OpenAI SDK, LangChain or LlamaIndex can use it with a base URL and a credential while prompts and answers stay inside your VPC.

This image runs Ollama on the CPU, so it needs no GPU. The Qwen2.5 1.5B Instruct model (qwen2.5:1.5b, Apache 2.0) is pulled at build time and ships on the image, so the API answers as soon as the instance is up; you pull larger models for real work (Step 9).

What is on the image:

  • Ollama 0.40.6 at /usr/local/bin/ollama, rebuilt by cloudimg from the upstream v0.40.6 source with Go 1.26.10 and patched Go modules (the build record is in /usr/share/doc/cloudimg/ollama/), run by systemd as the unprivileged ollama user
  • Upstream's 0.40.6 CPU runtime libraries in /usr/local/lib/ollama (the llama.cpp backends for every x86-64 CPU generation, with their licence texts), taken from the checksum-verified release archive; the CUDA, ROCm and Vulkan GPU runtimes are not installed
  • The qwen2.5:1.5b model in /var/lib/ollama/models, pinned to its published manifest and file digests, with its licence in /usr/share/doc/cloudimg/ollama-models/
  • Ollama bound to loopback only (127.0.0.1:11434) and nginx as the only network-facing service
  • HTTPS on port 443 with a self-signed certificate created on your instance's first boot; port 80 redirects to HTTPS
  • A per-instance login (HTTP Basic, user admin) on every API path, generated on first boot into a root-only file
  • ollama-set-hostname, a helper that serves Ollama under your own DNS name and re-issues the certificate for it

Default user: ubuntu · Architecture: x86_64 · Root volume: 30 GiB gp3 (models live in /var/lib/ollama/models)

Security posture

  • No default credential. The login password is generated on each instance's first boot and stored only as a bcrypt hash in nginx and in a root-only file. It is not in the image, so no two instances share one. Ollama's own identity key (/var/lib/ollama/.ollama/id_ed25519) is also created per instance.
  • Every path needs the credential. Ollama has no authentication of its own, so it listens on loopback only and nginx asks for the login on every path except the version check GET /api/version and nginx's GET /healthz. A wrong password is refused, and the client's Authorization header is never passed to Ollama.
  • Cloud features are off. OLLAMA_NO_CLOUD=1 turns off Ollama's cloud models, sign-in and the web search and web fetch proxies, so the instance only talks to the model registry when you pull a model.
  • No instance metadata for Ollama. The ollama service, every model runner it starts, and nginx are denied the instance metadata service by systemd IPAddressDeny, so no pull or create request can make Ollama read the instance's role credentials, even one sent with "insecure": true.
  • Registry redirects are checked. Ollama 0.40.6 refuses a registry redirect that downgrades HTTPS to HTTP or points a pull at a different, non-public host; older releases (before 0.34.2) followed such redirects to any address.
  • Fails closed. nginx will not start until first boot has created the certificate and the credential, and first boot stops nginx again if its own check of the credential gate fails.
  • Only nginx faces the network. It answers only for the configured hostname (any other Host header gets its connection closed), uses TLS 1.2 or 1.3, refuses cross-origin POST requests and WebSocket upgrades (403), strips CORS headers, and replaces any client-supplied X-Forwarded-For or X-Real-IP header with the real peer address. OLLAMA_ORIGINS is not set.
  • The hostname comes from configuration, never from a request. First boot reads it from EC2 instance metadata (IMDSv2), or from /etc/cloudimg/ollama-hostname if you set one.
  • Patched build. Every upstream Ollama release up to 0.40.6 is built with Go 1.26.0 and carries 84 known Go advisories (in net/http, crypto/tls, crypto/x509, golang.org/x/net and others). The cloudimg binary is the same 0.40.6 source built with Go 1.26.10 and the fixed module releases; one advisory remains, GO-2026-5932 for the deprecated golang.org/x/crypto/openpgp package, which has no fixed release and which Ollama does not use (Step 10 shows the record).
  • IMDSv2 required. The AMI is registered with ImdsSupport v2.0, so instances launched from it refuse token-less (IMDSv1) metadata requests.
  • Per-instance identity. SSH host keys and the machine ID are generated fresh on every instance.

Prerequisites

  1. An AWS account and permission to launch EC2 instances
  2. An EC2 key pair in the Region you launch in

Recommended instance type: m5.large (2 vCPU, 8 GiB) for the bundled 1.5B model. Ollama runs models on the CPU and keeps them in memory, and generation speed scales with vCPUs and memory bandwidth. As a measured reference, on 4 vCPUs (Intel Xeon Platinum 8259CL) the 1.5B model generates about 8 tokens per second and reads prompts at about 60 tokens per second; expect roughly half that on m5.large. For 3B to 8B models choose m5.xlarge (16 GiB) or larger, and for interactive use of 7B to 8B models a compute optimised type such as c5.4xlarge or c6i.4xlarge. The software charge is per vCPU.

Connecting to your instance

Version Operating system SSH user
Ollama 0.40.6 on Ubuntu 24.04 Ubuntu 24.04 LTS ubuntu

Step 1: Launch the instance

Option A: AWS Marketplace console

  1. Open the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
  2. Once the subscription is active, choose Launch your software.
  3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose Ollama 0.40.6 on Ubuntu 24.04, then choose your Region.
  4. Pick an instance type (m5.large for the starter model; larger for real models) and your key pair. Increase the root volume if you plan to pull several large models.
  5. Security group: the listing's default group opens SSH (22), HTTP (80) and HTTPS (443). Restrict SSH to your own IP address, and restrict 80 and 443 to the networks your users and applications connect from.
  6. Choose your VPC and subnet, leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and HTTPS from your application network, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDR ranges with your own:

SG=$(aws ec2 create-security-group --group-name ollama \
  --description "Ollama LLM API" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr 10.0.0.0/16
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80 --cidr 10.0.0.0/16
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --subnet-id subnet-0123456789abcdef0 \
  --security-group-ids "$SG" \
  --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=60,VolumeType=gp3}' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=ollama}]'

First boot takes about a minute after the instance is running: it creates the certificate and the login, starts Ollama and nginx, and checks the login gate before it finishes.

Step 2: Connect and find your credentials

Connect with SSH as ubuntu:

ssh -i my-key-pair.pem ubuntu@<public-dns-name>

The login is in a root-only file. These commands print the hostname, URL and user name, list the keys in the file without their values, and show its permissions:

sudo grep -E '^OLLAMA_(HOST|URL|USERNAME)=' /root/ollama-credentials.txt
sudo grep -oE '^OLLAMA_[A-Z]+=' /root/ollama-credentials.txt
sudo stat -c '%a %U:%G' /root/ollama-credentials.txt

OLLAMA_HOST=ec2-3-82-98-249.compute-1.amazonaws.com
OLLAMA_URL=https://ec2-3-82-98-249.compute-1.amazonaws.com/
OLLAMA_USERNAME=admin
OLLAMA_HOST=
OLLAMA_URL=
OLLAMA_USERNAME=
OLLAMA_PASSWORD=
600 root:root

To read the password, run sudo cat /root/ollama-credentials.txt in your own session and store it in your secrets manager. The self-signed certificate your clients need to trust is /etc/ssl/ollama/ollama.crt.

Step 3: Confirm Ollama is running

Ollama listens on loopback only; nginx serves ports 80 and 443. The version check is open:

systemctl is-active ollama.service nginx.service
ollama --version
sudo ss -Htlnp | grep -E ':(22|80|443|11434) ' | awk '{split($6,a,"\""); print $4, a[2]}' | sort -u
H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" "https://$H/api/version"; echo

active
active
ollama version is 0.40.6
0.0.0.0:22 sshd
0.0.0.0:443 nginx
0.0.0.0:80 nginx
127.0.0.1:11434 ollama
[::]:22 sshd
[::]:443 nginx
[::]:80 nginx
{"version":"0.40.6"}

An EBS volume created from an AMI fetches each block from Amazon S3 the first time it is read, so loading a model for the first time on a new instance is slow (we measured one to over three minutes for the 1 GB starter model). First boot therefore reads the shipped model once in the background (ollama-model-prewarm.service, at idle I/O priority); on a t3.large this took about three and a half minutes. A request sent before it finishes still works; it just waits for the read. This waits up to about three minutes for it to finish and shows how long it took:

for i in $(seq 1 34); do [ -f /var/lib/cloudimg/ollama-prewarm.done ] && break; sleep 5; done
cat /var/lib/cloudimg/ollama-prewarm.done 2>/dev/null || echo "model pre-read still running"

bytes=986062749 seconds=212

Models you pull later are written to the volume directly, so they load at full speed.

Step 4: Prove the credential gate

Requests without the login are refused on every API path, and so is a wrong password. An unknown Host header gets no answer (curl reports 000), and plain HTTP redirects to the configured HTTPS name:

H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
C=(-s -o /dev/null --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1")
for p in /api/tags /api/ps /v1/models; do echo "anonymous $p -> $(curl "${C[@]}" -w '%{http_code}' "https://$H$p")"; done
echo "wrong password -> $(curl "${C[@]}" -w '%{http_code}' -u admin:wrong-password "https://$H/api/tags")"
echo "unknown Host -> $(curl -sk -o /dev/null -w '%{http_code}' -H 'Host: evil.example' https://127.0.0.1/api/version || true)"
curl -s -o /dev/null -w 'plain HTTP -> %{http_code} %{redirect_url}\n' -H "Host: $H" http://127.0.0.1/api/tags

anonymous /api/tags -> 401
anonymous /api/ps -> 401
anonymous /v1/models -> 401
wrong password -> 401
unknown Host -> 000
plain HTTP -> 301 https://ec2-3-82-98-249.compute-1.amazonaws.com/api/tags

Step 5: List the installed models

H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" "https://$H/api/tags" \
  | jq '.models[] | {name, size, digest: .digest[0:12], details: {family: .details.family, parameter_size: .details.parameter_size, quantization_level: .details.quantization_level}}'

{
  "name": "qwen2.5:1.5b",
  "size": 986061892,
  "digest": "65ec06548149",
  "details": {
    "family": "qwen2",
    "parameter_size": "1.5B",
    "quantization_level": "Q4_K_M"
  }
}

The digest is the published manifest of qwen2.5:1.5b in the Ollama library; the image build refuses any other.

Step 6: Generate text

With Ollama's native API:

H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" "https://$H/api/generate" \
  -d '{"model":"qwen2.5:1.5b","prompt":"In one sentence, what is a reverse proxy?","stream":false,"options":{"seed":42,"temperature":0}}' \
  | jq '{response, eval_count, tokens_per_second: (.eval_count / (.eval_duration / 1e9) | floor)}'

{
  "response": "A reverse proxy is a server that acts as an intermediary between clients and other servers, managing requests and routing them to the appropriate destination.",
  "eval_count": 28,
  "tokens_per_second": 9
}

With the OpenAI-compatible API:

H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" "https://$H/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen2.5:1.5b","messages":[{"role":"user","content":"Name three planets. Answer with a comma separated list only."}],"temperature":0,"seed":42}' \
  | jq '{model, content: .choices[0].message.content, usage}'

{
  "model": "qwen2.5:1.5b",
  "content": "Mercury, Venus, Earth",
  "usage": {
    "prompt_tokens": 41,
    "prompt_tokens_details": {
      "cached_tokens": 24
    },
    "completion_tokens": 7,
    "total_tokens": 48
  }
}

The tokens_per_second figure is from a 4 vCPU instance; your figure depends on the instance type (see Prerequisites).

Step 7: Call Ollama from your application

Copy /etc/ssl/ollama/ollama.crt to your client (or replace it with a certificate from your own CA, Step 8) and send the login with every request. With the OpenAI Python SDK:

import httpx
from openai import OpenAI

client = OpenAI(
    base_url="https://ollama.example.com/v1",
    api_key="unused",  # Ollama ignores it; nginx checks the Basic login below
    http_client=httpx.Client(verify="ollama.crt", auth=("admin", "<OLLAMA_PASSWORD>")),
)
reply = client.chat.completions.create(
    model="qwen2.5:1.5b",
    messages=[{"role": "user", "content": "Summarise our VPN policy in two lines."}],
)
print(reply.choices[0].message.content)

Ollama's own client libraries and tools that speak the Ollama API (OLLAMA_HOST=https://ollama.example.com) work the same way when they can send HTTP Basic credentials. Keep stream on for chat front ends: nginx passes streamed responses through without buffering.

Step 8: Serve Ollama under your own DNS name

Point a DNS record at the instance, then run:

sudo ollama-set-hostname ollama.example.com

ollama-set-hostname: Ollama is served at https://ollama.example.com/

This re-issues the self-signed certificate for the new name (keeping the instance's own addresses in it), makes nginx answer for it and updates the credentials file. To use a certificate from your own CA or from Let's Encrypt instead, replace /etc/ssl/ollama/ollama.crt and /etc/ssl/ollama/ollama.key (keep the key mode 0600, owner root) and run sudo systemctl reload nginx.

Step 9: Add models

Pull models from the Ollama library through the API (any authenticated client) or with the ollama command on the instance, which talks to the local server:

ollama pull qwen2.5:7b
ollama list

Or remotely:

curl --cacert ollama.crt -u "admin:<OLLAMA_PASSWORD>" https://ollama.example.com/api/pull -d '{"model":"qwen2.5:7b"}'

Check each model's licence before you use it (ollama show <model> --license); many open-weights models carry their own terms. A 4-bit quantised model needs roughly 0.6 GB of memory per billion parameters plus the context, and its files go to /var/lib/ollama/models on the root volume; grow the volume in EC2 and run sudo growpart /dev/nvme0n1 1 && sudo resize2fs /dev/nvme0n1p1 if you need more space. Remove a model with ollama rm <model>.

A pull with "insecure": true (or ollama pull --insecure) is for your own private registry: it allows plain HTTP and turns off Ollama's private-address checks for that pull. Instance metadata stays blocked either way. If your Ollama should never reach your private networks at all, add a systemd drop-in that denies them:

sudo tee /etc/systemd/system/ollama.service.d/60-private-deny.conf >/dev/null <<'EOF'
[Service]
IPAddressDeny=10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 fd00::/8
EOF
sudo systemctl daemon-reload && sudo systemctl restart ollama.service

Step 10: Check the security controls

The cloud proxies are switched off, a pull aimed at the instance metadata address dies at the connection (systemd drops the packets), and the metadata service answers your shell but not the Ollama service:

H=$(sudo grep '^OLLAMA_HOST=' /root/ollama-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-)
C=(-s -o /dev/null --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW")
for p in /api/experimental/web_fetch /api/experimental/web_search; do
  echo "POST $p -> $(curl "${C[@]}" -w '%{http_code}' -X POST -d '{"url":"http://169.254.169.254/latest/meta-data/","query":"x"}' "https://$H$p")"
done
curl -s -m 60 --cacert /etc/ssl/ollama/ollama.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" "https://$H/api/pull" \
  -d '{"model":"169.254.169.254/library/probe:latest","insecure":true,"stream":false}'; echo
systemctl show -p IPAddressDeny --value ollama.service
CG=$(systemctl show -p ControlGroup --value ollama.service)
curl -s -m 5 -o /dev/null -w 'metadata from your shell -> %{http_code}\n' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token
sudo bash -c "echo \$\$ > /sys/fs/cgroup$CG/cgroup.procs; exec curl -s -m 5 -o /dev/null -w 'metadata from inside ollama.service -> %{http_code}\n' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token" || true

POST /api/experimental/web_fetch -> 403
POST /api/experimental/web_search -> 403
{"error":"pull model manifest: Get \"http://169.254.169.254/v2/library/probe/manifests/latest\": dial tcp 169.254.169.254:80: i/o timeout"}
168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128
metadata from your shell -> 200
metadata from inside ollama.service -> 000

The build record names the exact source, toolchain and module versions of the binary you are running:

cat /usr/share/doc/cloudimg/ollama/BUILD-RECORD.txt
sha256sum /usr/local/bin/ollama

cloudimg d-102 dependency rebuild of the ollama binary
source        https://github.com/ollama/ollama/archive/refs/tags/v0.40.6.tar.gz
source sha256 5adad63f326e5a3911d5248e923ac03def72b6ba5ba80fe1c6089cf26b9291c3
toolchain     go1.26.10 linux/amd64, official tarball https://go.dev/dl/go1.26.10.linux-amd64.tar.gz
toolchain sha256 ccfe7ae822d2c2f41f4fe7c956199967a4dac22d0ee0d2b2bfdb440aab3b20c5
cgo           CGO_ENABLED=1 CGO_CFLAGS=-O3 CGO_CXXFLAGS=-O3, gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
build         go build -trimpath -buildmode=pie -ldflags "-w -s -X version.Version=0.40.6 -X server.mode=release"
bumps         golang.org/x/net v0.46.0 v0.60.0; golang.org/x/crypto v0.43.0 v0.57.0; golang.org/x/image v0.22.0 v0.45.0; golang.org/x/text v0.30.0 v0.42.0; golang.org/x/mod v0.30.0 v0.41.0; github.com/klauspost/compress v1.18.3 v1.18.7; github.com/buger/jsonparser v1.1.1 v1.1.2; 
transitive    golang.org/x/sync golang.org/x/sys golang.org/x/term 
govulncheck   v1.9.0 -mode=binary: GO-2026-5932 
binary sha256 75ae92ec4a5748ff9c1ffc163086f77626aab4514f2efb19396dd893163b52a3
75ae92ec4a5748ff9c1ffc163086f77626aab4514f2efb19396dd893163b52a3  /usr/local/bin/ollama

The full govulncheck report is next to it in govulncheck.txt. The remaining GO-2026-5932 covers the deprecated golang.org/x/crypto/openpgp package, which has no fixed release; Ollama does not import it (the binary is stripped, so govulncheck reports the whole module).

Step 11: Rotate the login password

Generate a new password, write its bcrypt hash for nginx (the password goes in on standard input, never on a command line), update the credentials file and reload nginx:

NEW=$(openssl rand -base64 48 | tr -dc 'A-Za-z0-9' | cut -c1-24)
printf '%s\n' "$NEW" | sudo htpasswd -iB /etc/nginx/.ollama.htpasswd admin
sudo sed -i "s/^OLLAMA_PASSWORD=.*/OLLAMA_PASSWORD=$NEW/" /root/ollama-credentials.txt
sudo systemctl reload nginx
unset NEW

Maintenance

  • Operating system updates: unattended-upgrades applies Ubuntu security updates daily. Run sudo apt update && sudo apt full-upgrade for everything else and reboot when a new kernel is installed.
  • Ollama updates: the image runs exactly 0.40.6 and does not update itself. cloudimg publishes new versions of this listing as Ollama releases; to move to one, launch the new version and copy /var/lib/ollama/models across (or pull your models again).
  • Logs: journalctl -u ollama.service, journalctl -u ollama-firstboot.service and /var/log/nginx/.
  • Memory: sudo systemctl edit ollama.service lets you set Ollama's own variables, for example Environment=OLLAMA_KEEP_ALIVE=30m (how long a model stays loaded) or Environment=OLLAMA_CONTEXT_LENGTH=8192.

Troubleshooting

  • 401 with the right password: check you send it as HTTP Basic for user admin and that the client sends it to the configured hostname (sudo grep ^OLLAMA_HOST= /root/ollama-credentials.txt).
  • Connection closed with no response: the request's Host does not match a configured name. Use the instance's public DNS name or IP, or set your own with ollama-set-hostname.
  • Certificate errors: trust /etc/ssl/ollama/ollama.crt on the client, or install your own certificate (Step 8).
  • nginx is not running after boot: first boot did not finish; see journalctl -u ollama-firstboot.service. Remove /var/lib/cloudimg/ollama-firstboot.done and run sudo systemctl restart ollama-firstboot.service to retry.
  • Slow or failed generation: the model may not fit in memory (free -m, journalctl -u ollama.service); use a smaller model or a larger instance type.

Support

cloudimg provides support for this image by email at support@cloudimg.co.uk. Include the instance ID, the Region and the version you launched.