LocalAI 4.11.0 on AWS User Guide

Overview

This guide covers launching and verifying LocalAI 4.11.0 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other LLM servers, gateways and chat interfaces in the listing are described on the listing's guide hub.

LocalAI is an open-source (MIT) inference server that exposes an OpenAI-compatible REST API (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models and more) and a built-in web UI for chatting with your models. Code written for the OpenAI SDK, LangChain or LlamaIndex can point at it with a base URL and a credential, while prompts and answers stay inside your VPC.

This image runs LocalAI on the CPU with the llama.cpp backend, so it needs no GPU. A small open-weights instruct model, SmolLM2-135M-Instruct (Apache 2.0), is installed at build time so the API and the web UI answer as soon as the instance is up; you add your own GGUF models for real work (Step 9).

What is on the image:

• LocalAI 4.11.0 at /usr/local/bin/local-ai, compiled by cloudimg from the upstream v4.11.0 source with Go 1.26.9 and patched Go dependencies (the build record is in /usr/share/doc/cloudimg/localai/build-provenance/), run by systemd as the unprivileged localai user
• The upstream v4.11.0 cpu-llama-cpp backend, checked against its published image digest at build time, and the SmolLM2-135M-Instruct Q4_K_M model, both on a dedicated 40 GiB gp3 data volume at /var/lib/localai
• LocalAI bound to loopback only (127.0.0.1:8080) with its own per-instance API key, and nginx as the only network-facing service
• HTTPS on port 443 with a self-signed certificate created on your instance's first boot; port 80 redirects to HTTPS
• A per-instance login (HTTP Basic, user admin) on the web UI and on the API, generated on first boot into a root-only file
• localai-set-hostname, a helper that serves LocalAI under your own DNS name and re-issues the certificate for it

Default user: ubuntu · Architecture: x86_64 · Data volume: 40 GiB gp3 at /var/lib/localai

Security posture

• No default credential. The login password and LocalAI's internal API key are generated on each instance's first boot. The password is stored only as a bcrypt hash in nginx and in a root-only file; the key only in two root-only files. Neither is in the image, so no two instances share one.
• Every path needs the credential. nginx asks for the login on the web UI, on /v1 and on every other LocalAI path, then hands LocalAI its own API key; the client's Authorization header is never passed through. LocalAI itself also refuses every API route without its key (deny-by-default since LocalAI 4.9.0). Only nginx's own GET /health is open, for load balancer health checks.
• Model and backend installation is off. The model gallery and the backend gallery are empty and their endpoints disabled, the runtime settings API is off (so they cannot be switched back on over the API), backend installs require an integrity check, and nginx refuses every install route (/models/apply, /backends/apply, /api/backends/install-external, /api/models/install/..., /api/settings) with 403. Agents, MCP and the built-in assistant are disabled. Step 9 shows how to add models, and how to opt in to the gallery if you want it.
• No instance metadata and no private network for LocalAI. The localai service, and every backend process it starts, is denied the instance metadata service and the private address ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10, fd00::/8) by systemd IPAddressDeny, so no request (an image or audio URL, a model URL) can make it read the instance's role credentials or probe your VPC. nginx is denied the metadata service too.
• Fails closed. LocalAI will not start without its per-instance key, nginx will not start until first boot has created the certificate and the credentials, and first boot stops nginx again if its own check of the credential gate fails.
• Only nginx faces the network. It answers only for the configured hostname (any other Host header gets its connection closed), uses TLS 1.2 or 1.3, refuses cross-origin POST requests and WebSocket upgrades (403), strips CORS headers, and replaces any client-supplied X-Forwarded-For, X-Real-IP or Forwarded header with the real peer address.
• The hostname comes from configuration, never from a request. First boot reads it from EC2 instance metadata (IMDSv2), or from /etc/cloudimg/localai-hostname if you set one.
• Patched build. The upstream 4.11.0 release binary is built with Go 1.26.0 and carries 98 known Go advisories; the cloudimg build uses Go 1.26.9 and the fixed dependency releases, leaving 7 that have no fixed release upstream (Step 10 shows the record).
• IMDSv2 required. The AMI is registered with ImdsSupport v2.0, so instances launched from it refuse token-less (IMDSv1) metadata requests.
• Per-instance identity. SSH host keys and the machine ID are generated fresh on every instance.

Prerequisites

1. An AWS account and permission to launch EC2 instances
2. An EC2 key pair in the Region you launch in

Recommended instance type: m5.large for the bundled 135M demo model. LocalAI runs models on the CPU and keeps them in memory: for 1B to 3B parameter models choose m5.xlarge (16 GiB) or larger, and for 7B to 8B models m5.2xlarge (32 GiB) or a compute optimised type such as c5.4xlarge. The software charge is per vCPU.

Connecting to your instance

Version	Operating system	SSH user
LocalAI 4.11.0 on Ubuntu 24.04	Ubuntu 24.04 LTS	ubuntu

Step 1: Launch the instance

Option A: AWS Marketplace console

1. Open the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
2. Once the subscription is active, choose Launch your software.
3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose LocalAI 4.11.0 on Ubuntu 24.04, then choose your Region.
4. Pick an instance type (m5.large for the demo model; larger for real models) and your key pair.
5. Security group: the listing's default group opens SSH (22), HTTP (80) and HTTPS (443). Restrict SSH to your own IP address, and restrict 80 and 443 to the networks your users and applications connect from.
6. Choose your VPC and subnet, leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and HTTPS from your application network, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDR ranges with your own:

SG=$(aws ec2 create-security-group --group-name localai \
  --description "LocalAI inference API" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr 10.0.0.0/16
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80 --cidr 10.0.0.0/16
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=localai}]'

You do not need --metadata-options: the AMI already makes IMDSv2 required. An inference API usually serves applications inside your VPC; keep 443 closed to the internet unless users outside need it.

Step 2: Connect and find your credentials

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot finishes within a minute of launch. It sets the hostname LocalAI is served as (the instance's public DNS name when it has one, else its IP address), generates the login password and LocalAI's API key, creates the TLS certificate, starts LocalAI on loopback, then starts nginx and checks the credential gate. It writes /root/localai-credentials.txt (mode 0600, root). These are the entries in that file (the password is not shown here):

sudo grep -E '^localai\.(host|url|user)=' /root/localai-credentials.txt
sudo grep -oE '^localai\.[a-z]+=' /root/localai-credentials.txt
sudo stat -c '%a %U:%G' /root/localai-credentials.txt

localai.host=ec2-44-203-187-207.compute-1.amazonaws.com
localai.url=https://ec2-44-203-187-207.compute-1.amazonaws.com/
localai.user=admin
localai.host=
localai.url=
localai.user=
localai.password=
600 root:root

Value	What it is for
localai.url	The address of the web UI; the OpenAI-compatible API is under /v1
localai.user / localai.password	The login for the web UI and the API (HTTP Basic)

To read the password, run sudo cat /root/localai-credentials.txt in your own session and store it in your secrets manager. LocalAI's internal API key is in /etc/localai/localai-secrets.env; clients never need it, because nginx adds it after checking the login.

Step 3: Confirm LocalAI is running

The first-boot unit, LocalAI and nginx are active, and the installed release is 4.11.0. Only sshd and nginx listen beyond loopback; LocalAI itself is on 127.0.0.1:8080, so nginx is the only way in. nginx's open health check answers ok:

systemctl is-active localai-firstboot.service localai.service nginx.service
local-ai --version
sudo ss -Htlnp | grep -E ':(22|80|443|8080) ' | awk '{split($6,a,"\""); print $4, a[2]}' | sort -u
H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/localai/localai.crt --resolve "$H:443:127.0.0.1" "https://$H/health"

active
active
active
v4.11.0
0.0.0.0:22 sshd
0.0.0.0:443 nginx
0.0.0.0:80 nginx
127.0.0.1:8080 local-ai
[::]:22 sshd
[::]:443 nginx
[::]:80 nginx
ok

When a model is loaded, LocalAI also starts a llama.cpp backend process that listens on a loopback port for LocalAI only.

Step 4: Prove the credential gate and the web defaults

Requests without the login are refused on every path, including the version, metrics and API documentation pages, and so is a wrong password. LocalAI refuses a request on loopback that lacks its own key, an unknown Host header gets no answer (curl reports 000), and plain HTTP redirects to the configured HTTPS name:

H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
C=(--cacert /etc/ssl/localai/localai.crt --resolve "$H:443:127.0.0.1" -s -o /dev/null)
for p in / /v1/models /models /version /metrics /swagger/index.html; do
  echo "anonymous $p -> $(curl "${C[@]}" -w '%{http_code}' "https://$H$p")"
done
echo "wrong password -> $(curl "${C[@]}" -w '%{http_code}' -u admin:wrong-password "https://$H/v1/models")"
echo "LocalAI on loopback without its key -> $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8080/v1/models)"
echo "unknown Host -> $(curl -sk -o /dev/null -w '%{http_code}' -H 'Host: evil.example' https://127.0.0.1/ || true)"
curl -s -o /dev/null -w 'plain HTTP -> %{http_code} %{redirect_url}\n' -H "Host: $H" http://127.0.0.1/v1/models

anonymous / -> 401
anonymous /v1/models -> 401
anonymous /models -> 401
anonymous /version -> 401
anonymous /metrics -> 401
anonymous /swagger/index.html -> 401
wrong password -> 401
LocalAI on loopback without its key -> 401
unknown Host -> 000
plain HTTP -> 301 https://ec2-44-203-187-207.compute-1.amazonaws.com/v1/models

Step 5: List the served models

With the login, /v1/models lists what LocalAI serves. The bundled model is smollm2-135m-instruct:

H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^localai.password=' /root/localai-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/localai/localai.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" "https://$H/v1/models" | jq

{
  "object": "list",
  "data": [
    {
      "id": "smollm2-135m-instruct",
      "object": "model"
    }
  ]
}

Step 6: Run a chat completion

Send an OpenAI-style chat request. The first request loads the model into memory, which takes a few seconds; later ones answer at once:

H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^localai.password=' /root/localai-credentials.txt | cut -d= -f2-)
curl -s --cacert /etc/ssl/localai/localai.crt --resolve "$H:443:127.0.0.1" -u "admin:$PW" \
  "https://$H/v1/chat/completions" -H 'Content-Type: application/json' \
  -d '{"model":"smollm2-135m-instruct","messages":[{"role":"user","content":"Name the three primary colours of light."}],"max_tokens":40,"temperature":0}' \
  | jq '{model, reply: .choices[0].message.content, finish: .choices[0].finish_reason, usage}'

{
  "model": "smollm2-135m-instruct",
  "reply": "The three primary colours of light are:\n\n1. Red\n2. Blue\n3. Green\n\nThese colours are the primary colors of light because they are the only colors that can be produced",
  "finish": "length",
  "usage": {
    "prompt_tokens": 17,
    "completion_tokens": 40,
    "total_tokens": 57
  }
}

The demo model is deliberately tiny so the image boots fast on any instance size; use it to prove the plumbing, then add a capable model (Step 9).

Step 7: Call LocalAI from your application

Any OpenAI client works. Copy /etc/ssl/localai/localai.crt to the client machine (or install your own certificate, Step 8) and pass the login. With the OpenAI Python SDK, send the login as a Basic Authorization header (nginx checks it and replaces it with LocalAI's own key):

import base64, httpx
from openai import OpenAI

auth = base64.b64encode(b"admin:<password from /root/localai-credentials.txt>").decode()
client = OpenAI(
    base_url="https://<your-localai-host>/v1",
    api_key="unused",
    default_headers={"Authorization": f"Basic {auth}"},
    http_client=httpx.Client(verify="localai.crt"),
)
reply = client.chat.completions.create(
    model="smollm2-135m-instruct",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

With curl, -u admin:<password> does the same. Keep the password in your secrets manager, not in source code.

The web UI

Open https://<localai.host>/ in a browser and sign in as admin with the password from Step 2 (accept the self-signed certificate, or install your own). The home page has a chat box with the model already selected:

Chat keeps a conversation with any served model and shows the generation speed:

Models > Installed lists the models LocalAI serves and whether each is loaded:

Models > Explore is empty on purpose, because the model gallery is switched off on this image (Step 9):

Step 8: Serve LocalAI under your own DNS name

Point a DNS record at the instance (an Elastic IP keeps the address stable), then tell LocalAI the name. The helper re-issues the self-signed certificate for it, updates nginx's accepted names, records the name in /etc/cloudimg/localai-hostname (kept across reboots) and updates localai.url:

sudo localai-set-hostname llm.example.com

localai-set-hostname: LocalAI is served at https://llm.example.com/

To use a certificate from your own CA or from Let's Encrypt instead, replace /etc/ssl/localai/localai.crt and /etc/ssl/localai/localai.key (keep the key mode 0600, owner root) and run sudo systemctl reload nginx.

Step 9: Add models

Recommended: copy a GGUF model onto the data volume

LocalAI serves any GGUF model in /var/lib/localai/models that has a small YAML file next to it. Download the model from a publisher you trust on your own machine (Hugging Face lists the SHA-256 of each file), check the checksum, copy it to the instance, and describe it. For example, for a Qwen 2.5 1.5B instruct model:

scp -i ~/.ssh/my-key-pair.pem qwen2.5-1.5b-instruct-q4_k_m.gguf ubuntu@<public-ip>:/tmp/
sudo install -o localai -g localai -m 0640 /tmp/qwen2.5-1.5b-instruct-q4_k_m.gguf /var/lib/localai/models/
sudo tee /var/lib/localai/models/qwen2.5-1.5b-instruct.yaml >/dev/null <<'EOF'
name: qwen2.5-1.5b-instruct
backend: cpu-llama-cpp
context_size: 4096
parameters:
  model: qwen2.5-1.5b-instruct-q4_k_m.gguf
template:
  use_tokenizer_template: true
EOF
sudo chown localai:localai /var/lib/localai/models/qwen2.5-1.5b-instruct.yaml
sudo systemctl restart localai.service

The model then appears in /v1/models and in the web UI. use_tokenizer_template uses the chat template stored in the GGUF file. Size the instance for the model: a 4-bit quantised model needs roughly 0.6 GB of memory per billion parameters plus the context. Grow the data volume in EC2 and run sudo resize2fs on it if you need more space.

Optional: switch the model gallery back on

The upstream gallery lets the web UI and the API download models and backends by URL. It is off on this image because it fetches content from URLs the caller chooses and installs backend executables. If you want it, LocalAI 4.11.0 checks gallery URLs against private, loopback and metadata addresses, and the localai service stays blocked from the metadata service and your private ranges. To opt in, edit /etc/localai/localai.env (set LOCALAI_DISABLE_GALLERY_ENDPOINT=false, and set LOCALAI_GALLERIES and LOCALAI_BACKEND_GALLERIES to the galleries you trust, for example the upstream ones from the LocalAI documentation), empty nginx's install guard, then restart both services:

sudo nano /etc/localai/localai.env
sudo truncate -s 0 /etc/nginx/localai-install-guard.conf
sudo systemctl restart localai.service nginx.service

Anyone with the login can then install models and backends, so limit who holds it. To switch the gallery off again, restore the two files from a fresh instance of this image (or reverse the edits) and restart both services.

Step 10: Check the security controls

Every install route is refused at nginx even with the right login (403), and LocalAI's gallery, runtime settings, agents and MCP are off:

H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
PW=$(sudo grep '^localai.password=' /root/localai-credentials.txt | cut -d= -f2-)
C=(--cacert /etc/ssl/localai/localai.crt --resolve "$H:443:127.0.0.1" -s -o /dev/null -u "admin:$PW")
for p in /models/apply /backends/apply /api/backends/install-external /api/models/install/x /api/settings; do
  echo "POST $p -> $(curl "${C[@]}" -w '%{http_code}' -X POST -H 'Content-Type: application/json' -d '{"url":"http://169.254.169.254/latest/meta-data/"}' "https://$H$p")"
done
grep -E '^LOCALAI_(DISABLE_GALLERY_ENDPOINT|GALLERIES|BACKEND_GALLERIES|AUTOLOAD_GALLERIES|DISABLE_RUNTIME_SETTINGS|REQUIRE_BACKEND_INTEGRITY|DISABLE_AGENTS|DISABLE_MCP)=' /etc/localai/localai.env

POST /models/apply -> 403
POST /backends/apply -> 403
POST /api/backends/install-external -> 403
POST /api/models/install/x -> 403
POST /api/settings -> 403
LOCALAI_DISABLE_GALLERY_ENDPOINT=true
LOCALAI_GALLERIES=[]
LOCALAI_BACKEND_GALLERIES=[]
LOCALAI_AUTOLOAD_GALLERIES=false
LOCALAI_DISABLE_RUNTIME_SETTINGS=true
LOCALAI_REQUIRE_BACKEND_INTEGRITY=true
LOCALAI_DISABLE_AGENTS=true
LOCALAI_DISABLE_MCP=true

The certificate was made for this instance on first boot. The localai service is denied the metadata service and the private ranges (systemd lists them in its own order, folding fd00:ec2::254 into fd00::/8), and nginx the metadata service. The last two lines make requests from inside LocalAI's own service control group, exactly as a request handled by LocalAI would; both fail (000):

H=$(sudo grep '^localai.host=' /root/localai-credentials.txt | cut -d= -f2-)
openssl s_client -connect 127.0.0.1:443 -servername "$H" </dev/null 2>/dev/null | openssl x509 -noout -subject -enddate
systemctl show -p IPAddressDeny --value localai.service
systemctl show -p IPAddressDeny --value nginx.service
CG=$(systemctl show -p ControlGroup --value localai.service)
PRIV=$(hostname -I | awk '{print $1}')
sudo bash -c "echo \$\$ > /sys/fs/cgroup$CG/cgroup.procs; exec curl -s -m 5 -o /dev/null -w 'metadata from inside localai.service -> %{http_code}\n' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token" || true
sudo bash -c "echo \$\$ > /sys/fs/cgroup$CG/cgroup.procs; exec curl -sk -m 5 -o /dev/null -w 'private address $PRIV from inside localai.service -> %{http_code}\n' https://$PRIV/health" || true

subject=CN = ec2-44-203-187-207.compute-1.amazonaws.com, O = cloudimg
notAfter=Jan 22 02:34:05 2029 GMT
100.64.0.0/10 10.0.0.0/8 168.63.129.16/32 169.254.169.254/32 192.168.0.0/16 172.16.0.0/12 fd00::/8
fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32
metadata from inside localai.service -> 000
private address 172.31.91.17 from inside localai.service -> 000

The build record shows the toolchain and the Go advisories left in the binary (each has no fixed upstream release, or, for the Slack connector library, a fix that breaks the build; agents are off), and only the CPU backend is installed, with no CUDA, ROCm or Vulkan libraries:

grep -E '^(LocalAI|toolchain|govulncheck) ' /usr/share/doc/cloudimg/localai/build-provenance/BUILD-PROVENANCE.txt | cut -c1-120
grep -E '^Your code' /usr/share/doc/cloudimg/localai/build-provenance/govulncheck.txt
sudo ls /var/lib/localai/backends/
sudo find /var/lib/localai /usr/local \( -iname 'libggml-cuda*' -o -iname 'libcudart*' -o -iname 'libggml-vulkan*' -o -iname 'libamdhip*' \) | wc -l

LocalAI 4.11.0 on this image was compiled by cloudimg from the unmodified upstream source at
toolchain     go1.26.9 linux/amd64, https://go.dev/dl/go1.26.9.linux-amd64.tar.gz sha256 42d158b4d8f7b61ac0a830567c940a8
govulncheck   golang.org/x/vuln/cmd/govulncheck@v1.1.4 -mode=binary on the compiled binary: GO-2026-4883 GO-2026-4887 GO
Your code is affected by 7 vulnerabilities from 3 modules.
cpu-llama-cpp
0

Step 11: Rotate the login password or the API key

To change the login password, htpasswd reads it from standard input, so it never appears on a command line, in your shell history or in the system logs. Enter the new password when prompted, then reload nginx:

sudo htpasswd -i -B /etc/nginx/.localai.htpasswd admin
sudo systemctl reload nginx

To issue a new internal API key, write it to both root-only files and restart LocalAI, then reload nginx. The key is generated on the instance and handed to sudo tee on standard input, so it never appears on a sudo command line (which sudo writes to the system logs):

NEWKEY="sk-localai-$(openssl rand -hex 24)"
printf '%s=%s\n' LOCALAI_API_KEY "$NEWKEY" | sudo tee /etc/localai/localai-secrets.env >/dev/null
printf 'proxy_set_header Authorization "Bearer %s";\n' "$NEWKEY" | sudo tee /etc/nginx/localai-upstream-auth.inc >/dev/null
sudo systemctl restart localai.service && sudo systemctl reload nginx
unset NEWKEY

Both files keep mode 0600, owner root. Then put the new password in /root/localai-credentials.txt (or delete that file) so it does not hold a stale secret.

Maintenance

• Logs: sudo journalctl -u localai.service for LocalAI and its backends, /var/log/nginx/ for HTTPS access, sudo journalctl -u localai-firstboot.service for first boot.
• Restart: sudo systemctl restart localai.service (models load again on their next request), or sudo systemctl reload nginx.
• Operating system updates: Ubuntu's unattended security upgrades stay on, so nginx, OpenSSL and the rest of the OS keep receiving patches. Apply the rest with sudo apt update && sudo apt upgrade in a maintenance window.
• Upgrading LocalAI: the release is pinned and the instance never upgrades LocalAI or its backend on its own. To move to a newer release, launch the newer version from this listing when cloudimg publishes it and move your models across (they are plain files on the data volume), or test an upstream release on a copy of the instance first.
• Memory: LocalAI keeps loaded models in memory. If the instance runs short, set LOCALAI_MAX_ACTIVE_BACKENDS=1 in /etc/localai/localai.env and restart LocalAI, so only one model is resident at a time.

Troubleshooting

401 Unauthorized from every request. Send the login admin and the password from /root/localai-credentials.txt (curl -u, or a Basic Authorization header) over https://.

403 Forbidden. Install routes are refused by design (Step 9), and so are cross-origin POST requests from a browser page on another site. Same-origin requests and API clients that send no Origin header are accepted.

curl: (60) SSL certificate problem. The certificate is self-signed. Pass --cacert with a copy of /etc/ssl/localai/localai.crt, or install your own certificate (Step 8). Use the configured hostname (localai.host), which the certificate covers.

nginx or LocalAI will not start. Both wait for first boot to create the certificate and credentials, by design. Check first boot with sudo journalctl -u localai-firstboot.service --no-pager | tail -20.

A model you added is not listed or fails to load. Read sudo journalctl -u localai.service -n 50 --no-pager: the usual causes are a YAML model: name that does not match the file, a file the localai user cannot read, or too little memory for the model.

Support

cloudimg provides 24/7 support for this image by email and live chat: launching and connecting, API clients and SDKs, TLS and hostnames, adding models and sizing.

LocalAI is developed by Ettore Di Giacinto and contributors and is released under the MIT licence. llama.cpp is released under the MIT licence. SmolLM2 is published by Hugging Face under the Apache 2.0 licence. Ubuntu is a registered trademark of Canonical Ltd. OpenAI is a trademark of OpenAI, Inc. All other product and company names are trademarks or registered trademarks of their respective holders. cloudimg is not affiliated with or endorsed by them.