AnythingLLM 1.14.1 on AWS User Guide

Overview

This guide covers launching and using AnythingLLM 1.14.1 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace; you pick it as the software version when you subscribe and launch. The other LLM servers, gateways and chat interfaces in the listing are described on the listing's guide hub.

AnythingLLM, from Mintplex Labs, is an all-in-one AI application: you upload documents into workspaces and chat with them using retrieval-augmented generation (RAG), build agents, and give your team their own logins. It has a built-in vector database (LanceDB), a built-in embedding model that runs on the CPU, and a SQLite database for users, workspaces and settings. For the chat itself you connect the LLM provider of your choice: OpenAI, Anthropic, Amazon Bedrock, an Ollama server in your VPC, and many more.

The image ships no LLM provider and no model weights, and no GPU libraries: the language model runs at the provider you choose, so a small CPU instance is enough.

What is on the image:

• AnythingLLM 1.14.1 (MIT licence) on Node.js 22 LTS in /opt/anythingllm: the server (anythingllm.service) and the document collector (anythingllm-collector.service), both run as the unprivileged anythingllm user
• Both services bound to loopback only (127.0.0.1:3001 and 127.0.0.1:8888), with nginx as the only network-facing service
• HTTPS on port 443 with a self-signed certificate created on your instance's first boot; port 80 redirects to HTTPS
• Multi-user mode with a per-instance admin password generated on first boot, and no public sign-up
• A dedicated 40 GiB EBS data volume at /var/lib/anythingllm holding the database, the vector store, your documents and the per-instance settings
• anythingllm-set-hostname, a helper that serves AnythingLLM under your own DNS name and re-issues the certificate for it

Default user: ubuntu · Architecture: x86_64 · Root volume: 20 GiB gp3 · Data volume: 40 GiB gp3

Security posture

• No default password. The admin password and the application secrets (JWT_SECRET, SIG_KEY, SIG_SALT) are generated on each instance's first boot. None of them exists in the image, so no two instances share one. The server's web-push and internal key pairs are also created per instance.
• Fails closed. The server and the collector refuse to start unless first boot has completed and well-formed per-instance secrets exist, and nginx will not start before first boot has created the certificate. A failed first boot serves nothing, rather than an open instance.
• Every route needs a login. Multi-user mode is on from the first boot: anonymous API calls get 401, and there is no public sign-up. New users come only from an administrator (Settings > Admin > Users) or from an invite link an administrator issues.
• Only nginx faces the network. It answers only for the configured hostname (any other Host header gets its connection closed), uses TLS 1.2 or 1.3, refuses cross-origin POST requests and WebSocket upgrades (403), never passes AnythingLLM's permissive CORS headers on (except for the chat-embed widget API, which is cross-origin by design), and replaces any client-supplied X-Forwarded-For with the real peer address.
• The hostname comes from configuration, never from a request. First boot reads it from EC2 instance metadata (IMDSv2) or from /etc/cloudimg/anythingllm-hostname.
• No instance metadata access for the app. AnythingLLM fetches URLs on your behalf (agent web browsing, Agent Flow API calls, link ingestion), so both of its services are blocked from the instance metadata service (169.254.169.254 and fd00:ec2::254). A prompt or a document can never make it read the instance's role credentials. Amazon Time Sync (169.254.169.123) and the rest of your VPC are unaffected.
• IMDSv2 required. The AMI is registered with ImdsSupport v2.0, so instances launched from it refuse token-less (IMDSv1) metadata requests.
• Sandboxed services and per-instance identity. Both services run with NoNewPrivileges, ProtectSystem=full, ProtectHome and PrivateTmp. SSH host keys and the machine ID are generated fresh on every instance.

Prerequisites

1. An AWS account and permission to launch EC2 instances
2. An EC2 key pair in the Region you launch in
3. Access to an LLM: an API key for a hosted provider (OpenAI, Anthropic and so on), Amazon Bedrock, or a self-hosted model server such as Ollama

Recommended instance type: m5.large. AnythingLLM does not run the language model itself, so 2 vCPUs and 8 GiB of memory are enough for the app, the collector and the built-in CPU embedder. Choose more memory if you embed very large document sets.

Connecting to your instance

Step 1: Launch the instance

Option A: AWS Marketplace console

1. Open the cloudimg Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
2. Once the subscription is active, choose Launch your software.
3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose AnythingLLM 1.14.1 on Ubuntu 24.04, then choose your Region.
4. Pick an instance type (m5.large recommended; the software charge is $0.04 per vCPU per hour) and your key pair.
5. Security group: the listing's default group opens SSH (22), HTTP (80) and HTTPS (443). Restrict SSH to your own IP address, and restrict 80 and 443 to the networks your users connect from.
6. Choose your VPC and subnet, keep the 40 GiB data volume, leave the metadata defaults (the image already requires IMDSv2), then choose Launch.

Option B: AWS CLI

Run this on your own workstation. It creates a security group that allows SSH from your management address and HTTPS from your users' network, then launches the instance. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDR ranges with your own:

SG=$(aws ec2 create-security-group --group-name anythingllm \
  --description "AnythingLLM" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr 10.0.0.0/16
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80 --cidr 10.0.0.0/16
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=anythingllm}]'

The AMI's block device mapping already includes the 40 GiB data volume, and you do not need --metadata-options: the AMI already makes IMDSv2 required.

Step 2: Connect and find your admin password

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot finishes within a minute of launch. It sets the hostname AnythingLLM is served as (the instance's public DNS name when it has one), creates the TLS certificate for it, generates the secrets and the admin password, and writes /root/anythingllm-credentials.txt. These are the entries in that file (the password is not shown here):

sudo grep -E '^anythingllm\.(url|admin\.user)=' /root/anythingllm-credentials.txt
sudo grep -oE '^anythingllm\.[a-z.]+=' /root/anythingllm-credentials.txt

anythingllm.url=https://ec2-3-86-196-143.compute-1.amazonaws.com/
anythingllm.admin.user=admin
anythingllm.url=
anythingllm.admin.user=
anythingllm.admin.pass=

To read your password, run sudo cat /root/anythingllm-credentials.txt in your own session. Store it in your password manager; you can change it after signing in (your avatar, top right, then Account).

Step 3: Confirm AnythingLLM is running

The first-boot unit, the server, the collector and nginx are active, the installed version is 1.14.1 on Node.js 22, and only sshd and nginx listen beyond loopback. The server and the collector are on 127.0.0.1 only.

systemctl is-active anythingllm-firstboot.service anythingllm.service anythingllm-collector.service nginx.service
node -e 'console.log("AnythingLLM " + require("/opt/anythingllm/server/package.json").version)'
node --version
sudo ss -Htlnp | awk '{print $4, $6}' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort -u

active
active
active
active
AnythingLLM 1.14.1
v22.23.3
0.0.0.0:22 users:(("sshd"),("systemd"))
0.0.0.0:443 users:(("nginx"),("nginx"),("nginx"))
0.0.0.0:80 users:(("nginx"),("nginx"),("nginx"))
127.0.0.1:3001 users:(("node"))
127.0.0.1:8888 users:(("node"))
127.0.0.53%lo:53 users:(("systemd-resolve"))
127.0.0.54:53 users:(("systemd-resolve"))
[::]:22 users:(("sshd"),("systemd"))
[::]:443 users:(("nginx"),("nginx"),("nginx"))
[::]:80 users:(("nginx"),("nginx"),("nginx"))

The API answers over HTTPS, multi-user mode is on, and an anonymous request is refused:

H=$(sudo sed -n 's#^anythingllm.url=https://\(.*\)/$#\1#p' /root/anythingllm-credentials.txt)
curl -s --cacert /etc/ssl/anythingllm/anythingllm.crt "https://$H/api/ping"; echo
curl -s --cacert /etc/ssl/anythingllm/anythingllm.crt "https://$H/api/system/multi-user-mode"; echo
curl -s -o /dev/null -w 'anonymous GET /api/workspaces: %{http_code}\n' --cacert /etc/ssl/anythingllm/anythingllm.crt "https://$H/api/workspaces"

{"online":true}
{"multiUserMode":true}
anonymous GET /api/workspaces: 401

Step 4: Sign in

Open https://<anythingllm.url>/ from your credentials file in your browser. The certificate is self-signed for your instance, so the browser asks you to accept it the first time (Step 9 covers using your own name and a trusted certificate). Sign in as admin with the password from Step 2.

Step 5: Connect your LLM provider

AnythingLLM ships with no LLM provider configured. Open Settings (the wrench icon, bottom left) > AI Providers > LLM, choose your provider and enter its details:

• OpenAI, Anthropic, Gemini, Mistral and other hosted APIs: paste your API key and pick a model.
• Amazon Bedrock: enter an access key for an IAM user (or role credentials) with Bedrock access, the Region and the model ID. The app cannot use the instance's own role, because it is blocked from the instance metadata service.
• Ollama in your VPC (for example the Ollama version of this listing on another instance): choose Ollama and set the base URL to the server's private address, http://<ollama-private-ip>:11434. Open port 11434 on the Ollama instance's security group to this instance only.

Embeddings use AnythingLLM's built-in embedder, which runs on the CPU on this instance, so document upload and search work before you choose a chat provider. It downloads its small model (about 23 MB) from Hugging Face the first time you embed a document, so the instance needs outbound internet access for that, or choose another embedder under AI Providers > Embedder.

Step 6: Create a workspace and chat with your documents

Choose + next to Search to create a workspace, give it a name and save. Each workspace has its own documents, chat threads and settings.

To add documents, choose the upload icon next to the workspace name. Upload PDFs, Word files, text, CSV and other files, select them under My Documents, choose Move to Workspace, then Save and Embed. The collector extracts the text on this instance, the embedder turns it into vectors in the LanceDB store, and the workspace chat then answers from your documents and cites them.

The Fetch website link option uses a headless Chromium browser, which this image does not include; upload files instead, or install Chromium yourself if you need website scraping.

Step 7: Add users

Multi-user mode is already on. Under Settings > Admin > Users choose Add user to create accounts with the Default, Manager or Admin role, or issue single-use invite links under Invites. Workspace access is granted per user in each workspace's settings.

Step 8: Your data on the dedicated volume

Everything AnythingLLM stores lives on the 40 GiB EBS volume mounted at /var/lib/anythingllm: the SQLite database, the LanceDB vector store, your documents, the embedding model and the per-instance settings file (.env, mode 0600). /opt/anythingllm/server/storage is a link onto it.

df -h /var/lib/anythingllm
sudo ls /var/lib/anythingllm/storage
readlink -f /opt/anythingllm/server/storage

Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme1n1     40G  516K   38G   1% /var/lib/anythingllm
README.md
anythingllm.db
assets
comkey
documents
models
push-notifications
/var/lib/anythingllm/storage

The volume is mounted by filesystem UUID with nofail, so you can grow it in the EC2 console (then run sudo resize2fs on its device) or snapshot it on its own schedule with Amazon Data Lifecycle Manager.

Step 9: Serve AnythingLLM under your own DNS name

Port 80 redirects to HTTPS on the configured hostname, and the certificate covers that name, the instance's public and private addresses, localhost and 127.0.0.1:

H=$(sudo sed -n 's#^anythingllm.url=https://\(.*\)/$#\1#p' /root/anythingllm-credentials.txt)
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' "http://$H/workspaces"
openssl x509 -in /etc/ssl/anythingllm/anythingllm.crt -noout -subject -enddate -ext subjectAltName

301 https://ec2-3-86-196-143.compute-1.amazonaws.com/workspaces
subject=CN = ec2-3-86-196-143.compute-1.amazonaws.com, O = cloudimg
notAfter=Jan 21 18:48:46 2029 GMT
X509v3 Subject Alternative Name: 
    DNS:ec2-3-86-196-143.compute-1.amazonaws.com, IP Address:3.86.196.143, IP Address:172.31.88.48, IP Address:127.0.0.1, DNS:localhost

To use your own name, point a DNS record at the instance (an Elastic IP keeps the address stable across stops), then run the helper. It records the name in /etc/cloudimg/anythingllm-hostname, issues a new self-signed certificate for it, updates the allowed origins and the credentials file, and reloads nginx:

sudo anythingllm-set-hostname <your-dns-name>

For a publicly trusted certificate, put the instance behind an Application Load Balancer with an AWS Certificate Manager certificate, or replace /etc/ssl/anythingllm/anythingllm.crt and anythingllm.key with your own and run sudo systemctl reload nginx.

Step 10: Check the security controls

Both AnythingLLM services carry the metadata block, the settings file holds this instance's own secrets (names shown, values not), and nginx refuses cross-origin writes and requests for any hostname it was not configured with:

systemctl show -p IPAddressDeny anythingllm.service anythingllm-collector.service
sudo grep -oE '^(JWT_SECRET|SIG_KEY|SIG_SALT)=' /var/lib/anythingllm/.env
H=$(sudo sed -n 's#^anythingllm.url=https://\(.*\)/$#\1#p' /root/anythingllm-credentials.txt)
curl -s -o /dev/null -w 'cross-origin POST: %{http_code}\n' --cacert /etc/ssl/anythingllm/anythingllm.crt \
  -X POST -H 'Origin: https://evil.example' -H 'Content-Type: application/json' --data '{}' "https://$H/api/request-token"
code=$(curl -sk -o /dev/null -w '%{http_code}' -H 'Host: evil.example' https://127.0.0.1/ || true); echo "unknown Host: ${code} (connection closed by nginx)"

IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128

IPAddressDeny=168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128
JWT_SECRET=
SIG_KEY=
SIG_SALT=
cross-origin POST: 403
unknown Host: 000 (connection closed by nginx)

Maintenance

• Logs: sudo journalctl -u anythingllm.service -f for the server, sudo journalctl -u anythingllm-collector.service -f for document processing, /var/log/nginx/ for HTTPS access.
• Restart: sudo systemctl restart anythingllm-collector.service anythingllm.service.
• Operating system updates: Ubuntu's unattended security upgrades stay on. Apply the rest with sudo apt update && sudo apt upgrade in a maintenance window.
• Upgrading AnythingLLM: take a snapshot of the data volume first, then follow the bare-metal upgrade steps in the AnythingLLM documentation for /opt/anythingllm. Keep the cloudimg changes the image makes to bind the server and collector to 127.0.0.1, and run sudo systemctl restart anythingllm-collector.service anythingllm.service afterwards.
• Backups: the data volume holds everything stateful. Snapshot it (or the whole instance) on a schedule; /root/anythingllm-credentials.txt and /etc/ssl/anythingllm/ are on the root volume.

Support

cloudimg provides 24/7 support for this image by email and live chat: launching and connecting, provider configuration, users, TLS and hostnames, and upgrades.

Trademarks

AnythingLLM is a trademark of Mintplex Labs Inc. Ubuntu is a registered trademark of Canonical Ltd. All other product and company names are trademarks or registered trademarks of their respective holders. cloudimg is not affiliated with or endorsed by them.