AnythingLLM on AWS User Guide

Overview

This image runs AnythingLLM, the all-in-one open source application for chatting with your documents using any large language model. AnythingLLM turns any document, resource or piece of content into context that a chatbot can reference during conversation, with a clean workspace based UI, retrieval augmented generation (RAG), AI agents and a built-in vector database.

AnythingLLM is built bare-metal from the official source on Node.js 22 LTS and runs as a dedicated unprivileged anythingllm system account. Two systemd services run on boot and restart on failure: the application server (anythingllm.service, bound to 127.0.0.1:3001) and the document collector (anythingllm-collector.service, bound to 127.0.0.1:8888). All application state, the SQLite database, the LanceDB vector store, the embedder model cache and your uploaded documents, lives at /opt/anythingllm/storage, a dedicated, independently resizable EBS data volume that survives instance replacement.

An nginx reverse proxy publishes the workspace UI on port 80 with WebSocket support for streaming chat, and is the only public entry point: the security group opens only port 80 (and port 22 for SSH), so the application is always reached through the authenticating proxy. On the first boot of every deployed instance the image generates a fresh admin password, seeds it into the database as a bcrypt hash, and enables multi-user mode so the instance requires a login. Two instances launched from the same Amazon Machine Image never share a password. The password is written to /root/anythingllm-aws-credentials.txt with mode 0600 so that only the root user can read it.

The image ships RAG-ready: a built-in local embedding model (Xenova/all-MiniLM-L6-v2) and a built-in local vector database (LanceDB) are already configured, so document upload, chunking, embedding and semantic search all work on a fresh launch with no external service to configure. AnythingLLM is large language model provider agnostic and ships with no LLM key bundled; you choose your provider and supply your key once in Settings to enable chat.

Prerequisites

Before you deploy this image you need:

• An Amazon Web Services account where you can launch EC2 instances
• IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
• An EC2 key pair in the target Region for SSH access to the instance
• A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and port 80 for the workspace UI
• The AWS CLI (version 2) installed locally if you plan to deploy from the command line
• An API key for your preferred large language model provider (for example OpenAI, Anthropic or Azure OpenAI), or a reachable local model server such as Ollama, to enable chat

Step 1: Launch the Instance from the AWS Marketplace

Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images choose AWS Marketplace AMIs and search for AnythingLLM. Select the cloudimg listing and choose Select, then Continue on the subscription summary.

Pick an instance type of t3.large or larger. Choose your EC2 key pair under Key pair (login). Under Network settings select your VPC and subnet, and either create or select a security group that opens port 22 from your management network and port 80 for the workspace UI. Leave the root volume at the default size or larger.

Select Launch instance. First boot initialisation takes up to a minute or two after the instance state becomes Running and the status checks pass, while the admin password is generated and the application restarts.

Step 2: Launch the Instance from the AWS CLI

The following block launches an instance from the cloudimg AnythingLLM Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing, <key-name> with your EC2 key pair name, <subnet-id> with your subnet ID, and <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type t3.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=anythingllm}]'

When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name from the EC2 console or with aws ec2 describe-instances.

Step 3: Connect to Your Instance

Connect over SSH using your key pair and the login user for your operating system variant.

ssh -i <key-name>.pem ubuntu@<public-ip>

Step 4: Retrieve the Admin Password

The admin password is unique to your instance and was generated on first boot. Read it as root:

sudo cat /root/anythingllm-aws-credentials.txt

The file lists the workspace URL, the admin user (admin) and the generated password. Keep this password somewhere safe.

Step 5: Sign In to the Workspace UI

The AnythingLLM workspace UI is served on port 80 by nginx. In a browser, go to:

http://<instance-public-ip>/

You are prompted to sign in. Enter the user admin and the password from the credentials file. After signing in you land in the AnythingLLM workspace, where you create workspaces, upload documents and chat with them.

A workspace is a container for a set of documents and the conversations grounded in them. Create a new workspace from the sidebar, give it a name, and open it to start a chat thread.

Step 6: Confirm AnythingLLM Is Running

Over SSH, confirm the application server, the collector and the nginx proxy are active and that the ports are listening:

sudo systemctl is-active anythingllm anythingllm-collector nginx

sudo ss -tlnp | grep -E ':(80|3001|8888) '

You should see all three services reported as active, the application server listening on port 3001 and the collector on port 8888 on the instance, and nginx listening on port 80. The security group exposes only port 80 (and port 22 for SSH), so the application is reached exclusively through the authenticating nginx proxy; keep ports 3001 and 8888 closed in your security group.

You can also check the server health endpoint over loopback on the instance:

curl -s http://127.0.0.1:3001/api/ping

This returns {"online":true} when the server is up.

Step 7: Choose Your LLM Provider

AnythingLLM ships with the local embedder and vector database already configured, but it is provider agnostic for the chat model and ships with no LLM key. To enable chat, open Settings from the sidebar, then LLM Preference. Choose your provider, for example OpenAI, Anthropic, Azure OpenAI, or a local Ollama server, and supply your API key or endpoint. Save the setting.

Your prompts and documents stay on your instance for embedding and retrieval; only the final prompt and retrieved context are sent to the model provider you select.

Step 8: Upload Documents and Build Your Knowledge Base

Open a workspace and use the document manager to upload content. AnythingLLM accepts PDFs, text, Microsoft Office documents, web pages and more. The document collector extracts the text, the built-in local embedder chunks and embeds it, and the chunks are stored in the bundled LanceDB vector database, all on your instance.

Once a document is embedded into a workspace, ask questions in the chat thread. AnythingLLM retrieves the most relevant chunks and passes them to your chosen LLM, so answers are grounded in your own content.

Step 9: The Data Volume

All application state lives on a dedicated EBS volume mounted at /opt/anythingllm/storage. This keeps the database, the vector store and your uploaded documents off the operating system disk and lets you resize or snapshot them independently. Confirm the mount with:

df -h /opt/anythingllm/storage

To grow the storage, expand the EBS volume in the AWS console, then grow the filesystem on the instance with sudo resize2fs on the underlying device.

Step 10: Enable HTTPS

The workspace UI is served over plain HTTP on port 80 by nginx. For production use, place it behind TLS. Obtain a certificate for your domain (for example with a managed certificate on an Application Load Balancer in front of the instance, or with Certbot installed on the instance), then configure nginx to listen on 443 with your certificate and proxy to 127.0.0.1:3001 exactly as the bundled site does for port 80, keeping the WebSocket upgrade headers in place. Restrict the security group so ports 80 and 443 are reachable only from the networks that use the application.

Step 11: Backup and Maintenance

Back up the application by snapshotting the /opt/anythingllm/storage EBS volume, which captures the database, the vector store and your uploaded documents in one consistent set. Apply operating system security updates with sudo apt-get update && sudo apt-get upgrade and reboot when a new kernel is installed; AnythingLLM and nginx start automatically on boot.

Support

This image is published and supported by cloudimg. Support covers deployment, workspace and document configuration, embedding and vector database tuning, LLM provider setup, AI agents, API keys, TLS and storage tuning. Contact cloudimg through the support channel listed on the AWS Marketplace listing.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.