Chroma on AWS User Guide

Overview

This image runs Chroma 1.5, the open source vector database for AI - store embeddings and run fast similarity search for retrieval augmented generation - on Ubuntu 24.04 LTS. Chroma is installed into a dedicated Python virtual environment under /opt/chroma on Python 3.12 and run by an unprivileged chroma system account under a systemd service that starts the server on boot.

The server listens on the loopback address 127.0.0.1:8000; nginx fronts it on port 80 with HTTP Basic Authentication. The public /api/v2/heartbeat endpoint stays open; everything else requires the password. The default security group opens port 22 (SSH) and port 80 (HTTP) only, so 8000 is not reachable externally.

On the first boot of every deployed instance a one-shot service generates a fresh password, unique to that instance, and writes it to /root/chroma-credentials.txt (mode 0600, root only). Persistent data lives at /var/lib/chroma on a dedicated, independently resizable EBS data volume. The image ships no embedding model and is CPU only.

Prerequisites

• An AWS account subscribed to this product in AWS Marketplace.
• An EC2 key pair in your target region for SSH access.
• A security group allowing inbound TCP 22 (SSH) from your IP and TCP 80 (HTTP) from your users.
• Recommended instance type: m5.large or larger.

Connecting to your instance

Step 1 - Launch from the AWS Marketplace console

1. Open the product page in AWS Marketplace and choose Continue to Subscribe, then Continue to Configuration.
2. Select the Chroma 1.5 on Ubuntu 24.04 delivery option and your region, then Continue to Launch.
3. Choose your instance type, VPC/subnet, key pair and the security group described above, and launch.

Step 2 - Launch from the AWS CLI

aws ec2 run-instances \
  --image-id ami-xxxxxxxxxxxxxxxxx \
  --instance-type m5.large \
  --key-name your-key \
  --security-group-ids sg-xxxxxxxx \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=chroma}]'

Step 3 - Connect to your instance

ssh -i your-key.pem ubuntu@<instance-public-ip>

Step 4 - Confirm the services are running

systemctl is-active chroma.service nginx.service
curl -s http://127.0.0.1/api/v2/heartbeat

Expected output:

active
active
{"nanosecond heartbeat":1780999459324414192}

Step 5 - Retrieve your password

sudo cat /root/chroma-credentials.txt

# Chroma - generated on first boot by chroma-firstboot.service
CHROMA_URL=http://<instance-public-ip>/
CHROMA_USERNAME=admin
CHROMA_PASSWORD=<your-unique-password>

Step 6 - Connect with the Chroma client

The heartbeat is open; everything else is gated by HTTP Basic Authentication. The Chroma Python client has a built in basic auth provider:

import chromadb
from chromadb.config import Settings

client = chromadb.HttpClient(
    host="<instance-public-ip>", port=80,
    settings=Settings(
        chroma_client_auth_provider="chromadb.auth.basic_authn.BasicAuthClientProvider",
        chroma_client_auth_credentials="admin:<your-unique-password>",
    ),
)

col = client.create_collection("docs")
col.add(ids=["1"], embeddings=[[0.1, 0.2, 0.3]], documents=["hello world"])
print(col.query(query_embeddings=[[0.1, 0.2, 0.3]], n_results=1))

You can also call the REST API directly with -u admin:<password> (HTTP Basic). Bring your own embeddings, or configure an embedding function in the client.

Step 7 - Confirm the runtime

/opt/chroma/venv/bin/pip show chromadb | grep ^Version

Version: 1.5.9

Enabling HTTPS

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

certbot edits the nginx site at /etc/nginx/sites-available/cloudimg-chroma to add the TLS listener and arranges automatic renewal.

Backup and maintenance

• All Chroma data lives under /var/lib/chroma (the chroma.sqlite3 index and segment files) on its own EBS volume. Snapshot that volume to back up collections and embeddings.
• The password is in the nginx htpasswd file /etc/nginx/.chroma.htpasswd; rotate it with sudo htpasswd /etc/nginx/.chroma.htpasswd admin.
• Restart with sudo systemctl restart chroma.service; logs: sudo journalctl -u chroma.service.

Support

cloudimg provides 24/7 technical support for this image by email and chat, covering Chroma deployment, collection design, embedding functions, backups, TLS termination and scaling. Contact details are on the AWS Marketplace listing.