Evidently on AWS User Guide

Overview

This image runs Evidently, the open source platform for evaluating, testing and monitoring machine learning, large language model and data pipelines. Evidently computes data drift, data quality and model performance reports over your datasets and renders them in an interactive monitoring dashboard, where each report snapshot is stored as a versioned run inside a project so you can track how your data and models change over time.

Evidently is installed into a dedicated Python virtual environment at /opt/evidently/venv (pinned to version 0.7.21) and runs as a dedicated unprivileged evidently system account. One systemd service runs on boot and restarts on failure: the Evidently UI server (evidently.service), which is bound to 127.0.0.1:8000 and is never exposed directly. All workspace state, the projects and the report snapshots the dashboard renders, lives at /var/lib/evidently, a dedicated, independently resizable EBS data volume that survives instance replacement.

An nginx reverse proxy publishes the monitoring dashboard on port 80 and is the only public entry point. The open source Evidently UI has no built in authentication of its own, so this image gates every request behind nginx HTTP Basic Auth. On the first boot of every deployed instance the image generates a fresh dashboard password unique to that instance, stores it as a bcrypt hash in the nginx Basic Auth file, and writes the plaintext password to /root/evidently-credentials.txt with mode 0600 so that only the root user can read it. Two instances launched from the same Amazon Machine Image never share a password, and no shared or default credential ships in the image. An unauthenticated static health endpoint is exposed at /healthz for load balancer probes only; every other path requires the password.

To give you something real to explore on the first login, first boot also seeds a demo project named cloudimg-demo containing a genuine data drift and data quality report computed over a synthetic sample, so the dashboard shows an explorable report immediately rather than an empty workspace.

Prerequisites

Before you deploy this image you need:

• An Amazon Web Services account where you can launch EC2 instances
• IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
• An EC2 key pair in the target Region for SSH access to the instance
• A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and port 80 for the monitoring dashboard
• The AWS CLI (version 2) installed locally if you plan to deploy from the command line

Step 1: Launch the Instance from the AWS Marketplace

Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images choose AWS Marketplace AMIs and search for Evidently. Select the cloudimg listing and choose Select, then Continue on the subscription summary.

Pick an instance type of m5.large or larger. Choose your EC2 key pair under Key pair (login). Under Network settings select your VPC and subnet, and either create or select a security group that opens port 22 from your management network and port 80 for the dashboard. Leave the root volume at the default size or larger.

Select Launch instance. First boot initialisation takes up to a minute or two after the instance state becomes Running and the status checks pass, while the demo project is seeded, the dashboard password is generated and the proxy starts.

Step 2: Launch the Instance from the AWS CLI

The following block launches an instance from the cloudimg Evidently Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing, <key-name> with your EC2 key pair name, <subnet-id> with your subnet ID, and <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=evidently}]'

When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name from the EC2 console or with aws ec2 describe-instances.

Step 3: Connect to Your Instance

Connect over SSH using your key pair and the login user for your operating system variant.

ssh -i <key-name>.pem ubuntu@<public-ip>

Step 4: Retrieve the Dashboard Password

The dashboard password is unique to your instance and was generated on first boot. Read it as root:

sudo cat /root/evidently-credentials.txt

The file lists the dashboard URL, the user name (admin) and the generated password:

EVIDENTLY_URL=http://<instance-public-ip>/
EVIDENTLY_USERNAME=admin
EVIDENTLY_PASSWORD=<EVIDENTLY_PASSWORD>
EVIDENTLY_WORKSPACE=/var/lib/evidently/workspace

Keep this password somewhere safe.

Step 5: Sign In to the Monitoring Dashboard

The Evidently dashboard is served on port 80 by nginx. In a browser, go to:

http://<instance-public-ip>/

Your browser prompts you for a user name and password. Enter the user admin and the password from the credentials file. After you authenticate you land on the Evidently project list, which shows the seeded cloudimg-demo project.

A project is a container for a set of report snapshots and the monitoring panels built over them. Open cloudimg-demo to explore the seeded report, or create a new project with the plus button to organise your own reports.

Step 6: Explore the Seeded Demo Report

Open the cloudimg-demo project and select the Reports tab. The seeded snapshot appears in the report list, timestamped from first boot, with View, Download and delete actions.

Select View to open the report. The demo runs a data drift preset over a synthetic reference and current dataset that were deliberately shifted, so the report shows drift detected across every column, with per column statistical tests, drift scores and reference versus current distribution charts.

Scroll down for the data summary, which shows per column statistics such as count, missing values, min, max, mean and standard deviation, alongside distribution charts comparing the reference and current data for numerical and categorical features.

Step 7: Confirm Evidently Is Running

Over SSH, confirm the Evidently UI server and the nginx proxy are active:

sudo systemctl is-active evidently.service nginx.service

Both are reported as active:

active
active

Confirm the ports: the Evidently UI server listens on loopback 127.0.0.1:8000 and is never exposed, while nginx listens on port 80:

sudo ss -tlnp | grep -E ':(80|8000) '

LISTEN 0  2048  127.0.0.1:8000  0.0.0.0:*  users:(("evidently",...))
LISTEN 0  511     0.0.0.0:80     0.0.0.0:*  users:(("nginx",...))

The security group exposes only port 80 (and port 22 for SSH), so the dashboard is reached exclusively through the authenticating nginx proxy; keep port 8000 closed in your security group.

The unauthenticated health endpoint returns a plain ok for load balancer probes:

curl -s http://127.0.0.1/healthz

Every other path requires the password. An authenticated API call over loopback lists the seeded project:

PW=$(sudo grep '^EVIDENTLY_PASSWORD=' /root/evidently-credentials.txt | cut -d= -f2-)
curl -s -u "admin:$PW" http://127.0.0.1/api/projects

[{"id":"019f9530-02d8-7645-a203-988c05584973","name":"cloudimg-demo","description":"cloudimg demo: data drift and data quality over a synthetic sample",...}]

Step 8: Monitor Your Own Data

To monitor your own data, compute Evidently reports over your datasets with the bundled Evidently Python SDK and add their snapshots to a project in the workspace at /var/lib/evidently/workspace. The Evidently package is already installed in the virtual environment at /opt/evidently/venv, so you can run Python that imports it directly, for example as the evidently user.

The following example computes a data drift report over two pandas dataframes and stores it in a new project so it appears in the dashboard:

import pandas as pd
from evidently import Dataset, DataDefinition, Report
from evidently.presets import DataDriftPreset, DataSummaryPreset
from evidently.ui.workspace import Workspace

reference = pd.read_csv("reference.csv")
current = pd.read_csv("current.csv")

schema = DataDefinition(
    numerical_columns=["feature_a", "feature_b"],
    categorical_columns=["segment"],
)
ref_ds = Dataset.from_pandas(reference, data_definition=schema)
cur_ds = Dataset.from_pandas(current, data_definition=schema)

report = Report(metrics=[DataDriftPreset(), DataSummaryPreset()])
snapshot = report.run(current_data=cur_ds, reference_data=ref_ds)

ws = Workspace.create("/var/lib/evidently/workspace")
project = ws.create_project("my-model", description="production monitoring")
ws.add_run(project.id, snapshot)

Refresh the dashboard and your new project appears in the project list, with the report available under its Reports tab. Wire this into a scheduled job or your pipeline to add a snapshot on each run and build a monitoring history over time. Full API and metric documentation is at the Evidently documentation site linked from the dashboard header.

Step 9: The Data Volume

All workspace state lives on a dedicated EBS volume mounted at /var/lib/evidently. This keeps the projects and report snapshots off the operating system disk and lets you resize or snapshot them independently. Confirm the mount with:

df -h /var/lib/evidently

Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme1n1     20G  412K   19G   1% /var/lib/evidently

To grow the storage, expand the EBS volume in the AWS console, then grow the filesystem on the instance with sudo resize2fs on the underlying device.

Step 10: Enable HTTPS

The dashboard is served over plain HTTP on port 80 by nginx. For production use, place it behind TLS. Obtain a certificate for your domain, for example with a managed certificate on an Application Load Balancer in front of the instance, or with Certbot installed on the instance, then configure nginx to listen on 443 with your certificate and proxy to 127.0.0.1:8000 exactly as the bundled site does for port 80, keeping the Basic Auth directives in place. Restrict the security group so ports 80 and 443 are reachable only from the networks that use the dashboard.

Step 11: Backup and Maintenance

Back up the workspace by snapshotting the /var/lib/evidently EBS volume, which captures every project and report snapshot in one consistent set. Apply operating system security updates with sudo apt-get update && sudo apt-get upgrade and reboot when a new kernel is installed; Evidently and nginx start automatically on boot.

Support

This image is published and supported by cloudimg. Support covers deployment, workspace configuration, building data drift, data quality and model performance reports, wiring Evidently into batch or streaming pipelines with the Evidently Python SDK, nginx Basic Auth and TLS configuration, data volume resizing, and runtime tuning. Contact cloudimg through the support channel listed on the AWS Marketplace listing.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.