Dagster Data Orchestration Platform

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

Dagster Data Orchestration - Ready to Run on AWS

Launch a fully configured Dagster orchestration platform on EC2 and start building, scheduling, and observing data pipelines in minutes - not hours. This AMI eliminates the manual setup of Python environments, reverse proxies, WebSocket forwarding, and service management so your team can focus on defining assets and shipping data products.

---

Why This AMI Instead of a Manual Install

Setting up Dagster from scratch means installing Python dependencies, configuring systemd services, troubleshooting WebSocket proxy headers (the UI hangs without them), and hardening credentials. This image solves every one of those problems out of the box:

• UI accessible behind authenticated nginx proxy within minutes of first boot
• No hanging loading screens - WebSocket upgrade headers are pre-configured
• No shared or default credentials - a unique admin password is generated per instance
• No risk of data loss from root volume resizing - instance storage lives on a dedicated, independently resizable data disk

---

Application Stack

• Dagster installed in a dedicated Python virtual environment under /opt/dagster, run by an unprivileged service account
• Instance directory (configuration, run storage, event log, schedule storage) on a separate data disk for independent resizing
• Two systemd services: the webserver (UI and GraphQL API) and the daemon (schedules, sensors, run queue)
• Nginx reverse proxy on port 80 with HTTP Basic authentication and full WebSocket upgrade support
• Default SQLite storage suitable for development, demos, and small teams; upgradeable to PostgreSQL for production scale

---

Ready-to-Run Demo Project

A demo project ships at /opt/dagster/demo containing software-defined assets and a job with a pre-configured workspace file. It loads automatically on start, giving you a working example to explore asset materialization, lineage graphs, and scheduling. Point the workspace at your own code location and restart the services to go live with your pipelines.

---

Secure First Boot

On first launch, a one-shot service generates a fresh admin password unique to your instance, writes it to the nginx credentials file, and stores it in a root-only file. The Dagster webserver binds exclusively to loopback and is never exposed without authentication. No shared or default credentials ship in the image.

---

Use Cases

• ELT/ETL orchestration: Schedule nightly extract-load-transform pipelines across multiple data sources with full lineage visibility and automatic retry on failure
• Software-defined asset management: Define assets in plain Python, materialize them on demand or on schedule, and trace lineage from source to dashboard
• Event-driven data jobs: Use Dagster sensors to trigger pipelines when new files land in S3 or upstream tables update
• Analytics and ML workflows: Orchestrate feature engineering, model training, and reporting jobs from a single control plane
• Development and demo environments: Spin up a self-contained Dagster instance for prototyping pipelines before promoting to a shared environment

---

cloudimg Support

24/7 technical support by email and live chat. Our engineers assist with deployment, defining assets and jobs, code locations and workspaces, scheduling and sensors, migrating from SQLite to PostgreSQL, TLS configuration, and performance tuning. Critical issues receive a one-hour average response time.

To discuss your deployment needs or request a guided walkthrough, contact support@cloudimg.co.uk.

---

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.