Dagu - Self-Hosted DAG Workflow Orchestration

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

## Overview

Dagu is a self-contained workflow and cron orchestration engine. You describe your work as a DAG, a directed acyclic graph of steps, in a single readable YAML file, and Dagu runs it on demand or on a schedule, handling dependencies between steps, retries, timeouts, preconditions and lifecycle hooks. A built-in web UI lets you browse workflows, start and stop runs, watch each step of a run graph in real time, inspect per-run logs and browse execution history, while a REST API and a full command line interface drive the same engine from scripts and automation.

## Why This Image vs a Manual Install

Unlike heavyweight schedulers, Dagu needs no database, no message broker and no complex runtime. It ships as a single self-contained binary and keeps all of its state on the local filesystem, so it starts instantly and stays lightweight. This image eliminates the manual binary download and verification, systemd unit authoring, reverse-proxy configuration and credential hardening that a manual install requires. What normally takes several steps across download, service configuration and security hardening is completed for you at image build time. Launch the instance, read your unique credential, and start defining workflows. An example workflow is included so the engine has something to run on first boot.

## Application Stack

A single dagu start-all process runs the HTTP server, scheduler and executor together as a dedicated unprivileged system user under a hardened systemd unit, bound to the loopback interface only and fronted by an nginx reverse proxy on port 80. systemd starts the daemon and the web server on boot and restarts them on failure. The workflow definitions, run logs and execution history live on a dedicated data volume, independently resizable and separate from the OS disk.

## Security Hardening

This image follows a defence-in-depth approach:

• Network isolation: Dagu binds exclusively to the loopback interface; only the nginx reverse proxy is exposed
• Authentication enforced: the full web UI and REST API sit behind an nginx HTTP Basic Auth gate; only an unauthenticated health endpoint is served anonymously
• No default or shared credentials: a cryptographically random administrator password is generated on each instance's first boot and written only to a root-only file
• Least privilege: the engine runs as an unprivileged system user with NoNewPrivileges, PrivateTmp, ProtectHome and ProtectSystem set
• Dedicated data volume: workflow definitions, logs and run history live on their own disk, surviving OS-disk changes and resizable independently
• Automatic restart: systemd monitors both Dagu and nginx and restarts them on failure

Buyers requiring formal compliance documentation should contact cloudimg to discuss specific requirements.

## Concrete Use Cases

• Cron replacement and internal automation: Replace a sprawl of crontab entries with declarative YAML DAGs that carry dependencies, retries and a UI showing exactly which step failed and why
• Data pipelines and batch jobs: Orchestrate nightly extract-transform-load runs, database backups and report generation on a single machine, with cron scheduling, timezone support and per-run logs
• Edge and single-node automation: Run dependable scheduled automation on an edge or IoT host where a full distributed scheduler would be overkill, with all state on the local filesystem and nothing external to manage

## AWS Integration

Deploy on any EC2 instance type. Use the unauthenticated health endpoint with an Application Load Balancer for availability monitoring. Back up your workflow definitions and run history using EBS snapshots or AWS Backup. Resize the dedicated data volume independently of the OS disk as your run history grows.

## Getting Started

Read your credential from the root-only file, open the web interface in your browser, sign in as admin, and browse the included example workflow. Define your own workflows as YAML DAGs under the data directory, run and monitor them from the UI or the command line, and schedule them with cron expressions. To evaluate the AMI, launch it on a small instance type and run the included example workflow to confirm the full stack is working before building your own.

## Resources

cloudimg provides a step-by-step user guide for this image at https://www.cloudimg.co.uk/guides/dagu-aws/, covering credential retrieval, signing in, defining your first DAG and scheduling. Two example workflows - a hello DAG and an ETL-style pipeline - ship in the image under the data directory so the engine works out of the box.

## cloudimg Support

24/7 technical support by email and chat. Help with deployment, reverse-proxy termination with your own domain and certificate, workflow and schedule configuration, and backup planning for your workflow store.