Dagu Workflow Orchestration on AWS User Guide

Overview

Dagu is a self-contained workflow and cron orchestration engine. You describe your work as a DAG - a directed acyclic graph of steps - in a single, readable YAML file, and Dagu runs it on demand or on a schedule, resolving dependencies between steps and handling retries, timeouts, preconditions and lifecycle hooks. A built-in web UI lets you browse workflows, start and stop runs, watch each step of a run graph in real time and inspect per-run logs and execution history, while a REST API and a full command line interface drive the same engine from scripts. Because it needs no database and no message broker and keeps all of its state on the local filesystem, it starts instantly and is ideal for cron replacement, data pipelines, backups, batch jobs and edge automation on a single machine.

This cloudimg image installs the pinned Dagu 2.10.7 single Go binary running under systemd as dagu start-all (the HTTP server, scheduler and executor in one process), then locks it down for a marketplace appliance. Dagu binds to loopback 127.0.0.1:8080 so its UI and API are never exposed directly, and an nginx reverse proxy on port 80 adds a per-instance HTTP Basic Auth gate in front of it. Dagu's own web UI is unauthenticated by default, so this image closes that gap: nginx requires the admin credential (a unique password generated on the first boot of your instance) for the entire UI and API, while an unauthenticated /healthz endpoint is exposed for load balancer probes. Two example workflows are bundled so the engine has something to run immediately. Backed by 24/7 cloudimg support.

What is included:

• Dagu 2.10.7 installed as a single Go binary (/usr/local/bin/dagu) running as the dagu systemd service (dagu start-all)
• The Dagu web UI and REST API on :80, fronted by nginx with the binary bound to loopback only
• Per-instance HTTP Basic Auth (user admin) protecting the entire UI and API, with a unique password generated on first boot
• Dagu bound to 127.0.0.1:8080 - nothing is exposed to the network directly
• Dagu's own native basic auth also enabled as defence-in-depth on the loopback interface
• Example workflows at /var/lib/dagu/dags/ (hello.yaml and etl-pipeline.yaml) so the engine works out of the box
• DAGU_HOME=/var/lib/dagu on a dedicated data volume, holding your workflow definitions (dags/), run logs and state
• dagu.service and nginx.service as systemd units, enabled and active
• An unauthenticated /healthz endpoint for Elastic Load Balancing health checks
• Ubuntu 24.04 LTS base, fully patched at build time
• 24/7 cloudimg support

Prerequisites

An AWS account, an EC2 key pair in the target region, and a VPC with a public subnet. m5.large (2 vCPU / 8 GiB RAM) is the recommended instance type; Dagu is light on resources. Security group inbound: allow 22/tcp from your management network for SSH and 80/tcp for the web UI and API. Dagu serves plain HTTP on port 80; for production use, terminate TLS in front of it with your own domain and restrict access to trusted IP ranges (see Maintenance).

Step 1 - Launch the AMI

Subscribe to the listing in AWS Marketplace and choose Launch. Pick the Ubuntu 24.04 version, an m5.large instance type, your key pair and a public subnet, and a security group that allows 22/tcp (from your IP) and 80/tcp. Launch the instance and wait for the status checks to pass.

Step 2 - Connect to your instance

Connect over SSH as the default login user for the OS variant you launched.

ssh -i /path/to/key.pem ubuntu@<instance-public-ip>

Step 3 - Confirm the services are running

Dagu and nginx are managed by systemd and come up on boot. Confirm both are active:

systemctl is-active dagu nginx

Dagu binds only to the loopback interface - confirm nothing serves the raw application port to the network:

ss -tln | grep '127.0.0.1:8080'

Check the installed Dagu version:

/usr/local/bin/dagu version

Step 4 - Retrieve your unique admin credential

On the first boot of your instance, a cryptographically random admin password was generated and written to a root-only file, together with the URL to open. Read it (the password is redacted here; drop the sed filter to see it):

sudo sed 's/^DAGU_PASSWORD=.*/DAGU_PASSWORD=<redacted - remove this sed filter to reveal>/' /root/dagu-credentials.txt

The file is 0600 root:root, so only an administrator on the instance can read it. There is no default or shared password baked into the image.

Step 5 - Confirm authentication from the command line

Before opening a browser, prove the security model from the instance itself: the health endpoint is open, the UI is closed without credentials, and the per-instance password authenticates. This reads the password into a shell variable without printing it:

PW=$(sudo grep '^DAGU_PASSWORD=' /root/dagu-credentials.txt | cut -d= -f2-)
curl -s -o /dev/null -w 'health (unauthenticated): %{http_code}\n' http://127.0.0.1/healthz
curl -s -o /dev/null -w 'UI without credentials:    %{http_code}\n' http://127.0.0.1/
curl -s -o /dev/null -w 'UI with admin password:    %{http_code}\n' -u "admin:$PW" http://127.0.0.1/

You should see 200 for the health check, 401 for the unauthenticated UI, and 200 once the correct password is supplied.

Step 6 - Sign in to the web UI

Open http://<instance-public-ip>/ in your browser. nginx prompts for HTTP Basic Auth; sign in with the username admin and the password from Step 4. The dashboard opens on the Timeline, showing recent runs across all your workflows.

Open Workflows to see every DAG the engine knows about, each with its schedule, last status and controls to run, stop or refresh it. The image ships with two example workflows.

Select a workflow to see its run graph. Each step is a node, dependencies are edges, and colour shows status - here the etl-pipeline example fans out from extract to two transform steps and back into a single load, all succeeded.

The History view shows each step's status across every run, with the run graph, timeline and per-step logs a click away.

Step 7 - Run and define workflows

The bundled example workflows live under DAGU_HOME/dags. List them:

ls -1 /var/lib/dagu/dags/

Trigger a run from the command line (or press Start in the UI) and watch it succeed:

sudo -u dagu env DAGU_HOME=/var/lib/dagu /usr/local/bin/dagu start /var/lib/dagu/dags/hello.yaml

To create your own workflow, drop a YAML file into /var/lib/dagu/dags/. A DAG is a list of steps; each step has an id, a run command, and optional depends on other steps. An optional top-level schedule runs it automatically on a cron expression. For example:

schedule: "0 2 * * *"      # run at 02:00 every day
steps:
  - id: backup
    run: /usr/local/bin/my-backup.sh
  - id: notify
    run: echo "backup finished"
    depends: backup

Dagu picks up new and edited files automatically, so the workflow appears in the UI without a restart.

Step 8 - Where Dagu keeps its state

DAGU_HOME=/var/lib/dagu is a dedicated EBS data volume, separate from the OS disk, so your workflow definitions, run logs and execution history survive OS-disk changes and can be resized independently. Confirm it is its own filesystem:

findmnt /var/lib/dagu && df -h /var/lib/dagu

Everything under dags/ (definitions), logs/ (per-run logs) and data/ (run state) lives here. Back the volume up with EBS snapshots or AWS Backup.

Maintenance

• Health checks: point an Application Load Balancer target group at /healthz on port 80 - it returns 200 without authentication so probes never need the admin credential.
• Add TLS for production: Dagu is served over plain HTTP on port 80. For production, put your own domain and certificate in front of nginx (for example with Certbot, or by terminating TLS at an ALB) and restrict the security group to trusted IP ranges.
• Rotate the admin password: edit DAGU_AUTH_BASIC_PASSWORD in /etc/dagu/dagu.env, regenerate the nginx credential with sudo htpasswd -B /etc/nginx/.dagu.htpasswd admin, then sudo systemctl restart dagu nginx.
• Service management: sudo systemctl restart dagu and sudo systemctl restart nginx; logs via journalctl -u dagu and journalctl -u nginx.

Support

cloudimg provides 24/7 technical support for this product by email and live chat at support@cloudimg.co.uk. We help with deployment, reverse-proxy termination with your own domain and TLS certificate, workflow and schedule configuration, and backup planning for your workflow store. For refund requests, contact support with your AWS account ID and subscription details.