Hatchet - Distributed Task Queue and Workflow Orchestration

AWS Application Infrastructure

Hatchet, an open source distributed task queue and workflow orchestration engine, ready on first boot.

Base
Hardened build
minimal ports, security patches applied at build time
Access
Unique credentials
generated on first boot, readable only by root
Verified
Boots working
services pass a health gate before release
Support
24/7, 365 days
by email and live chat, 24 hour response SLA

Overview

Hatchet is an open source distributed task queue and background job engine for durable workflow orchestration. You define workflows and tasks in code with your language SDK, and Hatchet runs them across a fleet of workers with automatic retries, concurrency and rate limits, fan out and scheduling, then shows every run, worker, event and schedule in a modern web dashboard. Teams use it to run reliable background jobs, event driven pipelines and multi step AI and data workflows, owned inside their own cloud account rather than a hosted service.

Why the cloudimg image

cloudimg ships Hatchet hardened and fully patched, with a bundled PostgreSQL and RabbitMQ on the same instance. Security is enforced from first boot: a unique administrator, database password, message broker password and encryption keyset are generated on each instance, the default account and default secrets are never shipped, open sign up and tenant creation are disabled so the instance cannot be seized, and PostgreSQL and RabbitMQ are bound to the internal container network only. It comes with a paired deployment guide and 24/7 cloudimg support.

Common uses

  • Reliable background jobs and task queues
  • Event driven and scheduled workflow orchestration
  • Multi step AI and data processing pipelines

Key features

  • Prebuilt and secure in minutes: unlike a manual install that requires standing up Docker, PostgreSQL, RabbitMQ and the Hatchet engine, wiring them on a private network, generating encryption keysets and cookie secrets and removing the stock default administrator, this image completes every build and hardening step at image creation time. Launch an instance and reach a working, authenticated dashboard over TLS without editing a single configuration file.
  • Secure by default with unique per instance secrets: every instance generates its own administrator password, PostgreSQL password, RabbitMQ password and encryption keyset on first boot, and the well known default administrator is never created. Open sign up and tenant creation are disabled so the instance cannot be seized; the engine, database and broker bind to a private network and only the nginx TLS proxy is exposed.
  • Durable distributed task queue with a modern dashboard and 24/7 support: define workflows and tasks in code, run them across a fleet of workers with automatic retries, concurrency limits, fan out and scheduling, and observe every run, worker, event and schedule in the browser. The Docker data root, workflow database and broker state live on a dedicated independently resizable data volume. Backed by 24/7 cloudimg technical support via email and chat.

See it running

Real screenshots taken while testing this image against its deployment guide.

Hatchet - Distributed Task Queue and Workflow Orchestration screenshot 1 Hatchet - Distributed Task Queue and Workflow Orchestration screenshot 2 Hatchet - Distributed Task Queue and Workflow Orchestration screenshot 3 Hatchet - Distributed Task Queue and Workflow Orchestration screenshot 4

Description

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

## Overview

Hatchet is an open source distributed task queue and background job engine for durable workflow orchestration. You define workflows and tasks in code with your existing language SDK, and Hatchet runs them across a fleet of workers with automatic retries, concurrency and rate limits, fan out and scheduling, then shows you every run, worker, event and schedule in a modern web dashboard. This AMI delivers Hatchet fully installed and secured so a working orchestration engine is operational within minutes of launch.

## Why This Image vs a Manual Install

Running Hatchet yourself means standing up Docker, a PostgreSQL datastore, a RabbitMQ message broker and the Hatchet engine, wiring them together on a private network, generating encryption keysets and cookie secrets, and removing the well known default administrator the stock compose seeds. This image completes every one of those steps at build time. The engine, dashboard and API are bound to the loopback interface and fronted by an nginx reverse proxy with TLS; PostgreSQL and RabbitMQ are never exposed on a host port; a unique administrator, database password, broker password and encryption keyset are generated on each instance's first boot. Launch the instance, read your unique credential, and sign in.

## Application Stack

Hatchet is delivered as the official all in one engine, API and dashboard container, pinned by digest, alongside a bundled PostgreSQL and RabbitMQ, also pinned by digest and reachable only inside a private container network. Docker Engine and its data root, including every pre pulled image and the database and broker state, live on a dedicated independently resizable data volume separate from the OS disk. systemd brings the whole stack up on boot and restarts it on failure. The dashboard and API are served behind nginx over TLS.

## Security Hardening

This image follows a defence in depth approach:

  • No default or shared credentials: the well known default administrator the stock Hatchet compose seeds is never created; a unique administrator with a cryptographically random password is generated on each instance's first boot and written only to a root-only file
  • Per instance secrets: the PostgreSQL password, the RabbitMQ password, the encryption master keyset, the JWT keysets and the cookie secrets are all generated per instance on first boot, never shipped in the image
  • Closed registration: open sign up and tenant creation are disabled, so the instance cannot be seized; new members join by administrator invitation only
  • Network isolation: the engine, dashboard and API bind exclusively to the loopback interface; PostgreSQL and RabbitMQ are never published to a host port; only the nginx reverse proxy is exposed, over TLS
  • Dedicated data volume: the Docker data root, the workflow database and the broker state live on their own disk, surviving OS-disk changes and resizable independently
  • Automatic restart: systemd monitors the stack and restarts it on failure

Buyers requiring formal compliance documentation should contact cloudimg to discuss specific requirements.

## Concrete Use Cases

  • Reliable background jobs for a web application: offload email sending, PDF generation, billing and webhook delivery from the request path onto durable Hatchet tasks with automatic retries, so a transient failure never loses work
  • AI and data pipelines: orchestrate multi step retrieval, model inference and post processing as a workflow with concurrency limits and fan out, observing every run and worker in the dashboard
  • Scheduled and event driven automation: run cron style schedules and event triggered workflows across a fleet of workers, with per run visibility into status, timing and failures

## AWS Integration

Deploy on any EC2 instance type; the recommended size is m5.large. Use the unauthenticated liveness endpoint with an Application Load Balancer for health monitoring. Back up the workflow database and broker state using EBS snapshots or AWS Backup, and resize the dedicated data volume independently of the OS disk as your throughput grows. Connect external workers over the engine gRPC port once you open it in the security group.

## Getting Started

Read your unique administrator credential from the root-only file, open the web dashboard in your browser over TLS, sign in, create an API token and point your Hatchet SDK workers at the instance. Define your first workflow in code, run it, and watch every step in the dashboard.

## cloudimg Support

24/7 technical support by email and chat. Help with deployment, reverse-proxy termination with your own domain and certificate, connecting external workers, scaling PostgreSQL and RabbitMQ, and backup planning for your workflow database.

Related technologies

hatchettask queueworkflow orchestrationbackground jobsdistributedjob schedulerself-hosteddurable executionpostgresqlrabbitmq