Gearman Job Server on AWS User Guide

Overview

Gearman is a mature, language agnostic job server. Your application hands a unit of work to the job server, the job server hands it to whichever worker process is free, and the result comes back to the caller. Workers can be written in a different language from the client, can live on a different machine, and can be added or removed while the system is running.

Read this before anything else: Gearman has no authentication and no access control. Any client that can open a TCP connection to port 4730 can submit jobs, read and drain the queue, and register itself as a worker for any function, including functions your own application defines. An internet reachable Gearman job server is an open remote execution surface.

This image ships it closed. The job protocol is bound to 127.0.0.1 only. The recommended security group never opens 4730. Letting workers on other hosts reach the job server is a deliberate, documented two step change covered in the exposure section near the end of this guide.

This is a headless product. There is no web application to log in to. You administer it over SSH, and a read only HTTPS status console shows you queue depth and worker counts.

What is included:

• Gearman job server gearmand 1.1.20 and the gearman / gearadmin command line tools, installed from the signed distribution archive (BSD 3-Clause License), so the instance keeps receiving distribution security updates
• The job protocol bound to 127.0.0.1:4730 through /etc/default/gearman-job-server, managed by systemd
• A persistent SQLite background job queue at /var/lib/gearman/gearmand-queue.db on its own dedicated, independently resizable block storage volume
• A read only HTTPS status console on port 443, protected by HTTP basic authentication whose password is generated uniquely for your instance on first boot and stored bcrypt hashed
• An unauthenticated /healthz endpoint for load balancer and monitoring probes
• gearmand-firstboot.service, which mints the console password, issues a per instance TLS certificate and writes /root/gearmand-credentials.txt (mode 0600 root:root)
• /usr/local/sbin/gearman-exposure-check.sh, a fail closed audit of every listening socket on the instance
• 24/7 cloudimg support

Prerequisites

An AWS account, an EC2 key pair, and a VPC subnet. The security group should allow inbound TCP 22 (SSH) and TCP 443 (status console) from your trusted CIDR. Do not open TCP 4730 to the internet. m5.large is a comfortable default.

Connecting to your instance

SSH in as the default login user for the operating system variant you launched. This listing currently ships one variant:

OS variant	SSH login user
Ubuntu 24.04 LTS	ubuntu

ssh ubuntu@<public-ip>

Step 1: Launch from AWS Marketplace

Subscribe to the cloudimg Gearman Job Server listing and launch an instance. Pick m5.large, choose your VPC and subnet, and attach a security group allowing inbound TCP 22 and TCP 443 from your trusted CIDR only.

The instance boots with two volumes: the operating system disk, and a dedicated volume mounted at /var/lib/gearman that holds the persistent job queue.

Step 2: SSH In

ssh ubuntu@<public-ip>

The login banner prints the status console URL and the commands you need most.

Step 3: Confirm the Services Are Running

Two services make up the appliance: the job server itself, and the nginx front end that serves the status console.

sudo systemctl is-active gearman-job-server nginx

active
active

Step 4: Confirm the Job Protocol Is Loopback Only

This is the single most important check on the instance. gearmand must be listening on 127.0.0.1 and nowhere else.

BIND="$(sudo ss -Hltn | awk '{print $4}' | grep ':4730$')"
echo "Gearman job protocol is listening on: ${BIND}"
case "${BIND}" in
    127.0.0.1:4730) echo "CONFIRMED: loopback only - not reachable from any other host" ;;
    *) echo "This job server is NOT loopback bound; review /etc/default/gearman-job-server"; exit 1 ;;
esac

Gearman job protocol is listening on: 127.0.0.1:4730
CONFIRMED: loopback only - not reachable from any other host

The shipped configuration file records the same thing, along with the procedure for widening access safely.

sudo grep -E '^GEARMAND_' /etc/default/gearman-job-server

GEARMAND_LISTEN=127.0.0.1
GEARMAND_PORT=4730
GEARMAND_QUEUE_DB=/var/lib/gearman/gearmand-queue.db
GEARMAND_EXTRA_OPTS=

Step 5: Audit Every Listening Socket

The image ships a fail closed exposure audit. It lists every socket bound to a non loopback address and exits non zero if anything outside the allow list appears, or if the job protocol has been bound off loopback. The same check runs during the build, before the image is captured.

sudo /usr/local/sbin/gearman-exposure-check.sh "22 80 443"

--- non-loopback TCP listeners ---
  tcp 0.0.0.0:443
  tcp 0.0.0.0:22
  tcp 0.0.0.0:80
  tcp [::]:443
  tcp [::]:22
  tcp [::]:80
--- non-loopback UDP listeners ---
  udp 172.31.93.204%ens5:68
--- gearman job protocol binding ---
  127.0.0.1:4730
exposure check clean: only 22 80 443 (tcp) are non-loopback

Only SSH and the status console are reachable from the network. The job protocol is not.

Step 6: Read Your Per Instance Credentials

First boot writes a root only credentials file. It holds the status console user, the password generated for this instance, and the console URLs. Nothing here is shared with any other instance.

The command below prints the key names only, never the values, so it is safe to run in a shared terminal or paste into a ticket.

sudo grep -oE '^[A-Z_]+=' /root/gearmand-credentials.txt

GEARMAN_STATUS_USER=
GEARMAN_STATUS_PASSWORD=
GEARMAN_STATUS_URL=
GEARMAN_HEALTH_URL=
GEARMAN_JOB_SERVER=

To read the whole file, including the password, run sudo cat /root/gearmand-credentials.txt on your own instance.

Step 7: The Status Console Requires Authentication

The console is served over HTTPS. Without credentials it returns 401; with them it returns 200. The block below reads the password into a shell variable rather than printing it.

sudo bash -c '
U=$(grep -m1 "^GEARMAN_STATUS_USER=" /root/gearmand-credentials.txt | cut -d= -f2-)
P=$(grep -m1 "^GEARMAN_STATUS_PASSWORD=" /root/gearmand-credentials.txt | cut -d= -f2-)
UNAUTH=$(curl -sk -o /dev/null -w "%{http_code}" https://127.0.0.1/status)
AUTH=$(curl -sk -o /dev/null -w "%{http_code}" -u "$U:$P" https://127.0.0.1/status)
echo "status console, no credentials   : HTTP ${UNAUTH}"
echo "status console, with credentials : HTTP ${AUTH}"
[ "${UNAUTH}" = 401 ] && [ "${AUTH}" = 200 ] || { echo "the status console did not behave as documented"; exit 1; }
'

status console, no credentials   : HTTP 401
status console, with credentials : HTTP 200

The certificate is a per instance self signed certificate issued at first boot, with your instance address in its subject alternative name. -k is used here because it is self signed; see the certificate section below for replacing it with your own.

Step 8: Read the Status Console

The console shows the server version and process id, the queue (one row per registered function, with queued, running and worker counts) and the connected workers. It is strictly read only, and it never proxies the job protocol.

sudo bash -c '
U=$(grep -m1 "^GEARMAN_STATUS_USER=" /root/gearmand-credentials.txt | cut -d= -f2-)
P=$(grep -m1 "^GEARMAN_STATUS_PASSWORD=" /root/gearmand-credentials.txt | cut -d= -f2-)
curl -sk -u "$U:$P" https://127.0.0.1/status'

cloudimg Gearman Job Server - status
generated : 2026-07-26T08:39:13Z
host      : ip-172-31-93-204
listening : 127.0.0.1:4730

== server ==
version : 1.1.20+ds
pid     : 7458

== queue (function  queued  running  workers) ==
.

== workers ==
37 127.0.0.1 - :
.

From your own workstation, browse to the GEARMAN_STATUS_URL value in the credentials file and log in with the status user and the generated password.

Step 9: The Unauthenticated Health Endpoint

Load balancer and monitoring probes need a target that does not require a credential. /healthz is a static 200 on both port 80 and port 443, and it exposes nothing about the queue.

curl -s -o /dev/null -w 'health endpoint: HTTP %{http_code}\n' http://127.0.0.1/healthz

health endpoint: HTTP 200

Step 10: Submit Your First Job

A Gearman round trip needs two halves: a worker that registers a function name and does the work, and a client that submits a job for that function name and waits for the result.

The block below registers a worker for a function called reverse that runs the rev command, submits one job to it, and asserts the result came back correctly. -c 1 makes the worker exit after a single job so the example is self contained.

timeout 30 gearman -h 127.0.0.1 -p 4730 -w -c 1 -f reverse -- rev >/dev/null 2>&1 &
sleep 2
RESULT="$(printf 'gearman on cloudimg' | timeout 30 gearman -h 127.0.0.1 -p 4730 -f reverse)"
wait
echo "worker returned: ${RESULT}"
[ "${RESULT}" = "gmiduolc no namraeg" ] || { echo "the round trip did not return the reversed payload"; exit 1; }

worker returned: gmiduolc no namraeg

The payload went from the client, through the job server, into the worker's standard input; the worker's standard output came back as the job result.

Step 11: Background Jobs and the Persistent Queue

Background jobs are the reason most people run Gearman: the client submits and returns immediately, and a worker picks the job up whenever it is free. This image writes those jobs to a SQLite queue on the dedicated volume, so a job accepted before a restart is still there afterwards.

The block below submits a background job, proves it is both queued in memory and persisted to disk, then starts a worker to drain it and proves the queue is empty again.

printf 'nightly-report' | timeout 20 gearman -h 127.0.0.1 -p 4730 -b -f build_report
sleep 2
gearadmin --status | grep '^build_report' || { echo "the background job was not queued"; exit 1; }
sudo sqlite3 -readonly /var/lib/gearman/gearmand-queue.db "select function_name, unique_key from gearman_queue;"
timeout 30 gearman -h 127.0.0.1 -p 4730 -w -c 1 -f build_report -- cat >/dev/null 2>&1
sleep 1
echo "after the worker ran:"
gearadmin --status | grep '^build_report'
REMAIN="$(sudo sqlite3 -readonly /var/lib/gearman/gearmand-queue.db 'select count(*) from gearman_queue;')"
echo "rows still persisted: ${REMAIN}"
[ "${REMAIN}" = "0" ] || { echo "the queue did not drain"; exit 1; }

build_report    1   0   0
build_report|7aa82032-88cd-11f1-8d4f-12d811d5a21d
after the worker ran:
build_report    0   0   0
rows still persisted: 0

The four columns from gearadmin --status are function name, queued, running and connected workers. The job appeared as one queued item and one row in the SQLite queue, and both cleared once a worker had run it.

Step 12: Inspect the Queue

gearadmin is the administration client. It talks to the job server over the same loopback socket.

gearadmin --status

build_report    0   0   0
.

Other useful subcommands, all safe to run on a live server:

gearadmin --server-version
gearadmin --getpid
gearadmin --workers

1.1.20+ds
7458
37 127.0.0.1 - :
.

Step 13: The Dedicated Queue Volume

The persistent queue lives on its own block storage volume, not on the operating system disk, so you can resize it independently as your queue depth grows.

findmnt -no SOURCE,TARGET,FSTYPE,SIZE /var/lib/gearman
df -h /var/lib/gearman | tail -1

/dev/nvme1n1 /var/lib/gearman ext4   19.5G
/dev/nvme1n1     20G   36K   19G   1% /var/lib/gearman

The mount is recorded in /etc/fstab by filesystem UUID, so it reattaches on every boot. To grow it, expand the EBS volume in the AWS console and then run sudo resize2fs /dev/nvme1n1.

Step 14: Versions

/usr/sbin/gearmand -V
dpkg-query -W -f='${Package} ${Version}\n' gearman-job-server gearman-tools

gearmand 1.1.20+ds - https://github.com/gearman/gearmand/issues
gearman-job-server 1.1.20+ds-1.2build4
gearman-tools 1.1.20+ds-1.2build4

Exposing the Job Server to Your Own Private Subnet

Everything above ran on the instance itself, because that is the only place the job protocol is reachable from. If your workers run on other hosts, you can widen access, but do it deliberately.

Never open TCP 4730 to 0.0.0.0/0. There is no authentication behind it. A client that reaches it can register itself as a worker for one of your function names and receive your jobs, or drain your queue.

The safe pattern is two changes that must both be made:

1. Change the listen address on the instance. Edit /etc/default/gearman-job-server and set GEARMAND_LISTEN to the instance's private address (or 0.0.0.0 if you have several interfaces), then restart the service:

sudo sed -i 's/^GEARMAND_LISTEN=.*/GEARMAND_LISTEN=0.0.0.0/' /etc/default/gearman-job-server
sudo systemctl restart gearman-job-server
sudo ss -Hltn | awk '{print $4}' | grep ':4730$'

2. Add one security group rule scoped to your own network. In the AWS console, or with the CLI, allow TCP 4730 from your VPC or subnet CIDR and nothing wider:

aws ec2 authorize-security-group-ingress \
  --group-id sg-your-security-group \
  --protocol tcp --port 4730 \
  --cidr 10.0.0.0/16

Re-run the exposure audit afterwards with 4730 added to the allow list, so you have a record of exactly what is reachable:

sudo /usr/local/sbin/gearman-exposure-check.sh "22 80 443 4730"

If you want defence in depth, keep the job server on loopback and give each worker host an SSH tunnel to it instead. That keeps the protocol entirely off the network:

ssh -N -L 4730:127.0.0.1:4730 ubuntu@your-job-server

Connecting Your Own Workers and Clients

Gearman client libraries exist for most languages, and they all speak the same wire protocol as the gearman command line tool used above. Point them at the job server address and the function names you register.

Common libraries, installed from your own language's package manager on your own application hosts, not on this instance:

Language	Library
PHP	the gearman PECL extension
Python	python-gearman or gearman3
Perl	Gearman::Client and Gearman::Worker
Ruby	gearman-ruby
Java	java-gearman-service
Go	github.com/mikespook/gearman-go
Node.js	node-gearman
C / C++	libgearman, already present on this instance

The pattern is always the same: a worker calls add_function("name", handler) and loops; a client calls do("name", payload) for a synchronous result or do_background("name", payload) to submit and return. Scale by running more worker processes, on this instance or on other hosts, against the one job server.

Replacing the Status Console Certificate

The console uses a self signed certificate issued at first boot, which is why curl -k appears above. To use your own certificate, replace these two files and reload nginx:

sudo install -o root -g root -m 0644 your-certificate.crt /etc/nginx/tls/gearman.crt
sudo install -o root -g root -m 0600 your-private-key.key /etc/nginx/tls/gearman.key
sudo nginx -t
sudo systemctl reload nginx

Changing the Status Console Password

The console password lives in /etc/nginx/gearman-status.htpasswd, bcrypt hashed. To rotate it, generate a new entry and update the credentials file so the value on the instance stays accurate:

NEWPW="$(openssl rand -base64 48 | tr -dc 'A-Za-z0-9' | cut -c1-28)"
sudo htpasswd -nbB status "$NEWPW" | sudo tee /etc/nginx/gearman-status.htpasswd >/dev/null
sudo chown root:www-data /etc/nginx/gearman-status.htpasswd
sudo chmod 640 /etc/nginx/gearman-status.htpasswd
sudo systemctl reload nginx
echo "new password: $NEWPW"

Troubleshooting

The job server will not start. Check the unit log; the most common cause is a GEARMAND_LISTEN value that is not an address on this instance.

sudo systemctl is-active gearman-job-server

active

If it is not active, run sudo journalctl -u gearman-job-server -n 50 --no-pager on your instance to see why.

A client hangs and never gets a result. There is no worker registered for that function name. Check the queue: a function with a worker count of zero has nothing to run it.

The status console returns 401 with the right password. First boot may not have completed. Confirm the sentinel exists, then re-read the credentials file.

sudo test -f /var/lib/cloudimg/gearmand-firstboot.done && echo "first boot completed"

first boot completed

Background jobs disappear after a restart. They should not; that is what the persistent queue is for. Confirm the queue database is present and on the dedicated volume with the commands in Step 13.

Support

cloudimg provides 24/7 technical support by email and live chat for this product, covering deployment, exposing the job server safely to your own subnet, worker connectivity, persistent queue behaviour, status console access and sizing.

Email: support@cloudimg.co.uk

Gearman is distributed under the BSD 3-Clause License. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.