Gearman Job Server on Ubuntu 24.04 on Azure User Guide

Overview

This guide covers the deployment and use of Gearman Job Server on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images. Gearman is a mature, language-agnostic job server: an application hands a unit of work to the server, which dispatches it to whichever worker process is free and returns the result to the caller. Workers can be written in a different language from the client, can run on a different machine, and can be added or removed while the system is running.

The image installs gearman-job-server 1.1.20 (gearmand) and the gearman / gearadmin command line tools from the signed Ubuntu 24.04 archive (GPG-verified by apt), so the job server keeps receiving Ubuntu security updates on your running VM. The persistent background-job queue is stored on-box in SQLite, and a read-only status console is served over HTTPS.

The problem this image solves. Upstream Gearman has no authentication and no access control of any kind. Every client that can open a TCP connection to port 4730 can submit jobs, read and drain the queue, and register itself as a worker for any function, including functions your own application defines. Stock packages happily bind that port to every interface, which turns an internet-reachable Gearman server into an open remote-execution surface.

Closed by default. This image inverts that dangerous default:

• The job protocol is bound to 127.0.0.1 only (loopback). Nothing outside the instance can reach port 4730 until you deliberately change it.
• The recommended Network Security Group opens SSH and the status console only, never port 4730.
• Two independent start guards enforce the binding: the job server will not start until first boot has provisioned this instance (ConditionPathExists on a per-VM bootstrap marker), and a start-time guard refuses to start if the listen address is a world-open value such as 0.0.0.0.
• The build proves the job protocol refuses a connection on the instance's own routable address, and that nothing unexpected is listening on a non-loopback address.

Secure by default — no shared credential. The shipped image contains no valid status-console password: on the first boot of every VM a one-shot service mints a 28-character password unique to that instance, hashes it with bcrypt, issues a per-instance TLS certificate, and writes the details to /root/gearmand-credentials.txt (mode 0600, root only).

What is included:

• gearman-job-server 1.1.20 (gearmand) plus gearman-tools (the gearman client/worker CLI and gearadmin), installed from the signed Ubuntu archive and managed by systemd

• The job protocol bound to 127.0.0.1:4730 only, with a persistent SQLite background-job queue at /var/lib/gearman/gearmand-queue.db

• A read-only HTTPS status console on port 443 (/status) behind bcrypt basic auth with a per-instance password, plus an unauthenticated /healthz endpoint for load balancer and monitoring probes

• A per-instance TLS certificate and status-console password minted on first boot, recorded in /root/gearmand-credentials.txt (0600)

• Client libraries exist for PHP, Perl, Python, Ruby, Java, Go, C and Node.js, so existing workers connect without code changes

Recommended virtual machine size: Standard_B2s (2 vCPU, 4 GB RAM). Gearman is extremely light; scale by running more worker processes rather than resizing the job server.

Step 1: Deploy from the Azure Portal

Search Gearman Job Server in Marketplace, select the cloudimg publisher, and click Create. Configure the Network Security Group to allow TCP 22 for administration and TCP 443 for the status console, both restricted to trusted source addresses. Do not open TCP 4730 — the job protocol is loopback-only by design.

Step 2: Deploy from the Azure CLI

RG="gearmand-prod"; LOCATION="eastus"; VM_NAME="gearmand-01"
GALLERY_IMAGE_ID="/subscriptions/<sub-id>/resourceGroups/azure-cloudimg/providers/Microsoft.Compute/galleries/cloudimgGallery/images/gearmand/versions/<version>"
SSH_KEY="$(cat ~/.ssh/id_rsa.pub)"
az group create --name "$RG" --location "$LOCATION"
az vm create \
  --resource-group "$RG" --name "$VM_NAME" \
  --image "$GALLERY_IMAGE_ID" \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values "$SSH_KEY" \
  --public-ip-sku Standard
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 22 --priority 1001
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 443 --priority 1002

Deploy inside your own virtual network and restrict the Network Security Group to trusted networks.

Step 3: First boot and reading your credentials

On first boot the image generates this instance's status-console password, writes the bcrypt htpasswd entry, issues a per-instance TLS certificate whose subject alternative name carries this VM's address, seeds the status snapshot, and writes /root/gearmand-credentials.txt. This completes within a minute. SSH in as azureuser and read the details:

sudo cat /root/gearmand-credentials.txt

The file records GEARMAN_STATUS_USER, GEARMAN_STATUS_PASSWORD, and the console URLs. The password is unique to this instance and is never shared with any other VM.

Step 4: Confirm the service is running and bound to loopback

Both gearman-job-server.service and nginx.service are active. The job protocol is bound to 127.0.0.1:4730 only — ss shows it on loopback, while only SSH (22), HTTP (80) and HTTPS (443) are reachable off-box.

systemctl is-active gearman-job-server.service nginx.service
ss -Hltn | awk '{print $4}' | sort -u

Expected output — the job protocol appears as 127.0.0.1:4730, never 0.0.0.0:4730:

active
active
0.0.0.0:22
0.0.0.0:443
0.0.0.0:80
127.0.0.1:4730
[::]:22
[::]:443
[::]:80

The shipped exposure check enumerates every listening socket and fails closed on any unexpected non-loopback listener:

sudo /usr/local/sbin/gearman-exposure-check.sh '22 80 443'

Step 5: Submit a job and see the result

This is the core of the product: a worker registers for a named function, a client submits a job for that function, and the result is returned through the job server. Register a worker for a reverse function backed by the rev command, submit one job, and read the reversed result back. gearadmin --status then reflects the registered function and gearadmin --workers shows the connected worker.

# Register a worker for the 'reverse' function (backed by /usr/bin/rev), in the background
gearman -h 127.0.0.1 -p 4730 -w -f reverse -- /usr/bin/rev &
WORKER_PID=$!
sleep 2

# gearadmin reflects the registered function: name  queued  running  workers
echo "== gearadmin --status =="
gearadmin --status | grep '^reverse'

# Submit a job from a client and read the result returned through the job server
echo "== submit a job =="
printf 'Hello Gearman' | gearman -h 127.0.0.1 -p 4730 -f reverse
echo

# gearadmin shows the connected worker registered for 'reverse'
echo "== gearadmin --workers =="
gearadmin --workers | grep reverse

kill "$WORKER_PID" 2>/dev/null

Expected output — the client sends Hello Gearman, the worker reverses it, and the result namraeG olleH returns through the job server:

== gearadmin --status ==
reverse 0   0   1
== submit a job ==
namraeG olleH
== gearadmin --workers ==
37 127.0.0.1 - : reverse

Step 6: Background jobs and the persistent queue

Background jobs are accepted immediately and queued for a worker to pick up later. They are written to an on-box persistent SQLite queue, so work accepted before a restart is replayed after it rather than lost. Submit a background job, see it in the queue and persisted to SQLite, then drain it with a worker.

# Submit a BACKGROUND job (returns immediately with a job handle)
printf 'process-me' | gearman -h 127.0.0.1 -p 4730 -b -f demo_bg
sleep 1

# The job is queued (queued count is 1) ...
echo "== queued =="
gearadmin --status | grep '^demo_bg'

# ... and persisted to the on-box SQLite queue database
echo "== persisted to SQLite =="
sudo sqlite3 -readonly /var/lib/gearman/gearmand-queue.db \
  "select function_name, length(data) as bytes from gearman_queue where function_name='demo_bg';"

# Drain it with a one-shot worker
echo "== drain =="
gearman -h 127.0.0.1 -p 4730 -w -c 1 -f demo_bg -- /bin/cat >/dev/null && echo "background job drained by a worker"

Step 7: The read-only HTTPS status console

A read-only status console is served over HTTPS on port 443. It shows live queue depth, registered functions, connected workers and the running version — the numbers you need for capacity planning and alerting — without ever exposing the job protocol itself. It is protected by HTTP basic authentication whose password was generated for this instance on first boot. A separate unauthenticated /healthz endpoint is provided for load balancer and monitoring probes. Because the console serves the per-instance self-signed certificate, curl is used with -k here.

STATUS_USER=$(sudo grep '^GEARMAN_STATUS_USER=' /root/gearmand-credentials.txt | cut -d= -f2-)
STATUS_PASS=$(sudo grep '^GEARMAN_STATUS_PASSWORD=' /root/gearmand-credentials.txt | cut -d= -f2-)

echo -n 'healthz (unauthenticated): HTTP '; curl -sk -o /dev/null -w '%{http_code}\n' https://127.0.0.1/healthz
echo -n '/status without a password: HTTP '; curl -sk -o /dev/null -w '%{http_code}\n' https://127.0.0.1/status
echo '--- /status with this instance credentials ---'
curl -sk -u "$STATUS_USER:$STATUS_PASS" https://127.0.0.1/status | head -14

Expected output — /healthz returns 200 unauthenticated, /status returns 401 without the password and the live snapshot with it:

healthz (unauthenticated): HTTP 200
/status without a password: HTTP 401
--- /status with this instance credentials ---
cloudimg Gearman Job Server - status
generated : ...
host      : ...
listening : 127.0.0.1:4730

Step 8: The security posture, proven

The job protocol is loopback-only. Prove it: the port refuses a connection on this VM's own routable (private NIC) address — only 127.0.0.1 is bound. The start-time guard also refuses to start the service against a world-open listen address such as 0.0.0.0, the upstream default.

# The routable NIC address of this VM
PRIV=$(hostname -I | awk '{print $1}')
echo "routable address: $PRIV"

# A connection to 4730 there MUST be refused (the port is loopback-only)
if timeout 5 bash -c "exec 3<>/dev/tcp/$PRIV/4730" 2>/dev/null; then
  echo "UNEXPECTED: 4730 was reachable on $PRIV"
else
  echo "OK: 4730 refused on the routable address $PRIV (loopback only)"
fi

# The start-time guard refuses a world-open listen address (the upstream default)
GEARMAND_LISTEN=0.0.0.0 /usr/local/sbin/gearman-listen-guard.sh \
  || echo "OK: the start-time guard refuses GEARMAND_LISTEN=0.0.0.0"

# The credentials file is root-readable only
sudo stat -c '%n %a %U:%G' /root/gearmand-credentials.txt

Step 9: Exposing the job server to your own private subnet

Gearman has no native authentication, so exposing port 4730 is a deliberate decision, not a default. To let workers on other hosts in your own private subnet reach the job server, make a two-part change and scope it tightly to your own network — never to the internet.

First, change the listen address in /etc/default/gearman-job-server from loopback to this VM's private address (leave it a specific private IP, never 0.0.0.0; the start-time guard refuses a world-open value):

# /etc/default/gearman-job-server  (excerpt)
# Set to THIS VM's private NIC address, e.g. 10.0.0.14 — never 0.0.0.0.
GEARMAND_LISTEN=10.0.0.14

Then restart the service and add one Network Security Group rule allowing TCP 4730 only from your own VNet/subnet CIDR:

sudo systemctl restart gearman-job-server.service
az network nsg rule create --resource-group "$RG" --nsg-name "$NSG" \
  --name AllowGearmanFromSubnet --priority 1004 \
  --access Allow --protocol Tcp --direction Inbound \
  --destination-port-ranges 4730 --source-address-prefixes "<your-vnet-cidr>"

Because Gearman has no authentication, anyone who can reach port 4730 can submit and consume jobs. Keep the source scoped to your own private CIDR, and never add a rule for 4730 from 0.0.0.0/0 or Internet. For stronger isolation, front the job server with mutual TLS via a proxy such as stunnel, or place workers and the job server in a dedicated private subnet.

Step 10: Monitoring

Point your load balancer or monitoring system at the unauthenticated health endpoint, and read live queue metrics from the authenticated status console:

# Liveness for a load balancer or uptime monitor (no credential needed)
curl -sk -o /dev/null -w 'healthz: HTTP %{http_code}\n' https://127.0.0.1/healthz

# Live queue depth and worker counts (function  queued  running  workers)
gearadmin --status

Step 11: Writing your own worker

A worker connects to the job server, registers for one or more function names, and executes jobs. The CLI worker used above wraps any command; in production you use a client library. For example, in Python with python-gearman:

import gearman

class Worker(gearman.GearmanWorker):
    pass

worker = Worker(['127.0.0.1:4730'])

def reverse(worker, job):
    return job.data[::-1]

worker.register_task('reverse', reverse)
worker.work()

Client libraries exist for PHP, Perl, Python, Ruby, Java, Go, C and Node.js. Point each worker at 127.0.0.1:4730 on the job server host, or at the private address you configured in Step 9 for workers on other hosts in your subnet.

Troubleshooting

• First boot not finished. If /root/gearmand-credentials.txt still shows only comment lines, first boot has not completed. Check it with systemctl status gearmand-firstboot.service.
• The service will not start. The start-time guard refuses a world-open listen address. Check GEARMAND_LISTEN in /etc/default/gearman-job-server is 127.0.0.1 or a specific private address, then journalctl -u gearman-job-server.service.
• Workers on other hosts cannot connect. Confirm you changed GEARMAND_LISTEN to the VM's private address (Step 9), restarted the service, and added the scoped NSG rule for TCP 4730.
• Status console returns 401. Use the exact GEARMAN_STATUS_USER and GEARMAN_STATUS_PASSWORD from /root/gearmand-credentials.txt.

Support

cloudimg provides 24/7 technical support for this product by email at support@cloudimg.co.uk and via live chat. We help with deployment, exposing the job server to your own subnet safely, worker and client connectivity across languages, persistent queue behaviour and sizing, status console access, TLS certificate replacement, monitoring, Network Security Group scoping, and version and patch guidance.

Gearman is distributed under the BSD 3-Clause License. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.