Screwdriver 8.0.207 on AWS User Guide

Overview

This guide covers launching and verifying Screwdriver 8.0.207 on Ubuntu 24.04 from cloudimg on Amazon EC2. It is one version of the cloudimg Hardened Apache Subversion (SVN) & CI/CD Tools AMIs listing on AWS Marketplace; you pick it as the software version when you launch.

Screwdriver is an open source build and continuous delivery platform (BSD 3-Clause licence), first written at Yahoo. You describe a pipeline in a screwdriver.yaml file in your repository: jobs, the container image each job runs in, its steps and what triggers it. Screwdriver receives your GitHub webhooks, runs each job in a fresh container, streams the logs to its web interface and keeps build artifacts in its store.

What is on the image:

• The Screwdriver API 8.0.207, the web UI 1.0.1429 and the artifact and log store 7.0.6 from the official screwdrivercd images, plus PostgreSQL 16.14 from the official postgres image as the datastore. Each is pinned by its linux/amd64 digest and carries one cloudimg layer of operating system security updates; systemd starts each by image ID with --pull=never, so the instance never fetches a different build. The upstream image tagged v8.0.207 carries the 8.0.206 API package, so cloudimg adds the published 8.0.207 package on top, checked against the npm registry's own checksum: the API you run is 8.0.207
• A separate rootless Docker engine for builds (sdbuild-docker), run by the unprivileged system account sdbuild. Builds never run on the engine that runs Screwdriver itself (see Security posture)
• nginx serving HTTPS on port 443 with a certificate created on your instance. Port 80 only redirects to HTTPS
• screwdriver-allow-user (choose who may log in), screwdriver-configure-github (connect your GitHub OAuth app), screwdriver-set-host (serve Screwdriver under your own DNS name) and screwdriver-selftest (an end-to-end check)

Default user: ubuntu · Architecture: x86_64 · Root volume: 30 GiB gp3

Security posture

• Builds run on a rootless engine, never on the host's. Every build runs on a second Docker engine that runs rootless under the unprivileged sdbuild account, inside a user namespace mapped onto sdbuild's subordinate IDs. The Screwdriver API reaches only that engine's socket; it has no /var/run/docker.sock mount and no path to the root engine that runs the API, UI, store and PostgreSQL.
• Docker inside builds is off until you turn it on. Upstream, a job that sets the screwdriver.cd/dockerEnabled annotation gets a privileged container with the build engine's Docker socket, and no setting turns that off. All builds share that one engine, so a build holding the socket can see and exec into every other build running at the same time and read their tokens and secrets. This image ships with Docker-enabled builds off: the annotation is ignored, and the build gets no Docker socket and no privileged container. You can turn it on with one command (Step 8), which tells you the risk first. With it on, "root" and "privileged" inside a build are still root of the user namespace only. Before release cloudimg runs a real Docker-enabled build through the real executor with the switch on and confirms that from inside it a host /etc mount cannot read /etc/shadow, a root-owned host directory cannot be written, the root Docker socket cannot be reached, kernel modules cannot be loaded and instance metadata cannot be reached. It also confirms that with the switch off the same build gets no socket.
• Builds cannot open ports on the instance. The build engine has no port forwarding, so docker run -p inside a build opens no listener on the instance. Firewall rules on the sdbuild account and its subordinate IDs also stop builds from reaching the API, UI and store's private loopback ports. Builds reach the API and store only through a host-internal address. The launcher image every build starts from is pinned by digest, so a locally retagged image is never used in its place.
• Login is closed until you say who may log in. Screwdriver signs people in with GitHub. Upstream treats an empty allow-list as "everyone with a GitHub account". Here the allow-list is never empty: until /etc/screwdriver/allowed-users names an account, it holds a value no GitHub account can match, so nobody can log in. You add yourself with screwdriver-allow-user (Step 6), by login and numeric GitHub ID. Guest access is off. Pull requests from forks do not start builds automatically.
• Per-instance secrets. First boot creates the JWT signing key pair, the cookie, encryption and hashing passwords, the PostgreSQL password and the GitHub webhook secret on your instance; no two instances share one. They live only in /etc/screwdriver (mode 0700, files 0600) and never in a log.
• The services are private. The API, UI and store listen on the loopback interface only; PostgreSQL has no host port at all; the Docker daemons have no TCP socket and the docker group is empty. From outside the instance exactly three ports answer: 22, 80 (a redirect) and 443. Anonymous requests to change anything, and to read pipelines, builds, secrets, tokens and stored artifacts, are refused 401.
• HTTPS from first boot. TLS 1.2 and 1.3 only and Strict-Transport-Security. The host name comes from configuration, never from a request: first boot reads the instance's public DNS name from EC2 instance metadata (IMDSv2) and pins nginx, the API and UI base URLs and the GitHub OAuth callback to it as https. Requests for any other host name have their connection closed.
• Rate limits. Every client gets 30 requests per second (bursts of 200), and the login and token routes 20 requests per minute, after which nginx answers 429. A client is one IPv4 address or one IPv6 /64, taken from the connection, never from a header, and every spelling of a login or token URL counts against the same budget.
• Builds are refused instance metadata, but can reach your VPC and the internet. A rule in Docker's DOCKER-USER chain, applied on every boot before any container starts (Screwdriver does not start without it), refuses 169.254.169.254, fd00:ec2::254 and 168.63.129.16 to the service containers; firewall rules matched on the sdbuild account and its subordinate IDs, and a systemd address deny on the build engine, refuse the same addresses to every build. So a build cannot read the instance's IAM role credentials. Builds can reach your VPC and the internet: they clone code, pull images and fetch dependencies. Keep the security group's outbound rules as restrictive as your builds allow, and do not attach an IAM role the builds should not have.
• Verified build. The image digests were scanned for known vulnerabilities and their licences checked before the build. SSH allows keys only, no root login and no X11 forwarding; the AMI requires IMDSv2.

Trusted pipeline authors only. Anyone who can change a screwdriver.yaml that Screwdriver builds can run code on this instance's build engine, with its network access. Allow only people you trust, and keep fork builds off.

Prerequisites

1. An AWS account and permission to launch EC2 instances
2. An EC2 key pair in the Region you launch in
3. A GitHub account, and permission to create an OAuth app (personal, or in your organisation)

Recommended instance type: m5.large (2 vCPUs, 8 GiB). Screwdriver itself uses about 1 GiB at rest; your builds use the rest, and the build engine is capped at 70% of the instance's memory so that a runaway build cannot starve Screwdriver. Choose a larger type for heavy or parallel builds.

Connecting to your instance

Version	Operating system	SSH user
Screwdriver 8.0.207 on Ubuntu 24.04	Ubuntu 24.04 LTS	ubuntu

Step 1: Launch the instance

Option A: AWS Marketplace console

1. Open the cloudimg Hardened Apache Subversion (SVN) & CI/CD Tools AMIs listing on AWS Marketplace and choose View purchase options, then Subscribe. One subscription covers every version in the listing.
2. Once the subscription is active, choose Launch your software.
3. Choose One-click launch from AWS Marketplace (or Launch from EC2 Console for the full launch wizard). In Version, choose Screwdriver 8.0.207 on Ubuntu 24.04, then choose your Region.
4. Pick an instance type (m5.large recommended) and your key pair.
5. Security group: open SSH (22) to your own IP address, and HTTPS (443) and HTTP (80, redirect only) to the networks your users work from. GitHub must also reach 443 to deliver webhooks: add GitHub's published hook address ranges, or open 443 to the internet. Never open 9000, 9001 or 9002. Keep the group's outbound rules as narrow as your builds allow.
6. Choose your VPC and subnet. Under Advanced details, leave Metadata version at V2 only (the image already requires it) and set Metadata response hop limit to 1.
7. Choose Launch.

Option B: AWS CLI

Run this on your own workstation. Replace the AMI ID with the one shown for your Region on the Marketplace launch page, and the VPC, subnet, key pair and CIDRs with your own:

SG=$(aws ec2 create-security-group --group-name screwdriver \
  --description "Screwdriver" --vpc-id vpc-0123456789abcdef0 \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 22 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 80 --cidr <your-mgmt-cidr>
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 443 --cidr <your-mgmt-cidr>
aws ec2 run-instances \
  --image-id ami-0123456789abcdef0 \
  --instance-type m5.large \
  --key-name my-key-pair \
  --security-group-ids "$SG" \
  --subnet-id subnet-0123456789abcdef0 \
  --metadata-options HttpTokens=required,HttpPutResponseHopLimit=1,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=screwdriver}]'

Associate an Elastic IP if you want the address to survive a stop and start, and see Step 9 to serve Screwdriver under your own name.

Step 2: Connect and find your address

Connect as ubuntu with the key pair you chose:

ssh -i ~/.ssh/my-key-pair.pem ubuntu@<public-ip>

First boot takes two to three minutes after the instance starts. It sets the name Screwdriver is served as (the instance's public DNS name when it has one), creates the TLS certificate and this instance's secrets, starts PostgreSQL, the store, the API and the UI, and only then opens nginx. The details you need are in a root-only file:

sudo grep -E '^SCREWDRIVER_(URL|HOST|GITHUB_CALLBACK_URL|TLS_SHA256|ALLOWED_USERS_FILE)=' /root/screwdriver-credentials.txt
sudo stat -c '%a %U:%G %n' /root/screwdriver-credentials.txt

Value	What it is for
SCREWDRIVER_URL / SCREWDRIVER_HOST	The address of this instance's Screwdriver: the web interface, with the API under /v4 and the store under /v1
SCREWDRIVER_GITHUB_CALLBACK_URL	The authorization callback URL your GitHub OAuth app needs (Step 5)
SCREWDRIVER_TLS_SHA256	The fingerprint of this instance's certificate, to check the browser warning against
SCREWDRIVER_ALLOWED_USERS_FILE	The file that lists who may log in (Step 6)

There is no password to read: you sign in with GitHub.

Step 3: Confirm Screwdriver is running

The first-boot unit has finished; the walls, both Docker engines, the four Screwdriver services and nginx are active; the four containers run on the root engine with the API, store and UI published on the loopback interface only:

systemctl is-active screwdriver-firstboot cloudimg-sdbuild-walls docker sdbuild-docker screwdriver-postgres screwdriver-store screwdriver-api screwdriver-ui nginx
sudo docker ps --format '{{.Names}} {{.Ports}}' | sort
sudo ss -tlnH | awk '{print $4}' | sort -u

active
active
active
active
active
active
active
active
active
sd-api 127.0.0.1:9001->8080/tcp
sd-postgres 5432/tcp
sd-store 127.0.0.1:9002->80/tcp
sd-ui 127.0.0.1:9000->80/tcp
0.0.0.0:22
0.0.0.0:443
0.0.0.0:80
10.213.0.1:9101
10.213.0.1:9102
127.0.0.1:9000
127.0.0.1:9001
127.0.0.1:9002
127.0.0.53%lo:53
127.0.0.54:53
[::]:22
[::]:443
[::]:80

The listening sockets are sshd on 22, nginx on 80 and 443 on every address, nginx on 10.213.0.1 (the host-internal address builds use to reach the API and store), the three services on 127.0.0.1, and Ubuntu's local DNS resolver on 127.0.0.53 and 127.0.0.54. PostgreSQL is reachable only by the API, on the services' private container network.

Step 4: Run the self-test

sudo /usr/local/sbin/screwdriver-selftest

It checks, against the real HTTPS front door, that the API and UI answer, that port 80 only redirects, that anonymous writes and anonymous store reads are refused 401, that guest login is refused 403, that a token signed with this instance's key is accepted and one signed with any other key is refused, that the running allow-list is exactly what /etc/screwdriver/allowed-users says, that the API container has no Docker socket of the root engine, and whether Docker inside builds is on (Step 8; it ships off). It ends with SCREWDRIVER_SELFTEST_OK and prints nothing secret.

You can also confirm by hand that login is closed and guests are refused:

H=$(sudo sed -n 's/^SCREWDRIVER_HOST=//p' /root/screwdriver-credentials.txt)
curl -sk --resolve "$H:443:127.0.0.1" -o /dev/null -w 'guest login: %{http_code}\n' "https://$H/v4/auth/login/guest"
curl -sk --resolve "$H:443:127.0.0.1" -o /dev/null -w 'anonymous pipeline create: %{http_code}\n' -X POST -H 'Content-Type: application/json' -d '{}' "https://$H/v4/pipelines"
sudo screwdriver-allow-user list

guest login: 403
anonymous pipeline create: 401
(nobody: login is closed)

Step 5: Connect your GitHub OAuth app

Screwdriver signs people in with GitHub and receives GitHub webhooks, so it needs an OAuth app of your own:

1. In GitHub, open Settings, Developer settings, OAuth Apps, New OAuth App (or the same under your organisation's settings).
2. Homepage URL: your SCREWDRIVER_URL followed by /.
3. Authorization callback URL: your SCREWDRIVER_GITHUB_CALLBACK_URL (it ends in /v4/auth/login/github).
4. Register the app, then generate a client secret.

On the instance, give Screwdriver the client ID, then paste the client secret when asked (it is read from the terminal, never from the command line):

sudo screwdriver-configure-github <client-id>

The API restarts with the new app. Until you finish Step 6 nobody can complete a login, which is intended.

Step 6: Choose who may log in

Add each GitHub account that may use this Screwdriver. admin makes the account a Screwdriver administrator; leave it off for everyone else:

sudo screwdriver-allow-user add <your-github-login> admin
sudo screwdriver-allow-user list

The helper looks up the account's numeric GitHub ID and records login:id, so a renamed account cannot be taken over by someone who registers the old name. If the instance cannot reach api.github.com, give the ID yourself: sudo screwdriver-allow-user add <your-github-login>:<id> admin. Remove an account with sudo screwdriver-allow-user remove <login>. Each change re-renders the API configuration and restarts the API. If the file is ever empty, missing or unreadable, login closes again rather than opening to everyone.

The first admin account in the list is also the account Screwdriver uses to handle pushes and pull requests from GitHub users who have never signed in to it, so sign in with that account once (Step 7) before you create pipelines.

Step 7: Sign in and add a pipeline

Open your SCREWDRIVER_URL in a browser. The certificate is created on your instance, so the browser warns about it until you install your own (Step 9); the fingerprint is SCREWDRIVER_TLS_SHA256. Choose Sign in with GitHub and approve the OAuth app.

Add a screwdriver.yaml to a repository you own, for example:

jobs:
  main:
    image: node:22
    requires: [~pr, ~commit]
    steps:
      - install: npm ci
      - test: npm test

Then choose Create in Screwdriver, paste the repository's URL and create the pipeline. Screwdriver adds a webhook to the repository with this instance's own webhook secret, so a push or a pull request starts the main job.

The API's own documentation is served at your SCREWDRIVER_URL followed by /v4/documentation. Use it to look up a route before you script against the API:

Build images need bash. The Screwdriver launcher runs each step with /bin/bash, so choose an image that has it (for example node, python, ubuntu, debian or buildpack-deps). Minimal images such as alpine or busybox cannot run a job as they are.

Step 8: How builds run, and what they can reach

Every build runs on the rootless build engine. You can see that engine and confirm the API cannot reach the root one:

sudo docker --host unix:///run/sdbuild-docker/docker.sock info --format '{{.SecurityOptions}}'
sudo docker inspect sd-api --format '{{range .Mounts}}{{.Source}} -> {{.Destination}} {{end}}' | tr ' ' '\n' | grep -c docker.sock || true

The first line includes name=rootless; the second prints 0, because no Docker socket of the root engine is mounted into the API.

• Docker inside builds. Off by default; see the next section.
• Network. Builds reach the API and store over a host-internal address, and reach your VPC and the internet like any program on the instance. They cannot reach instance metadata. Keep the security group's outbound rules restrictive, and keep secrets your builds need in Screwdriver's secrets (per pipeline), not on the instance.
• Resources. The build engine is capped at 70% of the instance's memory. Build images and caches live under /var/lib/sdbuild; clean them with sudo docker --host unix:///run/sdbuild-docker/docker.sock system prune when the disk fills.

Turning on Docker inside builds

A job that sets annotations: { screwdriver.cd/dockerEnabled: true } gets no Docker socket while the switch is off: docker commands in it fail with "Cannot connect to the Docker daemon". Check the switch with:

sudo screwdriver-docker-builds status

Read this before you turn it on. Every build on this instance runs on ONE shared build engine. With the switch on, a Docker-enabled job gets that engine's socket. From there it can list, inspect and exec into every other build running at the same time, whether or not that build is Docker-enabled, and read its environment: build tokens, secrets and SCM credentials. It still cannot reach the host, the root engine, the Screwdriver services' private ports or instance metadata. Turn it on only when every pipeline author on this instance is trusted. Run untrusted pipelines on a separate instance with the switch left off.

If that is acceptable for your instance, turn it on with:

sudo screwdriver-docker-builds on

The command prints the same warning and restarts the API, which takes about a minute. Docker-enabled jobs can then run docker build and docker run on the rootless engine. Turn it off again with sudo screwdriver-docker-builds off.

Step 9: Serve Screwdriver under your own DNS name

Point a DNS record at the instance (an Elastic IP keeps the address stable), then set the name. This creates a new certificate for it, updates nginx, the API and UI base URLs and the GitHub callback, and restarts the services:

sudo screwdriver-set-host ci.example.com

Update the Homepage URL and Authorization callback URL of your GitHub OAuth app to the new name. The name is kept across reboots. To use a certificate from your own CA or from Let's Encrypt, replace the pair after setting the name, keeping the same paths and modes, and reload nginx:

sudo install -m 0644 fullchain.pem /etc/ssl/screwdriver/screwdriver.crt
sudo install -m 0600 privkey.pem  /etc/ssl/screwdriver/screwdriver.key
sudo systemctl reload nginx

Running screwdriver-set-host again creates a new self-signed certificate, so install your own pair again afterwards. Without a pinned name, the instance follows its public DNS name across a stop and start.

Step 10: Back up Screwdriver

Pipelines, jobs, builds, users and secrets live in the PostgreSQL database; build logs and artifacts in the store's volume. A database backup is one dump through the database container:

sudo docker exec sd-postgres pg_dump -Fc -U screwdriver screwdriver | sudo tee /var/tmp/screwdriver-db.dump > /dev/null && sudo ls -l /var/tmp/screwdriver-db.dump

Copy the file off the instance, or snapshot the root volume for everything at once. Pipeline secrets in the dump are encrypted with this instance's encryption password, which is in /etc/screwdriver/secrets.json: keep a copy of that file with the dump if you will restore onto another instance.

Maintenance

• Operating system updates: unattended security upgrades are on, and nginx and Docker are Ubuntu or Docker packages that they cover. Reboot when /var/run/reboot-required exists; the metadata rules, both engines, the four services and nginx start again by themselves.
• Screwdriver updates: the services are pinned container images, so apt never updates them. To move to a newer release, launch the newer version of this image when the listing offers it and restore your Step 10 backup into it. Do not run docker image prune -a on the root engine: the services start their pinned images only and never pull.
• Logs: sudo journalctl -u screwdriver-api (and -store, -ui, -postgres) for the services, /var/log/nginx/ for web requests, sudo journalctl -u sdbuild-docker for the build engine and sudo journalctl -u screwdriver-firstboot for first boot. Build logs are in the web interface.
• The end-to-end check: sudo /usr/local/sbin/screwdriver-selftest prints SCREWDRIVER_SELFTEST_OK when everything holds.

Troubleshooting

• The page does not load right after launch: first boot is still running. sudo journalctl -u screwdriver-firstboot -f shows its progress.
• Browser shows "connection closed": nginx answers only the configured name and the instance's own addresses. Use SCREWDRIVER_URL, or run sudo screwdriver-set-host <name> for a new name.
• GitHub sign-in returns 403: your account is not in the allow-list, or its GitHub ID does not match. Run sudo screwdriver-allow-user list and add the account (Step 6).
• GitHub says the redirect URI is not associated with the app: the OAuth app's callback must be exactly SCREWDRIVER_GITHUB_CALLBACK_URL.
• Pushes do not start builds: GitHub must reach port 443 of the instance; check the security group and the webhook's recent deliveries in the repository settings. Pull requests from forks do not start builds by design.
• A job fails at once with "bash: not found" or similar: the job's image has no /bin/bash; choose an image that has it.
• A build cannot reach 169.254.169.254: that is the metadata rule working. Give builds what they need through Screwdriver secrets.
• A Docker-enabled job says "Cannot connect to the Docker daemon": Docker inside builds is off by default. Read Step 8 before you run sudo screwdriver-docker-builds on.
• docker run -p inside a build publishes nothing on the instance: by design; the build engine has no port forwarding. Test a service from inside the same build instead.
• 429 Too Many Requests: the per-client rate limit; wait a minute.

Licensing

Screwdriver is licensed under the BSD 3-Clause License; its licence and the third-party notice for the container images are on the instance in /usr/share/doc/screwdriver/, and the image digests in /usr/share/screwdriver/IMAGES. PostgreSQL is licensed under the PostgreSQL Licence, nginx under the 2-clause BSD licence and slirp4netns under the GPL version 2. Screwdriver is a name of the screwdriver-cd project; cloudimg is not affiliated with, sponsored by or endorsed by the project or by Yahoo, and the name identifies the software this image contains.

Support

cloudimg provides 24/7 technical support for this image by email and live chat: deployment, GitHub OAuth and allow-list setup, HTTPS and DNS setup, pipelines and Docker-enabled builds, what builds can reach, backups, upgrades and troubleshooting. Contact support@cloudimg.co.uk.