Developer Tools Azure

Salt Master on Ubuntu 24.04 on Azure User Guide

| Product: Salt Master on Ubuntu 24.04 LTS on Azure

Overview

This guide covers the deployment and configuration of Salt Master on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images. Salt is an event driven configuration management and remote execution platform. A Salt master publishes commands and state to Salt minions and collects their returns, so you can describe how a fleet of machines should be configured, apply that description, and run ad hoc commands across the whole fleet in seconds.

This image is a Salt master appliance, and it ships as a working one. A local minion is already connected to the master and accepted, and a small state and pillar tree is in place, so salt '*' test.ping and salt '*' state.apply both do something real the moment the VM boots. You are not handed a bare install and left to wire it up.

The cloudimg image ships the free and open source, Apache-2.0 licensed Salt 3008.2 — the newest release of the current long term support branch — installed from the Salt Project's own packages and pinned by content hash, tied to an exact upstream commit. Backed by 24/7 cloudimg support.

The Salt master listing its accepted minion keys, returning True from test.ping, reporting the minion's operating system and Salt version from grains, and running the uptime command remotely on the minion

What is included:

  • Salt 3008.2 master and minion, from the Salt Project's own Debian packages, verified by SHA-256 content hash
  • A local minion already accepted on the master, so the appliance works end to end on first boot
  • A small state and pillar tree at /srv/salt and /srv/pillar that demonstrates the full render, publish, execute, return cycle
  • No key material in the image at all — the master key pair, the minion key pair and the accepted minion list are generated on your VM, at your first boot
  • The ZeroMQ transport bound to loopback, so SSH is the only routable listener on a freshly launched VM
  • auto_accept off, no autosign of any kind, open_mode off — enforced by a guard that stops the master from starting if any of it is weakened
  • A fingerprint helper that makes Salt's own key verification workflow actually work (see Adding a remote minion)
  • A full dependency licence inventory at /usr/share/saltstack/dependency-licences.txt

Security posture, stated plainly

A Salt master is a control plane: anything that can command it can command every machine it manages, as root. Internet exposed Salt masters have been compromised at scale — CVE-2020-11651 and CVE-2020-11652 gave unauthenticated root remote code execution on the master and were exploited in the wild. Salt's trust model is minion key based, not password based: there is no password on the wire, only key exchange and whatever the master has been told to accept.

This image is built around that. What cloudimg does for you, and what remains yours, is set out in /usr/share/saltstack/SECURITY-RESPONSIBILITIES on the VM and in Access control below. Securing master to minion trust and the network exposure of the Salt ports is your responsibility as the operator of this VM.

Prerequisites

  • An Azure subscription with permission to create virtual machines
  • An SSH key pair for VM access
  • A VM size of Standard_B2s (2 vCPU, 4 GiB) or larger

Standard_B2s is a genuine recommendation, not a squeeze. The master idles at roughly 200 MB and the local minion at under 100 MB, and no swap is created at any point. It is comfortable for a master with up to roughly 50 to 100 minions at the default worker_threads of 5. Beyond that, move to a larger size and raise worker_threads in /etc/salt/master.d/.

Ports this appliance actually serves

Port Protocol Purpose Reachable on a fresh VM
22 TCP SSH — how you administer the appliance Yes
4505 TCP Salt publish bus (master to minions) No — bound to 127.0.0.1
4506 TCP Salt request bus (minions to master) No — bound to 127.0.0.1

The offer advertises port 22 only, because that is the only port the appliance serves on a fresh VM. Salt's 4505 and 4506 are bound to 127.0.0.1, so the local minion reaches the master over loopback and nothing on the network can reach the transport at all. Opening it up for remote minions is a deliberate opt in, covered in Adding a remote minion.

There is no salt-api and no web server on this image. Salt's HTTP netapi is the component CVE-2021-3148 and CVE-2021-3197 are reached through, and the appliance's control plane is the salt command line over SSH, so it is not installed at all.

Launching the VM

From the Azure Portal

  1. Search the Azure Marketplace for Salt Master on Ubuntu 24.04 LTS by cloudimg
  2. Click Create
  3. Choose your subscription, resource group and region
  4. Set the VM size to Standard_B2s or larger
  5. Choose SSH public key authentication and supply your public key
  6. Under Inbound port rules, allow SSH (22) only
  7. Review and create

From the Azure CLI

az group create --name salt-master-rg --location eastus

az vm create \
  --resource-group salt-master-rg \
  --name salt-master \
  --image cloudimg:saltstack-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard

Only port 22 needs to be open. Do not open 4505 or 4506 unless and until you have read Adding a remote minion and decided you need them.

What happens on first boot

The shipped image contains no key material whatsoever: no master key pair, no minion key pair, no accepted or pending minion key, and no minion id. That is deliberate. A baked master private key would let anyone holding the image impersonate every customer's master; a baked accepted minion key would be a back door into every customer's master. So all of it is generated on your machine, at your first boot, by saltstack-firstboot.service.

On first boot the appliance:

  1. Verifies the image really did arrive with no key of any kind, and refuses to continue if it did not
  2. Sets this VM's minion id from its hostname
  3. Starts the master, which generates this VM's own master key pair
  4. Confirms the transport came up on 127.0.0.1 and that nothing unexpected is listening
  5. Pins the local minion to this master's public key (master_finger), so the minion will not talk to any other master
  6. Starts the local minion, which generates this VM's own minion key pair
  7. Accepts the local minion's key only after confirming it is byte for byte this machine's own public key — never simply whatever key arrived first
  8. Proves the round trip works, then proves the two negatives: a wrong master fingerprint is refused, and a freshly generated stranger key gets nothing at all
  9. Writes /root/salt-credentials.txt and its sentinel

First boot typically completes within a minute. Check it finished:

sudo systemctl is-active saltstack-firstboot.service
sudo test -f /var/lib/cloudimg/saltstack-firstboot.done && echo "first boot complete"

Expected output:

active
first boot complete

Retrieving your identity material

Salt does not authenticate with passwords, so what first boot records for you is identity material: the fingerprints of the key pairs this machine generated. They exist nowhere else, and they are how you verify identity when you add a minion.

sudo cat /root/salt-credentials.txt

The file is 0600 root:root and contains, among the explanatory notes:

SALT_MASTER_KEY_FINGERPRINT=<SALT_MASTER_KEY_FINGERPRINT>
SALT_MINION_ID=<SALT_MINION_ID>
SALT_MINION_KEY_FINGERPRINT=<SALT_MINION_KEY_FINGERPRINT>
SALT_MASTER_INTERFACE=127.0.0.1
SALT_AUTO_ACCEPT=False

Running your first commands

The local minion is already accepted, so everything below works immediately.

sudo salt-key -L

Expected output — one accepted key, which is this machine, and nothing pending:

Accepted Keys:
<SALT_MINION_ID>
Denied Keys:
Unaccepted Keys:
Rejected Keys:

Now the round trip. test.ping asks the minion to answer; it is the simplest proof that the publish and return buses both work.

sudo salt '*' test.ping
<SALT_MINION_ID>:
    True

Remote execution takes arguments and returns real data:

sudo salt '*' grains.item os osrelease saltversion num_cpus
sudo salt '*' cmd.run 'uptime'
sudo salt '*' disk.usage

grains are facts the minion reports about itself — operating system, CPU count, Salt version — and they are what you target and template on.

Applying state

State is the configuration management half of Salt: you describe the desired end state, Salt makes it so, and running it again changes nothing. The appliance ships a small, harmless, idempotent example so you can see the whole cycle.

cat /srv/salt/top.sls
cat /srv/salt/cloudimg/welcome.sls

top.sls assigns cloudimg.welcome to every minion. Always dry run first — test=True shows what would change without changing it:

sudo salt '*' state.apply test=True

Then apply it for real:

sudo salt '*' state.apply

Expected output ends with a summary like:

Summary for <SALT_MINION_ID>
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1

A dry run of the shipped state with test equals True, then the real state run reporting one state succeeded and zero failed, followed by the managed file it wrote containing the minion id, operating system and greeting rendered from real grains and pillar data

The state wrote a file, rendered on the master from the minion's own grains and from pillar data:

cat /etc/cloudimg-salt-demo.txt

Pillar is the master's private, per minion data store. It is rendered on the master and delivered only to the minions it is targeted at, which is why it is where secrets belong — /srv/pillar is mode 0750 for that reason.

sudo salt '*' pillar.items

Note that Salt 3008 redacts pillar values in returns, showing ********** rather than the value, so that pillar data does not end up in job returns and logs. The value itself is delivered to the minion normally — the file the state above wrote contains the greeting in full.

Replace /srv/salt/top.sls and /srv/salt/cloudimg/welcome.sls with your own state tree. That is what the machine is for.

Access control

auto_accept is off, and the master will not start without it

The single most dangerous setting on a Salt master is auto_accept. With it on, the master trusts any key that introduces itself — on a reachable master that is unauthenticated remote code execution across your whole estate. On this appliance it is False, there is no autosign_file and no autosign_grains_dir, and open_mode (which disables authentication outright) is False.

These are not merely written into a config file. /usr/local/sbin/salt-preflight.sh reads the effective merged configuration through Salt's own loader and runs as ExecStartPre on salt-master.service, so a master that has been edited into an unsafe state does not start at all.

sudo salt-preflight.sh --master && echo "configuration accepted"

The master's own key fingerprint, the effective security settings showing auto accept false and interface 127.0.0.1, the kernel's full listening socket table showing 4505 and 4506 bound to loopback with only port 22 routable, the socket audit passing, and the preflight guard refusing to start a master with auto accept set to True

Verifying the listening surface yourself

Do not take our word for it. Read it from the kernel:

ss -lntuH
sudo salt-socket-audit.sh --expect-app

The audit enumerates every listening socket and refuses if any routable listener exists outside the allowed set (TCP 22 and the DHCP client on UDP 68). It also confirms the Salt transport really is on loopback.

Adding a remote minion

This is the security decision that matters, and it is yours. Every minion you accept is a machine you are granting root equivalent control over.

Step 1 — open the transport. Edit /etc/salt/master.d/10-cloudimg-network.conf, change interface: 127.0.0.1 to interface: 0.0.0.0, and restart:

sudo systemctl restart salt-master

Step 2 — allow the ports, narrowly. In the Azure portal or CLI, allow inbound TCP 4505 and 4506 from the narrowest source that works — the subnet or address range your minions live in, never Any. Do not expose these ports to the public internet. Prefer a private VNet, a peered network or a VPN.

Step 3 — point the minion at this master. On the minion machine, install Salt, set master: <your-master-ip> in /etc/salt/minion, and start salt-minion. Its key will arrive at your master as unaccepted.

Step 4 — verify the fingerprint before you accept. On the minion:

sudo salt-call --local key.finger

On this master, substituting the id the minion registered under (sudo salt-key -L lists it):

sudo salt-fingerprint.sh <minion-id>

Accept only if the two values are identical:

sudo salt-key -a <minion-id>

Use salt-fingerprint.sh, not salt-key -f. This is a real trap in Salt itself, not a cloudimg quirk. Salt's pem_finger hashes only the key body when it is given a file path — which is what salt-call key.finger does on the minion — but hashes the whole PEM blob when it is given raw bytes, which is what salt-key -f and salt-key -F do on the master. The two are therefore different strings for the same key, every time, for every key. An operator who follows the widely documented salt-key -f comparison sees a mismatch on a perfectly good minion, concludes the check is meaningless, and starts accepting keys blind — which is exactly the risk auto_accept: False exists to keep in their hands. salt-fingerprint.sh prints the same form the minion does, so the comparison is real.

Do not turn auto_accept on, and do not configure an autosign file or autosign grains directory. A grain is self reported by the minion, so it is not a secret and cannot be an authentication factor.

Day to day operation

Services

sudo systemctl status salt-master
sudo systemctl status salt-minion

Both are enabled and start automatically on reboot.

Logs

sudo journalctl -u salt-master -n 50 --no-pager
sudo journalctl -u saltstack-firstboot -n 50 --no-pager

Salt also writes to /var/log/salt/.

Backup

The two things worth backing up are your state and pillar tree and the master's key material and accepted keys:

sudo tar czf /root/salt-backup.tar.gz /srv/salt /srv/pillar /etc/salt

/etc/salt/pki contains this master's private key. Treat that archive as a secret — anyone who holds it can impersonate your master.

Operating system updates

Ubuntu security updates are applied unattended. Salt itself is pinned at the version this image was built and verified against:

cat /etc/apt/preferences.d/salt-pin.pref

That pin is what stops an unattended run from replacing a verified artifact with an unverified one. To move to a newer Salt deliberately, remove that file, then upgrade and restart the services. Read the Salt release notes first — cloudimg's proofs apply to the version cloudimg built.

Provenance and licences

cat /usr/share/saltstack/source-provenance.txt
cat /usr/share/saltstack/SECURITY-RESPONSIBILITIES
head -20 /usr/share/saltstack/dependency-licences.txt

Salt reporting version 3008.2, the three installed packages at that exact version, the provenance record showing the upstream tag, commit, Apache-2.0 licence and the SHA-256 of every package, and the dependency licence inventory summarising 162 Python distributions examined with none restrictive

Every Python distribution in both of the image's Python environments is enumerated with its licence in /usr/share/saltstack/dependency-licences.txt, produced by an executable gate that fails the build on any network copyleft, source available or non commercial licence.

Troubleshooting

salt '*' test.ping returns "Minion did not return" — the minion is not running or its key is not accepted. Check sudo systemctl status salt-minion and sudo salt-key -L.

A remote minion's key never appears — the transport is still on loopback, or the network security group is closed. Check interface: in /etc/salt/master.d/10-cloudimg-network.conf, that you restarted salt-master, and that 4505 and 4506 are allowed from the minion's address.

salt-master will not start — run the guard by hand to see exactly which invariant it is refusing on:

sudo salt-preflight.sh --master

It refuses on auto_accept: True, open_mode: True, any autosign configuration, permissive_pki_access: True, key material readable beyond the master's own account, and on any file in /etc/salt/master.d/ that is not valid YAML. That last one matters: Salt does not abort on a malformed drop in, it silently discards the whole file, taking every setting in it with it — so the guard refuses rather than let your edit vanish unnoticed.

The fingerprints do not match when adding a minion — you are almost certainly comparing salt-key -f on the master with salt-call key.finger on the minion, which never match. Use sudo salt-fingerprint.sh <minion-id> on the master instead.

First boot did not complete — inspect it directly:

sudo journalctl -u saltstack-firstboot -n 80 --no-pager

Support

cloudimg provides 24/7 support for this image. Contact support@cloudimg.co.uk.

Salt is an independent open source project. This image is produced by cloudimg and is not affiliated with, endorsed by, or sponsored by the Salt Project or Broadcom. It ships the free and open source Apache-2.0 licensed software, unmodified.