fleet-manager: auto-renew the agent-plane server cert (step-ca sidecar + hot-reload)

The agent-plane server cert was issued by hand (server-pki.sh / docker exec
step ca certificate), which needs the step CLI on the host, re-specifies SANs
each time, and silently expires at the CA's 90-day cap — the exact failure that
just broke agent TLS (cert minted with a stale SAN, missing manager.fleet.lab).

Automate it:
- agent-cert-renewer sidecar (smallstep step-cli image, compose profile
  cert-auto): issues the cert once from the co-located fleet step-ca, then
  'step ca renew --daemon' rotates it before the cap. Renewal preserves the
  subject + SANs, so SAN drift cannot recur. Chowns to the manager's nonroot
  uid so the distroless manager reads the key. Opt-in + only valid where the CA
  is co-located; server-pki.sh stays as the manual/remote-CA path.
- manager hot-reloads via GetCertificate (certreload.go): a rotated cert on
  disk is picked up on the next handshake with no restart and no dropped agent
  streams; a half-written pair keeps the last good cert serving. Unit-tested.
- defaults.env documents the cert-auto knobs; DEPLOY.md §2a recommends the
  renewer and flags the mandatory manager.fleet.lab SAN.
This commit is contained in:
Audiobuzz
2026-07-10 07:42:18 +00:00
parent 1c68c37824
commit 1db3977238
8 changed files with 360 additions and 12 deletions
+72
View File
@@ -0,0 +1,72 @@
#!/bin/sh
# agent-cert-renewer.sh — issue-once + auto-renew the fleet-manager AGENT-PLANE
# server cert from the co-located fleet step-ca. Runs in the smallstep step-cli
# image as the `agent-cert-renewer` compose sidecar (profile cert-auto).
#
# This replaces the manual `server-pki.sh` + `docker exec … step ca certificate`
# dance and the 90-day re-mint chore:
# - FIRST run (no cert yet): issues the cert via the 'enrol' provisioner, with
# ALL SANs hosts might dial (manager.fleet.lab + the host FQDN).
# - THEREAFTER: `step ca renew --daemon` rotates it well before the CA's cap.
# Renewal RE-ISSUES THE SAME IDENTITY (subject + SANs preserved), so the SAN
# drift that once broke agent TLS cannot recur.
# The manager hot-reloads the rotated pair via GetCertificate (certreload.go) —
# no restart, no dropped agent streams. Files are chowned to the manager's
# nonroot uid so the distroless manager can read the key.
#
# Renewal authenticates to the CA with the CERT ITSELF (mTLS) — the provisioner
# password is only needed for the initial issuance.
set -eu
CRT=${AGENT_CERT:-/pki/agent/server.crt}
KEY=${AGENT_KEY:-/pki/agent/server.key}
BUNDLE=${AGENT_CLIENT_CA:-/pki/agent/fleet-ca.crt}
CA_URL=${CA_URL:-https://localhost:8443}
CA_ROOT=${CA_ROOT:-/ca/root_ca.crt}
CA_INT=${CA_INTERMEDIATE:-/ca/intermediate_ca.crt}
PROVISIONER=${PROVISIONER:-enrol}
PW_FILE=${PROVISIONER_PASSWORD_FILE:-/ca/password}
SANS=${AGENT_SANS:-manager.fleet.lab}
EXPIRES_IN=${RENEW_EXPIRES_IN:-720h} # renew when < this remains (of the CA's cap)
NOT_AFTER=${CERT_NOT_AFTER:-2160h} # first-issue lifetime (must be <= CA max)
OWNER=${OWNER:-65532:65532} # the distroless manager's nonroot uid:gid
primary=${SANS%%,*}
san_args=""
oldifs=$IFS; IFS=,
for s in $SANS; do san_args="$san_args --san $s"; done
IFS=$oldifs
fixperms() {
chown "$OWNER" "$CRT" "$KEY" 2>/dev/null || true
chmod 0644 "$CRT" 2>/dev/null || true
chmod 0600 "$KEY" 2>/dev/null || true
}
# Client-verify bundle (intermediate + root). Build it if absent so a fresh
# bring-up needs only this sidecar, not a separate server-pki.sh run. The manager
# loads it once at startup — the root/intermediate don't rotate on this cadence.
if [ ! -s "$BUNDLE" ]; then
{ [ -s "$CA_INT" ] && cat "$CA_INT"; cat "$CA_ROOT"; } > "$BUNDLE"
chown "$OWNER" "$BUNDLE" 2>/dev/null || true
chmod 0644 "$BUNDLE" 2>/dev/null || true
echo "agent-cert-renewer: wrote client-verify bundle $BUNDLE"
fi
# Issue once if there's no cert yet (provisioner + password). Renewal afterwards
# needs neither.
if [ ! -s "$CRT" ] || [ ! -s "$KEY" ]; then
echo "agent-cert-renewer: issuing CN=$primary (SANs=$SANS) from $CA_URL"
# shellcheck disable=SC2086 # san_args is intentionally word-split
step ca certificate "$primary" "$CRT" "$KEY" \
--ca-url "$CA_URL" --root "$CA_ROOT" \
--provisioner "$PROVISIONER" --provisioner-password-file "$PW_FILE" \
--not-after "$NOT_AFTER" --force $san_args
fixperms
fi
echo "agent-cert-renewer: watching $CRT (renew when < $EXPIRES_IN remains)"
exec step ca renew --daemon --expires-in "$EXPIRES_IN" \
--ca-url "$CA_URL" --root "$CA_ROOT" \
--exec "sh -c 'chown $OWNER $CRT $KEY; chmod 0644 $CRT; chmod 0600 $KEY'" \
"$CRT" "$KEY"