Audiobuzz
28d31e436c
F-02/P-17 SSH host-cert flow was proven end-to-end on comet-beacon-54
(2026-07-11); these fix the correctness/scoping/cleanup defects the bench
surfaced:
- ssh-hostcert: derive the FQDN principal <hostname>.<domain> from the DHCP
search domain at runtime (resolv.conf), alongside the short hostname, so an
admin `@cert-authority *.<domain>` line accepts by-FQDN logins instead of
"name is not a listed principal". Never baked (invariant #1). + tests.
- ssh-ca-bootstrap.sh: export the SSH host CA public key in OpenSSH wire format
(ssh-keygen -i -m PKCS8), not the PEM/PKCS8 that `step crypto keypair` writes
-- a PEM @cert-authority anchor is unparseable and silently falls back to TOFU.
- ssh-ca-bootstrap.sh: fail loudly (and verify enableSSHCA=true) if the
provisioner SSH-enable step fails, instead of swallowing it and shipping a CA
that 502s every runtime sign.
- dot1x-bench: point the co-located broker at https://localhost:8443, not the
fleet dnsmasq name ca.fleet.lab (unresolvable from the broker's host netns ->
"no such host" at sign time). localhost is in the CA cert SAN.
- serving: remove the mock-signer enrol-broker copy -- two host-networked
brokers collide on 9090/9443 and the mock issued untrusted certs. The real
broker lives with step-ca (dot1x-bench); nginx /enrol/ still proxies to it on
loopback.
- enrol-broker: retire the HTML admin console; the fleet-manager passkey SPA now
drives the mTLS JSON API (/pending, /approve|deny). Removes a weaker,
client-cert-only parallel human-approve path.
Design-doc bench note deferred: docs/ssh-host-certificate-design.md lives on the
secure-ntp branch (fbb8100), not here -- to be added there / on reconcile.
fleet-ssh-hostcert — SSH host certificate client
At boot, gets this host's per-boot SSH host key signed by the fleet SSH CA
(step-ca) so administrators trust every fleet host via a single @cert-authority
line in known_hosts — instead of trust-on-first-use (TOFU) on a key that changes
every boot. Closes F-02 / POA&M P-17. Full design: docs/ssh-host-certificate-design.md.
Why
The diskless root is a RAM overlay, so sshd-keygen@ regenerates the SSH host
keys every boot (see scripts/extract-artifacts.sh). Clients can therefore
never pin a host key — every admin SSH connection is TOFU, and a LAN MITM is
undetectable client-side. A CA-signed host certificate removes TOFU fleet-wide:
the key can stay ephemeral; trust comes from the CA, not the key.
How (Design A — per-boot key + boot-time cert)
- Load the TPM identity (persistent handle
0x81000100+ leaf cert in NV0x01800100) and build an mTLS client cert from it — the same identity used for 802.1X, the NVMe hostnqn and agent mTLS. The private key never leaves the TPM. - Read the fresh
/etc/ssh/ssh_host_ecdsa_key.pub. POSTit to the enrol-broker host plane (/ssh/hostcert) over mTLS. The broker verifies the client cert to the fleet CA (⇒ an enrolled host) and has step-ca sign an SSH host cert — the broker, not the host, holds the CA provisioner credential, so step-ca stays behind the single front door.- Write
/etc/ssh/ssh_host_ecdsa_key-cert.pub+ aHostCertificatesshd drop-in (60-hostcert.conf), thensystemctl reload sshd.
Fail-open: any failure (no network, broker/CA down, host not yet enrolled)
exits non-zero without touching sshd — the host keeps serving its bare per-boot
key (today's behaviour). It never installs a missing/broken HostCertificate that
would stop sshd from starting. ssh-hostcert.timer retries (boot + hourly); a
signed cert lands once the broker is reachable.
Identity model / invariants
- No per-host state in the image (invariant #1): the broker DNS name in
ssh-hostcert.envis fleet-generic; the host identity comes from the TPM at runtime. Principals default to the runtime hostname (never baked). - Reuses the one TPM identity ("one identity, N stages") as the mTLS client auth to the broker — the SSH host key itself is a separate ephemeral key, so this does not add cross-protocol signing exposure to the identity key (cf. F-06).
Config
Fleet-generic knobs in ssh-hostcert.env (installed to /etc/ssh-hostcert/):
SSH_HOSTCERT_BROKER_URL (the enrol-broker host-plane endpoint),
SSH_HOSTCERT_CACERT, SSH_HOSTCERT_HANDLE/_NVINDEX, SSH_HOSTCERT_PRINCIPALS.
All are also -flags for fleet-ssh-hostcert.
Build / test
Pure Go (go-tpm, CGO_ENABLED=0), one static reproducible binary like
fleet-enrol/fleet-agent; deps pinned by go.sum. Built out of the VM by
CI and file-provisioned into the rootfs (Packer 2b-ssh-hostcert, gated on
enable_ssh_hostcert).
cd files/ssh-hostcert
CGO_ENABLED=0 go test ./...
CGO_ENABLED=0 GOTOOLCHAIN=local go build -trimpath -buildvcs=false -ldflags '-s -w -buildid=' -o bin/fleet-ssh-hostcert .
Serving side (once per environment)
serving/dot1x-bench/step-ca/ssh-ca-bootstrap.sh enables step-ca's SSH CA, allows
the enrol provisioner to issue SSH host certs, issues the broker host-plane
server cert, and prints the admin @cert-authority line. The broker host plane is
enabled by BROKER_HOST_ADDR (wired in the dot1x-bench compose). Several steps are
BENCH-VERIFY against your step / step-ca version — notably the SSH host
principal authorisation policy. See serving/enrol-broker/ (host.go, sshsigner.go).
Bench checklist (BENCH-VERIFY)
- Broker host plane:
step ssh certificate --host --sign …(sshsigner.go) succeeds against the CA's SSH host CA; the exact flags are the tunable part of the contract. - Host principal policy: requested principals are authorised (tighten the broker to
validate
principalsagainst the peer cert). - SELinux: the (root) service writing
/etc/ssh/*and runningsystemctl reload sshdunder confinement — may need a small local TE module (à la the chrony read module). Watch for AVCs. - Client:
ssh <host>shows no host-key warning once@cert-authorityis set.