F-02/P-17 SSH host-cert flow was proven end-to-end on comet-beacon-54
(2026-07-11); these fix the correctness/scoping/cleanup defects the bench
surfaced:
- ssh-hostcert: derive the FQDN principal <hostname>.<domain> from the DHCP
search domain at runtime (resolv.conf), alongside the short hostname, so an
admin `@cert-authority *.<domain>` line accepts by-FQDN logins instead of
"name is not a listed principal". Never baked (invariant #1). + tests.
- ssh-ca-bootstrap.sh: export the SSH host CA public key in OpenSSH wire format
(ssh-keygen -i -m PKCS8), not the PEM/PKCS8 that `step crypto keypair` writes
-- a PEM @cert-authority anchor is unparseable and silently falls back to TOFU.
- ssh-ca-bootstrap.sh: fail loudly (and verify enableSSHCA=true) if the
provisioner SSH-enable step fails, instead of swallowing it and shipping a CA
that 502s every runtime sign.
- dot1x-bench: point the co-located broker at https://localhost:8443, not the
fleet dnsmasq name ca.fleet.lab (unresolvable from the broker's host netns ->
"no such host" at sign time). localhost is in the CA cert SAN.
- serving: remove the mock-signer enrol-broker copy -- two host-networked
brokers collide on 9090/9443 and the mock issued untrusted certs. The real
broker lives with step-ca (dot1x-bench); nginx /enrol/ still proxies to it on
loopback.
- enrol-broker: retire the HTML admin console; the fleet-manager passkey SPA now
drives the mTLS JSON API (/pending, /approve|deny). Removes a weaker,
client-cert-only parallel human-approve path.
Design-doc bench note deferred: docs/ssh-host-certificate-design.md lives on the
secure-ntp branch (fbb8100), not here -- to be added there / on reconcile.
Four fixes from the on-host agent bring-up:
1. Bake the real fleet CA into the image. New `fleet_ca` build var (default =
the committed placeholder) file-provisions the trust anchor to
/etc/dot1x/radius-ca.pem — used by wpa_supplicant (RADIUS) AND the fleet-agent
(verify the manager). CI materializes a FLEET_CA secret to a runner-temp file
and passes -var fleet_ca; unset => placeholder (lab). No CA material committed.
2. server-pki.sh installs the agent CLIENT CA as root+intermediate (not root
only). Host TPM certs are leaf-only in NV and signed by the step-ca
intermediate, so a root-only client CA makes the manager advertise only the
root in its CertificateRequest -> Go clients send no cert ("tls: certificate
required") and it couldn't verify a leaf-only cert anyway. bootstrap.sh now
exports intermediate_ca.crt so FLEET_CA_INTERMEDIATE resolves.
3. New `fleet-manager register-host <serial> [-name] [-hostnqn]` subcommand:
seeds/updates an inventory record for a host enrolled out of band (bench
broker, migrated fleet) through the real store, so its agent stream is
accepted. The manual counterpart to the authorise-approve flow.
4. The agent surfaces the authoritative recv-side gRPC status instead of the
bare io.EOF gRPC returns on the send side when the server ends the stream —
so "host is not enrolled" (PermissionDenied) shows up instead of "EOF".
The Dell 820 kept "finding a cert" and bypassing enrolment despite repeated BIOS
TPM clears. tpm2_nvreadpublic revealed 0x01800001 on that hardware is a FIRMWARE
platform index (platformcreate|policydelete|writelocked, 70 bytes — not our
cert; siblings at 0x01800003/4 + 0x01810000). platformcreate survives TPM2_Clear
and policydelete needs the platform's secret policy, so it's permanent and
unusable by us — the blind check false-positived on it, and enrolment could
never write there anyway. Same class of bug as the EK-handle collision.
Move the NV cert index default 0x01800001 -> 0x01800100 (free on the 820, owner
range, overridable via rd.dot1x.nvindex): cmdline (extract-artifacts.sh),
fleet-enrol -nvindex, the dracut hooks, bench scripts, CLAUDE.md invariant #1,
and design/SSP docs. The signing-key handle (0x81000100) is unchanged.
Harden WriteNVCert: refuse a platformcreate index with a clear "pick a free
owner index via rd.dot1x.nvindex" error instead of a confusing undefine
failure. Regression simtest/TestWriteNVCertRefusesPlatformIndex; NV roundtrip +
ReadNVCert tests moved to 0x01800100. All go tests + packer validate pass.
Bench (Proxmox vTPM, host cygnus-cruiser-99) confirmed via tpm2_readpublic that
0x81010001 — the handle our design used for the fleet signing key — is the
TCG-standard RSA Endorsement Key slot: the object there is the swtpm-provisioned
EK (restricted|decrypt|adminwithpolicy, RSA-2048/AES128CFB, canonical EK
authPolicy 837197…, EK cert at NV 0x1c00002). Most firmware/swtpm TPMs occupy
that slot, so EnsureKey could never create our key there — it errored reading
the EK as ECC and enrolment died before the CSR/POST.
Move the fleet signing key to 0x81000100, a non-reserved owner persistent
handle (clear of EK 0x81010001/2 and SRK 0x81000001/2). Threaded through:
- fleet-enrol -handle default + new rd.dot1x.handle= cmdline knob (baked by
extract-artifacts.sh, overridable via SMBIOS)
- files/etc/dot1x/wpa_supplicant-dot1x.conf private_key handle
- the dracut hook, bench tpm-provision.sh / tpm-selftest.sh
- CLAUDE.md invariant #1 + design/SSP docs + TODO #11
NV index 0x01800001 (owner range) is unchanged.
EnsureKey hardening: refuse to evict any restricted-decrypt object (EK/SRK) as
a safety net — never clobber the TPM endorsement/storage identity (also
protects the future EK/AK attestation path). Still self-heals a genuine stale
signing key. New regression TestEnsureKeyRefusesToEvictProtectedKey; the
stale-key regression and handle constants moved to 0x81000100.
Design note records the IDevID (0x81020001) option for the later EK/AK
attestation work. Follow-up items logged: content-aware NV presence check
(verify the cert chains to the fleet CA, so leftover/foreign NV can't suppress
enrolment) and a de-enrol/TPM-reset procedure.
The `step` CLI ran against --ca-url https://ca.fleet.lab:8443, but that name is
only served by the fleet dnsmasq to booting clients — the serving host's own
resolver can't resolve it ("no such host on 127.0.0.53"), so provisioner list +
RADIUS cert issuance failed. Connect over CA_CONNECT (default localhost, which
the compose puts in the CA cert SAN); CA_DNS stays the cert/SAN + RADIUS CN name.
Verifying the step signer against a real step-ca surfaced two real bugs the
mock signer had hidden, plus made the bench CA actually initialise:
- signer.go: move --not-after from `step ca token` (validity caps ~1h ->
"token validity too long") to `step ca sign` (it is the CERT lifetime).
Without this the step signer never issues a cert.
- dot1x-bench compose: DOCKER_STEPCA_INIT_PASSWORD_FILE pointed at a file that
does not exist at first init (it is an INPUT path), so `step ca init` failed
and never wrote secrets/password. Use DOCKER_STEPCA_INIT_PASSWORD (the value)
which the entrypoint writes to secrets/password, where bootstrap + broker read.
- bootstrap.sh: self-heal the root-owned ./pki Docker auto-creates; detect the
CA password at $STEPPATH/password or secrets/password (varies by image
version); raise authority.claims.maxTLSCertDuration so 720h fleet certs are
allowed (stock cap 24h, which also blocked bootstrap's own RADIUS cert); fix
RADIUS cert paths to write via the /export mount (were host paths run inside
the container, so they never resolved).
- serving stack: add the enrol-broker service (temporary home; nginx already
proxies /enrol/) defaulting to the zero-dependency lab path (mock signer +
insecure loopback admin), with ENROL_* knobs to switch to step + mTLS.
- verify-serving.sh: exercise request -> approve -> cert through the deployed
nginx /enrol/ proxy.
- TODO #11: record the step-signer verification + the two fixes.
Step signer VERIFIED end-to-end against a real step-ca: issued cert chains to
the fleet root, EKU clientAuth, hostnqn derived from serial.
Audiobuzz