141 lines
7.7 KiB
Markdown
141 lines
7.7 KiB
Markdown
# SSH host-certificate design — killing per-boot TOFU (F-02 / P-17)
|
|
|
|
> Status: **implemented (host + broker + serving scaffold), BENCH-VERIFY.** Branch
|
|
> `poam-hardening`. Closes SAR finding **F-02** / POA&M **P-17**. Signing is
|
|
> **broker-mediated** (host → enrol-broker host plane → step-ca), not host-direct.
|
|
|
|
## Problem
|
|
|
|
The diskless root is an immutable erofs + a **RAM overlay**, so
|
|
`sshd-keygen@.service` regenerates the SSH **host keys on every boot** (see
|
|
`scripts/extract-artifacts.sh`, which strips baked host keys — a baked key would
|
|
be shared fleet-wide, invariant #1). Consequences:
|
|
|
|
- Administrators can never pin a host key — every `ssh` to a fleet host is
|
|
**trust-on-first-use**, and the key *changes each reboot* so even a cached entry
|
|
triggers the "REMOTE HOST IDENTIFICATION HAS CHANGED" warning.
|
|
- A **LAN man-in-the-middle** of admin SSH is undetectable client-side.
|
|
|
|
## Approaches considered
|
|
|
|
- **A — per-boot key + boot-time CA-signed host cert (CHOSEN).** Keep the
|
|
ephemeral per-boot host key; at boot, get it **signed by the fleet SSH CA** over
|
|
a TPM-authenticated request. Clients trust via one `@cert-authority` line. The
|
|
TPM only *authenticates the signing request*; it is not the host key.
|
|
- **B — dedicated stable TPM host key + `HostKeyAgent` + cert at enrolment.**
|
|
Hardware-anchored, CA-independent, stable host identity — but needs a new
|
|
persistent TPM handle, a host-key agent daemon (SELinux/fapolicyd/ordering), and
|
|
more provisioning. Deferred; a follow-on if a CA-independent hardware host key is
|
|
specifically wanted.
|
|
|
|
**The certificate — not the key's origin — is what removes TOFU.** A per-boot key
|
|
*with a cert* is fine (short-lived keys are good hygiene), so A delivers the
|
|
security outcome with the least new attack surface and no added cross-protocol key
|
|
reuse. See the chat rationale; A was chosen 2026-07-11.
|
|
|
|
## Architecture (Design A, broker-mediated)
|
|
|
|
The signing request goes **through the enrol-broker**, not directly to step-ca —
|
|
so step-ca stays behind the single front door (consistent with enrolment), and the
|
|
proven `step` CLI does the SSH signing server-side. The host authenticates to the
|
|
broker's new **host plane** with its TPM identity over mTLS.
|
|
|
|
```
|
|
boot ─▶ sshd-keygen@ecdsa ─▶ /etc/ssh/ssh_host_ecdsa_key(.pub) (ephemeral)
|
|
│
|
|
ssh-hostcert.timer (OnBootSec + hourly)
|
|
└─▶ fleet-ssh-hostcert:
|
|
1. TPM: LoadSigner(0x81000100) + ReadNVCert(0x01800100) → mTLS client cert
|
|
2. read ssh_host_ecdsa_key.pub
|
|
3. POST {publicKey, principals, keyID} ─(mTLS)─▶ enrol-broker host plane
|
|
4. write ssh_host_ecdsa_key-cert.pub + 60-hostcert.conf; reload sshd
|
|
|
|
enrol-broker host plane (POST /ssh/hostcert):
|
|
verify client cert to the fleet CA (⇒ enrolled host)
|
|
└─▶ step ssh certificate --host --sign (broker holds the provisioner) ─▶ step-ca SSH CA
|
|
└─▶ returns {cert}
|
|
|
|
client: @cert-authority *.fleet.lab <ssh_host_ca.pub> (one line, once)
|
|
```
|
|
|
|
**Components**
|
|
|
|
| Piece | Where | Notes |
|
|
|---|---|---|
|
|
| `fleet-ssh-hostcert` | `files/ssh-hostcert/` (Go, static, go-tpm) | TPM mTLS client; out-of-VM CI build like fleet-enrol/agent; deps pinned by go.sum |
|
|
| `ssh-hostcert.service` + `.timer` | `files/ssh-hostcert/` | **root** oneshot (writes `/etc/ssh`, reloads sshd), sandboxed; timer = boot + hourly renew |
|
|
| `ssh-hostcert.env` | `files/ssh-hostcert/` → `/etc/ssh-hostcert/` | fleet-generic broker URL / CA / paths |
|
|
| Packer `2b-ssh-hostcert` | `diskless.pkr.hcl` | gated on `enable_ssh_hostcert`; fapolicyd-trusts the binary in 2c-STIG-postfix |
|
|
| broker **host plane** (`/ssh/hostcert`) | `serving/enrol-broker/host.go` + `sshsigner.go` | mTLS (fleet-CA host certs); `step ssh certificate` signer; enabled by `BROKER_HOST_ADDR` |
|
|
| step-ca SSH CA enablement | `serving/dot1x-bench/step-ca/ssh-ca-bootstrap.sh` | SSH CA keys + provisioner `--ssh` + broker host-plane cert; prints the admin `@cert-authority` line |
|
|
|
|
## Wire contract (the BENCH-VERIFY seam)
|
|
|
|
- **Host → broker** (mTLS): `POST /ssh/hostcert` `{publicKey, principals, keyID}`
|
|
→ `{cert}`. The host's mTLS client cert is its TPM identity; the broker verifies
|
|
it to the fleet CA (`RequireAndVerifyClientCert`) — *that* is the authorisation
|
|
(an enrolled host). No token, no CA secret on the host.
|
|
- **Broker → step-ca**: `step ssh certificate --host --sign …` with the broker's
|
|
provisioner credential. The exact flags are the tunable part
|
|
(`serving/enrol-broker/sshsigner.go`); the CA must have its **SSH host CA
|
|
enabled** and the provisioner allowed to issue SSH host certs
|
|
(`ssh-ca-bootstrap.sh`).
|
|
|
|
**Principal authorisation** is the remaining sharp edge: step-ca / the provisioner
|
|
must *authorise* the requested principals (the host's hostname). Tighten this in
|
|
the broker (validate `principals` against the peer cert) and/or via an SSH host
|
|
template on step-ca. Today the broker forwards the requested principals and logs
|
|
the peer CN.
|
|
|
|
## Security properties
|
|
|
|
- **Invariant #1 (no per-host state):** the image carries only a fleet-generic
|
|
broker DNS name + CA path. The host identity, and therefore the cert, derive from
|
|
the TPM at runtime. Principals default to the runtime hostname (not baked).
|
|
- **step-ca stays behind the front door:** hosts reach the broker, not step-ca. The
|
|
broker holds the provisioner credential; a host can never self-sign.
|
|
- **Fail-open:** never installs a missing/broken `HostCertificate` — a broker/CA
|
|
outage degrades to today's bare-key behaviour, not a boot/sshd failure.
|
|
- **Key reuse (cf. F-06):** the TPM identity authenticates the mTLS request; the
|
|
SSH host key is a separate ephemeral key.
|
|
- **Short-lived certs** + hourly renewal bound the blast radius of a leaked cert
|
|
(the key is ephemeral anyway).
|
|
|
|
## Client rollout
|
|
|
|
One line per admin (or a system `known_hosts`), from `ssh-ca-bootstrap.sh` output:
|
|
|
|
```
|
|
@cert-authority *.fleet.lab ecdsa-sha2-nistp256 AAAA... # fleet SSH host CA
|
|
```
|
|
|
|
## Bench checklist
|
|
|
|
1. Serving: run `ssh-ca-bootstrap.sh`; confirm `ca.json` has the `ssh` block, the
|
|
`enrol` provisioner is SSH-enabled, and `pki/broker/{host-server.crt,
|
|
fleet-client-ca.crt}` + `pki/ssh_host_ca.pub` exist. Bring up the broker with
|
|
`BROKER_HOST_ADDR` set; `journalctl` shows "host plane up".
|
|
2. Broker: from an enrolled host's identity, `curl --cert/--key … https://
|
|
boot.fleet.lab:9444/ssh/hostcert` returns a `{cert}`; an un-enrolled client is
|
|
rejected at the TLS handshake (`RequireAndVerifyClientCert`).
|
|
3. Host: `systemctl start ssh-hostcert.service`; `journalctl -u ssh-hostcert`
|
|
shows "installed SSH host certificate"; `/etc/ssh/ssh_host_ecdsa_key-cert.pub`
|
|
present; `sshd -T | grep hostcertificate`.
|
|
4. SELinux: watch for AVCs on the root service writing `/etc/ssh/*` and running
|
|
`systemctl reload sshd` under confinement — add a local TE module if needed
|
|
(like `fleet_agent_chronylog`). Reproducibility: a new `semodule` would rebuild
|
|
the policy store → re-check verity-hash stability across two builds.
|
|
5. Client: with the `@cert-authority` line set, `ssh <host>` shows **no** host-key
|
|
prompt/warning across reboots.
|
|
6. Confirm STIG crypto policy accepts the `ecdsa-sha2-nistp256-cert-v01` host cert
|
|
(same family already accepted for admin keys).
|
|
7. Principal policy: confirm the broker/step-ca only issue the principals a host is
|
|
entitled to (tighten `handleSSHHostCert` to validate against the peer cert).
|
|
|
|
## Deferred (Design B follow-on)
|
|
|
|
Stable, CA-independent hardware host key: a dedicated TPM key at a new persistent
|
|
handle (e.g. `0x81000101`, avoiding OEM/reserved NV like the identity key) served
|
|
to sshd via a small TPM `HostKeyAgent`, cert issued at enrolment. Only pursue if a
|
|
host identity that survives a CA outage is required.
|