diskless-workstation/docs/tpm-enrolment-design.md

# TPM identity enrolment + the fleet identity lifecycle — design note

**Status:** DESIGN (consolidated 2026-06-28). Supersedes the scattered decisions
in `docs/TODO.md` #11 (which it folds in) and closes the bootstrap gap left open
by `docs/fleet-management-design.md`. The bench stand-in
(`serving/dot1x-bench/enrol/tpm-provision.sh`) already proves the cryptography
end-to-end *minus* the human-approval queue; the only unbuilt piece is the
**enrolment broker**.

How a diskless host acquires — and keeps — its **one cryptographic identity**: a
signing key at TPM handle `0x81000100` and a cert at NV index `0x01800100`.
Invariant #1 forbids baking any per-host state into the image, so this identity
**must** be minted at enrolment, at runtime, from the TPM. That single identity is
then reused for everything: **802.1X EAP-TLS**, the **NVMe hostnqn** (derived from
the cert serial), and **mTLS to the fleet-manager**. This note is the authoritative
design for how it is born, used, renewed, and revoked.

## The gap this closes

`fleet-management-design.md` authenticates every host by **mTLS keyed on the TPM
cert** — but never says how the cert gets onto the host. That is a chicken-and-egg:
*mTLS needs a cert; the cert needs enrolment; enrolment is the one thing that
cannot itself be mTLS-authenticated* (an empty-NV host has no client identity yet).
Enrolment is therefore the **sole unauthenticated bootstrap** in the whole system,
and the management plane's entire trust model rests on it. Treating the enrolment
**broker as the fleet-manager's one unauthenticated endpoint** — rather than a
separate box — is what makes the two designs one coherent identity story.

## Design philosophy (forced by the architecture invariants)

1. **One identity, minted once, reused everywhere.** Key `0x81000100` + cert
   NV `0x01800100` = 802.1X identity = NVMe hostnqn (`…:host-<cert-serial>`) =
   mTLS client identity. There is no second secret. (Invariant #1.)
2. **The private key never leaves the TPM.** Generated *in* the TPM; only a CSR
   and the issued cert ever cross the wire — both are public. This is why the
   pre-auth, pre-MACsec enrolment link being plaintext is acceptable.
3. **Humans authorise; machines sign.** step-ca can sign but has no notion of
   "should I." The authorisation decision is a **human OOB act** (fingerprint
   cross-check). No auto-sign of unauthenticated CSRs, ever.
4. **Enrolment is the birth of the inventory record.** The fleet-manager keys
   every host by `cert-serial / hostnqn`; that key does not exist until enrolment.
   So approval is simultaneously `INSERT host` and the moment a human **binds
   physical identity → cryptographic identity** (asset tag / switch port /
   location → cert serial). Every downstream signal hangs off this record.
5. **Integrity of the pre-auth path comes from Secure Boot, not the network.**
   The UKI fetch and the enrolment exchange both happen pre-auth on an untrusted,
   unencrypted L2. The boot is trustworthy because the **UKI is Secure-Boot-signed**
   (#8), not because the wire is. Enrolment design and Secure Boot are two halves
   of the same "the pre-auth path is hostile" assumption.

## The identity lifecycle (the load-bearing state machine)

Triggered by the state of NV `0x01800100` and the cert within it. The three events
are deliberately **split** — conflating them is a security regression:

```
  EMPTY NV                  → ENROL    human OOB approval; broker→step-ca sign; write NV.
                                       Births the fleet-manager inventory record.
  VALID KEY + cert near/at  → RENEW    automatic, NO human. The host already holds a
  expiry                               CA-trusted key; re-sign the same key, rewrite NV.
  AUTH FAILS, cert present   → ALERT    retry + raise an alert. NEVER auto-re-enrol —
                                       that would mask a revocation and churn keys on a
                                       transient RADIUS/CA outage.
  Re-enrol after first time  → ADMIN    explicit admin action (clear/flag the host); not
                                       reachable from any automatic path.
```

- **Renew is safe to automate** precisely because the host proves possession of an
  already-trusted key — no human needed, no new trust established.
- **Auth-failure must not re-enrol.** A revoked host that re-enrolled itself would
  defeat revocation. Failures alert; they never mint new identity.

## Components

### 1. TPM key + NV cert (exists — `tpm-provision.sh`)

Key generated in the TPM, persisted at `0x81000100`; CSR built with it via the
openssl tpm2 provider (key never leaves the TPM); issued cert written DER to NV
`0x01800100`. At boot, `dot1x-macsec-up.sh` reads the cert from NV, wpa_supplicant
signs EAP-TLS with the handle key, and the hostnqn is derived from the cert serial.
All of this is built and bench-proven; enrolment just replaces the *manual* sign
step with a brokered, human-gated one.

### 2. The enrolment broker (the only new component)

A small service that turns "step-ca can sign" into "step-ca signs **only what a
human approved**." Stateless-leaning; pending state is short-lived and server-side.

**API (HTTPS; image trusts the serving CA):**

| Route | Auth | Action |
|---|---|---|
| `POST /enrol/request` | none (bootstrap) | accept CSR + self-reported device context; create a time-boxed **PENDING** record; return `enrol-id` + the **fingerprint** to display |
| `GET /enrol/status/{enrol-id}` | none (bootstrap) | host polls → `PENDING` \| `APPROVED`+cert \| `DENIED` \| `EXPIRED` |
| `GET /enrol/pending` | **mTLS, admin** | the approval queue (one entry per pending CSR, with context + fingerprint) |
| `POST /enrol/approve/{enrol-id}` | **mTLS, admin** | bind physical context → record; hand CSR to step-ca; store the issued cert |
| `POST /enrol/deny/{enrol-id}` | **mTLS, admin** | reject + audit |

The **request/status** routes are the sole unauthenticated surface; **pending/
approve/deny** are admin-only mTLS — i.e. they are already the fleet-manager's
authenticated plane. This is why the broker *is* the fleet-manager's bootstrap
endpoint, not a separate trust domain.

**Device context** (what an un-enrolled host can self-report, for the admin to
recognise *what* they are approving): NIC MAC, DMI system UUID/serial, TPM EK
public-key hash, and switch port via LLDP if available. Context is advisory — the
fingerprint is the cryptographic anchor.

### 3. OOB fingerprint approval (the human gate) — v1 assurance

**v1 ships fingerprint-only** (TPM attestation is the documented upgrade, §
"Stronger assurance"). The flow:

- `fingerprint = BIP39-encode( SHA-256( CSR_SubjectPublicKeyInfo_DER || server_nonce )[:N bits] )`
  — both broker and host compute it identically from the CSR public key and a
  **server-chosen per-`enrol-id` nonce**.
- The host shows it on the **Plymouth splash** (states only — invariant #1 — never
  per-host secrets; the fingerprint is public). The admin reads it off the physical
  device's screen and matches it to that device's entry in `GET /enrol/pending`.
- **What it proves:** the queue entry the admin approves carries the *same key* as
  the device physically in front of them. **What it does not prove:** that the key
  actually resides in a TPM (that's attestation, below).

Hardening, all required for v1:
- **Word-list encoding, not short hex.** Hex is grindable; ~6 BIP39 words (~66
  bits) reads short but resists collision-grinding.
- **Server nonce per `enrol-id`** so an attacker cannot pre-grind a key whose
  fingerprint collides with a target device's (the nonce is unknown until after
  submission, and is per-request).
- **Rate-limit + time-boxed pending** (expire unapproved requests) to bound queue
  flooding.
- **Show device context** beside the fingerprint so the admin approves a *specific*
  MAC/serial, not just a matching string.

### 4. step-ca (exists — `serving/dot1x-bench/step-ca/`)

The fleet CA. The broker calls it on approval (token-gated `step ca sign`, exactly
as `tpm-provision.sh` does today). Same root that is baked as `radius-ca.pem` and
pinned for mTLS — one trust anchor across 802.1X, NVMe, and management.

### 5. Relationship to the fleet-manager and host-api

- **Broker ⊂ fleet-manager.** `/enrol/request` + `/enrol/status` are the
  fleet-manager's *only* unauthenticated routes; `/enrol/pending|approve|deny` are
  its admin mTLS plane. Approval writes the inventory record the manager keys
  everything else on. Build the broker standalone first (fastest path to the state
  machine), but design it to fold into the mgmt service with no rewrite — same Go
  style, same step-ca anchor, same data model.
  - **AS-BUILT (2026-07): folded in "alongside", not merged.** The
    `serving/fleet-manager/` control plane drives the standalone broker's admin
    plane over mTLS (broker admin CA) rather than absorbing its code: the broker
    stays the enrolment **state machine + step-ca signer**, and gained an
    `Accept: application/json` approve response so the manager gets the decided
    enrolment back and **births the inventory record**, recording the human
    (passkey-authenticated) admin as approver. `verify-serving.sh` / browser
    callers are unchanged (still get the 204/redirect). The public
    `/enrol/request|status` edge is untouched. This keeps the one unauth,
    stdlib-only, reproducible pre-auth parser isolated while giving the manager
    the authenticated authorise pillar — same trust anchor, same data model, no
    rewrite. See `serving/fleet-manager/README.md` (Authorise pillar).
- **host-api is not in the enrolment path** (it is loopback/post-boot; the host has
  no cert during enrolment). The tie is downstream: the fleet-manager **agent** (the
  push sibling of `fleet-hostapi`) authenticates with the *enrolled* cert over mTLS,
  and inherits host-api's hardened posture (DynamicUser, fapolicyd-trusted,
  reproducible static build). One identity, three stages: **enrol** mints it →
  **host-api/agent** uses it → **renew** refreshes it.

## End-to-end enrolment flow

```
  empty-NV host (early-boot enrol branch) broker (= fleet-mgr bootstrap)     admin        step-ca
  ───────────────────────────────         ─────────────────────────────     ─────        ───────
  1. key in TPM @0x81000100
  2. build CSR (key never leaves)
  3. POST /enrol/request (CSR+context) ─▶  create PENDING, pick nonce,
                                           compute fingerprint
     ◀── enrol-id + fingerprint ─────────
  4. show fingerprint on Plymouth
                                           GET /enrol/pending ◀──────────── 5. read queue (mTLS)
                                                                            6. cross-check fp on
                                                                               the device screen
                                           POST /enrol/approve/{id} ◀────── 7. approve + bind
                                                                               physical→crypto
                                           bind context → INSERT host,
                                           sign CSR ──────────────────────────────────▶ issue cert
                                           store cert ◀───────────────────────────────
  8. GET /enrol/status/{id} (poll) ───▶
     ◀── APPROVED + cert ────────────────
  9. write cert DER → NV 0x01800100
  10. reboot into production UKI → 802.1X EAP-TLS → mTLS mgmt
```

## Implementation: language + transport (decided 2026-06-28)

**Go for all three fleet-authored components** (broker, `fleet-hostapi`, the
fleet-manager agent) — one language, one build/posture story. Transport is split by
*who the client is* and *where the code runs*, not by preference:

- **Enrolment path → plain HTTP/1.1 + JSON, stdlib-only. NOT gRPC.** It is two unary
  calls + a poll (`POST /enrol/request` → `GET /enrol/status` → cert), so gRPC's
  streaming/multiplexing buys nothing. It is the system's *one* unauthenticated,
  pre-auth, security-critical edge → the smallest, most auditable parser wins (a
  `net/http` handler over a JSON body, not HTTP/2 + protobuf). The **client runs in
  the early-boot initramfs** (the production UKI's empty-NV branch for now; a
  provisioning UKI later) → keep it stdlib-only and reproducible like `fleet-hostapi`
  (no `grpc-go`/protobuf vendoring in the minimal environment). And it
  must reverse-proxy through the existing serving nginx `/enrol/` location — a one-line
  `proxy_pass` for HTTP/1.1 vs `grpc_pass` + end-to-end HTTP/2 on the pre-auth edge.
  CSR/cert ride as base64/PEM in JSON. **Boring is the feature on the bootstrap path.**
- **Admin approve/deny routes → mTLS (HTTP/JSON or gRPC, builder's call).** These are
  already the fleet-manager's authenticated plane.
- **Steady-state agent ↔ manager (telemetry + reconciliation) → gRPC over mTLS,
  host-dialed.** Streaming fits here, and it stays compatible with "no inbound port on
  hosts" because the **host dials out** and holds one long-lived mTLS stream (health
  up / desired-state down). ~~OPEN FORK~~ **RESOLVED (2026-07):** a **bespoke gRPC
  bidi stream** (`serving/fleet-manager/proto/fleet_agent.proto` —
  `FleetAgent.Session`, health/metrics up + desired-state down), over the same
  reused TPM cert (fleet CA). Metrics land in **InfluxDB** via the manager (host
  holds no Influx credential), not OTLP→VictoriaMetrics; events stay on the Wazuh
  path. One stream, no polling, no inbound host port. (The on-host agent — the
  gRPC report/reconcile client that pairs with `fleet-hostapi` — is still to build.)
- **host-api stays HTTP/JSON — locked**, because its consumer is the Chromium kiosk
  (a browser, which cannot speak native gRPC).

## Where it runs: enrolment vehicle (build-flag, two modes)

Empty-NV and normal boot are **mutually exclusive** — a host with no cert cannot do
802.1X → MACsec → NVMe-root at all, so there is no normal hot-path sequence to
"protect" in the empty-NV case. The choice of vehicle is therefore *not* about
invariant #2 timing; it is about **whether enrolment capability ships in the
production artifact at all**. Two modes behind one build flag (`enrol_mode`, modelled
on the existing `disable_dot1x` / `boot_splash` cmdline toggles):

- **Inline single-UKI — CHOSEN FOR NOW (decided 2026-06-28).** The production UKI
  auto-detects empty-NV and branches into enrolment in early boot. Picked because it
  needs **no separate provisioning boot path** — everything stays purely
  network-booted (no USB / no provisioning media to manage). The empty-NV gate is the
  safety: NV-write at `0x01800100` needs TPM owner auth, so empty-NV is hard to
  coerce. Accepted trade-offs: production carries the enrolment + pre-auth broker code
  permanently, and there is no PCR-11 discrimination. The empty-NV branch is mutually
  exclusive with the secure path, so it does not endanger the load-bearing hook
  ordering. It needs a **pre-auth DHCP on the physical NIC** to reach the broker
  through the pre-auth permit-set (the same pre-auth network the firmware already used
  to fetch the UKI, like the `rd.dot1x=0` test path); on success it writes NV and
  **reboots into the normal secure path** (convergence = reboot).
- **Provisioning-UKI variant — PLANNED LATER UPGRADE.** `enrol_mode=true` conditionally
  `install()`s the enrolment scripts into the dracut module *and* bakes an immutable
  `rd.fleet.enrol=1` cmdline → a **separately signed UKI from the same source tree**.
  Production then lacks the code entirely (smaller initramfs, **physically cannot
  enrol**), matching the #10 Secure-Boot-enrolment decision and enabling PCR-11
  discrimination. Note the later separate UKI can still be **PXE/network-served**
  (dnsmasq `provisioning` tag / unknown-MAC) — it does **not** require USB media; the
  reason to defer is simply the extra boot-path + CI sign/artifact, not a media
  dependency. Revisit once the broker is built and the secure path is verified.

Either way, the production *boot* path only ever **reads** an already-present cert;
the human-gated, indefinitely-waiting enrolment lifecycle never interleaves with the
unattended, watchdog-tuned normal boot.

| | Inline (one UKI) | Provisioning variant (build-flag) |
|---|---|---|
| Ops | simplest — one artifact | +1 CI sign/artifact (same tree) |
| Prod attack surface | carries enrolment + pre-auth broker code | none — code absent from prod |
| "Prod can't enrol" guarantee | empty-NV gate (TPM owner auth) | structural (code/cmdline absent) |
| PCR-11 discrimination | no | yes |
| Consistency with #10 | diverges | matches |

## Pre-auth network placement

The firmware cannot do 802.1X, so the UKI fetch (DHCP/DNS/TFTP/HTTP to the serving
host) is **always pre-auth** — the secure model therefore already *requires* a
pre-auth permit-set (low-impact 802.1X / pre-auth dACL / boot→secure dynamic VLAN;
RADIUS returns the secure VLAN on Access-Accept). **Enrolment is just one more entry
in that set:** expose `/enrol/` on the **existing serving nginx**, reverse-proxied
to the broker. No new network mechanism. Pre-auth traffic has **no MACsec**
(plaintext L2) — fine here, because CSR/cert are public and the fingerprint supplies
authenticity, and because boot integrity comes from the SB-signed UKI.

**Bench caveat:** `hostapd` wired is strictly port-based (no pre-auth per-flow ACL):
before EAP success it passes nothing, so the rig cannot reproduce "boot + enrol
pre-auth on the same port." Model the pre-auth state with a separate provisioning
bridge/VLAN on the bench; validate the in-place low-impact pre-auth permit on the
820's real switch.

## Security posture / threat model

- **Sole unauthenticated surface = `/enrol/request|status`.** Scope the pre-auth
  permit to *only* these routes + the serving host; everything else is mTLS.
- **No auto-sign.** Every cert traces to a human OOB approval; the broker never
  signs on its own.
- **Fingerprint binds queue↔device** (v1). It does **not** prove TPM residency or
  prevent a determined insider — it raises the bar to "be physically at the device
  and have an admin approve it."
- **Rate-limit + time-box + audit** every request/approve/deny to the collector
  (the binding event is forensically important).
- **Revocation is real** only because auth-failure never auto-re-enrols (see
  lifecycle). Pair with step-ca CRL/short-lived certs + the renew path.

## Stronger assurance (documented upgrade, not v1)

Fingerprint proves "this physical device"; it does not prove "the key is in a TPM."
The residency proof is step-ca's ACME **`device-attest-01`** provisioner (TPM
EK/AK attestation): switch enrolment from `step ca sign` to an ACME exchange with a
TPM attestation provisioner on the same CA, anchored to the TPM vendors' EK roots.
Together — **fingerprint = a human authorised the device, attestation = the key is
genuinely TPM-resident** — this is the strong SSP R-01 story. Deferred from v1 to
keep the first build tractable; the broker's approve step is the natural insertion
point (verify the attestation statement before signing). The CA bootstrap already
carries a commented `acme-attest` provisioner block
(`serving/dot1x-bench/step-ca/bootstrap.sh`).

### Signing-key handle choice (and the IDevID option)

The fleet signing key is persisted at **`0x81000100`** — a non-reserved **owner**
persistent handle. It is deliberately **not** the EK slot `0x81010001` / `0x81010002`
nor the SRK `0x81000001` / `0x81000002`: firmware and swtpm provisioning routinely
occupy those (bench-confirmed 2026-07-02 — a Proxmox vTPM had the standard **RSA
Endorsement Key** at `0x81010001`, restricted-decrypt, with the canonical EK
authPolicy `837197…`, and its EK cert at NV `0x1c00002`). `EnsureKey` therefore
also refuses to evict any restricted-decrypt object as a safety net. The handle is
overridable via `rd.dot1x.handle=` / `fleet-enrol -handle`.

**Future option — IDevID (`0x81020001`):** if/when the EK/AK attestation path above
lands, the natural home for a TPM-resident *device identity* key is the TCG
**IDevID** handle `0x81020001` (with **LDevID** at `0x81030001`), per *TPM 2.0 Keys
for Device Identity and Attestation*. Moving there would align our key with the
IDevID template + EK-anchored certification flow (the key is certified by an AK that
chains to the EK) rather than a bare owner primary. Not adopted for v1 (it implies
the full IDevID template/cert conventions we don't yet follow); revisit together with
`device-attest-01`.

## Inventory record born at enrolment

```
Host (by cert-serial / hostnqn)            ← created at /enrol/approve
  ├─ identity:  cert serial, hostnqn, MAC, TPM EK hash, starship-name
  ├─ binding:   asset tag, switch port, location   ← the human OOB act records this
  ├─ enrolled:  approver, timestamp, fingerprint, attestation? (later)
  └─ (then fleet-manager fills) version / health / usage / desired-state
```

This is the join key the rest of `fleet-management-design.md` assumes already
exists. Enrolment is where it comes into being.

## Phasing

1. **Built / proven:** TPM key→CSR→sign→NV (manual, `tpm-provision.sh`);
   `tpm-selftest.sh` capability check; step-ca + EAP-TLS bench.
2. **Broker — BUILT (2026-06-28): [`serving/enrol-broker/`](../serving/enrol-broker/).**
   Go, stdlib-only. Public unauth listener (`POST /request`, `GET /status/{id}`)
   behind the serving nginx `/enrol/`; admin **mTLS web UI** (`/`, `/pending`,
   `/approve|deny/{id}`) on the mgmt network; wordlist fingerprint = `H(CSR-pubkey ‖
   server-nonce)`; time-boxed in-memory store + per-IP rate limit; signs via
   `step ca token`+`step ca sign` (broker holds the provisioner password) with a
   dev `mock` signer. **Fingerprint-only assurance.** Admin client CA is a
   **dedicated** CA (`admin-pki.sh`), separate from the fleet CA, so host certs
   can't authenticate to the admin plane. End-to-end smoke-tested (CSR → mTLS
   approve → cert; pubkey match). Ships in `serving/dot1x-bench/docker-compose.yml`.
3. **Host side — BUILT 2026-07-02 (awaiting bench):** the **inline empty-NV
   trigger** as a static **Go + go-tpm** binary (`files/enrol/`, `fleet-enrol`)
   talking to `/dev/tpmrm0` directly — ensures the key at `0x81000100`, builds the
   CSR, POSTs to the broker, shows the Plymouth fingerprint, polls, writes the leaf
   cert to NV `0x01800100`. A thin dracut GATE hook (`95dot1x-macsec/tpm-enrol.sh`,
   initqueue/settled prio 25) does the empty-NV check + pre-auth DHCP on the
   physical NIC + reboot-to-converge. No tpm2-tools key-gen / openssl provider /
   curl needed in the image (go-tpm does it all). Built **out of the VM** on the CI
   runner + file-provisioned in (like `fleet-hostapi`); `enrol_url` Packer var →
   `rd.fleet.enrol_url`. TESTED: transaction (mock broker) + full TPM ceremony vs
   the go-tpm simulator (`files/enrol/simtest/`). BENCH-VERIFY list in `docs/TODO.md` #11.
4. **Folds into fleet-manager:** the admin routes become the mgmt service's
   authenticated plane; approval writes the shared inventory record.
5. **Provisioning-UKI variant** (`enrol_mode`) — move enrolment out of the production
   artifact once the broker + secure path are proven (optional, network-served).
6. **Attestation upgrade:** ACME `device-attest-01` at the approve step.

## Open questions for build time

- **openssl TPM2 provider in the enrolling image** (the production image, inline) —
  `tpm2-tss` ≠ the provider;
  `tpm-selftest.sh` flags whether `tpm2-openssl` must be added (also affects the
  wpa_supplicant key-ref form: provider `handle:` vs tpm2-pkcs11 URI).
- **Pending store:** in-memory (simplest, lost on broker restart) vs a tiny
  persistent store. Time-boxed pending makes in-memory acceptable for v1.
- **Enrolment-vehicle** — inline single-UKI is **chosen for now** (network-boot only,
  no USB); the provisioning-UKI variant is the planned later upgrade behind the
  `enrol_mode` build flag (shares the #10 vehicle, not yet built) — see "Where it
  runs". Revisit when the broker is built + the secure path verified.
- **Steady-state transport fork** (fleet-manager, not enrolment) — bespoke gRPC vs
  OTLP-gRPC metrics + Wazuh events + a thin gRPC control stream. See
  [fleet-management-design.md](fleet-management-design.md).
- **Fingerprint length** — exact bit count / word count (entropy vs readability).
- **Open/test path identity** (`rd.dot1x=0`): MAC/fleet-name is a weak stand-in;
  gate real enrolment to the secure path, or accept lab-only there.