initial tpm broker

This commit is contained in:
Audiobuzz
2026-06-28 10:33:14 +00:00
parent 3bcf16e384
commit 9e1dbd3ef7
17 changed files with 1584 additions and 19 deletions
+20 -3
View File
@@ -982,9 +982,26 @@ UKI + isolated VLAN + read-back verify.**
[tpm-enrolment-design.md](tpm-enrolment-design.md)** — unifies this item with the
fleet-manager (the broker = the fleet-manager's one unauthenticated bootstrap
endpoint; approval births the inventory record) and the full identity lifecycle
(enrol / renew / alert). **v1 decision: fingerprint-only OOB assurance**; TPM
attestation (ACME `device-attest-01`) is the documented later upgrade. The notes
below are retained as the originating decisions; the design note is canonical.
(enrol / renew / alert). **v1 decisions (2026-06-28):** fingerprint-only OOB
assurance (TPM attestation via ACME `device-attest-01` is the documented later
upgrade); **inline single-UKI** vehicle for now (empty-NV early-boot branch — keeps
provisioning network-boot-only, no USB; a separate provisioning UKI behind an
`enrol_mode` build flag is the planned later upgrade); enrolment transport is plain
**stdlib HTTP/JSON** (Go, the one unauthenticated edge — gRPC reserved for the
steady-state agent path). The notes below are retained as the originating decisions;
the design note is canonical.
**Broker BUILT (server side, 2026-06-28): [serving/enrol-broker/](../serving/enrol-broker/)**
— Go/stdlib; public unauth `POST /request`+`GET /status/{id}` (behind the serving
nginx `/enrol/`) + admin **mTLS web UI** (`/`, `/pending`, `/approve|deny/{id}`)
with a **dedicated admin CA** (`admin-pki.sh`); wordlist fingerprint, time-boxed
store, per-IP rate limit; signs via `step` (dev `mock` signer). Ships in
`serving/dot1x-bench/docker-compose.yml`; end-to-end smoke-tested (CSR → mTLS
approve → cert). **BENCH-VERIFY:** run against real step-ca (`bootstrap.sh` root +
`enrol` provisioner password readable by the broker) and confirm an issued cert
authenticates EAP-TLS. **REMAINING:** the host-side inline empty-NV dracut branch
(generate key/CSR → POST → Plymouth fingerprint → poll → NV write → reboot) +
`tpm2-openssl` in the image.
How a host gets its **802.1X EAP-TLS client identity** the first time — the key at
TPM handle `0x81010001` + cert at NV `0x01800001` that invariant #1 says must come
+15
View File
@@ -44,6 +44,13 @@ endpoint; the server authenticates the host by cert CN/SAN, which is the canonic
fleet identity (= NVMe hostnqn = 802.1X identity). Reuses the **step-ca fleet CA**
(`serving/dot1x-bench/`). No baked secret; the host also pins the server cert.
> **Bootstrap (where the cert comes from):** mTLS here assumes the host already
> holds a TPM cert — but an un-enrolled host does not. That chicken-and-egg is
> resolved by **enrolment**, which is the management plane's *one* unauthenticated
> endpoint and the moment the inventory record is born. See
> **[tpm-enrolment-design.md](tpm-enrolment-design.md)** — the enrolment broker is
> designed to fold into this mgmt service as its bootstrap routes.
- **Secure path:** full mTLS, TPM cert. Strong, no secrets in the image.
- **Open/test path (`rd.dot1x=0`):** fall back to dnsmasq fleet-name + MAC as a
weak identity. **Lab only** — flagged as insufficient for production.
@@ -181,6 +188,14 @@ bus) already exist.
- **Telemetry encoding:** structured syslog/JSON into Wazuh (Phase 1) vs native
OTLP from the start. Leaning Wazuh-first for reuse.
- **Steady-state transport (agent ↔ manager):** the agent is **Go**, the path is
**mTLS + host-dialed** (preserves "no inbound port"). Fork: bespoke **gRPC**
stream (health up / desired-state down) vs reuse **OTLP-gRPC for metrics + Wazuh
for events** + a thin gRPC control stream — the latter gets VictoriaMetrics/
Grafana/Wazuh for free. `Connect` (connectrpc.com) if one protobuf IDL serving
gRPC + JSON + browser is wanted. Note **enrolment** deliberately stays plain
HTTP/JSON (stdlib, minimal, the one unauth edge) — see
[tpm-enrolment-design.md](tpm-enrolment-design.md) "Implementation".
- **Metrics store:** VictoriaMetrics vs Prometheus (VM is lighter for push +
long retention).
- **Desired-state store:** git (GitOps, aud/rollback for free, matches the
+98 -16
View File
@@ -156,7 +156,7 @@ pinned for mTLS — one trust anchor across 802.1X, NVMe, and management.
## End-to-end enrolment flow
```
empty-NV host (provisioning UKI) broker (= fleet-mgr bootstrap) admin step-ca
empty-NV host (early-boot enrol branch) broker (= fleet-mgr bootstrap) admin step-ca
─────────────────────────────── ───────────────────────────── ───── ───────
1. key in TPM @0x81010001
2. build CSR (key never leaves)
@@ -178,15 +178,78 @@ pinned for mTLS — one trust anchor across 802.1X, NVMe, and management.
10. reboot into production UKI → 802.1X EAP-TLS → mTLS mgmt
```
## Where it runs: a provisioning UKI/mode
## Implementation: language + transport (decided 2026-06-28)
Enrolment must **not** live in the production initramfs. The boot hook ordering is
load-bearing (invariant #2: NV-cert → supplicant → Authorized+macsec0 → DHCP →
hostnqn); parking that sequence on a multi-minute human-approval wait is
unacceptable. Enrolment runs in a separate **provisioning UKI / mode** — the same
vehicle as #10 (Secure Boot key enrolment) — entered deliberately (empty-NV detect,
or an explicit provisioning boot), never as a branch inside the normal hot path.
The production UKI only ever *reads* an already-present cert; it never enrols.
**Go for all three fleet-authored components** (broker, `fleet-hostapi`, the
fleet-manager agent) — one language, one build/posture story. Transport is split by
*who the client is* and *where the code runs*, not by preference:
- **Enrolment path → plain HTTP/1.1 + JSON, stdlib-only. NOT gRPC.** It is two unary
calls + a poll (`POST /enrol/request` → `GET /enrol/status` → cert), so gRPC's
streaming/multiplexing buys nothing. It is the system's *one* unauthenticated,
pre-auth, security-critical edge → the smallest, most auditable parser wins (a
`net/http` handler over a JSON body, not HTTP/2 + protobuf). The **client runs in
the early-boot initramfs** (the production UKI's empty-NV branch for now; a
provisioning UKI later) → keep it stdlib-only and reproducible like `fleet-hostapi`
(no `grpc-go`/protobuf vendoring in the minimal environment). And it
must reverse-proxy through the existing serving nginx `/enrol/` location — a one-line
`proxy_pass` for HTTP/1.1 vs `grpc_pass` + end-to-end HTTP/2 on the pre-auth edge.
CSR/cert ride as base64/PEM in JSON. **Boring is the feature on the bootstrap path.**
- **Admin approve/deny routes → mTLS (HTTP/JSON or gRPC, builder's call).** These are
already the fleet-manager's authenticated plane.
- **Steady-state agent ↔ manager (telemetry + reconciliation) → gRPC over mTLS,
host-dialed.** Streaming fits here, and it stays compatible with "no inbound port on
hosts" because the **host dials out** and holds one long-lived mTLS stream (health
up / desired-state down). OPEN FORK (a fleet-manager build decision, not enrolment):
bespoke gRPC vs reuse **OTLP-gRPC for metrics + Wazuh for events** + a thin gRPC
control stream — the latter gets VictoriaMetrics/Grafana/Wazuh for free. `Connect`
(connectrpc.com) is an option if one protobuf IDL serving gRPC + JSON + browser is
wanted across the authenticated surface.
- **host-api stays HTTP/JSON — locked**, because its consumer is the Chromium kiosk
(a browser, which cannot speak native gRPC).
## Where it runs: enrolment vehicle (build-flag, two modes)
Empty-NV and normal boot are **mutually exclusive** — a host with no cert cannot do
802.1X → MACsec → NVMe-root at all, so there is no normal hot-path sequence to
"protect" in the empty-NV case. The choice of vehicle is therefore *not* about
invariant #2 timing; it is about **whether enrolment capability ships in the
production artifact at all**. Two modes behind one build flag (`enrol_mode`, modelled
on the existing `disable_dot1x` / `boot_splash` cmdline toggles):
- **Inline single-UKI — CHOSEN FOR NOW (decided 2026-06-28).** The production UKI
auto-detects empty-NV and branches into enrolment in early boot. Picked because it
needs **no separate provisioning boot path** — everything stays purely
network-booted (no USB / no provisioning media to manage). The empty-NV gate is the
safety: NV-write at `0x01800001` needs TPM owner auth, so empty-NV is hard to
coerce. Accepted trade-offs: production carries the enrolment + pre-auth broker code
permanently, and there is no PCR-11 discrimination. The empty-NV branch is mutually
exclusive with the secure path, so it does not endanger the load-bearing hook
ordering. It needs a **pre-auth DHCP on the physical NIC** to reach the broker
through the pre-auth permit-set (the same pre-auth network the firmware already used
to fetch the UKI, like the `rd.dot1x=0` test path); on success it writes NV and
**reboots into the normal secure path** (convergence = reboot).
- **Provisioning-UKI variant — PLANNED LATER UPGRADE.** `enrol_mode=true` conditionally
`install()`s the enrolment scripts into the dracut module *and* bakes an immutable
`rd.fleet.enrol=1` cmdline → a **separately signed UKI from the same source tree**.
Production then lacks the code entirely (smaller initramfs, **physically cannot
enrol**), matching the #10 Secure-Boot-enrolment decision and enabling PCR-11
discrimination. Note the later separate UKI can still be **PXE/network-served**
(dnsmasq `provisioning` tag / unknown-MAC) — it does **not** require USB media; the
reason to defer is simply the extra boot-path + CI sign/artifact, not a media
dependency. Revisit once the broker is built and the secure path is verified.
Either way, the production *boot* path only ever **reads** an already-present cert;
the human-gated, indefinitely-waiting enrolment lifecycle never interleaves with the
unattended, watchdog-tuned normal boot.
| | Inline (one UKI) | Provisioning variant (build-flag) |
|---|---|---|
| Ops | simplest — one artifact | +1 CI sign/artifact (same tree) |
| Prod attack surface | carries enrolment + pre-auth broker code | none — code absent from prod |
| "Prod can't enrol" guarantee | empty-NV gate (TPM owner auth) | structural (code/cmdline absent) |
| PCR-11 discrimination | no | yes |
| Consistency with #10 | diverges | matches |
## Pre-auth network placement
@@ -249,21 +312,40 @@ exists. Enrolment is where it comes into being.
1. **Built / proven:** TPM key→CSR→sign→NV (manual, `tpm-provision.sh`);
`tpm-selftest.sh` capability check; step-ca + EAP-TLS bench.
2. **v1 (next build):** the broker state machine (request/status/pending/approve/
deny) + BIP39 fingerprint + Plymouth display + the `/enrol/` nginx permit +
the provisioning-mode trigger. **Fingerprint-only assurance.**
3. **Folds into fleet-manager:** the admin routes become the mgmt service's
2. **Broker — BUILT (2026-06-28): [`serving/enrol-broker/`](../serving/enrol-broker/).**
Go, stdlib-only. Public unauth listener (`POST /request`, `GET /status/{id}`)
behind the serving nginx `/enrol/`; admin **mTLS web UI** (`/`, `/pending`,
`/approve|deny/{id}`) on the mgmt network; wordlist fingerprint = `H(CSR-pubkey ‖
server-nonce)`; time-boxed in-memory store + per-IP rate limit; signs via
`step ca token`+`step ca sign` (broker holds the provisioner password) with a
dev `mock` signer. **Fingerprint-only assurance.** Admin client CA is a
**dedicated** CA (`admin-pki.sh`), separate from the fleet CA, so host certs
can't authenticate to the admin plane. End-to-end smoke-tested (CSR → mTLS
approve → cert; pubkey match). Ships in `serving/dot1x-bench/docker-compose.yml`.
3. **Host side (next build):** the **inline empty-NV trigger** — dracut early-boot
branch that generates key/CSR, POSTs to the broker, shows the fingerprint on
Plymouth, polls, writes NV, reboots; plus `tpm2-openssl` in the image.
4. **Folds into fleet-manager:** the admin routes become the mgmt service's
authenticated plane; approval writes the shared inventory record.
4. **Attestation upgrade:** ACME `device-attest-01` at the approve step.
5. **Provisioning-UKI variant** (`enrol_mode`) — move enrolment out of the production
artifact once the broker + secure path are proven (optional, network-served).
6. **Attestation upgrade:** ACME `device-attest-01` at the approve step.
## Open questions for build time
- **openssl TPM2 provider in the provisioning image** — `tpm2-tss` ≠ the provider;
- **openssl TPM2 provider in the enrolling image** (the production image, inline) —
`tpm2-tss` ≠ the provider;
`tpm-selftest.sh` flags whether `tpm2-openssl` must be added (also affects the
wpa_supplicant key-ref form: provider `handle:` vs tpm2-pkcs11 URI).
- **Pending store:** in-memory (simplest, lost on broker restart) vs a tiny
persistent store. Time-boxed pending makes in-memory acceptable for v1.
- **Provisioning UKI vehicle** — shared with #10 but not yet built; sequencing.
- **Enrolment-vehicle** — inline single-UKI is **chosen for now** (network-boot only,
no USB); the provisioning-UKI variant is the planned later upgrade behind the
`enrol_mode` build flag (shares the #10 vehicle, not yet built) — see "Where it
runs". Revisit when the broker is built + the secure path verified.
- **Steady-state transport fork** (fleet-manager, not enrolment) — bespoke gRPC vs
OTLP-gRPC metrics + Wazuh events + a thin gRPC control stream. See
[fleet-management-design.md](fleet-management-design.md).
- **Fingerprint length** — exact bit count / word count (entropy vs readability).
- **Open/test path identity** (`rd.dot1x=0`): MAC/fleet-name is a weak stand-in;
gate real enrolment to the secure path, or accept lab-only there.