diskless-workstation/serving/fleet-manager/README.md

# fleet-manager — server-side fleet control plane

The management service for the diskless kiosk fleet, and the backend for the
admin SPA. It is the central place to **monitor**, **authorise**, and
**lifecycle-control** enrolled hosts. See
[`docs/fleet-management-design.md`](../../docs/fleet-management-design.md) and
[`docs/tpm-enrolment-design.md`](../../docs/tpm-enrolment-design.md) §5.

Unlike `fleet-hostapi` / `fleet-enrol` (baked into the reproducible verity image,
stdlib-only), this runs **server-side** as its own container on the management
network, so external deps are fine (`grpc-go`, `influxdb-client-go`).

## Two mTLS planes (distinct trust anchors)

| Plane | Transport | Verifies | Who | Purpose |
|---|---|---|---|---|
| **admin** | HTTPS + mTLS | **dedicated admin CA** (`admin-pki.sh`) | admin browsers | JSON API + SPA (monitor / authorise / lifecycle) |
| **agent** | gRPC + mTLS | **fleet CA** (host TPM cert) | enrolled hosts | bidi stream: health/metrics up, desired-state down |

The split is load-bearing: the agent plane uses the host's **reused TPM identity
cert** (the same one used for 802.1X EAP-TLS and from which the NVMe `hostnqn`
derives — "one identity, three stages"), while the admin plane uses a **separate**
CA so an enrolled host can never reach approve/deny/lifecycle. See the identity
discussion in the enrolment design §5.

## Monitoring path (no host ever touches InfluxDB)

```
host agent ──gRPC bidi (mTLS, TPM cert)──▶ fleet-manager ──line protocol──▶ InfluxDB
           ◀──── desired-state (target ver / kiosk_url / reboot / drain) ──┘ (+ inventory health)
SPA ──GET /api/hosts/{serial}/metrics──▶ fleet-manager ──Flux──▶ InfluxDB ──JSON──▶ SPA renders charts
```

Hosts hold **no InfluxDB credential** and never reach it; the manager is the only
writer/reader, holding the Influx token server-side. Identity is taken from the
mTLS cert (anti-spoofing: a host can only write its own series). Push-only, no
inbound host port. Graphs are rendered **in the SPA**, not Grafana.

## Inventory record

Keyed by TPM **cert serial** (= hostnqn key). Born at enrolment approval, filled
by the agent stream and admin action:

```
Host (cert_serial)
  ├─ identity : cert_serial, hostnqn, mac, ek_hash, starship_name
  ├─ binding  : asset_tag, switch_port, location        (recorded at approval)
  ├─ enrolled : approver, at, fingerprint
  ├─ version  : image_version, rootfs_digest, kernel     (agent-reported)
  ├─ health   : load, cpu, temp, tmpfs%, nvme_root, auth, selinux, chrony, kiosk  (latest snapshot; series in InfluxDB)
  ├─ usage    : kiosk_url, session_uptime, chromium_crashes, display
  └─ desired  : target_version, kiosk_url, reboot_at, drain   (SERVER-AUTHORITATIVE)
```

Persisted as a single JSON file (atomic writes) on a mounted volume — stdlib-only,
inspectable, reproducible; a DB swap is a drop-in behind the `Store` interface.

## API (admin plane)

| Route | Status |
|---|---|
| `GET /healthz` | ✅ ckpt 1 |
| `GET /api/hosts` | ✅ ckpt 1 — fleet overview |
| `GET /api/hosts/{serial}` | ✅ ckpt 1 — host detail |
| `PUT /api/hosts/{serial}/desired` | ✅ ckpt 1 — set desired-state (audited to admin CN) |
| `GET /api/enrol/pending` | ✅ ckpt 3 — proxies the broker queue |
| `POST /api/enrol/{id}/approve` | ✅ ckpt 3 — signs via broker + **births inventory** (identity/binding/enrolled) |
| `POST /api/enrol/{id}/deny` | ✅ ckpt 3 |
| `GET /api/hosts/{serial}/metrics?field=&range=` | ✅ ckpt 4 — Flux time-series (field whitelist) |

The authorise pillar drives the enrol-broker's admin plane over mTLS (the manager
is a service client, gated by the dedicated admin CA) asking for JSON, so an
approve returns the decided enrolment; the manager records the **human** approver
(its own admin mTLS CN) and births the inventory record. The broker stays the
enrolment state machine + step-ca signer (v1 "alongside" fold-in).

**Agent plane** — gRPC `FleetAgent.Session` bidi over mTLS (fleet CA): ✅ ckpt 2.
`Hello`/`HealthSnapshot`/`CommandResult` up; `DesiredState`/`Command` down.
Identity is the peer cert serial (never self-asserted); an unenrolled serial is
refused. Desired-state is pushed live on connect + on change, and is durable in
inventory for offline delivery. Metrics writing is behind a `MetricsSink`
(no-op now; InfluxDB in ckpt 4).

## Config (env)

| Var | Default | Notes |
|---|---|---|
| `FLEET_MGR_ADMIN_ADDR` | `127.0.0.1:9500` | admin listener |
| `FLEET_MGR_ADMIN_CERT` / `_KEY` | — | admin server cert/key (mTLS) |
| `FLEET_MGR_ADMIN_CLIENT_CA` | — | **dedicated admin CA** (NOT the fleet CA) |
| `FLEET_MGR_ADMIN_INSECURE` | `false` | plaintext, no client-cert — **dev only** |
| `FLEET_MGR_AGENT_ADDR` | `127.0.0.1:9600` | gRPC agent listener |
| `FLEET_MGR_AGENT_CERT` / `_KEY` | — | agent-plane server cert/key |
| `FLEET_MGR_AGENT_CLIENT_CA` | — | **fleet CA** (verifies host TPM certs). Agent plane starts only when all three are set. |
| `FLEET_MGR_BROKER_URL` | — | enrol-broker admin base URL (e.g. `https://127.0.0.1:9443`). Authorise pillar enabled only when set. |
| `FLEET_MGR_BROKER_CLIENT_CERT` / `_KEY` | — | manager's client cert for the broker admin mTLS (admin CA) |
| `FLEET_MGR_BROKER_CA` | — | CA that signs the broker's admin server cert |
| `FLEET_MGR_INFLUX_URL` | — | InfluxDB URL (private mgmt net). Metrics enabled only when URL + token set. |
| `FLEET_MGR_INFLUX_TOKEN` | — | InfluxDB token — **server-side only**, never on hosts |
| `FLEET_MGR_INFLUX_ORG` | `fleet` | InfluxDB org |
| `FLEET_MGR_INFLUX_BUCKET` | `fleet_health` | InfluxDB bucket |
| `FLEET_MGR_INVENTORY` | `/var/lib/fleet-manager/inventory.json` | inventory file (mount a volume) |
| `FLEET_MGR_SEED` | `false` | inject demo hosts so the SPA has data — dev only |
| `HOSTNQN_BASE` | `nqn.2026-06.org.example.fleet:host-` | hostnqn prefix (match the broker) |

InfluxDB env lands in checkpoint 4.

## Dev run

```bash
FLEET_MGR_ADMIN_INSECURE=true FLEET_MGR_SEED=true \
  FLEET_MGR_ADMIN_ADDR=127.0.0.1:9500 FLEET_MGR_INVENTORY=/tmp/inv.json \
  go run .
curl -s localhost:9500/api/hosts | jq '.hosts[] | {name:.identity.starship_name, run:.version.image_version, want:.desired.target_version, online}'
```

## Build status

- **Checkpoint 1 (done):** Go skeleton, JSON-file inventory `Store`, admin JSON API
  (monitor + lifecycle), admin mTLS wiring, dev seed. Stdlib-only.
- **Checkpoint 2 (done):** gRPC agent plane — proto contract, mTLS (fleet CA)
  server, stream registry, identity-from-cert, snapshot→inventory, live
  desired-state push, `MetricsSink` seam. Full mTLS integration test.
- **Checkpoint 3 (done):** authorise pillar — the enrol-broker gained an
  `Accept`-gated JSON approve response (browsers/`verify-serving.sh` unchanged);
  the manager drives it over mTLS and births the inventory record on approve.
  httptest-mock-broker test.
- **Checkpoint 4 (done):** InfluxDB metrics gateway — snapshots written as points
  tagged with the authenticated identity (`MetricsSink`→`influxSink`), and a
  read API (`/api/hosts/{serial}/metrics`) that runs a Flux query with a **field
  whitelist** + hex-serial guard (no caller input reaches the query raw). Unit
  tests for point construction, query building, and unsafe-input rejection.
- Next: themed SPA (ckpt 5) → Docker/compose + PKI + docs (ckpt 6).