docs: reflect chrony/uptime metrics, agent-cert renewer, and the chrony-access findings

Recent work added telemetry fields and a cert path not yet in the docs:
- files/agent/README.md: add uptime to the health list; document that chrony is
  read from /var/log/chrony/tracking.log (not the command socket) and needs BOTH
  the chrony supplementary group (DAC) and the fleet_agent_chronylog SELinux allow
  (TE), plus a reachable lab NTP (123/udp firewall); note the chrony group on the
  unit row.
- serving/fleet-manager/README.md: add uptime + chrony detail to the health list;
  note the cert-auto agent-cert-renewer sidecar + GetCertificate hot-reload
  (certreload.go) in the bring-up and checkpoint 7.
- docs/TODO.md #14: probe list gains uptime + chrony-from-tracking.log; new dated
  block records uptime/reboot-pending, the 4-layer chrony-access chase (all
  silent-failing; inventory.json is the isolator), and the agent-plane cert
  auto-renewal, all bench-verified 2026-07-10.
- docs/fleet-management-design.md: health record gains uptime + chrony
  source/stratum/offset detail.
This commit is contained in:
Audiobuzz
2026-07-10 22:45:00 +00:00
parent a46e5edf69
commit 3284a2713b
4 changed files with 62 additions and 15 deletions
+29 -2
View File
@@ -1412,8 +1412,9 @@ the design forks resolved:
the host **TPM identity** for mTLS (signing key at `0x81000100` via
`tpmkey.LoadSigner` — a read-only copy of the enrol `tpmkey`; cert from NV
`0x01800100`), opens the long-lived `Session` stream, sends `Hello` + periodic
`HealthSnapshot` (load/cpu/temp/mem/tmpfs/nvme/auth/selinux/chrony/kiosk probed
read-only from `/proc`+`/sys` + hostapi facts), and — **telemetry-only v1** —
`HealthSnapshot` (load/cpu/temp/mem/tmpfs/**uptime**/nvme/auth/selinux/kiosk from
`/proc`+`/sys` + hostapi facts, plus **chrony** clock health — state/source/
stratum/offset — read from `/var/log/chrony/tracking.log`), and — **telemetry-only v1** —
**receives `DesiredState`/`Command` but only LOGS them** (zero host mutation).
Static `CGO_ENABLED=0` binary built out-of-VM in CI (like enrol/hostapi), staged
by the `2b-agent` provisioner (`enable_agent`, default on), fapolicyd-trusted,
@@ -1464,6 +1465,32 @@ the design forks resolved:
register-host <serial> -name <starship>` (writes through the real store to
`FLEET_MGR_INVENTORY`). The manager caches inventory in memory — any edit needs
a `--force-recreate`. Also: the agent now logs the real gRPC status, not `EOF`.
**Metrics + cert auto-renewal added 2026-07-10 (`fleet-manager` branch):**
- **uptime + reboot-pending**: the agent sends `uptime_seconds`; the manager/UI
derive "reboot pending in X / rebooted" from boot_time (= last_seen − uptime) vs
`reboot_at`, all in the manager clock frame (no host↔manager clock sync needed).
Monitor metric graphs auto-refresh (10 s poll).
- **chrony clock health** (sync state + NTP source/stratum/offset). The agent reads
`/var/log/chrony/tracking.log` (image bakes `log tracking`), NOT chronyd's command
socket (owner-write-only unix socket; STIG `cmdport 0` closes the UDP port).
**Reaching that log from the hardened DynamicUser agent took a 4-layer chase, every
layer silent-failing** (memory `fleet-agent-hardened-probe-access`): (i) lab NTP
reachable — a `123/udp` rule on the serving-host firewall (host-networked ntp
container; same trap as audit `514`); (ii) DAC — `chrony` supplementary group on the
unit to traverse `/var/log/chrony` (0750); (iii) SELinux TE —
`files/stig/fleet-agent-chronylog.te` allow for `chronyd_var_log_t` (permissive
masks this); (iv) the MANAGER image must be ≥ the chrony commit — one with
uptime-but-not-chrony silently drops the fields (manager & workstation are SEPARATE
images/deploys). Isolator: read the manager `inventory.json` off the `fleet-data`
volume via a throwaway busybox (distroless manager, no shell). BENCH-VERIFIED on
`comet-beacon-54` 2026-07-10.
- **agent-plane server-cert auto-renewal**: the `agent-cert-renewer` compose sidecar
(profile `cert-auto`) issues + `step ca renew --daemon` renews the cert from a
co-located step-ca; the manager hot-reloads via `GetCertificate` (`certreload.go`),
no restart. Replaces the manual `server-pki.sh` + `docker exec … step` dance and
the 90-day re-mint. The server cert SANs MUST include `manager.fleet.lab` (the
fleet-DNS name the agent dials) — a missing SAN fails the handshake.
- **STILL TO BUILD:** (b) **per-host boot-version assignment** — `target_version` is
stored + pushed but does not change what a host *boots*; that needs dnsmasq
cohort steering + multi-version UKI serving (**#6**). Kiosk URL / reboot / drain
+6 -4
View File
@@ -125,8 +125,9 @@ Reported facts:
- **Health:** CPU/load, GPU act-freq, temps, **tmpfs overlay usage** (the
RAM-exhaustion → NVMe-keepalive risk — the single most important kiosk health
signal), NVMe-root link state, 802.1X/MACsec `Authorized`, SELinux enforcing,
chrony sync, `kiosk.service` active.
signal), host **uptime** (server derives reboot-pending vs `reboot_at`),
NVMe-root link state, 802.1X/MACsec `Authorized`, SELinux enforcing, chrony
clock health (sync state + NTP source/stratum/offset), `kiosk.service` active.
- **Version:** running image version (`/etc/fleet-image`), rootfs digest, kernel.
- **Usage:** kiosk URL, session uptime, **Chromium reload/crash count** (from the
`kiosk-session` restart loop — the exact signal behind the GPU-bug diagnosis),
@@ -186,8 +187,9 @@ UKI).
Host (by cert-serial / hostnqn)
├─ identity: cert serial, hostnqn, MAC, starship-name (dnsmasq)
├─ version: image_version, rootfs digest, kernel, desired_version
├─ health: cpu/load, gpu_freq, temps, tmpfs_used%, nvme_root, auth/macsec,
│ selinux, chrony, kiosk_active, last_seen
├─ health: cpu/load, gpu_freq, temps, tmpfs_used%, uptime, nvme_root,
│ auth/macsec, selinux, chrony (sync + source/stratum/offset),
│ kiosk_active, last_seen
├─ usage: kiosk_url, session_uptime, chromium_crashes, input_active, display
└─ desired: target_version, kiosk_url, reboot_at, drain (server-authoritative)
```