Add Host API and fix kiosk performance

This commit is contained in:
Audiobuzz
2026-06-28 03:14:25 +00:00
parent b5163f958e
commit ad6b7e3366
12 changed files with 747 additions and 37 deletions
+105 -6
View File
@@ -685,12 +685,26 @@ must be built around **never restarting the process that holds live connections.
the same `/dev/disk/by-id/nvme-uuid.<uuid>` host symlink.
- **Bootstrap unchanged:** `docker compose up` with one version still works —
the `/artifacts` image volume is seeded into `./releases/${IMAGE_VERSION}`.
- **BENCH-VERIFY:** (a) `load` a 2nd version while a host is connected to the
1st → 1st's connection survives (no AER/reset on the running host); (b)
`unload` drain-gate actually blocks while CONNS>0; (c) serving-host reboot
reconciles all staged versions; (d) per-version UUID build is reproducible
(two builds, same version → identical `data_ns_uuid`/`hash_ns_uuid` +
`root_hash`). See `serving/spdk/README.md`.
- **PARTIALLY BENCH-VERIFIED 2026-06-28:** long-lived target boots a host end
to end — UKI served over HTTP → host boots → NVMe/TCP root connects to the
persistent `nvmf_tgt` (single version `spdk-permanence-02ffd7d`). Surfaced +
fixed several real bugs along the way: bootstrap seed must COPY not symlink
(a symlink into the version-pinned `/artifacts` re-pointed a stale release at
new data); `spdk-release` parses RPC JSON via **python3** (SPDK image has no
jq); legacy `verity.json` (no per-version UUIDs) falls back to the fixed pair;
mislabeled-dir guard (dir name vs verity.json `image_version`, case-insensitive);
dropped the redundant discovery-listener re-add (daemon logged "Listener
already exists"). **CI lowercases `image_version`** (gitea-main-ci vars step) —
a mixed-case branch (`SPDK-permanence`) had baked a capital
`fleet-SPDK-…​.efi` the lowercase boot URL couldn't fetch (iPXE "no such
file"); booted via a local retag workaround pending a lowercase rebuild.
- **STILL BENCH-VERIFY:** (a) `load` a 2nd version while a host is connected to
the 1st → 1st's connection survives (no AER/reset on the running host); (b)
`unload`/auto-reap drain-gate blocks while CONNS>0 and reaps after grace on a
LIVE host; (c) serving-host reboot reconciles all staged versions; (d)
per-version UUID build reproducible (two builds, same version → identical
`data_ns_uuid`/`hash_ns_uuid` + `root_hash`); (e) the full fix on a clean
LOWERCASE CI rebuild (no retag). See `serving/spdk/README.md`.
- **Capacity:** each concurrently-served version costs target memory/hugepages +
file-backed bdev overhead — plan for N live versions (current + previous +
canary) during long rollouts (raise `HUGEPAGES`).
@@ -864,6 +878,10 @@ smoke test.
`docs/stig-deviations.md` #6 to reflect encrypted offload.
(RELP+TLS via `omrelp`/`imrelp` is the alternative if guaranteed delivery is
wanted alongside encryption.)
- **fleet-hostapi: scope CORS to the kiosk origin.** The host-info agent ships
with `FLEET_HOSTAPI_ALLOW_ORIGIN=*` (lab). For prod set it to the exact kiosk
origin (default `http://boot.fleet.lab`) via the unit's commented
`Environment=` line (or a drop-in). See #12.
## [ ] 9. (OPTIONAL / nice-to-have) Convert provisioning to Ansible where it pays off
@@ -1024,3 +1042,84 @@ non-destructive TPM capability check (device, key persist, NV roundtrip, openssl
TPM2 provider, CSR signed by the TPM key) to de-risk the TPM half before any of
this. Expected gap: the image installs `tpm2-tss` but no openssl TPM2 **provider**
(`tpm2-openssl`) — add it to the build once the self-test confirms.
## [~] 12. Workstation host-info API (`fleet-hostapi`) for the kiosk
**IMPLEMENTED 2026-06-28.** A small **Go, read-only, loopback-only** HTTP agent on
the workstation that reports host/OS/hardware facts to the kiosk SPA, per the
[HOST-API.md](../HOST-API.md) contract (`GET http://localhost:9099/v1/host`).
Source: `files/hostapi/` (stdlib-only, reproducible static build). Built + enabled
by provisioner **2b-hostapi**; toolchain removed in 2d; binary fapolicyd-trusted
in 2c-STIG-postfix (register #10). Chromium policy grants Private Network Access
scoped to the kiosk origin (`files/kiosk/chromium-kiosk-policy.json`).
- **Per-host safety:** host-specific fields (kernel, hostname, IP, MAC, board,
uptime) are read at RUNTIME; only image-generic facts (image version +
weston/chromium versions) are baked, to `/etc/fleet-hostinfo.json` (invariant
#1 preserved). Deliberately does NOT expose DMI serial (root-only/per-host) or
any TPM/cert/identity material.
- **Hardening shipped:** loopback bind enforced in code; `DynamicUser`, no caps,
`ProtectSystem=strict`, `MemoryDenyWriteExecute`, `SystemCallFilter=@system-service`,
`IPAddressDeny=any` (loopback only). Runs unprivileged, reads world-readable
files only.
**Lab posture LEFT AS-IS (decided 2026-06-28) — revisit for prod:**
- **CORS `Allow-Origin=*`** → scope to the kiosk origin (`http://boot.fleet.lab`).
Consolidated into #8.
- **SELinux:** runs in the default service domain (no custom policy module).
**BENCH-VERIFY** no AVC denials for `fleet-hostapi` on an enforcing booted host
(Wazuh/`fleet_audit` rule 100120 will surface any). If denials appear, decide:
write a confined SELinux policy module vs adjust the sandbox.
- **No auth** (justified: loopback + read-only + single-user appliance). Option if
the threat model tightens: shared token in `kiosk.env` + header check.
- **Info sensitivity:** exposes hostname/IP/MAC/board/versions to the kiosk. Drop
MAC/IP if that's later deemed too much.
- **Reproducibility:** the `golang` toolchain version is a reproducibility-sensitive
input (a package bump changes the binary → verity hash); covered by the
build-twice gate. Pin the go version only if drift becomes a problem.
**BENCH-VERIFY (functional):** (a) service starts under enforcing SELinux +
fapolicyd (not blocked); (b) the kiosk SPA's System page fetches `/v1/host`
successfully through the PNA allowance (preflight + GET); (c) facts render
correctly on real hardware (board/CPU/GPU from DMI/DRM, not the virtio build VM).
## [~] 13. Kiosk GPU compositing + Chromium flag ownership
**GPU FIX APPLIED 2026-06-28.** The 820 (Intel HD620/i915) ran Chromium at ~150%
CPU with periodic crashes: `--disable-gpu` in our `kiosk-session` forced software
raster/compositing. Root cause confirmed from the live cmdline. Fixes in
`files/kiosk/kiosk-session` + `diskless.pkr.hcl`:
- **Runtime GPU probe** (fleet-generic image boots real HW *and* virtio VMs): a
real KMS driver (`i915`/`amdgpu`/…) + `/dev/dri/renderD128` → GPU path
(`--ignore-gpu-blocklist --enable-gpu-rasterization --enable-zero-copy`, weston
GL renderer); else software (`--disable-gpu` + `weston --use-pixman`) so the VMs
still render. No per-host state baked.
- **kiosk user added to `render`** (renderD128 isn't logind-seat-ACL'd on EL9;
`video` only covers card0/KMS for weston).
- **Merged the distro wrapper's `--disable-features` into ours** — Chromium takes
the LAST `--disable-features` (not the union), so ours was silently dropping the
wrapper's Lens/Gemini/Compose disables.
**BENCH-VERIFY (on the 820):** GPU process alive (`chrome://gpu`), Compositor/
ThreadPool threads near-idle + CPU drop, `gt_act_freq_mhz` rises under load, no
SELinux AVC for `dri_device_t` from the kiosk domain (Wazuh `fleet_audit` 100120
will surface any — if present, add a scoped SELinux allow or confirm the kiosk
user's SELinux mapping is unconfined). Confirm VMs still render (software path).
**DECISION OPEN — Chromium flag ownership (security).** The distro chromium
wrapper PREPENDS vendor defaults we don't control, including capability-broadening
ones a locked kiosk arguably shouldn't carry: `--enable-extensions`,
`--enable-user-scripts`, `--enable-plugins`, `--enable-printing`,
`--auto-ssl-client-auth`, `--enable-sync`. Today the managed policy neutralises
the security-relevant ones (sync off via `SyncDisabled`; sign-in/metrics/etc.),
but extensions/user-scripts/auto-ssl-client-auth are NOT policy-locked. Option to
**own the full flag set**: bypass/neutralise the wrapper (invoke the real binary,
or empty its default-flags source) and specify everything ourselves incl. the
VA-API `--enable-features` (now known:
`AllowQt,WaylandLinuxDrmSyncobj,WaylandPerSurfaceScale,WaylandUiScale,AcceleratedVideoDecodeLinuxGL,AcceleratedVideoDecodeLinuxZeroCopyGL,AcceleratedVideoEncoder`).
Tradeoff: full determinism + drop unwanted capabilities, vs we then track Chromium
feature renames ourselves instead of inheriting the distro's. Also fully dedupes
`--disable-features`/`--enable-sync`. Needs locating the wrapper's flag source on
the box first (so we don't lose VA-API or break its env setup). Pairs with policy
adds: `ExtensionInstallBlocklist=["*"]`, disable user scripts, restrict
AutoSelectCertificateForUrls.
+7 -4
View File
@@ -143,10 +143,13 @@ hotkey disabled (e.g. a physically exposed terminal), re-select both rule IDs.
- **SELinux enforcing** (`selinux --enforcing`; labels preserved into the erofs).
- **fapolicyd** kept **enabled** (NOT waived) — it's the real exec-allowlist guard
for the writable RAM overlay, which dm-verity does **not** cover. The only
adjustment: the one non-RPM executable, `/usr/local/bin/kiosk-session`, is added
to fapolicyd's trust DB at build (provisioner 2c-STIG-postfix) so the kiosk runs;
everything else stays RPM-trust-only. (fapolicyd blocking that binary was the
original kiosk-won't-start cause.)
adjustment: the few non-RPM executables we ship are added to fapolicyd's trust DB
at build (provisioner 2c-STIG-postfix) so they run; everything else stays
RPM-trust-only. Trusted custom binaries: `/usr/local/bin/kiosk-session`,
`/usr/local/bin/fleet-sb-enroll`, `/usr/local/bin/fleet-hostapi` (the loopback
host-info agent, built from `files/hostapi`). Each is built into the immutable,
dm-verity-protected root, so trusting it doesn't weaken the overlay exec guard.
(fapolicyd blocking `kiosk-session` was the original kiosk-won't-start cause.)
- **auditd** enabled; STIG audit rules applied; logs locally.
- **USBGuard** enabled + enforcing (`block` target) with a HID/hub allowlist and
`AuditBackend=LinuxAudit` (see #12) — blocks USB mass storage / unknown device