9.5 KiB
TODO — diskless fleet boot system
Persistent backlog, kept in sync with the working session task tracker. Status:
[ ] pending · [~] in progress · [x] done.
[ ] 1. Port the dracut module off network-legacy → network-manager
Durable replacement for the current dhcp-client stopgap. dhclient is
deprecated in RHEL 9 and will eventually leave the repos; the build only works
today because the provisioner installs dhcp-client so network-legacy passes
dracut's check().
- Rewrite both network bring-up paths in
../files/dracut/95dot1x-macsec/dot1x-macsec-up.sh:
the secure path's
ifup macsec0and therd.dot1x=0bypass path'sifup $NIC— usenmcli/NetworkManager-in-initramfs (orip=cmdline + the nm module) instead. - Change
depends()network-legacy→network-managerin ../files/dracut/95dot1x-macsec/module-setup.sh. - Update
add_dracutmodulesin ../files/etc/dracut.conf.d/90-diskless.conf. - Drop
dhcp-clientfrom the provisionerdnf installin ../diskless.pkr.hcl. - Re-verify the
lsinitrdassertion. - Gate: full boot-chain smoke test (swtpm + hostapd + FreeRADIUS) before merge, per CLAUDE.md.
[ ] 2. Apply RHEL9 STIG hardening to the built image
OpenSCAP + SCAP Security Guide (xccdf_org.ssgproject.content_profile_stig), run
as a Packer provisioner after dnf install and before the dracut/initramfs build
and cleanup.
- Tailoring (
autotailor): unselect rules this architecture can't satisfy — separate-partition rules (single erofs root + tmpfs overlay; satisfy intent withnodev,nosuid,noexectmpfs mounts), GRUB/kernel-cmdline rules (no GRUB), and AIDE/file-integrity rules (dm-verity is the documented compensating control). - Kernel params → UKI cmdline in
../scripts/extract-artifacts.sh before
signing:
audit=1 audit_backlog_limit=8192 slub_debug=P page_poison=1 vsyscall=none(+fips=1if FIPS). - Reproducibility guard: strip non-deterministic remediation output (SSH host keys → regen at boot into overlay, AIDE db, timestamps, fresh UUIDs) so the verity hash stays stable.
- Audit logging:
audisp-remote/rsyslog to a central collector addressed by DNS name (host-generic). - Optional FIPS:
fips=1cmdline +01fipsdracut module + FIPS crypto policy. - CI: run
oscap eval(no--remediate) to publish an ARF+HTML STIG report as a build artifact; optionally hard-fail below a score threshold. - Decisions needed before implementing: FIPS on/off · remote-audit collector DNS name (or ephemeral/none for lab) · CI fail-on-score threshold vs report-only.
- Note: orthogonal to the
rd.dot1x=0test toggle, but a full STIG production posture wants 802.1X re-enabled.
[ ] 3. Roll the errata into design doc v3
Once the boot path is validated and item 1 lands, fold
design-v2-errata.md into the source design document and
re-export a v3 PDF (key change: rootovl → systemd.volatile=overlay in §3
step 8 and §5). Retire/reset the errata afterward. Needs the design-doc source
(not in this repo).
[ ] 4. Kiosk mode — autologin + fullscreen Chromium loading a URL
Make the diskless workstation boot straight into a locked-down kiosk: a fullscreen Chromium showing a configured site (initial target: Reddit). All baked at build time into the root tree (immutable erofs/verity root; runtime is the tmpfs overlay), fleet-generic (no per-host state).
- Graphical stack (keep it minimal): add a Packer provisioner installing a
lightweight kiosk compositor + browser. Preferred:
cage(Wayland single-app kiosk compositor) + Mesa (software GL fallback for VMs/no-GPU); X11 + a minimal WM (matchbox/openbox) as the alternative. Chromium on Rocky 9 is in EPEL — enabling EPEL adds a repo/supply-chain surface (decision below). - Autologin + session: dedicated unprivileged
kioskuser; launch the session via akiosk.servicerunningcage -s -- chromium --kiosk …(orgreetd+ cage). Stage units/configs underfiles/like the other provisioned config. - Chromium flags:
--kiosk --noerrdialogs --disable-infobars --incognito --no-first-run --disable-session-crashed-bubble <URL>. Writable profile/cache land on thesystemd.volatile=overlaytmpfs (RAM-backed, ephemeral) — fine, but watch RAM sizing; pin--user-data-dir/--disk-cache-dirto a tmpfs path if needed. - Kiosk URL = Packer var (e.g.
kiosk_url, default the target site), baked into the launch command; keep it host-generic (per-env via DNS if ever needed). - Verify in boot test: graphical session comes up on real/VM display,
Chromium fullscreen loads the URL over the booted network (works with the
current
rd.dot1x=0test path / open network). - Decisions: Wayland(cage) vs X11; accept EPEL for Chromium (vs another source); confirm the target URL.
- ⚠ Conflict with item 2 (STIG): passwordless autologin and a desktop browser are STIG findings and enlarge the attack surface/image. Reconcile — e.g. a tightly-constrained kiosk session + documented deviation — when hardening lands.
[~] 5. Deploy & validate the end-to-end boot path (open-network test)
In progress. Stand up serving/ (dnsmasq + nginx + SPDK) per
serving/README.md, rebuild the UKI with
nvmet_traddr=nvme.fleet.lab (now a Packer var — older UKIs have the literal
NVMET_TRADDR placeholder and won't connect), then boot a UEFI HTTP-boot client
(Secure Boot OFF, unsigned UKI; no TPM needed in rd.dot1x=0 mode) on the
boot LAN.
- Pass criteria: UKI fetched over HTTP → initramfs → DHCP on NIC → NVMe/TCP
namespace connected → dm-verity root mapped →
systemd.volatile=overlaywritable → multi-user. - This validation gates item #1 (network-manager port) and item #3 (errata → design v3).
[ ] 6. Rolling fleet updates (multi-version serve → cohort reboots)
Image-based, reboot-to-update across many live workstations. Hosts are immutable (read-only erofs/verity root over NVMe/TCP, ephemeral overlay), so updates are atomic A/B reboots, never in-place. The released OCI artifact + digest-pinned manifest + roothash coupling already give the atomic unit and instant rollback; this item builds the rollout machinery around it.
- Serve versions side-by-side (must not overwrite a live rootfs): extend
serving/so SPDK/nvmet exposes each release as its own subsystem/NQN (nqn…:rootfs-<ver>) and nginx serves multiplefleet-<ver>.efi(keep current + previous). Make the UKI's NQN version-specific so each signed UKI connects only its matching namespace (constant discovery DNS name). - Version selection at boot: dnsmasq cohort tagging (MAC / vendor class / assigned tag) hands each cohort a different option-67 UKI URL. Moving a cohort's URL = assigning the release.
- Control plane + in-image reboot agent: a baked-in systemd timer polls a "desired version" endpoint and reboots when idle / in a maintenance window (or out-of-band via Redfish/BMC). Stagger + jitter reboots to avoid UKI-fetch / NVMe-connect storms.
- Staged rollout: canary → waves (10/50/100%) with health gates (multi-user reached, NVMe connected, dm-verity mapped, kiosk URL rendered, prod: MACsec/802.1X up); auto-halt on threshold.
- Rollback: repoint cohort URL to previous UKI + reboot (current+previous stay served); security rollback via dbx-revoke / TPM monotonic counter (design §6).
- Drain & retire an old version only when SPDK/nvmet reports zero connected controllers on it.
- Availability: run NVMe/TCP target replicas behind distinct discovery entries (design §7) — never serve the whole fleet from one unreplicated node, especially mid-rollout (N versions + reboot bursts).
- Note: kiosk content changes are server-side (the website) and need no fleet update — only OS/UKI/rootfs changes require a reboot.
[ ] 7. Harden / evaluate the NVMe-oF target (serving-node resilience)
There is no production-grade memory-safe NVMe-oF target — the Rust NVMe work
is all driver/initiator-side (vroom, RedLeaf, Rust-for-Linux); the only
production targets are C (Linux kernel nvmet, SPDK). So this is serving-node
hardening, not a language swap. dm-verity already removes the target from the
clients' integrity TCB (verify-on-read rejects tampered blocks), so the risk is
RCE/DoS on the node that holds the fleet's disk, not client integrity. The
export is read-only + single-namespace = inherently small attack surface.
- Evaluate kernel
nvmetvs SPDK for the target: kernelnvmetkeeps the data path in the hardened/fuzzed kernel with a tiny userspace footprint (arguably a better attack-surface story); SPDK is a userspace C daemon (current choice, needs hugepages + caps). Decide per environment. - Confine the target (whichever): unprivileged, drop caps, seccomp + SELinux/AppArmor profile, read-only container, minimal mounts; minimal immutable serving image, digest-pinned.
- Network-isolate :4420 so only the boot VLAN reaches the target (compose with 802.1X/MACsec segmentation in prod).
- Replicas behind distinct discovery entries (availability is the real risk — it's the fleet's disk). Ties into #6 (rolling updates) availability.
- Revisit a memory-safe target only if/when an off-the-shelf one matures
(e.g. a future Rust-for-Linux
nvmet).