# Design v2 — implementation errata (RHEL/Rocky 9)

Records where the as-built implementation diverges from
`diskless-fleet-design-v2.pdf`. The PDF remains authoritative for the
architecture; this note captures deltas discovered bringing the build up on
**Rocky 9.8** until they are folded into a consolidated **v3**. Last updated
2026-06-15.

The architecture is unchanged: 802.1X EAP-TLS with a TPM-held key, MACsec on the
access link, dm-verity-protected erofs root over NVMe/TCP, signed UKI, and the
FROM-scratch OCI release artifact. Only the items below differ.

## 1. Design-level change — writable root overlay

**PDF says (§3 step 8, §5):** runtime state lives on a tmpfs overlay enabled by
the kernel cmdline option **`rootovl`**.

**As built:** the cmdline uses **`systemd.volatile=overlay`** instead.

**Why:** `rootovl`/`overlayfs` is a Debian/third-party dracut module; RHEL/Rocky
dracut does not ship it (`dracut: module 'overlayfs' cannot be found`).
`systemd.volatile=overlay` is the RHEL-native, systemd-supported mechanism and
is the recommended overlay for dm-verity-protected read-only roots. Behaviour is
equivalent: the verity erofs root mounts read-only and a tmpfs overlay carries
runtime writes; per-host `machine-id` is still generated into the overlay.

**Caveat:** requires `systemd-volatile-root.service` in the initramfs (bundled by
recent dracut; `install_items` it if the root stays read-only at boot). The known
failure mode is NFS root — not applicable here (NVMe block root). Set in
[../scripts/extract-artifacts.sh](../scripts/extract-artifacts.sh); module list
in [../files/etc/dracut.conf.d/90-diskless.conf](../files/etc/dracut.conf.d/90-diskless.conf).

## 1b. Design-level change — dm-verity uses two namespaces, not in-file offset

**PDF says (§5):** the erofs data and the dm-verity hash tree share a single
NVMe/TCP namespace, appended in one file and split on the host with
`systemd.verity_root_options=hash-offset=`.

**As built:** data and hash are **two separate namespaces** — `rootfs.erofs`
(data) and `rootfs.hash` (hash tree) — each a file in the OCI artifact, served as
its own NVMe namespace with a distinct pinned UUID. The cmdline uses
`systemd.verity_root_data=` / `systemd.verity_root_hash=` by-id devices with no
offset.

**Why:** Rocky 9.8 `systemd-veritysetup` does not support `hash-offset=` — it
logs `Encountered unknown option 'hash-offset=…', ignoring` then `Failed to load
verity superblock: invalid argument`, so the single-namespace layout never maps
`/dev/mapper/root`. Two namespaces is the design's own documented fallback and
uses systemd's native verity (no custom activation code; keeps root-hash
signature support for production). Devices are referenced by stable
`/dev/disk/by-id/nvme-uuid.<uuid>` (not `/dev/nvmeNnM`) because controller
indices shift with the transient discovery controller.

**Touches:** [../scripts/extract-artifacts.sh](../scripts/extract-artifacts.sh)
(emits `rootfs.hash`, two UUIDs, offset-free cmdline),
[../scripts/build-oci.sh](../scripts/build-oci.sh) (artifact list + SHA256SUMS),
[../serving/spdk-setup.sh](../serving/spdk-setup.sh) (2nd bdev/namespace), and
CLAUDE.md invariant #9.

## 2. Implementation notes (not architectural)

These don't contradict the PDF (it doesn't specify them) but are worth recording.

- **Network backend.** The dracut hook uses the `network-manager` dracut module
  (ported 2026-06-17 from `network-legacy`/`ifup`/`dhclient`). NM hands its
  initramfs connections to the real-root NetworkManager, which **renews the DHCP
  lease** in the running system — network-legacy's one-shot lease was never
  renewed, so the address dropped at lease expiry and the NVMe/TCP root died.
  Managed/unmanaged is mode-aware: secure path → `macsec0` managed (NM DHCPs +
  renews it), physical `en*` unmanaged (wpa_supplicant owns EAPOL/MKA), enforced
  by the `nm-unmanaged.sh` cmdline hook writing a `/run/NetworkManager/conf.d`
  drop-in before NM starts; test path → boot NIC managed, hook is a no-op. No
  static unmanaged file (it couldn't be mode-aware). Awaiting boot-bench verify.
- **Build ISO.** Defaults to a version-pinned Rocky 9.x **minimal** ISO on the
  internal mirror (`iso_url` + `sha256` `iso_checksum`). `http/ks.cfg` installs
  only core+sshd+chrony from the ISO; the diskless tooling is pulled online by
  the provisioner, so the build VM needs outbound repo access.
- **Reference serving stack.** §7's SPDK + nginx + DHCP/DNS serving stack has a
  concrete docker-compose implementation in [../serving/](../serving/), incl.
  Docker image-volume mounting of the OCI artifact image.
- **STIG hardening.** DISA STIG (SSG) is applied with a diskless/UKI/kiosk
  tailoring; several rule families are deviated because the architecture makes
  them N/A (no partitions, no GRUB, AIDE→dm-verity) or by lab decision (FIPS off,
  firewall off, audit local-only). Full rationale + compensating controls:
  [stig-deviations.md](stig-deviations.md). STIG kernel params live in the UKI
  cmdline (the no-GRUB compensating control).
- Assorted build-time fixes (HCL `%%{` escaping, file-provisioner staging,
  dracut module naming) are documented in
  [../CLAUDE.md](../CLAUDE.md) → "RHEL 9.x build notes".

## 3. Temporary deviation — must be reverted before production

- **802.1X/MACsec test bypass.** `disable_dot1x=true` bakes `rd.dot1x=0` into the
  cmdline so the host skips authentication/MACsec and DHCPs the NIC directly, to
  validate the UKI/rootfs/NVMe-oF path on an open switch port. The auth stack
  stays in the image (dormant). This is **lab-only**; a production build must
  drop the flag (`DISABLE_DOT1X: "false"` in CI) to restore the secure boot
  chain described throughout the PDF.