Update Documents to latest implementation

CLAUDE.md

@@ -19,6 +19,8 @@ files/etc/dracut.conf.d/90-diskless.conf  no-hostonly + drivers + modules
 files/etc/NetworkManager/                 NM unmanaged-device config
 scripts/extract-artifacts.sh              qcow2 -> kernel/initramfs/erofs+verity/cmdline
 scripts/build-oci.sh                      ukify + sbsign + roothash gate + OCI image
+serving/                                  Lab serving stack: dnsmasq + nginx + SPDK
+.gitea/workflows/                         Gitea CI: lint -> packer build -> push OCI
 docs/                                     Design PDF and supporting notes
 ```
 
@@ -30,8 +32,11 @@ packer validate .                         # always before committing template ch
 packer build -var image_version=X.Y.Z \
   -var nic_drivers="ice mlx5_core" \
   -var oci_image=<registry>/fleet-boot .  # add sb_key/sb_cert for signed builds
+                                          # add disable_dot1x=true for an
+                                          # open-port test UKI (rd.dot1x=0)
 bash -n files/dracut/95dot1x-macsec/*.sh scripts/*.sh   # shell syntax gate
-shellcheck files/dracut/95dot1x-macsec/*.sh scripts/*.sh
+# .shellcheckrc disables the dracut false positives; CI passes them explicitly:
+shellcheck --exclude=SC1091,SC2154 files/dracut/95dot1x-macsec/*.sh scripts/*.sh
 ```
 
 Build-host deps: qemu-kvm, libguestfs-tools, erofs-utils, cryptsetup, jq,
@@ -113,8 +118,40 @@ instead of working around it.
   journal-less ext4 in `extract-artifacts.sh`).
 - `systemd.verity_root_options=hash-offset` support in the shipped systemd
   (fallback: split data/hash into two namespaces).
-- `network-legacy` dracut module availability (the hook’s `ifup` step needs
-  porting if a future release drops it).
+- `network-legacy` dracut module needs `dhclient`; RHEL 9 omits it from the
+  minimal install, so the provisioner installs `dhcp-client` (without it dracut
+  fails "module network-legacy can't be installed"). dhclient is deprecated —
+  the durable fix is porting the hook to the `network-manager` dracut module.
+- Writable root overlay is `systemd.volatile=overlay`, **not** `rootovl`:
+  RHEL/Rocky dracut ships no `overlayfs`/`rootovl` module. Needs
+  `systemd-volatile-root.service` in the initramfs (bundled by recent dracut;
+  `install_items` it if the root stays read-only). Misbehaves with NFS root —
+  N/A here (NVMe block root).
+
+## RHEL 9.x build notes (learned — don't re-discover)
+
+These bit us bringing the build up on Rocky 9.8; they are easy to regress.
+
+- **ISO**: default is a version-pinned Rocky 9.x **minimal** ISO on the internal
+  mirror (`iso_url` + `sha256` in `iso_checksum`; bump both together on a point
+  release). `http/ks.cfg` installs only `@^minimal-environment`+sshd+chrony from
+  the ISO; the diskless tooling is pulled **online** by the provisioner, so the
+  build VM needs outbound repo access. Keeps the build ISO-variant-agnostic.
+- **`%{...}` in provisioner shell** (e.g. `rpm --qf '%{VERSION}'`) must be
+  escaped `%%{...}` — Packer's HCL2 engine parses a bare `%{` as a template
+  directive and `packer validate` fails otherwise.
+- **File provisioner staging**: pre-create the destination dir (`mkdir -p
+  /tmp/etc-stage`) before a trailing-slash *contents* upload; Packer requires
+  the dest to exist or the tree lands elsewhere and later `install` steps fail.
+- **dracut module name** in `add_dracutmodules` omits the `NN` dir-prefix: the
+  directory is `95dot1x-macsec` but the module is `dot1x-macsec` (cf. `nvmf`).
+- **`network-legacy` → `dhcp-client`** and **overlay → `systemd.volatile=overlay`**:
+  see the bench-verify items above.
+- **Test toggle**: `disable_dot1x=true` bakes `rd.dot1x=0`; the hook then skips
+  802.1X/MACsec and DHCPs the NIC directly for NVMe-oF on an open port. The auth
+  stack stays in the image, dormant. The matching lab DHCP/HTTP/NVMe-oF serving
+  stack lives in `serving/` (docker compose). Re-enable security by dropping the
+  var / setting `DISABLE_DOT1X: "false"` in the CI workflow.
 
 ## Testing changes
 

README.md

@@ -23,6 +23,11 @@ files/etc/dracut.conf.d/90-diskless.conf  no-hostonly + nvmf/verity/erofs + driv
 files/etc/NetworkManager/                 marks NIC/macsec0 unmanaged post-pivot
 scripts/extract-artifacts.sh              qcow2 -> vmlinuz + initramfs.img
                                                  + rootfs.erofs (+verity) + cmdline
+scripts/build-oci.sh                      ukify (+sbsign) + roothash gate -> OCI image
+serving/                                  Lab serving stack (docker compose):
+                                          dnsmasq DHCP/DNS + nginx (UKI/HTTP) + SPDK
+.gitea/workflows/                         Gitea CI: lint -> packer build -> push OCI
+docs/diskless-fleet-design-v2.pdf         Architecture design (authoritative)
 ```
 
 ## Build host requirements
@@ -33,6 +38,12 @@ scripts/extract-artifacts.sh              qcow2 -> vmlinuz + initramfs.img
 - **systemd-ukify** (+ **sbsigntools** when signing) — UKI assembly
 - **buildah** (or docker) — OCI release-artifact image build
 
+The default `iso_url`/`iso_checksum` point at a version-pinned **Rocky 9.x
+minimal** ISO on the internal mirror. `http/ks.cfg` installs only core+sshd from
+that ISO; the diskless tooling is pulled online by the provisioner, so the build
+VM needs **outbound network to the distro repos** (and a fast mirror helps).
+Bump the URL and the pinned sha256 together when moving to a new point release.
+
 ## Build
 
 ```bash
@@ -58,6 +69,13 @@ Substitute the real NVMe target address for `NVMET_TRADDR` in
 `scripts/extract-artifacts.sh` (or post-edit `cmdline.txt`) **before** the
 OCI step: the cmdline is immutable once signed into the UKI.
 
+**Test mode (auth disabled).** Add `-var disable_dot1x=true` to bake
+`rd.dot1x=0` into the cmdline. At boot the hook then skips 802.1X/MACsec and
+DHCPs the physical NIC directly to reach the NVMe/TCP target — for validating
+the UKI/rootfs/NVMe-oF path on an **open switch port**. The auth stack stays in
+the image (dormant); drop the var to restore the secure boot. See `serving/`
+for the matching DHCP/HTTP/NVMe-oF lab stack.
+
 Outputs: `artifacts/<version>/` gains `fleet-<version>.efi` (the signed
 UKI) alongside `rootfs.erofs`/`verity.json`, and the build produces the
 release-artifact OCI image `<oci_image>:<version>` containing only
@@ -88,7 +106,7 @@ release only changes which artifact image feeds them:
 ## Kernel command line (per build — emitted in cmdline.txt)
 
 ```
-rd.neednet=1 rootovl \
+rd.neednet=1 systemd.volatile=overlay \
 rd.dot1x.nvindex=0x01800001 rd.dot1x.timeout=60 \
 rd.nvmf.discover=tcp,NVMET_TRADDR,4420 \
 root=/dev/mapper/root rootfstype=erofs rootflags=ro \
@@ -98,9 +116,12 @@ systemd.verity_root_hash=/dev/nvme0n1 \
 systemd.verity_root_options=hash-offset=<from verity.json>
 ```
 
-Single-file layout: data and hash tree share one namespace, split by
-`hash-offset`. `rd.dot1x.if=` may pin the NIC; otherwise the first
-non-loopback interface is used.
+`systemd.volatile=overlay` gives the writable tmpfs overlay over the read-only
+verity root — RHEL/Rocky dracut has **no** `rootovl`/`overlayfs` module (that is
+a Debian/third-party feature), and systemd's volatile overlay is the supported
+mechanism for dm-verity roots. Single-file layout: data and hash tree share one
+namespace, split by `hash-offset`. `rd.dot1x.if=` may pin the NIC; otherwise the
+first non-loopback interface is used. Test builds prepend `rd.dot1x=0`.
 
 ## Bench-verify before fleet rollout
 
@@ -118,14 +139,23 @@ non-loopback interface is used.
 5. **Pivot survival**: `@wpa_supplicant` survives switch-root; MKA continuity
    across it. NVMe/TCP keepalives must also ride out the pivot (they will,
    if the link never drops — that is the point of the @-daemon).
-6. **dracut module choice**: skeleton uses `network-legacy`; if removed in
-   your release, port the hook's ifup step to the available backend.
+6. **network backend**: the hook uses `network-legacy`/`ifup`, which requires
+   `dhclient`. RHEL 9 deprecated it and the minimal ISO omits it, so the
+   provisioner installs `dhcp-client` explicitly (without it dracut fails with
+   "module network-legacy can't be installed"). dhclient is on its way out; the
+   durable fix is to port the hook to the `network-manager` dracut module.
 7. **hostnqn allow-listing**: if the nvmet target filters hosts, the
    allowed-NQN list must be fed from the enrolment pipeline (NQNs are
    derived from certificate serials — see the hook).
 8. **CI smoke test**: boot artifacts in qemu/Proxmox with swtpm against
    hostapd-as-authenticator + FreeRADIUS (remember bridge group_fwd_mask=8
    for EAPOL); assert macsec0 up, dm-verity root mapped, overlay rw.
+9. **writable root overlay**: provided by `systemd.volatile=overlay`, which
+   needs `systemd-volatile-root.service` in the initramfs (bundled by recent
+   dracut). If the root stays read-only at boot, add `install_items+="
+   /usr/lib/systemd/systemd-volatile-root
+   /usr/lib/systemd/system/systemd-volatile-root.service "` to
+   `90-diskless.conf`. Known to misbehave with NFS root — N/A here (NVMe block).
 
 ## What is deliberately NOT in the image