# Diskless Fleet Image Build (Packer) Builds the fleet-generic boot artifacts for PXE/HTTP-booted diskless hosts that authenticate with 802.1X EAP-TLS (TPM-held key, certificate in TPM NV) and run MACsec on the access link. The root filesystem is an immutable, reproducible **erofs image with an appended dm-verity hash tree**, served as a single shared **read-only NVMe/TCP namespace**, with a tmpfs overlay for runtime state. Companion to `docs/diskless-fleet-design-v2.pdf`. ## Layout ``` diskless.pkr.hcl Packer template (qemu builder) http/ks.cfg Kickstart for the throwaway build VM files/dracut/95dot1x-macsec/ Custom dracut module module-setup.sh module definition dot1x-macsec-up.sh initqueue hook: NV cert -> @wpa_supplicant -> wait authorized + macsec0 -> DHCP -> hostnqn derived from cert files/etc/dot1x/wpa_supplicant-dot1x.conf Fleet-generic supplicant config files/etc/dot1x/radius-ca.pem PLACEHOLDER - replace with real CA chain files/etc/dracut.conf.d/90-diskless.conf no-hostonly + nvmf/verity/erofs + drivers files/etc/NetworkManager/ marks NIC/macsec0 unmanaged post-pivot scripts/extract-artifacts.sh qcow2 -> vmlinuz + initramfs.img + rootfs.erofs (+verity) + cmdline ``` ## Build host requirements - Packer >= 1.10, qemu plugin (`packer init .`) - qemu-kvm with KVM available - libguestfs-tools, **erofs-utils**, **cryptsetup**, jq (artifact extraction) - **systemd-ukify** (+ **sbsigntools** when signing) — UKI assembly - **buildah** (or docker) — OCI release-artifact image build ## Build ```bash # 1. Replace the CA placeholder cp /path/to/radius-ca-chain.pem files/etc/dot1x/radius-ca.pem # 2. Set the NIC drivers for your real fleet hardware (default covers # common Intel + Mellanox); the build VM only has virtio, so anything # not listed here will not be in the initramfs. packer init . packer build \ -var image_version=1.0.0 \ -var nic_drivers="ice mlx5_core" \ -var oci_image=registry.example.org/fleet-boot \ -var oci_push=true \ -var sb_key=/secure/ci/fleet-sb.key \ -var sb_cert=/secure/ci/fleet-sb.crt \ . ``` Omit `sb_key`/`sb_cert` for unsigned lab builds (the pipeline warns). Substitute the real NVMe target address for `NVMET_TRADDR` in `scripts/extract-artifacts.sh` (or post-edit `cmdline.txt`) **before** the OCI step: the cmdline is immutable once signed into the UKI. Outputs: `artifacts//` gains `fleet-.efi` (the signed UKI) alongside `rootfs.erofs`/`verity.json`, and the build produces the release-artifact OCI image `:` containing only `/artifacts/{fleet-.efi, rootfs.erofs, verity.json, SHA256SUMS}` with the roothash recorded as an image label. The build hard-fails if the roothash baked into the UKI cmdline does not match `verity.json`. ## Publishing a version The OCI image **is** the release. The serving stack stays constant (separate, independently-patched SPDK and nginx/caddy images); each fleet release only changes which artifact image feeds them: 1. Push `:` (or let `oci_push=true` do it) and record the **digest**. 2. In the fleet-boot deployment manifest (k8s Deployment or compose), bump the artifact initContainer / image-volume reference to that digest — one commit per fleet release; this manifest is the single place coupling UKI <-> root namespace. 3. The initContainer copies `/artifacts` into the shared volume; nginx serves `fleet-.efi` over HTTPS (firmware validates against the TlsCaCertificate CA, then Secure Boot verifies the UKI signature); SPDK exposes `rootfs.erofs` as a **read-only** NVMe/TCP namespace. 4. Rollout = digest bump; rollback = revert the commit. cosign the image so the deploy pipeline verifies provenance (sbsign is for firmware, cosign is for your cluster). ## Kernel command line (per build — emitted in cmdline.txt) ``` rd.neednet=1 rootovl \ rd.dot1x.nvindex=0x01800001 rd.dot1x.timeout=60 \ rd.nvmf.discover=tcp,NVMET_TRADDR,4420 \ root=/dev/mapper/root rootfstype=erofs rootflags=ro \ systemd.verity=yes roothash= \ systemd.verity_root_data=/dev/nvme0n1 \ systemd.verity_root_hash=/dev/nvme0n1 \ systemd.verity_root_options=hash-offset= ``` Single-file layout: data and hash tree share one namespace, split by `hash-offset`. `rd.dot1x.if=` may pin the NIC; otherwise the first non-loopback interface is used. ## Bench-verify before fleet rollout 1. **erofs module present** in the target RHEL kernel (`modprobe erofs`) — enabled in later RHEL 9 minors for image mode; if absent, fall back to journal-less ext4 (`mkfs.ext4 -O ^has_journal -d `) in `extract-artifacts.sh`. 2. **systemd.verity_root_options support** in the shipped systemd/dracut (needs a reasonably recent systemd; otherwise publish data and hash as two separate namespaces and drop the hash-offset option). 3. **TPM key reference**: confirm wpa_supplicant accepts `private_key="handle:0x81010001"` via the OpenSSL tpm2 provider; if not, switch to the tpm2-pkcs11 option in the supplicant config. 4. **CONFIG_MACSEC** in the shipped wpa_supplicant build. 5. **Pivot survival**: `@wpa_supplicant` survives switch-root; MKA continuity across it. NVMe/TCP keepalives must also ride out the pivot (they will, if the link never drops — that is the point of the @-daemon). 6. **dracut module choice**: skeleton uses `network-legacy`; if removed in your release, port the hook's ifup step to the available backend. 7. **hostnqn allow-listing**: if the nvmet target filters hosts, the allowed-NQN list must be fed from the enrolment pipeline (NQNs are derived from certificate serials — see the hook). 8. **CI smoke test**: boot artifacts in qemu/Proxmox with swtpm against hostapd-as-authenticator + FreeRADIUS (remember bridge group_fwd_mask=8 for EAPOL); assert macsec0 up, dm-verity root mapped, overlay rw. ## What is deliberately NOT in the image - Private keys, certificates, host names, per-host config of any kind. All identity lives in each host's TPM (persistent handle + NV index); the NVMe hostnqn is derived from the certificate at boot.