Follow-up to the linux-firmware strip (prev commit put it in the 2d hygiene
block, which runs AFTER the dracut build — so it only shrank the rootfs erofs,
not the initramfs). Move the strip to step 2, before the dracut build: with
--no-hostonly, plymouth's drm dep instmods a broad DRM set and dracut bundles
each module's firmware into the initramfs (amdgpu being the classic UKI
bloater). Trimming first means dracut never sees it, so both the rootfs erofs
AND the UKI initramfs shrink. Keep-list is unchanged (i915 + intel/ + mellanox
+ intel-ucode), so the fleet splash GPU (i915) and ice DDP firmware the
initramfs actually needs are retained; dracut just warns 'firmware not found'
for the removed drm modules and continues.
Add a post-build diagnostic (initramfs .img size + bundled firmware, expected
i915 only) so the initrd win is measured in the build log, not estimated.
The 2b rpm-size manifest revealed the biggest packages aren't the kiosk but
base bloat: linux-firmware 678 MB and glibc-all-langpacks 228 MB (vs chromium
370 + llvm 129). The uncompressed erofs is read over NVMe/TCP on every boot,
so this is fleet-wide dead weight + attack surface.
glibc: swap glibc-all-langpacks (every language) for glibc-minimal-langpack +
glibc-langpack-en. ~220 MB; image is English-only.
linux-firmware: a live fleet audit (modinfo -F firmware over loaded modules +
sysfs/lspci driver bindings on the real 820-class hosts and the VMs) proved the
only firmware any host loads is Intel i915 (GPU). Every NIC is firmware-free
(e1000e reports on-chip NVM, not a fw file; virtio_net loads nothing), so the
catastrophic missing-NIC-firmware case is off the table. Keep i915 + intel/
(ice DDP insurance) + mellanox + intel-ucode; strip the ~600 MB of foreign
vendor blobs (amdgpu/nvidia/radeon/ath/qcom/...). Remove iwl*-firmware (Wi-Fi,
~83 MB) on a wired-only fleet. Strip-in-place since the package can't be
removed without losing i915; du reports the result in the build log.
BENCH-VERIFY: kiosk still renders on the 820 (i915 DMC/GuC load) — the one real
risk; widen the keep-list if any host has a non-Intel GPU or an ice/mlx5 NIC.
The kiosk dnf transaction was measured at 229 pkgs / 1.0 GB. Nearly all
of the non-core packages are Recommends: a locked-down single-URL kiosk
never uses flatpak + its ostree/bubblewrap sandbox runtime, tracker-miners
(a filesystem indexer — dead weight and a threat surface), the pipewire/
pulseaudio audio stack, or libproxy-webkitgtk4 -> webkit2gtk3-jsc (an 8.9M
second browser engine dragged into a Chromium kiosk for libproxy's PAC
runner).
Add --setopt=install_weak_deps=False to cut them, and an rpm -qa size
manifest to the build log so the next CI build reports the exact new
footprint (turning 'seems unnecessary' into data for the next trim round:
gstreamer/media codecs, or mesa-dri-drivers if the GPU path is abandoned).
The immovable floor is chromium + chromium-common + llvm-libs (mesa sw-GL)
+ gtk3/adwaita + libicu, all hard deps. BENCH-VERIFY the kiosk still
renders with weak deps off (watch for a missing font or libva).
The build failed: checkmodule rejected `policy_module(chrony_dhcp_dispatcher,
1.0.0)` with "Building a policy module, but no module specification found".
policy_module() is a refpolicy m4 macro expanded only by the refpolicy Makefile
(/usr/share/selinux/devel), not by checkmodule. Use the raw module-policy
declaration `module chrony_dhcp_dispatcher 1.0.0;`, which is exactly what
checkmodule -M -m parses. require{} + allow rule unchanged.
The Dell 820 kept "finding a cert" and bypassing enrolment despite repeated BIOS
TPM clears. tpm2_nvreadpublic revealed 0x01800001 on that hardware is a FIRMWARE
platform index (platformcreate|policydelete|writelocked, 70 bytes — not our
cert; siblings at 0x01800003/4 + 0x01810000). platformcreate survives TPM2_Clear
and policydelete needs the platform's secret policy, so it's permanent and
unusable by us — the blind check false-positived on it, and enrolment could
never write there anyway. Same class of bug as the EK-handle collision.
Move the NV cert index default 0x01800001 -> 0x01800100 (free on the 820, owner
range, overridable via rd.dot1x.nvindex): cmdline (extract-artifacts.sh),
fleet-enrol -nvindex, the dracut hooks, bench scripts, CLAUDE.md invariant #1,
and design/SSP docs. The signing-key handle (0x81000100) is unchanged.
Harden WriteNVCert: refuse a platformcreate index with a clear "pick a free
owner index via rd.dot1x.nvindex" error instead of a confusing undefine
failure. Regression simtest/TestWriteNVCertRefusesPlatformIndex; NV roundtrip +
ReadNVCert tests moved to 0x01800100. All go tests + packer validate pass.
Captures the decision to re-enable a CONFIGURED firewalld (currently removed),
the required .pyc/-T0 root-cause fix (compileall --unchecked-hash), the 4 STIG
waivers it converts, and the load-bearing NVMe/TCP-survives-firewalld bench gate.
http/ks.cfg excludes firewalld (-firewalld) and sets firewall --disabled — the
fleet's security boundary is the network (802.1X/MACsec + switch); the host
firewall is a documented OFF item (stig-deviations #4). But a provisioner dnf
pulled firewalld back as a weak dep (Recommends) and it starts by default. Its
Python then tries to rewrite site-package .pyc bytecode, which the reproducible
erofs -T 0 mtime reset makes look stale, so firewalld_t gets write/unlink AVCs
on lib_t that flood the SIEM.
Remove firewalld in 2d hygiene (weak dep, nothing hard-needs it) and mask it as
a backstop against future weak-dep reinstalls. No firewalld => no daemon, no
AVC, matches the kickstart intent.
The NM dispatcher 10-chrony-dhcp-ntp runs `chronyc reload sources` to load DHCP
option-42 NTP servers. That chronyc runs in the confined custom-dispatcher
domain (NetworkManager_dispatcher_custom_t), so its reply socket carries that
label and the shipped policy has no rule for chronyd_t to sendto it. Under
enforcing, chronyd does the reload but its ack is denied — chronyc reports
failure and the AVC floods the SIEM (seen 2026-07-03; permissive=1 only because
the box was booted enforcing=0).
Ship a minimal local allow module (files/stig/chrony-dhcp-dispatcher.te):
allow chronyd_t NetworkManager_dispatcher_custom_t:unix_dgram_socket sendto;
Built + semodule -i'd in the 2c-STIG provisioner (checkpolicy installed if
absent), with an install assertion. `allow` not `dontaudit` — it also lets the
reload ack succeed so chronyc returns cleanly. Documented as
docs/stig-deviations.md #14 (not a rule waiver — a local functional allow).
BENCH-VERIFY: enforcing boot shows no AVC, chronyc reload returns 0, a
DHCP-handed NTP server appears in `chronyc sources`; and re-check verity-hash
reproducibility (semodule rebuilds the policy store, like the semanage step).
The nvmeof (spdk-target) service carried a versioned bootstrap mount
(type: image, source ${OCI_IMAGE}:${IMAGE_VERSION} -> /artifacts) plus an
IMAGE_VERSION env. Bumping IMAGE_VERSION therefore changed the service config,
so `docker compose up -d` recreated the container -> nvmf_tgt restarted ->
every connected host's NVMe/TCP root dropped at once. That is precisely the
fleet-wide outage the long-lived-target design exists to prevent.
Remove the /artifacts image-volume and IMAGE_VERSION env from nvmeof so its
config no longer depends on the version — compose leaves it running across a
bump. Bootstrap and rolling updates are already fully covered by spdk-loader
(stages the current release into ./spdk/releases and adds it online via
JSON-RPC); spdk-target.sh's seed block is guarded, so it just no-ops when
/artifacts isn't mounted. Verified nvmeof no longer references IMAGE_VERSION
(only spdk-loader does); compose YAML validates.
BENCH-VERIFY: bump IMAGE_VERSION + up -d with a host connected -> nvmeof
container not recreated, host root survives, loader adds the new version.
The enrol trigger keyed on "is NV 0x01800001 defined/readable", so leftover
state (an earlier tpm-provision run — bench-seen on the 820: "cert present —
normal boot" with no real identity) or random bytes (tpm-selftest) silently
suppressed enrolment. Now content-aware:
- tpmkey.ReadNVCert reads the full NV contents (chunked).
- classifyNVCert (TPM-agnostic, unit-tested): parses X.509, checks validity
window, and when a real fleet CA bundle is present verifies the cert chains
to it. Only a valid fleet cert = enrolled.
- New `fleet-enrol -check-only` (no network/no key-gen) exits 0=valid cert,
20=enrol needed, 1=error. The full enrol path uses the same classification
instead of NVPresent.
- The dracut hook calls -check-only as the single gate, replacing the blind
tpm2_nvread (and its now-dead TPM2TOOLS_TCTI export). tpm2_nvread stays in
the initramfs for the secure path (dot1x-macsec-up.sh).
Lab fallback: the shipped radius-ca.pem is a placeholder (not a cert) ->
loadCAPool returns nil -> chain-less "parseable + unexpired" acceptance, so
reboot-converge doesn't loop; production replaces radius-ca.pem with the real
fleet CA for strict chaining.
Tests: classifyNVCert table (valid/expired/garbage/foreign/no-CA) + simtest
ReadNVCert roundtrip across the 512B chunk. All pass; packer validate clean.
Bench (Proxmox vTPM, host cygnus-cruiser-99) confirmed via tpm2_readpublic that
0x81010001 — the handle our design used for the fleet signing key — is the
TCG-standard RSA Endorsement Key slot: the object there is the swtpm-provisioned
EK (restricted|decrypt|adminwithpolicy, RSA-2048/AES128CFB, canonical EK
authPolicy 837197…, EK cert at NV 0x1c00002). Most firmware/swtpm TPMs occupy
that slot, so EnsureKey could never create our key there — it errored reading
the EK as ECC and enrolment died before the CSR/POST.
Move the fleet signing key to 0x81000100, a non-reserved owner persistent
handle (clear of EK 0x81010001/2 and SRK 0x81000001/2). Threaded through:
- fleet-enrol -handle default + new rd.dot1x.handle= cmdline knob (baked by
extract-artifacts.sh, overridable via SMBIOS)
- files/etc/dot1x/wpa_supplicant-dot1x.conf private_key handle
- the dracut hook, bench tpm-provision.sh / tpm-selftest.sh
- CLAUDE.md invariant #1 + design/SSP docs + TODO #11
NV index 0x01800001 (owner range) is unchanged.
EnsureKey hardening: refuse to evict any restricted-decrypt object (EK/SRK) as
a safety net — never clobber the TPM endorsement/storage identity (also
protects the future EK/AK attestation path). Still self-heals a genuine stale
signing key. New regression TestEnsureKeyRefusesToEvictProtectedKey; the
stale-key regression and handle constants moved to 0x81000100.
Design note records the IDevID (0x81020001) option for the later EK/AK
attestation work. Follow-up items logged: content-aware NV presence check
(verify the cert chains to the fleet CA, so leftover/foreign NV can't suppress
enrolment) and a de-enrol/TPM-reset procedure.
Bench boot failed with "fleet-enrol: ensure key at 0x81010001: key at handle
is not ECC: did not contain eccDetail (selector value was 1)": a real host had
an RSA key persisted at 0x81010001 (left by an older tpm2-tools/openssl
provisioning path), and EnsureKey fatally errored trying to read it as our ECC
signing key — so enrolment died before the CSR/POST and the host fell through
to a normal boot (no fingerprint, nothing at the broker).
EnsureKey now evicts a wrong-type/unusable key at the handle and creates a
fresh ECC key. Safe on the enrol path: it only runs on an empty-NV host, so no
issued cert references the old key. A valid ECC key is still reused (idempotent
retried enrol).
Regression: simtest TestEnsureKeyEvictsStaleNonECC pre-plants an RSA key and
asserts EnsureKey recreates a usable ECC signer (passes against the go-tpm
simulator).
Captures the design + quantified STIG payoff (~21 mount/partition rules
notapplicable/waived -> pass, per reports/stig-arf.xml), the writable-set +
noexec/nosuid/nodev flag matrix, the kiosk-/tmp-noexec prerequisite, the
fstab-vs-runtime OVAL note, and that it's bench-gated after TPM enrolment (#11).
getfattr --only-values emits the NUL-terminated security.selinux value;
bash's $(...) then warns "ignored null byte in input". Pipe through
tr -d '\0' to silence the cosmetic warning. The label assertion above is
unaffected.
The out-of-VM Go build step used `set -euo pipefail`, but the self-hosted
Gitea/act runner runs `run:` steps under sh (dash), which rejects
`-o pipefail` ("Illegal option"). Add `shell: bash` to the step. Keeping
`set -e` is load-bearing: without it a failed go build/test in the first
subshell group is masked by the trailing `file` command's success, letting
a broken binary reach the Packer build.
Removes the hard no-op on the test path so the enrolment flow can be exercised
on a host with a TPM + a reachable broker WITHOUT standing up 802.1X/MACsec/
RADIUS. Networking is now mode-aware: in test mode ip=dhcp already owns the NIC
(and wrote resolv.conf), so the hook uses the existing lease instead of running
its own nmcli DHCP — avoiding the dracut "address already assigned" race
(CLAUDE.md "don't double-configure the NIC in test mode"). Secure mode still does
the nm-unmanaged override for the pre-auth lease. rd.fleet.enrol=0 disables
enrolment in either mode; production intent stays secure-path-only.
Replaces the shell/openssl-provider host enrolment (a35f2d6) with a static
Go + go-tpm binary, and moves BOTH host Go binaries out of the Packer VM.
fleet-enrol (files/enrol/, Go + go-tpm, no CGO):
- talks to /dev/tpmrm0 DIRECTLY via transport.FromReadWriter(os.File) — no
linuxtpm, so go-tpm is the only dep and there is NO tpm2-tools key-gen /
openssl tpm2 provider / curl in the image (removes the biggest bench risk,
the provider handle: key-ref).
- ensures the ECDSA key @0x81010001, builds the CSR (crypto.Signer over the
TPM key), POSTs CSR+context to the broker, shows the Plymouth fingerprint,
polls /status, writes the leaf cert to NV 0x01800001.
- tpmkey/ is an exported package so the TPM ceremony is tested against the
go-tpm software simulator from a SEPARATE simtest/ module (CGO stays out of
the shipped build). Transaction logic (mock broker) tested in-package.
- deps pinned by go.sum (not vendored): the binary now builds on the CI
runner, which has network — reproducible via pinned Go 1.26.4 + go.sum.
Out-of-VM build (decision): both fleet-enrol AND fleet-hostapi compile in the
CI "Build host Go binaries" step (Go 1.26.4, CGO off, -trimpath) and Packer
FILE-PROVISIONS the static binaries in. Dropped the in-VM `dnf install golang`
+ the 2d removal — the Rocky build image carries no Go toolchain / go-tpm tree.
Wiring:
- tpm-enrol.sh: now a thin GATE (empty-NV check, pre-auth DHCP on en*,
exec /usr/sbin/fleet-enrol, reboot-to-converge). module-setup.sh bundles the
binary (asserted) + reverts the tpm2-openssl/curl installs.
- diskless.pkr.hcl: file provisioners for both binaries; enrol installed to
/usr/sbin before the initramfs build + fapolicyd-trusted; lsinitrd asserts
fleet-enrol; dnf reverted to baseline (no tpm2-openssl/curl).
- gitea-main-ci.yaml: Set up Go + build both binaries + run the simulator
ceremony test, before packer validate.
TESTED here: transaction unit tests + the full TPM ceremony (key-gen -> persist
-> ECDSA sign -> CSR verify -> chunked NV write -> re-enrol) against the go-tpm
simulator. BENCH-VERIFY (real/swtpm, empty owner auth, pre-auth DHCP, Plymouth,
reboot-converge, prod HTTPS) tracked in docs/TODO.md #11.
Builds the host side of TPM identity enrolment — the "inline single-UKI"
vehicle from docs/tpm-enrolment-design.md. A production (secure) UKI now
auto-enrols a fresh host with no client cert, then reboots into the normal
secure path.
New dracut hook files/dracut/95dot1x-macsec/tpm-enrol.sh (initqueue/settled
prio 25, BEFORE dot1x-macsec-up at 30):
- no-op when a cert is already present in NV 0x01800001 (normal boot) or
under rd.dot1x=0 (test path has no trustworthy identity to enrol);
- on EMPTY NV: generate the signing key in the TPM @0x81010001 (openssl tpm2
provider builds the CSR; key never leaves the TPM), pre-auth DHCP on the
PHYSICAL NIC (nmcli re-manages en*, overriding the secure-mode unmanaged
drop-in), POST CSR+context to rd.fleet.enrol_url, show the broker
fingerprint on Plymouth, poll /status, write the issued leaf cert (DER) to
NV, and reboot to converge.
- Empty-NV and the secure path are mutually exclusive, so this never reorders
the load-bearing boot sequence (invariant #2).
Wiring:
- module-setup.sh: require+install the tpm2 key-gen/NV tools, curl, awk, and
the openssl tpm2 PROVIDER (ossl-modules/tpm2.so, asserted present — tpm2-tss
alone can't sign with a handle: key); install the hook.
- diskless.pkr.hcl: install tpm2-openssl + curl; assert hook/tools/provider in
the initramfs (invariant #9). New enrol_url var threaded to extract-artifacts.
- extract-artifacts.sh: bake rd.fleet.enrol_url into the cmdline.
- fleet.script: enrol-active/enrol-wait/enrol-done banner; fingerprint on the
message line (public — states only, invariant #1).
Shell JSON build/parse/cert-extract pipeline (no jq in the initramfs) verified
end-to-end against a live broker. BENCH-VERIFY list in docs/TODO.md #11 (tpm2
provider key-ref on real/swtpm, empty-NV detect, pre-auth DHCP override, Plymouth
render, NV write + reboot convergence, prod HTTPS).
The `step` CLI ran against --ca-url https://ca.fleet.lab:8443, but that name is
only served by the fleet dnsmasq to booting clients — the serving host's own
resolver can't resolve it ("no such host on 127.0.0.53"), so provisioner list +
RADIUS cert issuance failed. Connect over CA_CONNECT (default localhost, which
the compose puts in the CA cert SAN); CA_DNS stays the cert/SAN + RADIUS CN name.
Verifying the step signer against a real step-ca surfaced two real bugs the
mock signer had hidden, plus made the bench CA actually initialise:
- signer.go: move --not-after from `step ca token` (validity caps ~1h ->
"token validity too long") to `step ca sign` (it is the CERT lifetime).
Without this the step signer never issues a cert.
- dot1x-bench compose: DOCKER_STEPCA_INIT_PASSWORD_FILE pointed at a file that
does not exist at first init (it is an INPUT path), so `step ca init` failed
and never wrote secrets/password. Use DOCKER_STEPCA_INIT_PASSWORD (the value)
which the entrypoint writes to secrets/password, where bootstrap + broker read.
- bootstrap.sh: self-heal the root-owned ./pki Docker auto-creates; detect the
CA password at $STEPPATH/password or secrets/password (varies by image
version); raise authority.claims.maxTLSCertDuration so 720h fleet certs are
allowed (stock cap 24h, which also blocked bootstrap's own RADIUS cert); fix
RADIUS cert paths to write via the /export mount (were host paths run inside
the container, so they never resolved).
- serving stack: add the enrol-broker service (temporary home; nginx already
proxies /enrol/) defaulting to the zero-dependency lab path (mock signer +
insecure loopback admin), with ENROL_* knobs to switch to step + mTLS.
- verify-serving.sh: exercise request -> approve -> cert through the deployed
nginx /enrol/ proxy.
- TODO #11: record the step-signer verification + the two fixes.
Step signer VERIFIED end-to-end against a real step-ca: issued cert chains to
the fleet root, EKU clientAuth, hostnqn derived from serial.
Audiobuzz