nvmf: targeted per-version connect (rd.nvmf.nqn) instead of connect-all
Hosts connected with 'nvme connect-all', which attaches EVERY subsystem in the discovery log. Each host then mounts only its own by-UUID verity namespaces but leaves a live controller on every other served version's subsystem. Observed via spdk-release list: more controllers than workstations, each box on every version (1/3/3/1/3 across 5 versions for ~3 boxes). Beyond wasted connections (O(hosts x versions)), it defeated the rolling-update drain/reap: no version ever reached 0 connections while any host was up, so spdk-loader's auto-reap could never fire and old versions accumulated without bound. Fix — targeted connect to this build's subsystem only: - extract-artifacts.sh bakes the per-version SUBNQN (already in verity.json .subnqn, matched by the serving side) into the signed UKI cmdline as rd.nvmf.nqn=. NQN is version identity, not per-host state, and the cmdline is already per-version + immutable-per-signed-UKI, so invariant #1 holds. - nvmf-connect.sh (test path) uses 'nvme connect -n <nqn>'; legacy UKIs without rd.nvmf.nqn fall back to connect-all with a warning so they still boot. - dot1x-macsec-up.sh (secure path) gains an explicit step-6 targeted connect over macsec0 (after it's addressed, per invariant #2), replacing the delegation to the dracut nvmf module's discovery connect-all. Now each host = 1 controller on its own subsystem; versions drain to 0 when their hosts migrate, so auto-reap works and connections scale O(hosts). Requires a rebuild + re-sign (cmdline change, invariant #6). BENCH-VERIFY both paths; confirm the dracut nvmf module isn't also connect-all-ing on the secure path. CLAUDE.md invariant #2 + TODO #6 updated.
This commit is contained in:
@@ -115,9 +115,13 @@ instead of working around it.
|
||||
`rd.dot1x.nvindex=` / `fleet-enrol -handle -nvindex`.)
|
||||
1. **The hook ordering in `dot1x-macsec-up.sh` is load-bearing.**
|
||||
NV-cert extraction → supplicant start → wait for `Authorized` + macsec0 →
|
||||
DHCP **on macsec0** → hostnqn derivation. Never DHCP before auth
|
||||
DHCP **on macsec0** → hostnqn derivation → **targeted NVMe/TCP connect**
|
||||
(`nvme connect -n $rd.nvmf.nqn`, step 6). Never DHCP before auth
|
||||
completes (pre-auth leases are invalid after VLAN assignment); never
|
||||
connect NVMe before macsec0 is addressed.
|
||||
connect NVMe before macsec0 is addressed. The connect is **targeted to this
|
||||
version's subsystem** (`rd.nvmf.nqn`, baked per-version into the signed
|
||||
cmdline) — never a discovery `connect-all`, which attaches every served
|
||||
version and defeats the per-version drain/reap gate (see TODO #6).
|
||||
1. **The supplicant must survive switch-root.** It is launched with
|
||||
`exec -a @wpa_supplicant` (systemd root-storage-daemon convention) and
|
||||
its state lives under `/run`. Do not “clean up” the `@`, move state off
|
||||
|
||||
Reference in New Issue
Block a user