sunjizzle/diskless-workstation
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

serving: make the SPDK nvmeof target version-agnostic (stop recreating it on IMAGE_VERSION bump)

Browse Source 
The nvmeof (spdk-target) service carried a versioned bootstrap mount
(type: image, source ${OCI_IMAGE}:${IMAGE_VERSION} -> /artifacts) plus an
IMAGE_VERSION env. Bumping IMAGE_VERSION therefore changed the service config,
so `docker compose up -d` recreated the container -> nvmf_tgt restarted ->
every connected host's NVMe/TCP root dropped at once. That is precisely the
fleet-wide outage the long-lived-target design exists to prevent.

Remove the /artifacts image-volume and IMAGE_VERSION env from nvmeof so its
config no longer depends on the version — compose leaves it running across a
bump. Bootstrap and rolling updates are already fully covered by spdk-loader
(stages the current release into ./spdk/releases and adds it online via
JSON-RPC); spdk-target.sh's seed block is guarded, so it just no-ops when
/artifacts isn't mounted. Verified nvmeof no longer references IMAGE_VERSION
(only spdk-loader does); compose YAML validates.

BENCH-VERIFY: bump IMAGE_VERSION + up -d with a host connected -> nvmeof
container not recreated, host root survives, loader adds the new version.
This commit is contained in:
Audiobuzz
2026-07-03 01:07:16 +00:00
parent 86a9839cc4
commit 0e139f5346
2 changed files with 28 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/TODO.md
+17
View File
@@ -675,6 +675,23 @@ must be built around **never restarting the process that holds live connections.
the current UKI), so auto-reap is safe; removal is still drain-gated +
grace-debounced. The loader + target share the RPC socket on a named volume
(`nvmf_tgt -r`); the loader reuses the SPDK image (has python3 + rpc.py).
- **BENCH FINDING 2026-07-03 (FIXED) — the target WAS still recreated on an
`IMAGE_VERSION` bump.** Despite the above, the `nvmeof` (spdk-target) service
still carried a versioned `type: image` bootstrap mount
(`source: ${OCI_IMAGE}:${IMAGE_VERSION}` → `/artifacts`) **and** an
`IMAGE_VERSION` env, so bumping the version changed THIS service's config and
`compose up -d` recreated it → `nvmf_tgt` restarted → **every connected host's
NVMe/TCP root dropped** (the exact fleet-wide outage the long-lived-target
design exists to prevent). Fix: **made the target version-agnostic** — removed
the `/artifacts` image-volume + `IMAGE_VERSION` env from `nvmeof` in
`serving/docker-compose.yml`. Bootstrap AND rolling updates are now solely the
`spdk-loader`'s job (stages the current release into `./spdk/releases`, adds it
online); `spdk-target.sh`'s seed block is guarded
(`[ -s /artifacts/verity.json ] && [ -n IMAGE_VERSION ]`) so it no-ops when
unmounted. Verified `nvmeof` no longer references `IMAGE_VERSION` (only
`spdk-loader` does). **BENCH-VERIFY:** bump `IMAGE_VERSION` + `up -d` with a
host connected → `nvmeof` container NOT recreated (`docker ps` ID/uptime
unchanged), host root survives, loader adds the new version online.
- **No jq dependency:** `spdk-release.sh` parses RPC JSON with **python3** (the
SPDK image ships no jq); legacy artifacts (no per-version UUIDs in
verity.json) fall back to the fixed UUID pair with a warning.