serving: make the SPDK nvmeof target version-agnostic (stop recreating it on IMAGE_VERSION bump)
The nvmeof (spdk-target) service carried a versioned bootstrap mount
(type: image, source ${OCI_IMAGE}:${IMAGE_VERSION} -> /artifacts) plus an
IMAGE_VERSION env. Bumping IMAGE_VERSION therefore changed the service config,
so `docker compose up -d` recreated the container -> nvmf_tgt restarted ->
every connected host's NVMe/TCP root dropped at once. That is precisely the
fleet-wide outage the long-lived-target design exists to prevent.
Remove the /artifacts image-volume and IMAGE_VERSION env from nvmeof so its
config no longer depends on the version — compose leaves it running across a
bump. Bootstrap and rolling updates are already fully covered by spdk-loader
(stages the current release into ./spdk/releases and adds it online via
JSON-RPC); spdk-target.sh's seed block is guarded, so it just no-ops when
/artifacts isn't mounted. Verified nvmeof no longer references IMAGE_VERSION
(only spdk-loader does); compose YAML validates.
BENCH-VERIFY: bump IMAGE_VERSION + up -d with a host connected -> nvmeof
container not recreated, host root survives, loader adds the new version.
This commit is contained in:
+11
-13
@@ -306,23 +306,21 @@ services:
|
||||
# Multi-version release store: each ./spdk/releases/<ver>/ holds that
|
||||
# version's rootfs.erofs + rootfs.hash + verity.json. Staged by the
|
||||
# spdk-loader (or spdk-ctl.sh on the host); reconciled into the target at
|
||||
# (re)start and added/removed live thereafter. RW so the bootstrap seed +
|
||||
# loader can write here. Adding a version does NOT recreate this service.
|
||||
# (re)start and added/removed live thereafter. RW so the spdk-loader can
|
||||
# stage here. Adding a version does NOT recreate this service.
|
||||
- ./spdk/releases:/releases
|
||||
# Bootstrap convenience (optional): a single OCI release mounted as an
|
||||
# image volume. spdk-target seeds it into /releases as ${IMAGE_VERSION} if
|
||||
# not already staged — preserves the simple one-version `compose up` flow.
|
||||
# Remove this block for a pure multi-version (spdk-ctl-managed) deployment.
|
||||
- type: image
|
||||
source: ${OCI_IMAGE}:${IMAGE_VERSION}
|
||||
target: /artifacts
|
||||
image:
|
||||
subpath: artifacts
|
||||
# Fallback (Docker < 28 / offline): - ${ARTIFACT_SRC}:/artifacts:ro
|
||||
# NOTE: the target is deliberately VERSION-AGNOSTIC. It mounts NO
|
||||
# ${IMAGE_VERSION} artifact image and carries no IMAGE_VERSION env, so a
|
||||
# version bump never changes THIS service's config — `docker compose up -d`
|
||||
# after an IMAGE_VERSION bump recreates the loader/http/dnsmasq but leaves
|
||||
# nvmf_tgt running (recreating it would tear down EVERY host's NVMe/TCP root
|
||||
# at once). Bootstrap AND rolling updates are the spdk-loader's job: it
|
||||
# stages the current release into ./spdk/releases and adds it online via
|
||||
# JSON-RPC. (spdk-target still seeds from /artifacts IF one is manually
|
||||
# mounted + IMAGE_VERSION set — a legacy/standalone convenience, off here.)
|
||||
# Legacy single-version entrypoint (configure-at-start, restart-to-change):
|
||||
# - ./spdk-setup.sh:/spdk-setup.sh:ro
|
||||
environment:
|
||||
IMAGE_VERSION: ${IMAGE_VERSION} # bootstrap-seed label for /artifacts
|
||||
RELEASES_DIR: /releases
|
||||
RPC_SOCK: /var/run/spdk/spdk.sock # shared with spdk-loader (-r / -s)
|
||||
SUBNQN_BASE: ${SUBNQN_BASE} # per-version NQN base (falls back per release to verity.json .subnqn)
|
||||
|
||||
Reference in New Issue
Block a user