sunjizzle/diskless-workstation
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Adding spdk loader

Browse Source
This commit is contained in:
Audiobuzz
2026-06-27 10:44:42 +00:00
parent e43112b0f1
commit 02ffd7d885
9 changed files with 324 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serving/spdk/README.md
+28 -2
View File
@@ -23,7 +23,8 @@ process stays up.** Routine rollouts never restart SPDK.
|---|---|
| [`spdk-target.sh`](spdk-target.sh) | **Persistent entrypoint.** Starts `nvmf_tgt` once, creates the TCP transport + discovery listener, then reconciles every staged release. Never restarted to change versions. On (re)start it re-adds everything under `./releases/`, so a serving-host reboot restores all served versions. |
| [`spdk-release.sh`](spdk-release.sh) | **In-container control** (JSON-RPC). `add`/`remove`/`list`/`drain-status`/`reconcile` a version **live**. One **subsystem per version** (distinct NQN) so adding/removing one never disturbs the subsystem other hosts are attached to. Removal is **drain-gated**. |
| [`spdk-ctl.sh`](spdk-ctl.sh) | **Host-side ops.** Stages a release's artifacts from its OCI image into `./releases/<ver>/` and drives the live target via `docker compose exec`. |
| [`spdk-ctl.sh`](spdk-ctl.sh) | **Host-side ops** (manual / cleanup). Stages a release's artifacts from its OCI image into `./releases/<ver>/` and drives the live target via `docker compose exec`. |
| [`spdk-loader.sh`](spdk-loader.sh) | **Declarative rolling updates** (compose service `spdk-loader`, the "init container that stays up"). On `compose up` after an `IMAGE_VERSION` bump it stages + adds the new rootfs online, then auto-reaps old versions once they drain. |
| `./releases/<version>/` | Per-version `rootfs.erofs` + `rootfs.hash` + `verity.json`. `verity.json` is the **single source of truth** for that version's namespace UUIDs + subsystem NQN (written by `scripts/extract-artifacts.sh`), so the host's by-id verity refs and the target's namespace UUIDs always agree. |
Per-version namespace UUIDs are required: two versions served at once must have
@@ -32,7 +33,32 @@ globally-distinct namespace UUIDs or they collide on the same
derives them deterministically (UUIDv5 of the version) and bakes them into both
the immutable UKI cmdline and `verity.json`.
## Rolling update (add new, drain old, clean up)
## Rolling update — the declarative way (recommended)
Bump one tag and bring the stack up:
```bash
# in serving/.env
IMAGE_VERSION=1.3.0
docker compose up -d
```
Compose recreates the **`spdk-loader`** and the **http/dnsmasq** UKI servers (their
image volumes changed), but **not** the `nvmeof` target (its definition is
unchanged) — so no connections drop. The loader adds 1.3.0 to the live target,
serves the new UKI to all hosts, and then **auto-reaps** 1.2.0 once every host
has rebooted onto 1.3.0 and 1.2.0 has been drained for `REAP_GRACE_SECONDS`.
Rollback = set `IMAGE_VERSION` back + `up -d`. Set `REAP_OLD=false` to keep old
versions and reap them yourself.
Why auto-reap is safe here: there is a **single served UKI**, so a powered-off
host that returns boots the *current* UKI — nothing can come back to an old
rootfs once its live connections (CONNS) hit 0. Removal is still drain-gated and
debounced by the grace window (ride out transient nvme-tcp reconnects — keep
`REAP_GRACE_SECONDS` above the host `ctrl_loss_tmo`).
## Rolling update — the manual way (`spdk-ctl.sh`)
```bash
cd serving/spdk