diskless-workstation/serving/fleet-manager/DEPLOY.md

fleet-manager — deployment runbook

Bring-up of the fleet-manager (control plane + admin SPA) and its private InfluxDB on the management host. This is a lab/on-prem deployment; adapt the PKI/ACME steps to your environment. See README.md for the design and docs/fleet-management-design.md for the architecture.

Where it runs. The management host — the trusted side, alongside the enrol-broker and step-ca (serving/dot1x-bench/). Never on the boot LAN. The admin plane (passkey UI) faces admins; the agent plane (gRPC/mTLS) faces enrolled hosts; both are segmented from the boot network.

Prerequisites

• Docker + compose, and the step CLI (smallstep) on the host.
• The fleet step-ca already bootstrapped (serving/dot1x-bench/ — docker compose up -d step-ca && ./step-ca/bootstrap.sh), so ../dot1x-bench/pki/root_ca.crt and the enrol provisioner exist.
• The enrol-broker running (serving/dot1x-bench/ compose) with its admin plane on 127.0.0.1:9443.
• A DNS zone you control for the UI hostname (bitasylum.org) and an ACME client that can do DNS-01 (e.g. lego or certbot + your DNS provider).
• A YubiKey (or other WebAuthn authenticator) for the bootstrap admin.

1. Configure

cd serving/fleet-manager
cp .env.example .env
# Edit .env:
#   FLEET_MGR_RP_ID / _ORIGIN   -> your UI hostname (diskless-server.bitasylum.org)
#   FLEET_MGR_INFLUX_TOKEN      -> a long random secret (openssl rand -hex 32)
#   INFLUX_ADMIN_PASSWORD       -> a strong password

The Influx token is handed to both InfluxDB (its init admin token) and the manager, so one value wires them with no copy step. .env is gitignored.

2. PKI

Three certs, three trust roots — do not conflate them (see README "two planes"):

2a. Agent-plane server cert (fleet CA)

Enrolled hosts dial the gRPC agent plane and verify the manager against the fleet root they hold, so this cert is issued by the fleet CA:

# SANs = every name/IP a host might dial the agent plane on.
AGENT_SANS=diskless-server.bitasylum.org,10.0.0.1 \
  ./server-pki.sh
# -> ./pki/agent/{server.crt,server.key,fleet-ca.crt}

2b. Broker admin-client cert (broker admin CA)

The manager is an mTLS client of the enrol-broker's admin plane. Mint its client cert from the broker's dedicated admin CA (NOT the fleet CA — reusing the fleet CA would let any enrolled host authenticate to the admin plane):

../enrol-broker/admin-pki.sh ../dot1x-bench/pki/broker fleet-manager
cp -r ../dot1x-bench/pki/broker ./pki/broker
# -> ./pki/broker/{admin-ca.crt, clients/fleet-manager.{crt,key}}

2c. Admin UI cert (Let's Encrypt DNS-01)

The UI hostname resolves on the mgmt net and is not internet-reachable, so HTTP-01/TLS-ALPN-01 can't validate. Use DNS-01, which proves control of the zone via a TXT record (and supports wildcards). Example with lego:

# Credentials for your DNS provider are provider-specific (env vars).
lego --dns <provider> --domains diskless-server.bitasylum.org \
     --email you@example.org --accept-tos run
# Point the manager at the issued files (it hot-reloads on renewal):
mkdir -p ./pki/admin
cp .lego/certificates/diskless-server.bitasylum.org.crt  ./pki/admin/fullchain.pem
cp .lego/certificates/diskless-server.bitasylum.org.key  ./pki/admin/privkey.pem

Automate renewal (cron/systemd-timer) to rewrite those two files; the manager's certReloader notices the mtime bump and swaps the cert live — no restart. A publicly-trusted cert also keeps the WebAuthn secure-context clean (no browser warnings). Alternative: front the loopback admin listener with your existing mgmt reverse proxy and terminate the cert there; then set FLEET_MGR_ADMIN_INSECURE=false still but let the proxy hold the cert.

3. Start InfluxDB

docker compose up -d influxdb
docker compose ps    # wait for healthy

First start runs setup with the org/bucket/token/retention from .env.

4. Bootstrap the first admin (passkey)

Admin auth is closed — there is no password. Mint a one-time enrolment link and register your passkey in the browser:

docker compose run --rm fleet-manager register-admin "your-name"
# prints: https://diskless-server.bitasylum.org/#/enrol?token=...

The register-admin subcommand generates the shared enrol-HMAC key on first run (into the fleet-data volume at /data/enroll.key) and mints an HMAC-signed, single-use token (jti ledger prevents replay). Open the link on a machine with your YubiKey, tap + PIN, and the passkey is registered. Further admins can be enrolled from the UI once one exists.

5. Start the manager

docker compose up -d fleet-manager
docker compose logs -f fleet-manager

Healthy startup logs the four planes: admin HTTPS+passkey, agent gRPC/mTLS, authorise (broker), and the InfluxDB gateway. Any plane whose config is absent logs DISABLED and the rest still run.

Then browse to https://diskless-server.bitasylum.org/, log in with the passkey, and you should see Monitor / Authorise / Lifecycle.

Smoke test without hardware

To render the SPA against demo data with no passkey/PKI:

docker compose run --rm -e FLEET_MGR_ADMIN_INSECURE=true -e FLEET_MGR_SEED=true \
  -e FLEET_MGR_ADMIN_ADDR=0.0.0.0:9500 -p 9500:9500 fleet-manager
# http://localhost:9500/  (whoami returns "dev-insecure"; two seeded hosts)

FLEET_MGR_ADMIN_INSECURE=true is dev/lab only — plaintext, no passkey.

Backups

Back up the fleet-data volume (inventory.json, admins.json, enroll.key) and the influx-data volume. Losing admins.json locks every admin out — you'd re-bootstrap with register-admin. Losing enroll.key invalidates any outstanding (unused) enrol links.

What is NOT yet wired (recorded intent)

Per-host boot-version assignment (desired.target_version) is stored and pushed to the agent, but does not yet change what a host boots: that needs dnsmasq cohort steering + multi-version UKI serving (docs/TODO.md #6). Kiosk URL / reboot / drain do reconcile via the agent today. The Lifecycle page labels the version pin as recorded intent.