Commit Graph
91 Commits
Author SHA1 Message Date
Audiobuzz 5bc6d27f77 Adding rsyslog tls 2026-07-12 14:04:19 +09:30
Audiobuzz 28d31e436c ssh-hostcert + enrol-broker: fix defects surfaced by the F-02 bench
F-02/P-17 SSH host-cert flow was proven end-to-end on comet-beacon-54
(2026-07-11); these fix the correctness/scoping/cleanup defects the bench
surfaced:

- ssh-hostcert: derive the FQDN principal <hostname>.<domain> from the DHCP
  search domain at runtime (resolv.conf), alongside the short hostname, so an
  admin `@cert-authority *.<domain>` line accepts by-FQDN logins instead of
  "name is not a listed principal". Never baked (invariant #1). + tests.
- ssh-ca-bootstrap.sh: export the SSH host CA public key in OpenSSH wire format
  (ssh-keygen -i -m PKCS8), not the PEM/PKCS8 that `step crypto keypair` writes
  -- a PEM @cert-authority anchor is unparseable and silently falls back to TOFU.
- ssh-ca-bootstrap.sh: fail loudly (and verify enableSSHCA=true) if the
  provisioner SSH-enable step fails, instead of swallowing it and shipping a CA
  that 502s every runtime sign.
- dot1x-bench: point the co-located broker at https://localhost:8443, not the
  fleet dnsmasq name ca.fleet.lab (unresolvable from the broker's host netns ->
  "no such host" at sign time). localhost is in the CA cert SAN.
- serving: remove the mock-signer enrol-broker copy -- two host-networked
  brokers collide on 9090/9443 and the mock issued untrusted certs. The real
  broker lives with step-ca (dot1x-bench); nginx /enrol/ still proxies to it on
  loopback.
- enrol-broker: retire the HTML admin console; the fleet-manager passkey SPA now
  drives the mTLS JSON API (/pending, /approve|deny). Removes a weaker,
  client-cert-only parallel human-approve path.

Design-doc bench note deferred: docs/ssh-host-certificate-design.md lives on the
secure-ntp branch (fbb8100), not here -- to be added there / on reconcile.
2026-07-11 09:44:33 +00:00
Audiobuzz 2e318fba00 adding ssh host signing 2026-07-11 16:20:21 +09:30
Audiobuzz 3284a2713b docs: reflect chrony/uptime metrics, agent-cert renewer, and the chrony-access findings
Recent work added telemetry fields and a cert path not yet in the docs:
- files/agent/README.md: add uptime to the health list; document that chrony is
  read from /var/log/chrony/tracking.log (not the command socket) and needs BOTH
  the chrony supplementary group (DAC) and the fleet_agent_chronylog SELinux allow
  (TE), plus a reachable lab NTP (123/udp firewall); note the chrony group on the
  unit row.
- serving/fleet-manager/README.md: add uptime + chrony detail to the health list;
  note the cert-auto agent-cert-renewer sidecar + GetCertificate hot-reload
  (certreload.go) in the bring-up and checkpoint 7.
- docs/TODO.md #14: probe list gains uptime + chrony-from-tracking.log; new dated
  block records uptime/reboot-pending, the 4-layer chrony-access chase (all
  silent-failing; inventory.json is the isolator), and the agent-plane cert
  auto-renewal, all bench-verified 2026-07-10.
- docs/fleet-management-design.md: health record gains uptime + chrony
  source/stratum/offset detail.
2026-07-10 22:45:00 +00:00
Audiobuzz 8eb28c0a6b serving: document opening 123/udp on the host firewall for the NTP server
The ntp service is host-networked, so (like the audit collector's 514/tcp)
Docker doesn't open the port — the serving host's own ufw/firewalld governs
123/udp. Without a rule, workstations are handed 192.168.50.10 as their NTP
server via DHCP opt 42 but their packets are dropped: chrony logs 'Added source
… / unreachable' and never syncs (seen on comet-beacon-54). Add the boot-LAN
firewall rule to the ntp service NOTE, mirroring the audit-collector README.
2026-07-10 09:42:26 +00:00
Audiobuzz 1db3977238 fleet-manager: auto-renew the agent-plane server cert (step-ca sidecar + hot-reload)
The agent-plane server cert was issued by hand (server-pki.sh / docker exec
step ca certificate), which needs the step CLI on the host, re-specifies SANs
each time, and silently expires at the CA's 90-day cap — the exact failure that
just broke agent TLS (cert minted with a stale SAN, missing manager.fleet.lab).

Automate it:
- agent-cert-renewer sidecar (smallstep step-cli image, compose profile
  cert-auto): issues the cert once from the co-located fleet step-ca, then
  'step ca renew --daemon' rotates it before the cap. Renewal preserves the
  subject + SANs, so SAN drift cannot recur. Chowns to the manager's nonroot
  uid so the distroless manager reads the key. Opt-in + only valid where the CA
  is co-located; server-pki.sh stays as the manual/remote-CA path.
- manager hot-reloads via GetCertificate (certreload.go): a rotated cert on
  disk is picked up on the next handshake with no restart and no dropped agent
  streams; a half-written pair keeps the last good cert serving. Unit-tested.
- defaults.env documents the cert-auto knobs; DEPLOY.md §2a recommends the
  renewer and flags the mandatory manager.fleet.lab SAN.
2026-07-10 07:42:18 +00:00
Audiobuzz 1c68c37824 fleet-manager: chrony tracking metrics relative to the NTP source
The agent only reported chrony as synced|unsynced — nothing about how well the
clock tracks. Parse `chronyc tracking` for the source-relative metrics.

- proto: HealthSnapshot.chrony_offset_seconds (16, signed system-time offset,
  fast=+), chrony_source (17, reference name/IP), chrony_stratum (18); regenerated.
- agent: replace chronyState() with chronyTracking()/parseChrony() — one exec,
  parses Leap status + Reference ID + Stratum + System time offset (+ test).
- manager: Health.{ChronyOffset,ChronySource,ChronyStratum}; snapshotToHealth maps
  them; influx writes chrony_offset_seconds/_stratum/_source and whitelists the
  offset + stratum as graphable fields.
- UI: chronySummary() + offsetMs() — the chrony row now shows
  "synced · 192.168.50.10 · s3 · +0.01ms" instead of just "synced".
2026-07-10 06:47:27 +00:00
Audiobuzz df7dad81b7 fleet-manager: host uptime metric + reboot-pending status
The manager lets an admin schedule a reboot (desired.reboot_at) but never showed
whether it had happened. Add host uptime to the health snapshot and derive a
reboot status from it.

- proto: HealthSnapshot.uptime_seconds (15); regenerated pb.go (manager + agent copy).
- agent: probe /proc/uptime -> uptime_seconds (with parser + test).
- manager: Health.UptimeSeconds cached in inventory; snapshotToHealth maps it;
  influx writes + whitelists uptime_seconds as a queryable field.
- UI: fmt.js dur() + rebootStatus() — boot_time = last_seen - uptime (manager clock
  frame, no host/manager sync needed), compared to reboot_at. Monitor host detail
  shows an "uptime" row and a "reboot" row: pending in X / overdue by X / rebooted.

Observability-only — works before the reboot ACTUATOR lands (deferred). Once a host
boots after the scheduled time (BMC/manual now, agent later), the badge flips to
"rebooted" automatically.
2026-07-10 06:41:09 +00:00
Audiobuzz e6ae8c4b23 fleet-manager UI: auto-refresh the selected host's metric graphs
The Monitor tab polled /api/hosts every 10s (roster + host-detail refresh) but
only fetched a host's metric series on click, so the tmpfs/cpu/temp graphs went
stale until you re-selected. Extract loadMetrics() and call it from the same 10s
tick when a host is selected, so the graphs stay live.
2026-07-10 06:22:33 +00:00
Audiobuzz d85a1e5b23 serving: defaults.env header + Makefile + .gitignore + doc refs
Completes the rename commit (which carried only the file move): defaults.env
layering header, docker-compose usage comment, serving/Makefile (sets
COMPOSE_ENV_FILES), serving/.gitignore (.env), and the README/TODO references.
2026-07-10 06:19:11 +00:00
Audiobuzz cbc679c5bd serving: rename .env.example -> defaults.env (committed defaults + .env overrides)
Same layering as the fleet-manager stack: defaults.env holds the committed
defaults, .env holds only per-LAN overrides + secrets, layered via
COMPOSE_ENV_FILES=defaults.env,.env. Changing a default here reaches every run
without re-copying.

- git mv serving/.env.example -> serving/defaults.env; header explains the layering.
- docker-compose.yml usage comment -> the COMPOSE_ENV_FILES / Makefile flow.
- serving/Makefile: sets COMPOSE_ENV_FILES; help/up/down/restart/pull/logs/ps/config
  with a clear .env-missing guard (mirrors serving/fleet-manager/Makefile).
- serving/.gitignore: ignore .env (defaults.env is committed).
- README + TODO references updated. serving/siem/.env.example is a separate
  wazuh1-node file, left as-is.
2026-07-10 06:18:15 +00:00
Audiobuzz 1b8cf2f344 agent: dial the manager by a fleet DNS name (manager.fleet.lab)
The agent's baked default pointed at the PUBLIC admin-UI name
(diskless-server.bitasylum.org), which resolves to the serving host's outbound IP
— unreachable from the fleet LAN. There's no split-horizon to "automate": the
manager agent plane is just another *.fleet.lab, served by the fleet dnsmasq like
boot/nvme/audit.

- serving/.env.example: MGMT_HOST=manager.fleet.lab.
- serving/docker-compose.yml: dnsmasq --address=/${MGMT_HOST}/${SERVER_IP}.
- files/agent/agent.env + main.go default: FLEET_AGENT_MANAGER=manager.fleet.lab:9600.
- server-pki.sh: AGENT_SANS default -> manager.fleet.lab (cert SAN matches the
  baked name, so no FLEET_AGENT_SERVER_NAME override).
- docs/TODO #14 gotcha #4 updated: fleet name, not the IP+server-name workaround.

The public diskless-server.bitasylum.org stays the ADMIN-UI name (Traefik/:443).
2026-07-10 06:13:46 +00:00
Audiobuzz a89c3d2af1 fleet-manager: fix CA install, add register-host, surface agent status
Four fixes from the on-host agent bring-up:

1. Bake the real fleet CA into the image. New `fleet_ca` build var (default =
   the committed placeholder) file-provisions the trust anchor to
   /etc/dot1x/radius-ca.pem — used by wpa_supplicant (RADIUS) AND the fleet-agent
   (verify the manager). CI materializes a FLEET_CA secret to a runner-temp file
   and passes -var fleet_ca; unset => placeholder (lab). No CA material committed.

2. server-pki.sh installs the agent CLIENT CA as root+intermediate (not root
   only). Host TPM certs are leaf-only in NV and signed by the step-ca
   intermediate, so a root-only client CA makes the manager advertise only the
   root in its CertificateRequest -> Go clients send no cert ("tls: certificate
   required") and it couldn't verify a leaf-only cert anyway. bootstrap.sh now
   exports intermediate_ca.crt so FLEET_CA_INTERMEDIATE resolves.

3. New `fleet-manager register-host <serial> [-name] [-hostnqn]` subcommand:
   seeds/updates an inventory record for a host enrolled out of band (bench
   broker, migrated fleet) through the real store, so its agent stream is
   accepted. The manual counterpart to the authorise-approve flow.

4. The agent surfaces the authoritative recv-side gRPC status instead of the
   bare io.EOF gRPC returns on the send side when the server ends the stream —
   so "host is not enrolled" (PermissionDenied) shows up instead of "EOF".
2026-07-10 05:51:57 +00:00
Audiobuzz 8ab11a74c5 fleet-manager: add Makefile that sets COMPOSE_ENV_FILES for you
`make up` / `logs` / `restart` / `pull` / `build` / `config` / `register-admin`
/ `backup` — each exports COMPOSE_ENV_FILES=defaults.env,.env so plain `make ...`
layers the committed defaults under .env with no manual export or --env-file.
Guards on .env existing (COMPOSE_ENV_FILES requires it) with a clear message.
`make help` lists targets. DEPLOY §1 points at it.
2026-07-05 02:14:22 +00:00
Audiobuzz 45d7101472 fleet-manager: layer config as defaults.env (committed) + .env (overrides)
Rename .env.example -> defaults.env and make it the committed defaults layer.
Compose reads defaults.env then .env (later wins) via COMPOSE_ENV_FILES, feeding
BOTH container env and ${...} interpolation — so a changed default here reaches
every run without re-copying, and .env shrinks to just overrides + secrets.

No compose service restructure: the existing interpolated environment: blocks
already build the container env; the layering just supplies their values from two
files. (env_file: directive alone wouldn't work — it doesn't feed the image-tag /
cross-service Influx / ACME_EMAIL interpolation this compose relies on.)

Docs (DEPLOY §1, compose header, README) updated: export
COMPOSE_ENV_FILES=defaults.env,.env, keep only secrets/overrides in .env.
2026-07-05 02:08:25 +00:00
Audiobuzz 688ff1aa88 fleet-manager: mirror the distroless runtime base through Harbor
Route the gcr.io distroless runtime image through the Harbor mirror
(harbor.bitasylum.org/gcr.io/distroless/static-debian12:nonroot) per the standing
rule, completing the fleet-manager Dockerfile's base-image mirroring.
2026-07-05 01:47:28 +00:00
Audiobuzz e6b1f0308e fleet-manager: finish self-TLS cleanup (compose, env, docs, delete acme/)
Follow the code removal through the rest of the tree now that Traefik owns UI TLS:
- compose: drop FLEET_MGR_ADMIN_BEHIND_PROXY + ADMIN_CERT/_KEY from the manager
  env; correct the ./pki mount comment (agent + broker certs only).
- .env.example: drop the BEHIND_PROXY + cert vars + self-TLS note.
- DEPLOY.md / README: remove the self-TLS "alternative" and the BEHIND_PROXY /
  ADMIN_CERT config rows.
- server-pki.sh: point the admin UI cert at Traefik, not ./pki/admin.
- delete serving/fleet-manager/acme/ (the retired ACME side-car).
2026-07-05 01:46:52 +00:00
Audiobuzz 76c3a6aaa7 fleet-manager: drop self-TLS admin mode — the manager never terminates TLS
Traefik (and the k8s Ingress later) owns UI TLS, so remove the vestigial
self-TLS path: adminTLS(), the certReloader type, the adminCert/adminKey config
+ FLEET_MGR_ADMIN_CERT/_KEY env, and the now-redundant adminBehindProxy flag.
The admin plane always serves plaintext now — two modes: INSECURE (dev, no auth)
or default (passkey behind a trusted TLS proxy). Drops the crypto/tls, fmt, sync
imports and the errString helper.
2026-07-05 01:44:23 +00:00
Audiobuzz 24d67d07db serving: mirror the remaining base/service images through Harbor
Route the pre-existing non-CI image pulls through harbor.bitasylum.org per the
standing rule (harbor.bitasylum.org/<registry>/<namespace>/<image>):
- enrol-broker/Dockerfile: golang:1.23-alpine -> dockerhub/library/...,
  smallstep/step-cli:latest -> dockerhub/smallstep/...
- dot1x-bench/docker-compose.yml: smallstep/step-ca + freeradius/freeradius-server
  -> dockerhub/<namespace>/...
CI-built git.bitasylum.org/audiobuzz/* images and ${OCI_IMAGE}/${IPXE_IMAGE}
type:image volume mounts are unchanged.
2026-07-05 01:34:10 +00:00
Audiobuzz 08847532ee fleet-manager: pull non-CI images through the Harbor mirror
Route the ad-hoc images I introduced through harbor.bitasylum.org (the standing
rule): backup.sh uses harbor.bitasylum.org/dockerhub/library/alpine:3.20 (via an
overridable ALPINE_IMAGE); the DEPLOY.md volume-chown fix + the goacme/lego prose
use the Harbor paths too. Compose (influxdb/traefik) + Dockerfile bases are already
harbor-mirrored; git.bitasylum.org/audiobuzz/* stays (CI-built).
2026-07-05 01:28:06 +00:00
Audiobuzz 46d54798b7 fleet-manager: docs for the Traefik ingress + behind-proxy mode
- .env.example: RP_ORIGIN port-less, FLEET_MGR_ADMIN_BEHIND_PROXY=true, cert vars
  labelled self-TLS-only; ACME vars are now Traefik's (dropped sidecar-only ones).
- DEPLOY.md §2c rewritten for Traefik (443 + Route 53 DNS-01), plus a "Network
  paths" table answering how each plane is reached (enrolment via boot nginx,
  agent direct mTLS via OPNsense L3, admin via Traefik). Start sequence + hardening
  status updated (admin-exposure item resolved).
- README: admin transport = Traefik; behind-proxy explained; config table gains
  BEHIND_PROXY; checkpoint 8 (hardening + Traefik ingress).
2026-07-05 01:00:51 +00:00
Audiobuzz 4c1db3f765 fleet-manager: front the admin UI with Traefik (443 + Route 53 ACME), retire acme side-car
Management-plane ingress: a host-networked Traefik terminates the LE cert on :443
(Route 53 DNS-01, public-resolver propagation check) and forwards plaintext to
the manager running FLEET_MGR_ADMIN_BEHIND_PROXY=true on loopback :9500. Only the
admin plane rides Traefik — the agent gRPC/mTLS plane (:9600) stays direct (Traefik
can't terminate fleet-CA mTLS; hosts reach it via OPNsense L3 routing). This is the
k8s ingress-controller + cert-manager role in miniature.

- traefik/dynamic/fleet-manager.yml: file-provider router (Host rule -> manager),
  passHostHeader so the WebAuthn RP origin matches.
- compose: traefik service (static config as CLI flags so ${ACME_EMAIL} interpolates)
  + traefik-acme volume; manager switched to behind-proxy defaults, RP_ORIGIN now
  the port-less https origin.
- Retire the acme side-car (Traefik owns ACME now): remove the service + acme-state
  volume; renew-loop.sh kept + relabelled as the self-TLS-mode cert helper.
- :443 only (admin UI is https-only) so it can't collide with a boot-LAN nginx :80.
2026-07-05 00:57:04 +00:00
Audiobuzz 20acdb0254 fleet-manager: add behind-proxy admin mode (passkey over plaintext behind a TLS proxy)
Decouples "passkey auth ON" from "the listener terminates TLS". A new
FLEET_MGR_ADMIN_BEHIND_PROXY=true keeps WebAuthn/session auth fully on but serves
plaintext, for running behind a trusted TLS-terminating ingress (Traefik now, a
k8s Ingress later) that owns the LE cert. WebAuthn only needs RP_ORIGIN to match
the browser's HTTPS origin, not our transport; session cookies stay Secure. The
listener must be bound private (loopback/mgmt) so only the proxy reaches it —
logged as a warning. Insecure (dev, no auth) and self-TLS (default) modes
unchanged.
2026-07-05 00:52:19 +00:00
Audiobuzz 3140d51d95 fleet-manager: document hardening status + remaining per-env items
DEPLOY.md gains a "Hardening status" section (done vs operator-to-close), and
TODO #14 records the deployment as CI-built + acme-cert + backup, with the open
items called out: network-segment the :9500/:9600 listeners to the mgmt net, run
server-pki.sh when the agent lands, choose :9500 vs a 443 proxy, and digest-pin
the images.
2026-07-05 00:36:01 +00:00
Audiobuzz f72cea6b17 fleet-manager: add backup.sh for the durable-state volumes
Snapshot fleet-data (inventory + admins.json + enroll.key — the admin-lockout
risk) and influx-data to timestamped tarballs via a throwaway alpine container
(read-only, live-safe), with BACKUP_KEEP retention. Documented in DEPLOY.md
Backups with a restore pointer.
2026-07-05 00:34:23 +00:00
Audiobuzz 487c62c53f fleet-manager: run the CI-built image (FLEET_MANAGER_VERSION), keep local build
Point the compose service at the registry image the new CI workflow pushes
(git.bitasylum.org/audiobuzz/fleet-manager:${FLEET_MANAGER_VERSION:-latest}) so
mgmt hosts `docker compose pull` instead of building at deploy time — matching
dnsmasq/spdk/kiosk. `build: .` stays for local dev. Add FLEET_MANAGER_VERSION to
.env.example.
2026-07-05 00:32:58 +00:00
Audiobuzz cf86e615ea fleet-manager: go fix — strings.Split -> strings.SplitSeq in newWebAuthn
`go fix` modernised the comma-origin loop to the Go 1.24+ iterator idiom
(strings.SplitSeq). No behaviour change.
2026-07-05 00:26:41 +00:00
Audiobuzz 2b3708c718 fleet-manager: register-admin link uses the first of multiple RP origins
Follow-up to comma-separated RP_ORIGIN support: the enrolment URL builder used
the raw origin string, which would embed the whole comma list. Take the first
origin for the link.
2026-07-05 00:24:10 +00:00
Audiobuzz 76446edd75 fleet-manager: allow multiple WebAuthn origins (comma-separated RP_ORIGIN)
Passkeys bind to the exact scheme://host:port the browser used, so a single
configured origin fails "Error validating origin" when reached via a different
form (e.g. :9500 direct vs a 443 proxy). Split FLEET_MGR_RP_ORIGIN on commas so
several exact origins can be allowed. Single-value config is unchanged.
2026-07-04 11:07:15 +00:00
Audiobuzz 643ba6be2d fleet-manager: actionable error when the enrol key is missing at startup
Flipping FLEET_MGR_ADMIN_INSECURE=false before bootstrapping an admin crash-looped
on a cryptic "enrol key: open /data/enroll.key: no such file or directory". The
key is created by `register-admin`, so on ErrNotExist say exactly that (and note
the insecure escape hatch) instead of surfacing the raw errno.
2026-07-04 10:55:24 +00:00
Audiobuzz 693cf94695 fleet-manager: point acme DNS-01 propagation check at a public resolver
Split-horizon DNS made lego's propagation check query the container's internal
resolver, which doesn't serve the public _acme-challenge TXT record, so the
challenge timed out. Add ACME_RESOLVERS (comma-separated host:port, default
8.8.8.8:53) → one --dns.resolvers flag each. Documented in .env + DEPLOY.md.
2026-07-04 10:24:42 +00:00
Audiobuzz d31c0f18eb fleet-manager: locate lego at /lego in the acme side-car
The goacme/lego image ships the binary at /lego, not on PATH, so the bare `lego`
call failed ("not found") once the entrypoint was overridden with the shell.
Detect it (command -v, then /lego fallback) and invoke via $LEGO.
2026-07-04 10:20:48 +00:00
Audiobuzz 510f4f6b25 fleet-manager: pass optional AWS_HOSTED_ZONE_ID to the acme side-car
lego auto-discovers the Route 53 zone via ListHostedZonesByName, so the zone ID
is optional — but pinning it disambiguates overlapping public/private zones and
lets the IAM policy drop the list permission. Thread it through compose + .env
+ DEPLOY.md (no script change; lego reads it from the environment).
2026-07-04 10:14:51 +00:00
Audiobuzz 9479b70b04 fleet-manager: add acme side-car for the admin-UI cert (LE DNS-01 via Route 53)
Auto-issue + auto-renew the admin-UI TLS cert with a long-running compose
side-car instead of a host cron. The `acme` service (goacme/lego image, running
acme/renew-loop.sh) obtains the cert over DNS-01 via the AWS Route 53 API,
writes fullchain/privkey into ./pki/admin/ — the SAME dir the manager reads
(./pki:/pki:ro) — and chowns them to the nonroot runtime uid so the read-only
container can load the key. The manager's certReloader hot-swaps on the mtime
bump, no restart. Loop renews within 30 days of expiry; ACME_STAGING=1 gives a
dry-run vs LE staging.

- docker-compose.yml: `acme` service + `acme-state` volume (lego account/cert
  store). ACME_DOMAIN reuses FLEET_MGR_RP_ID so cert host == passkey origin.
- .env.example: ACME_EMAIL + AWS creds (or instance role) + ACME_STAGING.
- DEPLOY.md §2c: rewritten to the side-car flow — scoped Route 53 IAM policy,
  bring-acme-up-first ordering, flip off insecure, and an origin-match note.

(The docker-compose.yml commit also carries the local harbor.bitasylum.org
influxdb image mirror edit.)
2026-07-04 10:13:51 +00:00
Audiobuzz bf546a80a6 fleet-manager ui: render a never-reported host's last_seen as "never"
A just-approved inventory record has a zero-value last_seen (Go 0001-01-01),
which parsed to a pre-epoch instant and slipped past the !t guard, rendering as
"739800d ago". Treat any at-or-before-epoch timestamp as "never" — the honest
state for a born-but-never-connected host (its health/version stay "—" until an
agent dials the gRPC plane).
2026-07-04 10:03:28 +00:00
Audiobuzz 0360c1c6d1 fleet-manager: fix nonroot data-volume ownership + document broker wiring
The distroless nonroot runtime (uid 65532) couldn't write /data/inventory.json
because a named volume is created root-owned — approve failed with permission
denied on the inventory birth. Pre-create /data owned by nonroot in the image so
a fresh volume inherits it via Docker copy-up; an already-created root-owned
volume still needs a one-time chown (documented).

DEPLOY.md:
- 2b now covers BOTH broker deployments — the main serving stack
  (enrol-pki/broker, ENROL_ADMIN_INSECURE=true by default → must flip to false)
  and the bench stack — with the mint-once-into-broker-dir, copy-CA-to-manager
  flow and an mTLS verify curl.
- Troubleshooting section: the data-volume permission-denied fix, the
  "authorise plane not configured" (recreate + single-instance) fix, and the
  "wrong version number" (broker plaintext) fix.
2026-07-04 09:56:09 +00:00
Audiobuzz 014f43cb5a fleet-manager: log a positive line when the authorise pillar enables
Every plane logged its state EXCEPT the broker success path, which was silent —
so "authorise pillar working" was indistinguishable from "not logged". Add an
ENABLED line matching the InfluxDB/agent/admin convention.
2026-07-04 09:25:10 +00:00
Audiobuzz ea29d0d12e fleet-manager: soft-disable optional planes when their PKI isn't staged yet
The compose wires the authorise (broker) and agent planes to fixed cert paths,
so a first bring-up that hasn't run the PKI steps yet crash-looped on the first
missing file (e.g. /pki/broker/clients/fleet-manager.crt) instead of booting.

Treat a configured-but-absent cert as "plane not yet provisioned": log a loud
WARNING and disable that plane, so the manager boots and each plane lights up
as its PKI appears. A malformed/unreadable cert still hard-fails (real
misconfig). The admin plane stays a hard requirement (FLEET_MGR_ADMIN_INSECURE
is its bench escape). Adds a missingFiles() helper; hub push path already
nil-guards a disabled agent plane.
2026-07-04 08:58:38 +00:00
Audiobuzz 2ef0c1747d fleet-manager: checkpoint 7 — deployment (compose, PKI, deploy runbook, design-doc updates)
Complete the deployment story for the server-side control plane:

- docker-compose.yml: private InfluxDB (published to loopback only) + a
  host-networked fleet-manager (reaches the loopback enrol-broker on
  127.0.0.1:9443 and binds a routable gRPC agent listener, like the broker).
- .env.example: RP origin (diskless-server.bitasylum.org), Influx token/DB
  password, cert/PKI paths; the Influx token wires influxdb + manager with
  one value.
- server-pki.sh: issue the agent-plane server cert from the fleet CA (step)
  and stage the fleet root as the client-verify CA.
- DEPLOY.md: end-to-end runbook — three-cert PKI (fleet-CA agent cert, broker
  admin-client cert, Let's Encrypt DNS-01 UI cert), register-admin passkey
  bootstrap, compose up, hardware-free smoke test, backups.
- .dockerignore: keep host node_modules/dist/pki/secrets out of the build
  context so the embedded SPA is the one built in-image.
- .gitignore: never commit .env / issued PKI / data.

Design docs marked as-built: fleet-management-design.md as-built note
(resolved forks — gRPC bidi transport, InfluxDB metrics, SPA graphs, passkey
admin auth, broker fold-in "alongside"); tpm-enrolment-design.md (broker
fold-in + agent transport fork resolved); TODO.md #14 → partly built, with
the on-host agent + per-host boot-version assignment (#6) called out as the
remaining work.
2026-07-04 08:44:31 +00:00
Audiobuzz f62c668ce3 fleet-manager: add Dockerfile (checkpoint 7 start)
Two-stage build: node builds the Svelte SPA to dist, then golang compiles the
binary with the SPA embedded (-tags embedspa). Committed .pb.go stubs mean no
protoc. Ships as a small static distroless/nonroot image; same binary serves the
register-admin subcommand. (Not build-verified here — no docker in the
devcontainer — but the go build -tags embedspa + npm build are both proven.)
2026-07-04 06:19:21 +00:00
Audiobuzz 12c4296f9d fleet-manager ui: label target-version pin as recorded intent (not yet enforced)
Per the option-A decision (deploy current design without per-host versioning):
add an honest note on the Lifecycle panel that kiosk-url/reboot/drain reconcile
via the agent but per-host boot-version assignment (dnsmasq cohort steering) is
not yet wired, so a version pin is stored + pushed but does not change what the
host boots. Avoids implying the pin takes effect.
2026-07-04 06:19:21 +00:00
Audiobuzz f71a55eab5 fleet-manager ui: target version as a dropdown (fleet-known + custom)
Replace the free-text target-version field with a select populated from versions
known across the fleet (each host's running + targeted version, newest first),
plus a 'custom...' escape to a text input for a version not yet seen. Avoids
typos on the pin. (Authoritative serving-side source is a follow-up.)
2026-07-04 06:08:15 +00:00
Audiobuzz 20b29d9990 fleet-manager ui: readability + reboot-now
- Contrast: lighten the muted/faint text tokens (AA on the hull panels) and make
  Monitor host-card data values bright ink against dim faint labels, so version/
  tmpfs/seen details read clearly on the blue-grey panels (was grey-on-grey).
- Lifecycle: reboot selector now offers none | now | at... — 'now' sets
  reboot_at to the current time (agent reboots on reconcile); still confirm-gated
  as a high-consequence action.
2026-07-04 06:05:29 +00:00
Audiobuzz fae3624c58 fleet-manager: checkpoint 6 — the admin SPA (fleet command deck)
The themed single-page admin UI (serving/fleet-manager/ui/), embedded into the
binary and served same-origin over the admin plane.

- Svelte 5 + Tailwind 4 + Vite; a sibling of serving/kiosk-app. Theme = LYSANDRA
  'restricted-violet' (high-security evolution of the kiosk's flight-deck DNA):
  amber deck, VIOLET reserved for privileged/destructive + auth surfaces, teal
  telemetry, red alert, clearance chrome (corner-reticle panels, scanlines, mono
  clearance labels, secure-link status bar).
- Passkey gate: Login (usernameless discoverable WebAuthn) + Enrol (from the
  register-admin one-time link) via navigator.credentials, with base64url<->buffer
  helpers and the go-webauthn response shapes.
- Three pillars over the JSON API: Monitor (fleet roster + version-drift + health
  + per-host detail with bespoke SVG telemetry charts), Authorise (pending queue:
  fingerprint cross-check, approve+bind in violet, deny), Lifecycle (desired-state
  form; drain/reboot confirm-gated). Session-aware fetch client (cookie + CSRF).
- go:embed behind the  build tag (spa_embed.go / spa_stub.go) so the
  default go build/test need no dist/; mounted at / (api/auth/healthz win).

Verified: npm build clean (71KB JS / 33KB CSS + self-hosted fonts); default +
-tags embedspa go builds; smoke test serves index + assets + API through the SPA
origin. Against an insecure+seeded backend the deck renders passkey-free for
theme work. dist/ + node_modules gitignored.
2026-07-04 05:54:41 +00:00
Audiobuzz c40dd5ac93 fleet-manager: checkpoint 5 — admin WebAuthn/passkey auth (replaces browser mTLS)
The human->UI hop now authenticates with a hardware passkey (YubiKey), not an
mTLS client cert: phishing-resistant, origin-bound, better UX. (manager->broker
stays mTLS — service-to-service.)

- webauthn.go: credential store (registered admins + passkeys + a one-time-token
  jti replay ledger; JSON file, atomic writes) + the webauthn.User impl + RP setup.
- cli.go: 'fleet-manager register-admin <name>' mints a short-lived HMAC-signed
  enrolment link over a shared enrol key; the browser completes the WebAuthn
  create ceremony. No anyone-can-register endpoint; single-use via the ledger.
- authapi.go: register/login ceremonies (usernameless discoverable login,
  userVerification required) + logout + whoami; short-lived in-memory ceremony
  store between begin/finish.
- session.go: opaque server-side sessions (Secure/HttpOnly/SameSite=Strict cookie)
  + requireSession middleware gating /api/*, with a CSRF header check on mutations.
  adminName now comes from the session (audit attribution).
- main.go: register-admin subcommand dispatch; admin TLS switched to SERVER-AUTH
  (no client cert) with a HOT-RELOADING cert (certReloader) so a Let's Encrypt
  DNS-01 renewal is picked up without a restart. RP = diskless-server.bitasylum.org.
  /api/* mounted behind requireSession; /auth/* + /healthz public.

Cert strategy (per design decision): the UI HTTPS cert is a publicly-trusted LE
cert via DNS-01 (mgmt-network UI isn't internet-reachable, so HTTP-01/TLS-ALPN-01
can't validate; DNS-01 proves control of bitasylum.org via TXT + supports
wildcards). External ACME client issues+renews to the cert path; manager reloads.

Deps: github.com/go-webauthn/webauthn (server-side). Tests: enrol-token
mint/verify/tamper/expiry, session middleware (401/403/CSRF/expiry), cred-store
persistence + jti ledger. build/test/vet/gofmt clean. Live passkey ceremony
(virtual authenticator / real YubiKey) is a bench item.
2026-07-04 04:26:30 +00:00
Audiobuzz 9c86cd4c5a fleet-manager: checkpoint 4 — InfluxDB metrics gateway (write + Flux read)
The manager is the ONLY thing that talks to InfluxDB, both directions; hosts hold
no token and never reach it.

- influx.go: influxSink implements MetricsSink — each agent HealthSnapshot is
  written as a fleet_health point, identity (serial/hostnqn/starship) as TAGS
  from the AUTHENTICATED stream (a host can only write its own series), telemetry
  as fields. QueryField answers the SPA's read: a Flux query built with a strict
  FIELD WHITELIST + hex-serial guard + parsed-duration range, so no caller input
  is interpolated raw (injection-safe). Server-side influxdb-client-go/v2.
- admin.go: GET /api/hosts/{serial}/metrics?field=&range= -> time-series JSON;
  503 when InfluxDB isn't wired, 400 on a bad range, 404 for unknown host.
- main.go: sink created when FLEET_MGR_INFLUX_URL+_TOKEN set, fed to the agent
  hub; nil -> no-op sink so the manager runs without metrics.

Tests: snapshotPoint tags/fields/time, fluxQuery contents + %q quoting, and
QueryField rejects a non-hex serial + non-whitelisted field. build/test/vet/gofmt
clean. Live InfluxDB path validated in the compose bench (ckpt 6).
2026-07-04 03:59:18 +00:00
Audiobuzz df4fa95bdd fleet-manager: checkpoint 3 — authorise pillar (broker fold-in, births inventory)
The AUTHORISE pillar: the manager owns the enrolment queue UX, the enrol-broker
stays the state machine + step-ca signer (v1 'alongside' fold-in).

enrol-broker (minimal, backward-compatible change):
- admin approve/deny now return the decided Enrolment as JSON when the caller
  sends Accept: application/json. Browsers still redirect; other API callers
  (verify-serving.sh) still get 204. The cert PEM stays unexported (only the
  requesting host gets it via /status) — the JSON carries cert_serial, hostnqn,
  context, fingerprint, cn.

fleet-manager:
- broker.go: mTLS client for the broker admin plane (Pending/Approve/Deny),
  nil when unconfigured so the manager runs standalone.
- admin.go: GET /api/enrol/pending (proxy), POST /api/enrol/{id}/approve
  (drives broker, then BIRTHS the inventory record — identity from the returned
  serial/hostnqn/context, binding from the operator's optional body, enrolled
  with the HUMAN approver = the manager's admin mTLS CN), POST .../deny.
  503 when the broker isn't wired.

Test: httptest mock broker asserts the Accept:application/json approve, pending
proxy, inventory birth (identity+binding+enrolled), and the disabled-503 path.
Broker + manager both build/vet/test/gofmt clean.
2026-07-04 03:51:26 +00:00
Audiobuzz 80cbb833cc fleet-manager: checkpoint 2 — gRPC agent plane (bidi mTLS stream)
The steady-state host<->manager transport, resolving the design's OPEN FORK to
bespoke gRPC (not OTLP+Wazuh). One long-lived, host-dialed, mTLS bidi stream:
health/metrics up, desired-state/commands down — no polling, real-time control,
still no inbound host port.

- proto/fleet_agent.proto: FleetAgent.Session(stream AgentUp) -> stream ManagerDown.
  AgentUp = Hello | HealthSnapshot | CommandResult; ManagerDown = DesiredState |
  Command(reboot/drain/resync). int64 unix times (no well-known-type dep).
  Generated stubs committed (CI needs no protoc).
- agent.go: gRPC server with mTLS verified by the FLEET CA (the host's reused TPM
  identity cert — distinct from the admin plane's admin CA). Identity is the peer
  cert SERIAL, taken from the verified cert = inventory key, never self-asserted;
  an unenrolled serial is refused (enrolment gates the stream). Per-conn send
  channel decouples admin pushes from stream.Send. Stream registry (serial->conn)
  with reconnect eviction. On connect + on change, desired-state is pushed live;
  inventory stays authoritative so offline hosts reconcile on reconnect. Metrics
  go through a MetricsSink seam (no-op now; InfluxDB in ckpt 4).
- main.go: agent plane starts only when its cert/key/CA are set; admin can run
  standalone. Lifecycle PUT now live-pushes and reports pushed_live.

Full mTLS integration test (in-process fleet CA + host client cert): initial
desired-state on connect, Hello+snapshot -> inventory + online presence, live
desired-state push, and unenrolled-serial rejection. go build/test/vet/gofmt clean.

Deps (server-side only): google.golang.org/grpc + protobuf.
2026-07-04 03:40:55 +00:00
Audiobuzz 0572d21c07 fleet-manager: drop accidentally-committed binary + add .gitignore
The checkpoint-1 commit included the 9.1M compiled 'fleet-manager' binary. Remove
it from tracking and gitignore it (/fleet-manager + *.test), matching the
enrol-broker convention. Build artifacts don't belong in git.
2026-07-04 03:26:44 +00:00
Audiobuzz 915a15b6a1 fleet-manager: checkpoint 1 — service skeleton + inventory store + admin API
First slice of the server-side fleet control plane (serving/fleet-manager/), the
backend for the admin SPA (monitor / authorise / lifecycle). See
docs/fleet-management-design.md + tpm-enrolment-design.md §5.

This checkpoint (stdlib-only):
- Inventory record (identity/binding/enrolled/version/health/usage/desired),
  keyed by TPM cert serial = hostnqn key, per the design's 'inventory record born
  at enrolment'. JSON-file Store (atomic writes, mutex) behind an interface so a
  DB swap is a drop-in.
- Admin plane: mTLS server verified by the DEDICATED admin CA (never the fleet CA,
  so a host cert can't reach admin) + a plaintext dev mode. JSON API:
  GET /api/hosts, GET /api/hosts/{serial}, PUT /api/hosts/{serial}/desired
  (desired-state audited to the admin client-cert CN).
- Dev seed (FLEET_MGR_SEED) so the SPA has data before real enrolment/agent flow.

Smoke-tested: list/get/desired-PUT (incl. version-drift + drain), 404s, atomic
persistence. go build + go vet + gofmt clean.

Agent gRPC plane, broker-fold-in authorise, InfluxDB metrics, and the themed SPA
follow in checkpoints 2-6.
2026-07-04 03:25:54 +00:00