sunjizzle/diskless-workstation
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fleet-manager: docs for the Traefik ingress + behind-proxy mode

Browse Source 
- .env.example: RP_ORIGIN port-less, FLEET_MGR_ADMIN_BEHIND_PROXY=true, cert vars
  labelled self-TLS-only; ACME vars are now Traefik's (dropped sidecar-only ones).
- DEPLOY.md §2c rewritten for Traefik (443 + Route 53 DNS-01), plus a "Network
  paths" table answering how each plane is reached (enrolment via boot nginx,
  agent direct mTLS via OPNsense L3, admin via Traefik). Start sequence + hardening
  status updated (admin-exposure item resolved).
- README: admin transport = Traefik; behind-proxy explained; config table gains
  BEHIND_PROXY; checkpoint 8 (hardening + Traefik ingress).
This commit is contained in:
Audiobuzz
2026-07-05 01:00:51 +00:00
parent 4c1db3f765
commit 46d54798b7
3 changed files with 107 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serving/fleet-manager/DEPLOY.md
+70 -62
View File
@@ -93,19 +93,21 @@ curl -s --cert ./pki/broker/clients/fleet-manager.crt \
https://127.0.0.1:9443/pending ; echo # expect [] or null (empty queue)
```
### 2c. Admin UI cert — Let's Encrypt DNS-01 via Route 53 (the `acme` side-car)
### 2c. Admin UI ingress — Traefik on :443 (Let's Encrypt DNS-01 via Route 53)
The UI hostname resolves on the mgmt net and is **not internet-reachable**, so
HTTP-01/TLS-ALPN-01 can't validate. Use **DNS-01**, which proves control of the
zone via a TXT record. The `acme` compose service does this automatically: a
long-running side-car (`acme/renew-loop.sh`, on the `goacme/lego` image) issues
the cert on first start and renews within 30 days of expiry, writing it into
`./pki/admin/` — the same dir the manager reads — so the manager's `certReloader`
hot-swaps it live, no restart.
The admin UI is fronted by the **`traefik`** compose service: it terminates the LE
cert on :443 and forwards plaintext to the manager (behind-proxy mode, loopback
:9500). The UI hostname resolves on the mgmt net and is **not internet-reachable**,
so only **DNS-01** can validate — Traefik automates the `_acme-challenge` TXT
record via the AWS Route 53 API and auto-renews. **Only the admin plane rides
Traefik**; the agent gRPC/mTLS plane (:9600) is direct (see [Network paths](#network-paths)).
**IAM policy** — give the side-car AWS creds scoped to the `bitasylum.org` hosted
zone (put `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` in `.env`, or attach an
instance role and omit them):
**Set the UI hostname** in `traefik/dynamic/fleet-manager.yml` — the `Host()` rule
(file config isn't env-interpolated), matching `FLEET_MGR_RP_ID`.
**IAM policy** — AWS creds scoped to the `bitasylum.org` hosted zone (put
`AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` in `.env`, or attach an instance role
and omit them):
```json
{
@@ -120,49 +122,53 @@ instance role and omit them):
}
```
`AWS_HOSTED_ZONE_ID` is **optional**: lego auto-discovers the zone via
`ListHostedZonesByName` (hence that statement). Set it in `.env` to pin the zone
— required if a **public + private** zone share `bitasylum.org` (auto-discovery
may pick the wrong one), and it lets you **drop the `ListHostedZonesByName`
statement** for a tighter policy.
`AWS_HOSTED_ZONE_ID` is **optional**: the ACME client auto-discovers the zone via
`ListHostedZonesByName` (hence that statement). Set it (Traefik reads it from the
env) to pin the zone — required if a **public + private** zone share `bitasylum.org`,
and it lets you **drop the `ListHostedZonesByName` statement** for a tighter policy.
**Run it** (bring the side-car up first so the cert exists before the manager):
**Run it:**
```bash
# In .env: set ACME_EMAIL + the AWS creds above. ACME_DOMAIN is taken from
# FLEET_MGR_RP_ID. Optionally ACME_STAGING=1 for a first dry-run (untrusted, no
# rate limit) to prove Route 53 access, then re-run without it for a real cert.
docker compose up -d acme
docker compose logs -f acme # wait for "staged /out/{fullchain,privkey}.pem"
ls -l pki/admin/ # fullchain.pem + privkey.pem, owned by uid 65532
# .env: ACME_EMAIL + the AWS creds above. For a first DRY-RUN vs LE staging
# (untrusted, no rate limit), add the caserver flag to the traefik `command:` in
# docker-compose.yml, prove issuance, then remove it for the real cert.
docker compose up -d traefik
docker compose logs -f traefik # watch the DNS-01 challenge succeed + cert stored
```
Then turn off insecure mode and (re)start the manager — the defaults
`FLEET_MGR_ADMIN_CERT=/pki/admin/fullchain.pem` / `_KEY=/pki/admin/privkey.pem`
already point at what the side-car wrote:
Traefik runs the propagation check against **public** resolvers
(`8.8.8.8:53,1.1.1.1:53` in the command) — needed because split-horizon DNS won't
serve the public `_acme-challenge` record. The cert persists in the `traefik-acme`
volume (`acme.json`) and Traefik renews it automatically. A publicly-trusted cert
also keeps the WebAuthn secure-context clean.
```bash
# ensure FLEET_MGR_ADMIN_INSECURE=false in .env, then:
docker compose up -d --force-recreate fleet-manager
docker compose logs fleet-manager | grep -i listening # "admin (HTTPS + passkey) listening"
```
> **Origin match:** passkeys are origin-bound, so `FLEET_MGR_RP_ORIGIN` must equal
> the browser's origin. With Traefik on :443 that's the **port-less**
> `https://diskless-server.bitasylum.org`; `FLEET_MGR_RP_ID` is the host. The
> manager binds loopback (`127.0.0.1:9500`) — only Traefik reaches it; never expose
> that port directly.
A publicly-trusted cert also keeps the WebAuthn secure-context clean (no browser
warnings). The side-car keeps renewing on its 12-hour loop; the manager reloads
each new cert automatically.
> **Self-TLS alternative (no Traefik):** set `FLEET_MGR_ADMIN_BEHIND_PROXY=false`
> and give the manager a cert at `/pki/admin/{fullchain,privkey}.pem` (e.g. via the
> `acme/renew-loop.sh` helper under a `goacme/lego` container). The manager then
> terminates TLS itself and the origin includes the port it listens on.
> **Split-horizon DNS:** the side-car defaults `ACME_RESOLVERS=8.8.8.8:53` so
> lego checks TXT propagation against **public** DNS. Without it, lego uses the
> container's internal resolver, which doesn't serve the public
> `_acme-challenge` record, and the challenge times out. Override if 8.8.8.8 is
> blocked (e.g. `1.1.1.1:53`, comma-separated for several).
<a name="network-paths"></a>
### Network paths (three planes, three trust stages)
> **Origin match:** the cert CN/SAN (`FLEET_MGR_RP_ID`) must equal the origin the
> browser uses *and* `FLEET_MGR_RP_ORIGIN` — passkeys are origin-bound. The admin
> listener binds loopback (`127.0.0.1:9500`) by default; expose it as
> `https://diskless-server.bitasylum.org` either by binding a routable
> `FLEET_MGR_ADMIN_ADDR` or fronting it with a mgmt proxy that passes TLS through
> (don't re-terminate, or the passkey origin/cert won't match).
| Plane | When | From | Path | Auth |
|---|---|---|---|---|
| Enrolment (`/enrol/`) | pre-auth (no cert yet) | boot LAN | boot nginx passthrough → broker :9090 | none (the one unauth edge) |
| Agent (gRPC :9600) | post-auth (802.1X'd, holds TPM cert) | authenticated host VLAN | **direct** → manager :9600, via **OPNsense L3** (allow only host-VLAN→:9600) | **mTLS, TPM cert (fleet CA)** |
| Admin (UI :443) | anytime | mgmt/admin network | **Traefik** → manager :9500 | WebAuthn passkey |
The agent plane does **not** go through the boot nginx or Traefik: it connects
only after EAP-TLS/MACsec, and Traefik can't terminate the fleet-CA client-cert
mTLS. Segment it with an **OPNsense firewall rule** allowing only the authenticated
host VLAN → `manager:9600`. In the k8s target this becomes a dedicated TCP/TLS-
**passthrough** ingress (the pod still sees the client cert), separate from the
admin Ingress.
## 3. Start InfluxDB
@@ -189,16 +195,17 @@ single-use token (jti ledger prevents replay). Open the link on a machine with
your YubiKey, tap + PIN, and the passkey is registered. Further admins can be
enrolled from the UI once one exists.
## 5. Start the manager
## 5. Start the manager + ingress
```bash
docker compose up -d fleet-manager
docker compose up -d fleet-manager traefik
docker compose logs -f fleet-manager
```
Healthy startup logs the four planes: admin HTTPS+passkey, agent gRPC/mTLS,
authorise (broker), and the InfluxDB gateway. Any plane whose config is absent
logs `DISABLED` and the rest still run.
Healthy startup logs the four planes: admin (`passkey, behind TLS proxy` — plaintext
on loopback), agent gRPC/mTLS, authorise (broker), and the InfluxDB gateway. Any
plane whose config is absent logs `DISABLED` and the rest still run. Traefik
terminates :443 and forwards to the manager's loopback listener.
Then browse to `https://diskless-server.bitasylum.org/`, log in with the passkey,
and you should see **Monitor / Authorise / Lifecycle**.
@@ -262,24 +269,25 @@ If you run compose under a non-default project name, pass
## Hardening status
Done: distroless-nonroot runtime; passkey admin auth over HTTPS with an
auto-renewing LE cert (`acme` side-car); mTLS on both the agent (fleet CA) and
manager→broker (admin CA) planes; secrets kept server-side (`.env`/PKI gitignored);
optional planes soft-disable instead of crash-looping; CI builds + pushes the
image (`fleet-manager-image.yaml`); `backup.sh` for the durable volumes.
Done: distroless-nonroot runtime; passkey admin auth on :443 via the **Traefik**
mgmt ingress with an auto-renewing LE cert (Route 53 DNS-01); mTLS on both the
agent (fleet CA) and manager→broker (admin CA) planes; secrets kept server-side
(`.env`/PKI gitignored); optional planes soft-disable instead of crash-looping; CI
builds + pushes the image (`fleet-manager-image.yaml`); `backup.sh` for the durable
volumes.
Still open (operator to close per environment):
- **Network segmentation.** The admin (`:9500`) and agent (`:9600`) listeners must
be reachable only from the **mgmt network** — never the boot LAN. Enforce with a
host firewall / security group; this repo can't.
- **Network segmentation.** The admin (`:443` Traefik, forwards to loopback `:9500`)
and agent (`:9600`) listeners must be reachable only from their zones — admin from
the mgmt network, agent from the authenticated host VLAN via **OPNsense L3** (allow
only host-VLAN → `:9600`). Never the boot LAN. Enforce at the firewall; this repo
can't. See [Network paths](#network-paths).
- **Agent-plane server cert.** `server-pki.sh` isn't run yet (nothing connects to
the agent plane until the on-host agent exists — it soft-disables meanwhile).
Issue it when the agent lands.
- **Admin exposure.** Decide `:9500` direct vs a 443 mgmt proxy (TLS passthrough)
and align `FLEET_MGR_RP_ORIGIN` — see §2c.
- **Digest pinning.** For a locked deployment, pin `fleet-manager`, `influxdb`,
and the `goacme/lego` images by `@sha256:…` (CLAUDE.md invariant #12) rather than
the tags used here for convenience.
and `traefik` by `@sha256:…` (CLAUDE.md invariant #12) rather than the tags used
here for convenience.
## What is NOT yet wired (recorded intent)