Tightening Stig hardening

docs/TODO.md

@@ -5,7 +5,7 @@ Persistent backlog, kept in sync with the working session task tracker. Status:
 
 ---
 
-## Current build posture (LAB / test) — as of 2026-06-18
+## Current build posture (LAB / test) — as of 2026-06-19
 
 The image currently builds in **open-network test mode**. These toggles must be
 flipped back for a secure/production build (consolidated as item #8):
@@ -15,12 +15,24 @@ flipped back for a secure/production build (consolidated as item #8):
   per-env DNS name like `nvme.fleet.lab` once DNS-at-boot is trusted).
 - Unsigned UKI (`sb_key`/`sb_cert` not passed) → Secure Boot must be OFF on the
   test target. Production needs CI/HSM signing.
-- `enable_kiosk=false` by default (CI may set true); `enable_stig=true`;
-  `boot_splash=true`; NTP via DHCP→serving-host chrony.
+- `enable_kiosk=true`; `boot_splash=true`; NTP via DHCP→serving-host chrony.
+- **`enable_stig` / CI `ENABLE_STIG="false"` RIGHT NOW** — temporarily disabled
+  to get a working box while the STIG regressions were diagnosed (now fixed,
+  below). **Flip back to `true`** and bench-verify the hardened build.
+- **`debug_shell=true` (DEBUG HOLE — passwordless root on tty2). REMOVE before
+  prod** (`-var debug_shell=false`). Note: it didn't actually come up on tty2 in
+  testing; diagnosis ended up done via offline `rootfs.erofs` mount + SMBIOS
+  `enforcing=0`. SMBIOS cmdline-extra works on Proxmox only after a full VM
+  **stop/start** (not `reset`).
+- **Admin SSH key MUST be ECDSA or RSA, NOT ed25519**, when STIG is on — the STIG
+  crypto policy drops ed25519 from sshd's accepted pubkey algorithms. Set the
+  `SSH_AUTHORIZED_KEY` CI secret to an ECDSA/RSA key (see #2).
 - **Hostname:** image is hostname-neutral (empty `/etc/hostname`,
   `hostname-mode=default`); the serving dnsmasq's `fleet-namer` (`--dhcp-script`)
   assigns a stable creative starship name per MAC (e.g. `nebula-vanguard-07`) via
   DHCP option 12 + DNS A/PTR. Host picks it up on next DHCP renewal/reboot.
+- **Serving `.env`:** must set `AUDIT_HOST=audit.fleet.lab` (+ `docker compose up
+  -d dnsmasq` to reload) for remote log offload to resolve/flow.
 
 What's working/verified vs awaiting bench is per-item below. Cross-cutting
 bench-verify checklist lives in `CLAUDE.md` → "Known bench-verify items".
@@ -78,8 +90,13 @@ in **[stig-deviations.md](stig-deviations.md)** (curated register) and reported
 per-build by **[../scripts/stig-deviations.sh](../scripts/stig-deviations.sh)**
 (non-pass rules tagged intentional vs REVIEW); CI publishes
 `stig-report.html` + `stig-arf.xml` + `stig-deviations.md`. Scanner tooling is
-removed from the shipped image. **BENCH-VERIFY:** boot/login still work;
-reproducible verity hash across two builds; triage REVIEW rows.
+removed from the shipped image. **BENCH-VERIFY:** boot/login **VERIFIED ON THE
+BENCH 2026-06-19** — hardened build with `enable_stig=true`, host
+`cygnus-cruiser-99`: kiosk auto-starts + renders, admin SSH (ECDSA) logs in,
+audit offload reaches the collector, DHCP creative hostname applied, NVMe/TCP
+verity root mounts and the host stays up. **STILL OPEN:** reproducible verity
+hash across two builds; triage the REVIEW rows (below); the secure path (#8,
+802.1X/MACsec — this was the `disable_dot1x=true` test path).
 
 **FINDING 2026-06-18 (FIXED) — false-clean scan.** The first hardened build
 (branch `hardening`, `ada4aa4`) produced a report reading "0 REVIEW / clean",
@@ -107,34 +124,101 @@ two appliance accounts — `chage -M -1 -m 0 -I -1 -E -1` + `faillock --reset`;
 kiosk uses a dedicated `/etc/pam.d/kiosk` (`PAMName=kiosk`, permissive account /
 full session for seat); sshd gets an `account sufficient pam_succeed_if.so user
 = fleetadmin` first-rule bypass. Scoped to those two accounts; everyone else
-keeps full STIG. Justified as register deviation #10. **Bench-verify on rebuild:
-kiosk renders + admin SSH works with hardening on.**
+keeps full STIG. Justified as register deviation #10. **VERIFIED ON THE BENCH 2026-06-19**
+(`enable_stig=true`, host `cygnus-cruiser-99`): kiosk auto-starts + renders and
+admin SSH (ECDSA key) logs in with hardening on.**
 
 **UPDATE 2026-06-19 — offline erofs inspection of the hardened image (mounted
 `rootfs.erofs` read-only) pinned the real causes; the PAM-account theory above
 was incomplete:**
-- **Kiosk = fapolicyd (CONFIRMED).** fapolicyd is `enabled` and `kiosk-session`
-  was not in its trust → it blocks `/usr/local/bin/kiosk-session`. Fix: KEEP
-  fapolicyd (it's the exec-guard for the writable overlay dm-verity can't cover)
-  and add the kiosk binary to its trust DB at build (2c-STIG-postfix). Not waived.
-- **SSH = NOT a STIG/PAM/SELinux issue.** `enforcing=0` didn't help; pam_namespace
-  is inert (empty namespace.conf); account bypass present; key file + perms +
-  sshd config all correct. Remaining cause: **baked key mismatch** — the
-  `SSH_AUTHORIZED_KEY` CI secret bakes `audiobuzz@s3`
-  (`SHA256:30p4vOQdWk0TdRQIR3VMo95yc6I7cRmCwIBRVjIHaV4`); if the client uses a
-  different private key → `Permission denied (publickey)`. AWAITING client
-  fingerprint / `ssh -vvv` to confirm; fix is the secret, not code.
-- Side note: SMBIOS cmdline-extra injection works only after a Proxmox VM
-  **stop/start** (not `reset`); and the debug-shell-on-tty2 attempt did not come
-  up — diagnosis ended up done via offline erofs mount instead.
+- **Kiosk had TWO causes, both FIXED:**
+  1. **fapolicyd** is `enabled` and `kiosk-session` wasn't in its trust → it
+     blocked `/usr/local/bin/kiosk-session`. Fix: KEEP fapolicyd (it's the
+     exec-guard for the writable overlay dm-verity can't cover) and add the kiosk
+     binary to its trust DB at build (2c-STIG-postfix). Not waived.
+  2. **Auto-start:** kiosk was `WantedBy=graphical.target` but the host boots to
+     `multi-user.target` (graphical never reached) → unit `inactive (dead)` at
+     boot, fine on manual `restart`. Fix: `WantedBy=multi-user.target` + dropped
+     `set-default graphical.target` (kiosk launches weston directly, needs no DM
+     target).
+- **SSH = STIG crypto policy excludes ed25519 (CONFIRMED via `ssh -vvv`).** The
+  key fingerprint MATCHED (not a mismatch); the server's `server-sig-algs`
+  advertised only ecdsa + rsa-sha2 — **no `ssh-ed25519`** — so the ed25519 admin
+  key is rejected at the signature stage (before PAM/SELinux, which is why
+  `enforcing=0` and the account-bypass didn't help). **Fix (your action): set the
+  `SSH_AUTHORIZED_KEY` CI secret to an ECDSA (`ssh-keygen -t ecdsa -b 384`) or RSA
+  key**, then rebuild. No code change. (Alt: re-allow ed25519 via
+  `PubkeyAcceptedAlgorithms +ssh-ed25519` in the sshd drop-in — but that's a
+  non-approved algo and would re-fail a STIG rule; prefer the ECDSA key.)
+- **rsyslog TLS-offload rules waived (lab):** `rsyslog_encrypt_offload_*` force
+  gtls+x509+CA and broke the plaintext `2f` forward → unselected in the
+  tailoring; production TLS offload tracked under #8.
+- Diagnosis tooling notes: SMBIOS cmdline-extra works only after a Proxmox VM
+  **stop/start** (not `reset`); the debug-shell-on-tty2 never came up — the
+  decisive diagnosis was **offline `rootfs.erofs` mount** + `ssh -vvv` +
+  `enforcing=0`.
+
+**REVIEW-ROW TRIAGE — DONE 2026-06-19 (scan `app-9f7d8d1`, 21 REVIEW → 1 open).**
+The post-fix hardened scan was valid (398 pass / 12 fail / 9 notchecked). All 21
+REVIEW rows dispositioned (full detail: [stig-deviations.md](stig-deviations.md)
+#12). Mechanics: a rule only stops being flagged REVIEW if it's `selected="false"`
+in `diskless-tailoring.xml` (the register prose alone doesn't clear it) or made to
+pass / N/A.
+  - **Bucket A (waived, register-justified):** `encrypt_partitions`,
+    `auditd_audispd_configure_sufficiently_large_partition` (#1); `grub2_password`
+    (#2); `sysctl_crypto_fips_enabled`, `harden_sshd_macs_openssh_conf_crypto_policy`,
+    `harden_sshd_macs_opensshserver_conf_crypto_policy` (#5);
+    `configured_firewalld_default_deny` (#4); `sssd_enable_certmap`,
+    `sssd_has_trust_anchor` (#7).
+  - **Bucket B (waived N/A):** `account_temp_expire_date`,
+    `accounts_user_home_paths_only`, `libreswan_approved_tunnels`,
+    `security_patches_up_to_date`, `ssh_keys_passphrase_protected`.
+  - **Bucket C (decisions applied):** `installed_OS_is_vendor_supported` (Rocky IS
+    supported; RHEL-datastream CPE artifact — waived), `network_configure_name_resolution`
+    (single DHCP DNS by design — waived), `sysctl_user_max_user_namespaces_no_remediation`
+    (KEEP userns for Chromium sandbox — waived), `accounts_authorized_local_users`
+    (appliance accounts authorized, #10 — waived; prod: set the authorized-users
+    regex var instead), `package_postfix_installed` + `postfix_prevent_unrestricted_relay`
+    (**postfix removed** in 2c-STIG — no MTA needed; audit goes off-box via 2f),
+    **USBGuard** `configure_usbguard_auditbackend` + `usbguard_allow_hid`/`_hub`/
+    `_hid_and_hub` (**ENABLED + enforcing**: LinuxAudit backend + hardware-independent
+    HID/hub allowlist written in 2c-STIG; ⚠ BENCH-VERIFY physical console keyboard/
+    mouse on real hardware — the build-VM-generated policy + default block target
+    would otherwise lock out console input).
+  - **`selinux_context_elevation_for_sudo` — REMEDIATED WITH CONFINEMENT
+    (decision 2026-06-20).** 2c-STIG now rewrites the admin sudoers to
+    `TYPE=sysadm_t ROLE=sysadm_r NOPASSWD:ALL` and maps the admin to **`staff_u`**
+    (`semanage login`) so the sudo role transition is authorized. ⚠ **TWO BENCH
+    GATES:** (a) admin SSH login + sudo + privileged ops must work under `staff_u`
+    confinement (escape hatch: `enable_stig=false` → reverts to a justified
+    deviation); (b) **reproducibility** — `semanage` rebuilds the policy store;
+    confirm the verity hash is stable across two builds (normalize in 2d if it
+    drifts). Needs `policycoreutils-python-utils` (installed in 2c-STIG if absent).
+
+  **Expected next-build report: 0–1 REVIEW** (0 if both postfix→N/A and
+  usbguard→pass land and sudo confinement holds; the triage was validated by
+  re-running `scripts/stig-deviations.sh` against the app-9f7d8d1 ARF with the new
+  tailoring — 21 REVIEW → 3, where the 3 are exactly postfix/usbguard/sudo, all
+  addressed for the next build).
 
 Remaining (future hardening, currently waived — see the register): FIPS,
 firewall, per-host/rotatable admin access. **Remote audit offload — WIRED both
 ends** (2026-06-19): receiver `serving/audit-collector/` (rsyslog TCP/514,
 per-source-host files) + sender in the image (provisioner 2f, `audit_host` var:
 audisp-syslog→LOCAL6 + rsyslog `omfwd` to `audit.fleet.lab`). Lab is plaintext;
-TLS/RELP is the remaining hardening. **Bench-verify:** booted host's auditd AVCs
-land in `audit-collector/logs/<ip>/audit.log` and journal in `messages.log`.
+TLS/RELP is the remaining hardening. **VERIFIED END-TO-END ON THE BENCH
+2026-06-19** — booted host `cygnus-cruiser-99` (192.168.50.148) forwards to the
+collector, which writes `audit-collector/logs/192.168.50.148/{audit,messages}.log`.
+**Serving-host firewall gotcha hit + fixed:** the collector is host-networked so
+the serving host's **ufw** governs :514 (Docker doesn't manage iptables for
+host-net containers); 514/tcp was bound to the wrong NIC (`ens18` instead of the
+boot-LAN `ens19`), so SYNs were silently dropped (`SYN-SENT` / "connection timed
+out", not refused). Fix: `ufw allow in on ens19 to any port 514 proto tcp`. The
+sender's `action.resumeRetryCount="-1"` buffered the backlog and drained on
+reconnect — no records lost. Documented in
+[../serving/audit-collector/README.md](../serving/audit-collector/README.md).
+When the **workstation** firewall is re-enabled (#4 / register #4) it needs the
+reciprocal **egress** allow for 514/tcp to the collector.
 
 ### Original spec (retained for reference)