Adding rsyslog tls
This commit is contained in:
Audiobuzz
@@ -15,6 +15,11 @@ keys
|
||||
#/serving/tftp/*.efi
|
||||
/serving/tftp/*.kpxe
|
||||
|
||||
# Audit-collector syslog-TLS server cert/key (issued per-environment from the
|
||||
# fleet step-ca by serving/audit-collector/issue-cert.sh). Key material — never
|
||||
# commit it.
|
||||
/serving/audit-collector/tls/
|
||||
|
||||
# Out-of-VM CI-built host binaries (staged into the image by the Packer file
|
||||
# provisioner; built by the "Build host Go binaries" CI step).
|
||||
files/enrol/bin/
|
||||
|
||||
@@ -254,7 +254,17 @@ instead of working around it.
|
||||
`sourcedir /run/chrony-dhcp` (the provisioner appends it if absent). If sources
|
||||
are empty, the dispatcher didn't fire or the env var name differs — check
|
||||
`journalctl -u NetworkManager` and `ls /run/chrony-dhcp/`.
|
||||
- `fleet` Plymouth theme: confirm (a) the script plugin package actually
|
||||
- TLS audit offload (2f → `serving/audit-collector`, gtls/6514, since
|
||||
2026-07-12): the plaintext-514 path was bench-verified; the TLS upgrade is
|
||||
NOT yet. On a booted host confirm the sender is ESTABLISHED to **:6514**
|
||||
(not the legacy :514), records land in `audit-collector/logs/<ip>/`, and a
|
||||
wrong CA is rejected (sender spools + logs a cert-validation error — proves
|
||||
x509/name is enforced). Prereqs: collector cert issued
|
||||
(`audit-collector/issue-cert.sh`, SAN = `audit.fleet.lab`, same step-ca as
|
||||
the build's `fleet_ca`), 6514/tcp open on the boot NIC (host-networked —
|
||||
same ufw trap as 514), image built with the real fleet CA. The three
|
||||
`rsyslog_encrypt_offload_*` STIG rules are enforced (waivers dropped) and
|
||||
must PASS in the report.
|
||||
installed (`/usr/lib64/plymouth/script.so` — else the build fell back to
|
||||
`spinner`, logged as a WARNING), (b) the scripted theme + a font got bundled
|
||||
into the initramfs (`lsinitrd | grep -E 'fleet.script|script.so'`), and (c) it
|
||||
@@ -316,7 +326,9 @@ These bit us bringing the build up on Rocky 9.8; they are easy to regress.
|
||||
dangerous-to-remediate rules; every waiver is justified in
|
||||
`docs/stig-deviations.md`). STIG kernel params ride the **UKI cmdline**
|
||||
(`extract-artifacts.sh`, gated by the same flag) since there's no GRUB.
|
||||
**FIPS is OFF and audit offload is LOCAL-only** (decisions). CI is
|
||||
**FIPS is OFF** (decision); audit offload is **remote over syslog-TLS**
|
||||
(2f → `serving/audit-collector`, gtls/6514, fleet-CA x509/name; empty
|
||||
`audit_host` = local-only). CI is
|
||||
**report-only**: it publishes `stig-report.html` + `stig-arf.xml` +
|
||||
`stig-deviations.md` (generated by `scripts/stig-deviations.sh`) as artifacts
|
||||
and never fails on score. The scanner tooling is removed from the shipped image
|
||||
|
||||
+38
-14
@@ -150,7 +150,7 @@ variable "boot_splash" {
|
||||
|
||||
variable "enable_stig" {
|
||||
type = bool
|
||||
description = "MUST be true (enforced). Applies DISA STIG hardening (OpenSCAP + SSG) with a diskless/UKI/kiosk tailoring (files/stig/diskless-tailoring.xml), bakes STIG kernel params into the UKI cmdline, and emits a deviations report. FIPS is OFF and audit offload is LOCAL-only regardless (see docs/stig-deviations.md). The former enable_stig=false escape hatch is removed for production hardening (POA&M P-03) — a build with enable_stig=false now fails validation. To debug a hardening regression, revert this validation on a throwaway branch, never on a release build."
|
||||
description = "MUST be true (enforced). Applies DISA STIG hardening (OpenSCAP + SSG) with a diskless/UKI/kiosk tailoring (files/stig/diskless-tailoring.xml), bakes STIG kernel params into the UKI cmdline, and emits a deviations report. FIPS is OFF (see docs/stig-deviations.md); audit offload is remote over syslog-TLS (2f, audit_host). The former enable_stig=false escape hatch is removed for production hardening (POA&M P-03) — a build with enable_stig=false now fails validation. To debug a hardening regression, revert this validation on a throwaway branch, never on a release build."
|
||||
default = true
|
||||
validation {
|
||||
condition = var.enable_stig
|
||||
@@ -160,7 +160,7 @@ variable "enable_stig" {
|
||||
|
||||
variable "audit_host" {
|
||||
type = string
|
||||
description = "Remote rsyslog/audit collector the image forwards auditd (via audisp-syslog) + the journal (rsyslog imjournal) to. Host-generic DNS name resolved per-environment by the serving dnsmasq to the collector (serving/audit-collector); matches serving/.env AUDIT_HOST. The diskless root is a RAM-only overlay, so local logs die on reboot — this ships them off-box. Empty = local-only (no remote offload)."
|
||||
description = "Remote rsyslog/audit collector the image forwards auditd (via audisp-syslog) + the journal (rsyslog imjournal) to, over syslog-TLS (gtls :6514, server verified x509/name against the baked fleet CA — the collector cert's SAN must be this name; see serving/audit-collector/issue-cert.sh). Host-generic DNS name resolved per-environment by the serving dnsmasq to the collector (serving/audit-collector); matches serving/.env AUDIT_HOST. The diskless root is a RAM-only overlay, so local logs die on reboot — this ships them off-box. Empty = local-only (no remote offload)."
|
||||
default = "audit.fleet.lab"
|
||||
}
|
||||
|
||||
@@ -179,7 +179,7 @@ variable "sb_cert" {
|
||||
|
||||
variable "fleet_ca" {
|
||||
type = string
|
||||
description = "Path on the BUILD HOST to the fleet CA PEM baked to /etc/dot1x/radius-ca.pem — the trust anchor wpa_supplicant uses for RADIUS EAP-TLS AND the fleet-agent uses to verify the fleet-manager. Defaults to the COMMITTED PLACEHOLDER (files/etc/dot1x/radius-ca.pem), which is NOT a usable CA (auth/telemetry can't succeed with it). Override per-environment (CI secret -> temp file) with your real step-ca root (or a root+intermediate bundle for robustness). Never commit real internal CA material (CLAUDE.md)."
|
||||
description = "Path on the BUILD HOST to the fleet CA PEM baked to /etc/dot1x/radius-ca.pem — the trust anchor wpa_supplicant uses for RADIUS EAP-TLS, the fleet-agent uses to verify the fleet-manager, AND rsyslog uses to verify the audit collector (TLS offload, 2f). Defaults to the COMMITTED PLACEHOLDER (files/etc/dot1x/radius-ca.pem), which is NOT a usable CA (auth/telemetry/log-offload can't succeed with it). Override per-environment (CI secret -> temp file) with your real step-ca root (or a root+intermediate bundle for robustness). Never commit real internal CA material (CLAUDE.md)."
|
||||
default = "files/etc/dot1x/radius-ca.pem"
|
||||
}
|
||||
|
||||
@@ -348,6 +348,14 @@ build {
|
||||
# talks to the TPM directly via go-tpm. So this list stays at the baseline.)
|
||||
"dnf -y install wpa_supplicant tpm2-tools tpm2-tss nvme-cli cryptsetup dracut-network NetworkManager zstd openssl",
|
||||
|
||||
# rsyslog + its gnutls netstream driver, installed HERE (not in 2f) so
|
||||
# they're present when the 2c-STIG scan runs: the re-selected
|
||||
# rsyslog_encrypt_offload_* rules need the rsyslog package installed to be
|
||||
# applicable, and their remediation writes the gtls/x509 globals that the
|
||||
# clean eval of record then passes. 2f (after STIG) writes the actual TLS
|
||||
# forward action; rsyslog-gnutls provides lmnsd_gtls.so it loads at runtime.
|
||||
"dnf -y install rsyslog rsyslog-gnutls",
|
||||
|
||||
# Locale trim: @^minimal-environment pulls glibc-all-langpacks — ~228 MB of
|
||||
# locale data for EVERY language (measured 2026-07-03, 3rd-largest package in
|
||||
# the whole image). This fleet is English-only (we also strip
|
||||
@@ -804,7 +812,7 @@ build {
|
||||
# tailoring (files/stig/diskless-tailoring.xml) unselects rules that are N/A to
|
||||
# this architecture or dangerous to auto-remediate (FIPS off, firewall off,
|
||||
# AIDE, partitions, smartcard, GRUB) — each justified in docs/stig-deviations.md.
|
||||
# FIPS=off, audit offload=local per decision. Gated by enable_stig.
|
||||
# FIPS=off per decision; audit offload is remote/TLS (2f). Gated by enable_stig.
|
||||
provisioner "file" {
|
||||
source = "files/stig"
|
||||
destination = "/tmp/"
|
||||
@@ -1085,25 +1093,41 @@ build {
|
||||
|
||||
# --- 2f. Remote log / audit offload (sender) -----------------------------
|
||||
# Forward auditd (via the audisp-syslog plugin, on LOCAL6) AND the journal
|
||||
# (rsyslog's stock imjournal) to the remote collector (serving/audit-collector).
|
||||
# The diskless root is a RAM-only overlay so local logs vanish on reboot; this
|
||||
# ships them off-box (and gives remote visibility into boot/login failures).
|
||||
# Host-generic: targets ${audit_host} (DNS, resolved per environment). Empty
|
||||
# audit_host => local-only. Implements the remote-offload half of TODO #2 /
|
||||
# docs/stig-deviations.md #6. Disk-assisted queue so a network blip won't drop
|
||||
# records (action.resumeRetryCount=-1 retries forever).
|
||||
# (rsyslog's stock imjournal) to the remote collector (serving/audit-collector)
|
||||
# over syslog-TLS (gtls, :6514). The diskless root is a RAM-only overlay so
|
||||
# local logs vanish on reboot; this ships them off-box (and gives remote
|
||||
# visibility into boot/login failures).
|
||||
# TLS trust: the sender verifies the collector x509/name against the FLEET CA
|
||||
# already baked at /etc/dot1x/radius-ca.pem (var.fleet_ca) — host-generic
|
||||
# public material, no per-host state (invariant #1). No client cert: hosts are
|
||||
# fleet-generic and a baked fleet-wide client key would be shared secret
|
||||
# material in the image; the collector's listener is authMode=anon and access
|
||||
# is constrained by the boot LAN + serving-host firewall. With the default
|
||||
# PLACEHOLDER CA the TLS handshake can't succeed (same posture as RADIUS) —
|
||||
# the action suspends/retries and the disk-assisted queue spools until a real
|
||||
# CA + collector cert exist.
|
||||
# Host-generic: targets ${audit_host} (DNS, resolved per environment; the
|
||||
# collector cert's SAN must match — issued by audit-collector/issue-cert.sh).
|
||||
# Empty audit_host => local-only. Implements the remote-offload half of TODO
|
||||
# #2 / docs/stig-deviations.md #6; satisfies the rsyslog_encrypt_offload_*
|
||||
# STIG rules (re-selected in the tailoring). Disk-assisted queue so a network
|
||||
# blip won't drop records (action.resumeRetryCount=-1 retries forever).
|
||||
# rsyslog + rsyslog-gnutls are installed in step 2 (before the STIG scan).
|
||||
provisioner "shell" {
|
||||
environment_vars = ["AUDIT_HOST=${var.audit_host}"]
|
||||
inline = [
|
||||
"set -eux",
|
||||
"if [ -z \"$AUDIT_HOST\" ]; then echo 'audit_host empty -> local-only logging, no remote offload.'; exit 0; fi",
|
||||
"dnf install -y rsyslog",
|
||||
"rpm -q rsyslog rsyslog-gnutls", # installed in step 2; fail loudly if not
|
||||
# auditd -> syslog on LOCAL6 (the collector routes LOCAL6 to a per-host
|
||||
# audit.log; see serving/audit-collector/rsyslog.conf).
|
||||
"if [ -f /etc/audit/plugins.d/syslog.conf ]; then sed -i -e 's/^active.*/active = yes/' -e 's/^args.*/args = LOG_INFO LOG_LOCAL6/' /etc/audit/plugins.d/syslog.conf; fi",
|
||||
# rsyslog -> collector over TCP, disk-assisted queue.
|
||||
# rsyslog -> collector over TLS (gtls), disk-assisted queue. The explicit
|
||||
# StreamDriver* action params take precedence over the legacy globals the
|
||||
# STIG remediation appended to /etc/rsyslog.conf (same values).
|
||||
"cat > /etc/rsyslog.d/60-fleet-remote.conf <<EOF",
|
||||
"*.* action(type=\"omfwd\" target=\"$AUDIT_HOST\" port=\"514\" protocol=\"tcp\" queue.type=\"LinkedList\" queue.fileName=\"fleetfwdq\" queue.maxDiskSpace=\"256m\" queue.saveOnShutdown=\"on\" action.resumeRetryCount=\"-1\")",
|
||||
"global(DefaultNetstreamDriverCAFile=\"/etc/dot1x/radius-ca.pem\")",
|
||||
"*.* action(type=\"omfwd\" target=\"$AUDIT_HOST\" port=\"6514\" protocol=\"tcp\" StreamDriver=\"gtls\" StreamDriverMode=\"1\" StreamDriverAuthMode=\"x509/name\" StreamDriverPermittedPeers=\"$AUDIT_HOST\" queue.type=\"LinkedList\" queue.fileName=\"fleetfwdq\" queue.maxDiskSpace=\"256m\" queue.saveOnShutdown=\"on\" action.resumeRetryCount=\"-1\")",
|
||||
"EOF",
|
||||
"chmod 0644 /etc/rsyslog.d/60-fleet-remote.conf",
|
||||
"rsyslogd -N1",
|
||||
|
||||
+59
-28
@@ -32,7 +32,10 @@ flipped back for a secure/production build (consolidated as item #8):
|
||||
assigns a stable creative starship name per MAC (e.g. `nebula-vanguard-07`) via
|
||||
DHCP option 12 + DNS A/PTR. Host picks it up on next DHCP renewal/reboot.
|
||||
- **Serving `.env`:** must set `AUDIT_HOST=audit.fleet.lab` (+ `docker compose up
|
||||
-d dnsmasq` to reload) for remote log offload to resolve/flow.
|
||||
-d dnsmasq` to reload) for remote log offload to resolve/flow. Since the TLS
|
||||
upgrade (#8, 2026-07-12) the collector also needs its server cert issued
|
||||
(`audit-collector/issue-cert.sh`) + 6514/tcp open on the boot NIC, and the
|
||||
image must be built with the real fleet CA (`fleet_ca`) or senders spool.
|
||||
|
||||
What's working/verified vs awaiting bench is per-item below. Cross-cutting
|
||||
bench-verify checklist lives in `CLAUDE.md` → "Known bench-verify items".
|
||||
@@ -152,7 +155,8 @@ was incomplete:**
|
||||
non-approved algo and would re-fail a STIG rule; prefer the ECDSA key.)
|
||||
- **rsyslog TLS-offload rules waived (lab):** `rsyslog_encrypt_offload_*` force
|
||||
gtls+x509+CA and broke the plaintext `2f` forward → unselected in the
|
||||
tailoring; production TLS offload tracked under #8.
|
||||
tailoring; production TLS offload tracked under #8. **(DONE 2026-07-12 — 2f is
|
||||
now gtls/6514 and the three waivers are dropped; see #8.)**
|
||||
- Diagnosis tooling notes: SMBIOS cmdline-extra works only after a Proxmox VM
|
||||
**stop/start** (not `reset`); the debug-shell-on-tty2 never came up — the
|
||||
decisive diagnosis was **offline `rootfs.erofs` mount** + `ssh -vvv` +
|
||||
@@ -205,8 +209,9 @@ Remaining (future hardening, currently waived — see the register): FIPS,
|
||||
firewall, per-host/rotatable admin access. **Remote audit offload — WIRED both
|
||||
ends** (2026-06-19): receiver `serving/audit-collector/` (rsyslog TCP/514,
|
||||
per-source-host files) + sender in the image (provisioner 2f, `audit_host` var:
|
||||
audisp-syslog→LOCAL6 + rsyslog `omfwd` to `audit.fleet.lab`). Lab is plaintext;
|
||||
TLS/RELP is the remaining hardening. **VERIFIED END-TO-END ON THE BENCH
|
||||
audisp-syslog→LOCAL6 + rsyslog `omfwd` to `audit.fleet.lab`). **UPGRADED TO TLS
|
||||
2026-07-12** (gtls/6514, fleet-CA x509/name; see #8 — BENCH-VERIFY pending; the
|
||||
plaintext-era notes below record the original bring-up). **VERIFIED END-TO-END ON THE BENCH
|
||||
2026-06-19** — booted host `cygnus-cruiser-99` (192.168.50.148) forwards to the
|
||||
collector, which writes `audit-collector/logs/192.168.50.148/{audit,messages}.log`.
|
||||
**Serving-host firewall gotcha hit + fixed:** the collector is host-networked so
|
||||
@@ -218,7 +223,7 @@ sender's `action.resumeRetryCount="-1"` buffered the backlog and drained on
|
||||
reconnect — no records lost. Documented in
|
||||
[../serving/audit-collector/README.md](../serving/audit-collector/README.md).
|
||||
When the **workstation** firewall is re-enabled (#4 / register #4) it needs the
|
||||
reciprocal **egress** allow for 514/tcp to the collector.
|
||||
reciprocal **egress** allow for 6514/tcp (TLS; was 514/tcp) to the collector.
|
||||
|
||||
**SIEM (Wazuh) added 2026-06-27 — [serving/siem/](../serving/siem/).** Upgrades the
|
||||
"Detect" pillar from file-collection-only to parse/search/correlate/alert.
|
||||
@@ -241,7 +246,10 @@ and a live `sudo` on a booted host (`cygnus-cruiser-99`) surfaced `USER_CMD` →
|
||||
rule `100130` and `CRED_*`/`USER_ACCT` → rule `100110` (level 5) in the dashboard
|
||||
Threat Hunting view. Wazuh runs multi-node on `wazuh1.bitasylum.org`.
|
||||
**REMAINING (prod):** the relay→wazuh1 hop is **plaintext** — **TLS it** (gtls/6514,
|
||||
fleet CA; ISM protect-logs-in-transit) since it's now on the wire; rotate all Wazuh
|
||||
fleet CA; ISM protect-logs-in-transit) since it's now on the wire (NB: the
|
||||
host→collector hop is TLS since 2026-07-12, #8 — this is the separate
|
||||
collector→wazuh1 hop, and the collector-side `omfwd` must stay/`ptcp` until
|
||||
wazuh1 has a TLS listener); rotate all Wazuh
|
||||
creds; restrict dashboard/API to admin hosts; set `allowed-ips`→/32 of the serving
|
||||
host; ISM retention/archival; then update the SSP "Detect" + ISM event-logging
|
||||
controls.
|
||||
@@ -917,28 +925,51 @@ smoke test.
|
||||
secure `nvme connect` as the test path has.
|
||||
- **Future hardening waived for now** (from the STIG register): FIPS, firewall,
|
||||
per-host/rotatable admin (SSH CA / config-agent).
|
||||
- **TLS-encrypted audit/log offload (production).** The lab forwards logs to
|
||||
`serving/audit-collector` in **plaintext**, and the STIG TLS-offload rules
|
||||
(`rsyslog_encrypt_offload_defaultnetstreamdriver`,
|
||||
`..._actionsendstreamdrivermode`, `..._actionsendstreamdriverauthmode`) are
|
||||
**waived** in the tailoring because applying them forces `gtls` + x509 peer
|
||||
auth + a CA file and breaks the plaintext `2f` forward. For production, do it
|
||||
properly and DROP the waivers:
|
||||
1. Stand up a CA (or reuse the boot-infra CA); issue the collector a server
|
||||
cert (CN/SAN = `audit.fleet.lab`).
|
||||
2. Configure the collector's rsyslog for TLS: `imtcp` with
|
||||
`StreamDriver.Name="gtls"`, `StreamDriver.Mode="1"`, server cert/key + CA;
|
||||
listen on the syslog-TLS port (6514) — add a TLS receiver to
|
||||
`serving/audit-collector/rsyslog.conf` + compose.
|
||||
3. Bundle the CA into the image (host-generic, no per-host state — same model
|
||||
as the RADIUS CA) and point `$DefaultNetstreamDriverCAFile` at it.
|
||||
4. Set the `2f` `omfwd` action to `StreamDriver="gtls" StreamDriverMode="1"
|
||||
StreamDriverAuthMode="x509/name"` + the CA file, target port 6514.
|
||||
5. Re-select the three `rsyslog_encrypt_offload_*` rules (remove from
|
||||
`files/stig/diskless-tailoring.xml`) — they should then PASS, and update
|
||||
`docs/stig-deviations.md` #6 to reflect encrypted offload.
|
||||
(RELP+TLS via `omrelp`/`imrelp` is the alternative if guaranteed delivery is
|
||||
wanted alongside encryption.)
|
||||
- **[x] TLS-encrypted audit/log offload — IMPLEMENTED 2026-07-12 (awaiting
|
||||
bench).** The host→collector hop is now syslog-over-TLS; the plaintext `2f`
|
||||
forward and the three `rsyslog_encrypt_offload_*` waivers are gone. As built
|
||||
(followed the original 5-step plan):
|
||||
- **CA reused, not new:** the sender trusts the collector via the fleet CA
|
||||
already baked at `/etc/dot1x/radius-ca.pem` (`var.fleet_ca` — now the trust
|
||||
anchor for RADIUS EAP-TLS, fleet-agent AND log offload). Collector server
|
||||
cert (SAN = `${AUDIT_HOST}`) issued from the bench step-ca by
|
||||
`serving/audit-collector/issue-cert.sh` → `tls/` (gitignored), mounted at
|
||||
`/etc/rsyslog-tls`; renew by re-running it (720h default — expiry shows as
|
||||
handshake errors + senders spooling).
|
||||
- **Collector** (`serving/audit-collector/`): gtls listener on **6514**
|
||||
(per-listener `streamDriver.*` params; `authMode="anon"` — hosts carry NO
|
||||
client cert: fleet-generic, invariant #1; client mTLS with the TPM identity
|
||||
was CONSIDERED + DEFERRED 2026-07-12 — rationale + revisit trigger in
|
||||
audit-collector/README.md "Decision: no client mTLS"). Plaintext :514 kept as a LEGACY
|
||||
listener for pre-TLS images during migration — close it once the fleet is on
|
||||
TLS images. Deliberately no global `DefaultNetstreamDriver="gtls"` (would
|
||||
silently flip the plaintext wazuh1 relay `omfwd` to TLS and break it).
|
||||
Alpine image needs `rsyslog-tls` (ships `lmnsd_gtls.so`; verified v3.20).
|
||||
- **Sender (2f):** `omfwd` → `$AUDIT_HOST:6514`, `StreamDriver="gtls"` mode 1
|
||||
`StreamDriverAuthMode="x509/name"` + `StreamDriverPermittedPeers=$AUDIT_HOST`,
|
||||
CA via `global(DefaultNetstreamDriverCAFile="/etc/dot1x/radius-ca.pem")`.
|
||||
With the placeholder CA the handshake can't succeed → action suspends and the
|
||||
disk-assisted queue spools (same lab posture as RADIUS). `rsyslog` +
|
||||
`rsyslog-gnutls` moved to the step-2 baseline install so they're present at
|
||||
STIG-scan time.
|
||||
- **STIG:** the three `rsyslog_encrypt_offload_*` selects removed from
|
||||
`files/stig/diskless-tailoring.xml` (rules now evaluated; SSG remediation
|
||||
writes the legacy gtls/x509 globals, eval of record should PASS);
|
||||
register #6 rewritten (encrypted offload; audisp-remote rules +
|
||||
`rsyslog_remote_loghost` stay waived — intent met via rsyslog).
|
||||
- **Firewall:** serving host now needs **6514/tcp** open on the boot NIC
|
||||
(same host-networking trap as 514 — see audit-collector/README.md); the
|
||||
future workstation egress rule is 6514 too.
|
||||
- **BENCH-VERIFY (the gate):** build with the real fleet CA + run
|
||||
`issue-cert.sh` + `docker compose up -d --build audit`; on a booted host
|
||||
confirm records land in `audit-collector/logs/<ip>/audit.log`, `ss -tnp`
|
||||
on the serving host shows the sender ESTABLISHED to **:6514** (not :514),
|
||||
a tcpdump of :6514 shows TLS (no cleartext syslog), a wrong/placeholder CA
|
||||
is REJECTED (sender spools, `rsyslogd` logs a cert-validation error — proves
|
||||
x509/name is actually enforced), and the STIG report shows the three
|
||||
`rsyslog_encrypt_offload_*` rules PASS (0 new REVIEW rows).
|
||||
(RELP+TLS via `omrelp`/`imrelp` remains the alternative if guaranteed delivery
|
||||
is wanted alongside encryption.)
|
||||
- **fleet-hostapi: scope CORS to the kiosk origin.** The host-info agent ships
|
||||
with `FLEET_HOSTAPI_ALLOW_ORIGIN=*` (lab). For prod set it to the exact kiosk
|
||||
origin (default `http://boot.fleet.lab`) via the unit's commented
|
||||
|
||||
@@ -0,0 +1,186 @@
|
||||
# OQE Collection Plan — Automated Objective Quality Evidence for the IRAP Assessment
|
||||
|
||||
> **Protective marking:** `STARFLEET RESTRICTED — CLEARANCE LEVEL 2 — BRIDGE CREW / SECURITY DIVISION`
|
||||
> *(marking scheme per the [SAR](irap-sar.md) banner.)*
|
||||
>
|
||||
> **Status:** working plan (v0.1, 2026-07-11) — recorded from assessor guidance;
|
||||
> not yet implemented. Companion to the [SAR](irap-sar.md) and
|
||||
> [POA&M](irap-poam.md); feeds the continuous-monitoring plan (POA&M P-08 M2)
|
||||
> and the delta-assessment witness points (SAR §7.2).
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Define how Objective Quality Evidence (OQE) for the assessment is captured
|
||||
**automatically**, so that control effectiveness is demonstrated by
|
||||
machine-collected, attributable artifacts rather than screenshots and prose
|
||||
assertions — and so the delta assessment (and later, ongoing authorisation)
|
||||
can sample and re-verify evidence instead of regenerating it by hand.
|
||||
|
||||
This system is unusually well-positioned for automated OQE because it already
|
||||
has the three prerequisites:
|
||||
|
||||
1. **A digest that uniquely names the thing being assessed** (one release =
|
||||
one OCI digest = one manifest commit).
|
||||
2. **An authenticated channel from every host** (`fleet-agent` over TPM-mTLS —
|
||||
evidence is cryptographically attributable to a specific hardware host).
|
||||
3. **A CI pipeline that produces the security-relevant state** (signing, STIG
|
||||
scan, verity coupling, content assertions).
|
||||
|
||||
## 2. What makes an artifact OQE
|
||||
|
||||
Every evidence artifact must answer four questions **without a human
|
||||
interpreting it**:
|
||||
|
||||
| Question | Requirement |
|
||||
|---|---|
|
||||
| **What** does it demonstrate? | Mapped to a control/claim (ISM control ID + SSP/Annex A row + POA&M milestone). |
|
||||
| **Which** system produced it? | Exact attribution: OCI digest for builds; host serial (TPM cert serial) for hosts — never "the test box". |
|
||||
| **When** was it captured? | Timestamp from the authenticated time source (NTS, post-P-18). |
|
||||
| **How** was it produced? | The collection method is **in-repo and inspectable** — the assessor reads the collector before trusting its output. |
|
||||
|
||||
Evidence strength ordering (prefer higher): **signed attestation** > raw
|
||||
command output with attribution > screenshot/prose assertion. Automation
|
||||
removes the human from the capture path — but only if the automation itself is
|
||||
auditable.
|
||||
|
||||
## 3. The four collection planes
|
||||
|
||||
### 3.1 Build-time evidence — attached to the OCI artifact
|
||||
|
||||
CI already produces most of it. Formalise into a per-build **evidence bundle**,
|
||||
emitted as a CI artifact and (target state) attached to the OCI image as a
|
||||
**signed attestation** (cosign/in-toto via the registry referrers API), so OQE
|
||||
is *keyed to the digest*: assessing release `sha256:abc…` means pulling its
|
||||
evidence chain from the registry.
|
||||
|
||||
| Evidence | Source | Exists today? |
|
||||
|---|---|---|
|
||||
| STIG ARF + HTML report + deviations register (0 unjustified REVIEW) | CI artifacts (`stig-report.html`, `stig-arf.xml`, `stig-deviations.md`) | Yes |
|
||||
| roothash coupling (UKI cmdline ↔ `verity.json`) | `build-oci.sh` hard gate result | Yes (gate; log it) |
|
||||
| Initramfs content assertions (drivers, module, nvmf unit, NM hook) | `lsinitrd` assertions in the template | Yes (assert; capture listing) |
|
||||
| SELinux labels survived export | `getfattr` build assertion | Yes (assert; capture output) |
|
||||
| UKI + iPXE signature verification | `sbverify` output | Add |
|
||||
| Extracted (immutable) cmdline + its hash | `extract-artifacts.sh` dump | Add |
|
||||
| RPM manifest (+ size manifest) | `rpm -qa` dump (size manifest exists) | Partial |
|
||||
| Reproducibility proof | second build → identical `verity.json` root hash | Add (CI job) |
|
||||
| Production-profile assertion (which toggles/gates were active) | CI Production build gate output | Partial (gate exists — emit a machine-readable record) |
|
||||
| Signed provenance over the whole bundle | cosign/in-toto attestation on the OCI image | Add (target state) |
|
||||
|
||||
Supports: SAR §7.2 items 2 and 7; POA&M P-03 M2 (profile-tagged artifacts),
|
||||
P-04 M2, P-05.
|
||||
|
||||
### 3.2 Bench evidence — the harness emits structured results
|
||||
|
||||
Bench gates are witnessed live, but the harness should capture as it runs: a
|
||||
**JSON results file per run** — each pass criterion with its raw output,
|
||||
stamped with host serial, UKI digest, and timestamps. The assessor verifies the
|
||||
harness once, watches one live run, and keeps the bundle.
|
||||
|
||||
| Bench gate | Captured criteria (raw outputs) | POA&M |
|
||||
|---|---|---|
|
||||
| Secure path (802.1X/MACsec) | `wpa_cli status` = `Authorized`; `ip macsec show`; macsec0 DHCP + renewal journal lines; `nvme list-subsys` (targeted NQN); supplicant PID alive post-switch-root | P-01 |
|
||||
| Application control | novel non-RPM binary in `/tmp` → fapolicyd `FAN_DENY` audit record | SAR §7.2(7) |
|
||||
| Device control | USB mass-storage block event (LinuxAudit) + console HID still functional | SAR §7.2(7) |
|
||||
| SSH host certificates | `ssh -v` host-cert offer; `@cert-authority` validation; broker signing log line | P-17 M3 |
|
||||
| Authenticated time | `chronyc authdata` (NTS cookies present); tcp/4460 handshake; cold-clock `nocerttimecheck` path | P-18 M3 |
|
||||
| Agent actuators inert | desired-state/command received → log-only, no mutation | P-23 / SAR §7.2(7) |
|
||||
|
||||
### 3.3 Runtime/continuous evidence — the fleet-agent as evidence pipeline
|
||||
|
||||
The agent already streams SELinux state, chrony health, image version, and
|
||||
cert serial over TPM-mTLS — continuous, hardware-attributed OQE for those
|
||||
controls. Extending it per POA&M P-21 (SAR F-07) makes the InfluxDB store a
|
||||
rolling posture record: **every host, hourly, attributed to its TPM.**
|
||||
|
||||
Facts to add (P-21): Secure Boot state (`SecureBoot` EFI var), fapolicyd
|
||||
enforcement mode, kernel cmdline hash, USBGuard status.
|
||||
|
||||
This time-series **is** the substance of the continuous-monitoring plan (P-08
|
||||
M2) and the ongoing-authorisation story; its retention follows the P-08/P-09
|
||||
log-retention policy so the evidence itself survives. It also provides
|
||||
machine-checkable closure for other items — e.g. P-05 M2 closes when telemetry
|
||||
shows `mokutil --sb-state` = enabled for 100% of enrolled hosts.
|
||||
|
||||
### 3.4 Detect-pillar evidence — synthetic control tests
|
||||
|
||||
Exercise the detection path on a schedule and record the result: a probe runs
|
||||
`sudo` on a canary host, then confirms the `USER_CMD` event (a) landed in the
|
||||
collector's per-host file and (b) fired the Wazuh `fleet_audit` rule, within N
|
||||
seconds. Log each probe result. A control tested weekly with recorded outcomes
|
||||
is far stronger OQE than "the rule existed at assessment time."
|
||||
|
||||
Supports: Annex A §2.6 rows; POA&M P-09.
|
||||
|
||||
## 4. The mapping layer (where automation usually fails)
|
||||
|
||||
Evidence without a control map is a folder of files. Maintain a
|
||||
machine-readable matrix in-repo — `docs/oqe-matrix.yaml` (to be scaffolded) —
|
||||
one row per claim:
|
||||
|
||||
```yaml
|
||||
- id: OQE-013
|
||||
control: ISM-1542 # key against the ACSC OSCAL ISM edition
|
||||
claim: "Only fleet-signed UKIs boot under Secure Boot" # SSP/Annex A row
|
||||
plane: build # build | bench | runtime | detect
|
||||
collector: "sbverify --cert fleet-db.pem fleet-<ver>.efi"
|
||||
frequency: per-build
|
||||
closes: P-05 # POA&M milestone this evidences
|
||||
```
|
||||
|
||||
Two decisions recorded with it:
|
||||
|
||||
1. **Key controls against the ACSC OSCAL edition of the ISM.** The ISM is
|
||||
published in OSCAL; stable machine-readable identifiers per release make the
|
||||
May-2026 renumbering reconciliation (P-10) a computable diff instead of a
|
||||
manual crosswalk.
|
||||
2. **Define closure evidence per POA&M milestone.** Each M-milestone states
|
||||
its machine-checkable "done" signal in the matrix, so POA&M status stops
|
||||
being self-asserted.
|
||||
|
||||
## 5. Integrity and chain of custody
|
||||
|
||||
Evidence about a security system is itself a target. Proportionate measures at
|
||||
this classification:
|
||||
|
||||
- **Build evidence:** covered by the CI attestation signature (target state);
|
||||
until then, SHA256 manifests of each bundle **committed to git** — the
|
||||
one-commit-per-release convention makes git history the tamper-evident
|
||||
ledger.
|
||||
- **Runtime evidence:** inherently attributed by the agent's mTLS session +
|
||||
TPM-bound identity.
|
||||
- **Bench bundles:** hashed into a committed manifest per run.
|
||||
- **Timestamps:** ride the NTS-verified clock (P-18).
|
||||
- Storage: append-only files on the collector + committed manifests. An
|
||||
elaborate WORM store is **not** proportionate at STARFLEET RESTRICTED.
|
||||
|
||||
## 6. What the assessor still does in person
|
||||
|
||||
Automation changes the character of the SAR §7.2 witness points, it does not
|
||||
replace them. At the delta assessment the assessor will:
|
||||
|
||||
1. **Read the collectors** — confirm they measure what they claim.
|
||||
2. **Watch one live run** of each critical bench (P-01 above all).
|
||||
3. **Independently re-run a sample** of collection commands and diff against
|
||||
the automated bundles.
|
||||
|
||||
If the automated evidence survives that sampling, everything it produced
|
||||
*between* assessments is credible — which is what makes ongoing authorisation
|
||||
cheap instead of an annual re-assessment.
|
||||
|
||||
## 7. Implementation backlog (feeds TODO.md / POA&M)
|
||||
|
||||
| # | Task | Plane | Pairs with |
|
||||
|---|---|---|---|
|
||||
| O-1 | CI `evidence` job: bundle existing artifacts + add `sbverify`/cmdline/rpm-manifest dumps; emit machine-readable production-gate record | Build | P-03 M2 |
|
||||
| O-2 | Reproducibility CI job (double build, compare root hash) | Build | SAR §7.2(7) |
|
||||
| O-3 | Signed attestation on the OCI image (cosign/in-toto, referrers API) | Build | P-03 M2 |
|
||||
| O-4 | Bench harness structured JSON output (`serving/dot1x-bench/` first) | Bench | P-01 |
|
||||
| O-5 | Agent posture facts: SB state, fapolicyd mode, cmdline hash, USBGuard | Runtime | P-21 |
|
||||
| O-6 | Synthetic detect probe (canary `sudo` → collector file + Wazuh alert, scheduled) | Detect | P-09 |
|
||||
| O-7 | Scaffold `docs/oqe-matrix.yaml` from current POA&M rows; adopt OSCAL ISM keys | Mapping | P-10, P-08 M2 |
|
||||
| O-8 | Define closure-evidence signal for every open POA&M milestone in the matrix | Mapping | all |
|
||||
|
||||
---
|
||||
|
||||
*End of OQE collection plan. Review alongside the POA&M at each monthly
|
||||
review; promote O-items into TODO.md as they are scheduled.*
|
||||
@@ -26,7 +26,7 @@ Posture decisions (2026-06-18, lab phase):
|
||||
| 3 | **AIDE / file integrity** (`package_aide_installed`, `aide_build_database`, `aide_periodic_cron_checking`, `aide_scan_notification`, `aide_verify_*`) | The root is read-only and cryptographically verified; an on-disk AIDE db would also be non-deterministic and break the reproducible verity hash. | **dm-verity** verifies the whole root filesystem against a signed root hash on every read — continuous, hardware-backed integrity that supersedes AIDE's periodic scan. |
|
||||
| 4 | **Firewall** (`service_firewalld_enabled`, `package_firewalld_installed`, `set_firewalld_default_zone`) | Currently `firewall --disabled`; auto-enabling firewalld/nftables risks the NVMe/TCP (`:4420`) and NetworkManager paths during bring-up. | Isolated boot LAN; egress controlled at the serving-stack / network firewall layer. **Future hardening** — re-enable with explicit allow rules once the secure path is verified. |
|
||||
| 5 | **FIPS** (`enable_fips_mode`, `grub2_enable_fips_mode`, `package_dracut-fips_installed`) | Decision: off for the lab phase (interacts with the still-unsigned UKI / Secure Boot story). | Revisit with signing. No weakened crypto is configured in the meantime. |
|
||||
| 6 | **Remote audit offload** (`auditd_audispd_configure_remote_server`, `..._encrypt_sent_records`, `..._network_failure_action`, `rsyslog_remote_loghost`) **+ rsyslog TLS offload** (`rsyslog_encrypt_offload_defaultnetstreamdriver`, `..._actionsendstreamdrivermode`, `..._actionsendstreamdriverauthmode`) | Offload is done via **rsyslog over TCP** (simpler, container-friendly receiver); the `audisp-remote` native-protocol rules stay waived. The **TLS-offload rules are waived** because the isolated lab boot LAN is plaintext (mirrors HTTP boot / no MACsec) and applying them forces `gtls` + x509 peer auth + a CA file, which **breaks** the plaintext `2f` forward (no CA, plaintext `:514`). | **Wired both ends, plaintext.** Sender (image, provisioner 2f, `audit_host`): audisp-syslog → LOCAL6 + rsyslog `omfwd` (disk-assisted queue) → host-generic `audit.fleet.lab`. Receiver: `serving/audit-collector/` (rsyslog TCP/514, per-source-host files). No per-host state; isolated LAN. **Production (tracked in docs/TODO.md #2):** issue the collector a TLS cert, bundle the CA in the image, switch `2f` + the collector to `gtls`/6514, then **re-select** the three `rsyslog_encrypt_offload_*` rules (drop the waivers) so they pass. |
|
||||
| 6 | **Remote audit offload** (`auditd_audispd_configure_remote_server`, `..._encrypt_sent_records`, `..._network_failure_action`, `rsyslog_remote_loghost`) | Offload is done via **rsyslog over TLS** rather than the `audisp-remote` native protocol (simpler, container-friendly receiver), so the audisp-remote rules + `rsyslog_remote_loghost` (whose remediation would inject a bogus loghost) stay waived — the *intent* of all four is met by the rsyslog path. **The three `rsyslog_encrypt_offload_*` rules are NO LONGER waived** (re-selected 2026-07-12): the offload is now encrypted end-to-end and they are evaluated + expected to **pass** (rsyslog+rsyslog-gnutls installed before the scan; remediation writes the gtls/x509 globals). | **Wired both ends, TLS.** Sender (image, provisioner 2f, `audit_host`): audisp-syslog → LOCAL6 + rsyslog `omfwd` over **gtls :6514**, server verified `x509/name` against the baked fleet CA (`/etc/dot1x/radius-ca.pem`, `var.fleet_ca`), disk-assisted queue → host-generic `audit.fleet.lab`. Receiver: `serving/audit-collector/` (gtls listener :6514, server cert from the fleet step-ca via `issue-cert.sh`; legacy plaintext :514 kept only for pre-TLS images during migration — close it once the fleet is on TLS images). No client certs: hosts are fleet-generic (invariant #1 — a baked fleet-wide client key would be shared secret material), and client mTLS with the per-host **TPM identity was considered + deferred 2026-07-12** (redundant with 802.1X/MACsec on the secure path, which authenticates the same TPM credential at L2; rsyslog can't use a non-exportable TPM key without a delegated-cert or PKCS#11-tunnel contraption; hard-failing the forensic safety net on client-auth errors darkens exactly the hosts whose logs matter most — full rationale + revisit trigger in `serving/audit-collector/README.md`). Sender access constrained by the isolated boot LAN + serving-host firewall. No per-host state. |
|
||||
| 7 | **Smart card login** (`require_smart_card_login`, `sssd_enable_smartcard`, `install_smartcard_packages`) | Diskless hosts have no smartcard reader; admin access is SSH key-based. Enforcing smartcard login would lock the host out. | Admin auth is an SSH **key-only** account; root is locked; password auth disabled. |
|
||||
| 8 | **NOPASSWD sudo** (`sudo_remove_nopasswd`) | The baked admin is a **passwordless, key-only** account, so `sudo` cannot prompt for a password — NOPASSWD is required for it to function at all. | Single fleet-generic admin, key-only, `wheel`; no interactive password exists to protect. Tracked for the eventual per-host/rotatable access model (SSH CA / config-agent). |
|
||||
| 9 | **GUI / autologin / GDM** (gnome screensaver, GDM banner, automatic-login rules) | No GNOME/GDM is installed; the kiosk is an **intentional** autologin single-app (weston + Chromium) session on kiosk builds. | Kiosk user is unprivileged and passwordless; the session runs a single locked-down browser (no shell, no nav). Applies to kiosk builds only; non-kiosk builds don't install the GUI stack. |
|
||||
|
||||
@@ -79,15 +79,13 @@
|
||||
<xccdf:select idref="xccdf_org.ssgproject.content_rule_auditd_audispd_network_failure_action" selected="false"/>
|
||||
<xccdf:select idref="xccdf_org.ssgproject.content_rule_rsyslog_remote_loghost" selected="false"/>
|
||||
|
||||
<!-- rsyslog TLS offload: these force gtls + x509 peer auth + a CA file for ALL
|
||||
forwarding. The lab boot LAN is isolated and plaintext (HTTP boot, no
|
||||
MACsec); applying them makes rsyslog require TLS and BREAKS the plaintext
|
||||
2f forward to serving/audit-collector (no CA, plaintext :514). Waived for
|
||||
the lab; TLS offload is tracked as production hardening (docs/TODO.md #2).
|
||||
Re-select once the collector has a cert + the CA is bundled in the image. -->
|
||||
<xccdf:select idref="xccdf_org.ssgproject.content_rule_rsyslog_encrypt_offload_defaultnetstreamdriver" selected="false"/>
|
||||
<xccdf:select idref="xccdf_org.ssgproject.content_rule_rsyslog_encrypt_offload_actionsendstreamdrivermode" selected="false"/>
|
||||
<xccdf:select idref="xccdf_org.ssgproject.content_rule_rsyslog_encrypt_offload_actionsendstreamdriverauthmode" selected="false"/>
|
||||
<!-- rsyslog TLS offload (rsyslog_encrypt_offload_*): SELECTED/enforced since
|
||||
2026-07-12 — the 2f forward is now syslog-over-TLS (gtls :6514, x509/name
|
||||
vs the baked fleet CA /etc/dot1x/radius-ca.pem) to serving/audit-collector.
|
||||
rsyslog+rsyslog-gnutls are installed in step 2 (before this scan) so the
|
||||
rules are applicable; remediation writes the legacy gtls/x509 globals and
|
||||
the eval of record passes. Do NOT re-waive: a plaintext forward is a real
|
||||
finding now (docs/stig-deviations.md #6). -->
|
||||
|
||||
<!-- Smart card: diskless, key-based admin; no PKI/smartcard reader. Enabling
|
||||
smartcard-required login would lock the host out. -->
|
||||
|
||||
+6
-3
@@ -74,9 +74,12 @@ sudo nvme discover -t tcp -a ${SERVER_IP} -s 4420 # lists the
|
||||
# DNS: discovery name resolves to the serving host
|
||||
dig +short @${SERVER_IP} nvme.fleet.lab # -> ${SERVER_IP}
|
||||
|
||||
# Audit collector: rsyslog receiver is listening on :514 (booted hosts offload
|
||||
# auditd here; records land in audit-collector/logs/<sender-ip>/)
|
||||
ss -lntp | grep ':514'
|
||||
# Audit collector: syslog-TLS receiver on :6514 (booted hosts offload auditd
|
||||
# here; records land in audit-collector/logs/<sender-ip>/). Needs the server
|
||||
# cert issued first: audit-collector/issue-cert.sh (fleet step-ca).
|
||||
ss -lntp | grep ':6514'
|
||||
openssl s_client -connect ${SERVER_IP}:6514 -CAfile audit-collector/tls/ca.crt </dev/null # Verify return code: 0
|
||||
# End-to-end record via the LEGACY plaintext :514 intake (kept for pre-TLS images)
|
||||
logger -n ${SERVER_IP} -P 514 -T -t audispd "collector smoke test" # -> logs/<ip>/audit.log
|
||||
|
||||
# DHCP/boot: watch leases + bootfile handout
|
||||
|
||||
@@ -6,12 +6,15 @@
|
||||
|
||||
FROM harbor.bitasylum.org/dockerhub/library/alpine:3.20
|
||||
|
||||
RUN apk add --no-cache rsyslog tzdata \
|
||||
# rsyslog-tls ships lmnsd_gtls.so (gnutls netstream driver) for the :6514 TLS
|
||||
# listener (verified present in Alpine 3.20's rsyslog-tls subpackage).
|
||||
RUN apk add --no-cache rsyslog rsyslog-tls tzdata \
|
||||
&& mkdir -p /var/log/remote /var/spool/rsyslog
|
||||
|
||||
COPY rsyslog.conf /etc/rsyslog.conf
|
||||
|
||||
EXPOSE 514/tcp
|
||||
# 6514 = syslog-over-TLS (primary), 514 = legacy plaintext (migration only).
|
||||
EXPOSE 6514/tcp 514/tcp
|
||||
# Foreground (-n), our config only (-f). rsyslog logs its own startup to stderr,
|
||||
# captured by `docker compose logs audit`.
|
||||
ENTRYPOINT ["rsyslogd", "-n", "-f", "/etc/rsyslog.conf"]
|
||||
|
||||
@@ -4,28 +4,93 @@ An **rsyslog** container that receives logs forwarded by booted diskless hosts
|
||||
and writes them to durable, per-source-host files. Diskless hosts run on an
|
||||
immutable erofs root with a **RAM-only volatile overlay**, so their local
|
||||
`/var/log/audit/audit.log` is lost on every reboot — this gives auditd somewhere
|
||||
permanent to offload to. It implements the remote-audit-offload that STIG
|
||||
deviation #6 (`docs/stig-deviations.md`) lists as future hardening.
|
||||
permanent to offload to. It implements the remote-audit-offload of STIG
|
||||
deviation #6 (`docs/stig-deviations.md`); since 2026-07-12 the hop is
|
||||
**syslog-over-TLS** and the `rsyslog_encrypt_offload_*` STIG rules are enforced
|
||||
(no longer waived).
|
||||
|
||||
```
|
||||
booted host serving host
|
||||
┌───────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ auditd → audisp-syslog │ TCP/514 │ audit (this container) │
|
||||
│ → rsyslog (forward) ├──────────►│ rsyslogd → /var/log/remote/ │
|
||||
│ to ${AUDIT_HOST} │ syslog │ <sender-ip>/audit.log │
|
||||
└───────────────────────────┘ │ <sender-ip>/messages.log │
|
||||
└──────────────────────────────┘
|
||||
booted host serving host
|
||||
┌────────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ auditd → audisp-syslog │ TLS/6514 │ audit (this container) │
|
||||
│ → rsyslog (omfwd gtls) ├───────────►│ rsyslogd (gtls, fleet-CA │
|
||||
│ x509/name vs the │ syslog │ server cert) → │
|
||||
│ baked fleet CA │ │ /var/log/remote/ │
|
||||
└────────────────────────────┘ │ <sender-ip>/audit.log │
|
||||
(legacy plaintext :514 │ <sender-ip>/messages.log │
|
||||
kept for pre-TLS images) └──────────────────────────────┘
|
||||
```
|
||||
|
||||
## What this provides (the receiver)
|
||||
|
||||
- Listens on **TCP/514** (plain syslog). Disk-assisted queue so bursts / brief
|
||||
stalls don't drop records.
|
||||
- Listens on **TCP/6514, syslog-over-TLS** (rsyslog `gtls` driver). The server
|
||||
cert is issued from the **fleet step-ca** (below); senders verify it
|
||||
`x509/name` against the fleet CA baked in the image
|
||||
(`/etc/dot1x/radius-ca.pem`). Clients are **not** cert-authenticated
|
||||
(`authMode="anon"`): hosts are fleet-generic (no per-host state, invariant
|
||||
#1) and a baked fleet-wide client key would be shared secret material —
|
||||
sender access is constrained by the isolated boot LAN + the firewall below.
|
||||
- Still listens on **TCP/514 plaintext** as a **legacy/migration** intake for
|
||||
already-released pre-TLS images. Disable it (comment the input out + close
|
||||
the firewall port) once the fleet is on TLS-sending images.
|
||||
- Disk-assisted queue so bursts / brief stalls don't drop records.
|
||||
- Writes per **sender IP** (not the claimed hostname): audit records →
|
||||
`audit.log`, everything else → `messages.log`, under
|
||||
`./logs/<ip>/` (bind-mounted, inspectable on the host).
|
||||
`audit.log`, everything else → `messages.log`, under `./logs/<ip>/`
|
||||
(bind-mounted, inspectable on the host).
|
||||
- Host-networked, so hosts reach it at `${AUDIT_HOST}` (resolved to the serving
|
||||
host by the stack's dnsmasq), matching the boot/NVMe DNS pattern.
|
||||
- Relays a **copy** of audit records to the Wazuh SIEM node (see
|
||||
`rsyslog.conf`) — that hop is still plaintext (tracked in docs/TODO.md; it
|
||||
must stay `ptcp` until wazuh1 has a TLS listener).
|
||||
|
||||
## Decision: no client mTLS with the TPM identity (2026-07-12, considered + deferred)
|
||||
|
||||
Hosts already carry a fleet-CA-signed TPM identity (key at `0x81000100`,
|
||||
usable for mTLS — the fleet-agent uses it). We deliberately do **not** use it
|
||||
to client-authenticate the log stream:
|
||||
|
||||
- **Low marginal gain.** On the production/secure path the collector hop rides
|
||||
a LAN already gated by **802.1X EAP-TLS with the exact same TPM identity** +
|
||||
MACsec — client mTLS at layer 7 would re-prove a credential the network
|
||||
proved at layer 2. It also doesn't close the residual threat: a
|
||||
compromised-but-enrolled host has a valid cert and can still inject records
|
||||
about itself. And rsyslog can *validate* a client cert but can't template on
|
||||
its identity, so per-host files would still be keyed by sender IP — no
|
||||
attribution upgrade. The STIG `rsyslog_encrypt_offload_*` rules require
|
||||
server-auth TLS only; no compliance delta.
|
||||
- **Disproportionate cost.** The TPM key is non-exportable by design and
|
||||
rsyslog's gtls driver only takes PEM key *files* (no PKCS#11 URI), so this
|
||||
needs either a boot-time delegated cert (online CA round-trip *ahead of* log
|
||||
offload; key lands in tmpfs anyway — "TPM-held" degrades to "TPM-issued") or
|
||||
a PKCS#11-capable fronting daemon (ghostunnel etc.: new non-RPM daemon in the
|
||||
immutable image + fapolicyd/SELinux/reproducibility surface).
|
||||
- **Failure-mode asymmetry.** Offload is the forensic safety net; hard-failing
|
||||
it on client-auth problems makes a host with a broken TPM/cert — exactly the
|
||||
host whose logs matter most — go dark (RAM-backed spool, lost on reboot).
|
||||
|
||||
**Revisit trigger:** if cryptographically-attributed log transport becomes a
|
||||
hard requirement (ISM/SSP), route logs through the **fleet-agent's existing
|
||||
TPM-mTLS channel** (go-tpm, no PKCS#11 gymnastics) instead of teaching rsyslog
|
||||
client certs. Until then: `authMode="anon"` + boot-LAN firewall scoping (+
|
||||
`$AllowedSender` as cheap depth), with 802.1X/MACsec as the sender-authentication
|
||||
layer.
|
||||
|
||||
## TLS server cert (required before first start)
|
||||
|
||||
Issue it from the fleet CA (the dot1x-bench step-ca) into `./tls/`
|
||||
(**gitignored** — key material never enters the repo):
|
||||
|
||||
```bash
|
||||
cd serving/audit-collector
|
||||
./issue-cert.sh # SAN = ${AUDIT_HOST} from serving/.env; writes tls/{server.crt,server.key,ca.crt}
|
||||
```
|
||||
|
||||
This MUST be the same CA whose root the image was built with (`var.fleet_ca` →
|
||||
`/etc/dot1x/radius-ca.pem`) or every sender rejects the handshake and spools.
|
||||
Certs expire (default 720h): re-run `issue-cert.sh` and
|
||||
`docker compose up -d audit` to renew. Expiry symptom: collector logs TLS
|
||||
handshake errors, sender dirs stop growing, hosts spool
|
||||
(`action.resumeRetryCount="-1"` — records drain after renewal, not lost).
|
||||
|
||||
Run it with the rest of the stack:
|
||||
|
||||
@@ -37,82 +102,77 @@ docker compose logs -f audit # startup + connection messages
|
||||
tail -F audit-collector/logs/*/audit.log
|
||||
```
|
||||
|
||||
Quick self-test (send a line as if from a host):
|
||||
Quick self-tests:
|
||||
|
||||
```bash
|
||||
# TLS handshake + cert chain (expect Verify return code: 0 against the fleet CA)
|
||||
openssl s_client -connect 127.0.0.1:6514 -CAfile audit-collector/tls/ca.crt </dev/null
|
||||
|
||||
# End-to-end record over the LEGACY plaintext intake (still simplest smoke test)
|
||||
logger -n 127.0.0.1 -P 514 -T -t audispd "test audit record"
|
||||
ls audit-collector/logs/ # a 127.0.0.1/ dir appears
|
||||
```
|
||||
|
||||
## Serving-host firewall — open 514/tcp (easy to miss)
|
||||
## Serving-host firewall — open 6514/tcp (easy to miss)
|
||||
|
||||
This container uses **host networking**, so the **serving host's own firewall
|
||||
governs port 514** — Docker does *not* manage iptables for host-net containers
|
||||
(the usual "Docker bypasses the host firewall" behaviour only applies to
|
||||
*published* bridge ports, which insert rules into the `DOCKER` chain ahead of the
|
||||
host firewall). The symptom when it's closed is deceptive: the container is
|
||||
healthy, `ss -tlnp` shows `0.0.0.0:514` listening, DNS/DHCP from the same host
|
||||
work, but booted hosts sit in `SYN-SENT` and rsyslog logs `cannot connect to
|
||||
audit.fleet.lab:514 ... Connection timed out` (timed out = **dropped**, not
|
||||
refused). The logs aren't lost — the sender's `action.resumeRetryCount="-1"`
|
||||
buffers them and drains once the port opens.
|
||||
governs port 6514** (and legacy 514) — Docker does *not* manage iptables for
|
||||
host-net containers (the usual "Docker bypasses the host firewall" behaviour
|
||||
only applies to *published* bridge ports, which insert rules into the `DOCKER`
|
||||
chain ahead of the host firewall). The symptom when it's closed is deceptive:
|
||||
the container is healthy, `ss -tlnp` shows `0.0.0.0:6514` listening, DNS/DHCP
|
||||
from the same host work, but booted hosts sit in `SYN-SENT` and rsyslog logs
|
||||
`cannot connect to audit.fleet.lab:6514 ... Connection timed out` (timed out =
|
||||
**dropped**, not refused). The logs aren't lost — the sender's
|
||||
`action.resumeRetryCount="-1"` buffers them and drains once the port opens.
|
||||
(This exact trap was hit live on :514 — 2026-06-19, wrong NIC in the ufw rule.)
|
||||
|
||||
Open 514/tcp **scoped to the boot LAN** (don't expose it globally):
|
||||
Open 6514/tcp (+ 514/tcp while legacy senders remain) **scoped to the boot
|
||||
LAN** (don't expose it globally):
|
||||
|
||||
```bash
|
||||
# ufw (Debian/Ubuntu serving host):
|
||||
sudo ufw allow from 192.168.50.0/24 to any port 514 proto tcp
|
||||
sudo ufw allow from 192.168.50.0/24 to any port 6514 proto tcp
|
||||
sudo ufw allow from 192.168.50.0/24 to any port 514 proto tcp # legacy, until migrated
|
||||
sudo ufw reload && sudo ufw status numbered
|
||||
|
||||
# firewalld (RHEL/Rocky serving host):
|
||||
sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=192.168.50.0/24 port port=514 protocol=tcp accept'
|
||||
sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=192.168.50.0/24 port port=6514 protocol=tcp accept'
|
||||
sudo firewall-cmd --reload
|
||||
```
|
||||
|
||||
Verify SYNs now complete (SYN → SYN-ACK) and a sender dir appears:
|
||||
|
||||
```bash
|
||||
sudo tcpdump -ni any 'tcp port 514 and host <sender-ip>' # SYN-ACK now returns
|
||||
ls -R audit-collector/logs/ # <sender-ip>/ appears within ~1 min
|
||||
sudo tcpdump -ni any 'tcp port 6514 and host <sender-ip>' # SYN-ACK + TLS, no cleartext
|
||||
ls -R audit-collector/logs/ # <sender-ip>/ appears within ~1 min
|
||||
```
|
||||
|
||||
> If you ever move the collector from host networking to **published ports**
|
||||
> (`514:514`), this firewall rule stops applying — published Docker ports bypass
|
||||
> ufw/firewalld and must be controlled via Docker (`DOCKER-USER` chain) instead.
|
||||
> (`6514:6514`), this firewall rule stops applying — published Docker ports
|
||||
> bypass ufw/firewalld and must be controlled via Docker (`DOCKER-USER` chain)
|
||||
> instead.
|
||||
|
||||
## What's still needed (the sender — in the image)
|
||||
## The sender (in the image — wired)
|
||||
|
||||
This container only **receives**. To make hosts actually send, the hardened
|
||||
image needs (not yet wired — keep `no per-host state`, use the generic
|
||||
`${AUDIT_HOST}` name, never a per-host address):
|
||||
Provisioner **2f** in `diskless.pkr.hcl` (gated on the `audit_host` var; empty =
|
||||
local-only) configures each host to send — host-generic, no per-host state:
|
||||
|
||||
1. **auditd → syslog**: enable the `audisp-syslog` plugin
|
||||
(`/etc/audit/plugins.d/syslog.conf`: `active = yes`), so audit events also go
|
||||
to the local syslog socket.
|
||||
2. **rsyslog forward rule**: a drop-in (e.g. `/etc/rsyslog.d/60-fleet-remote.conf`)
|
||||
forwarding to the collector with a disk-assisted action queue, so a network
|
||||
blip doesn't drop records:
|
||||
1. **auditd → syslog**: the `audisp-syslog` plugin is enabled on LOCAL6, so
|
||||
audit events also go to the local syslog socket.
|
||||
2. **rsyslog forward**: `/etc/rsyslog.d/60-fleet-remote.conf` forwards `*.*` to
|
||||
`${AUDIT_HOST}:6514` over `gtls` (mode 1, `x509/name`,
|
||||
`StreamDriverPermittedPeers=${AUDIT_HOST}`), trusting
|
||||
`/etc/dot1x/radius-ca.pem` (the baked fleet CA), with a disk-assisted queue
|
||||
so a network blip / collector outage doesn't drop records.
|
||||
|
||||
```rsyslog
|
||||
*.* action(type="omfwd" target="audit.fleet.lab" port="514" protocol="tcp"
|
||||
queue.type="LinkedList" queue.fileName="fwdq"
|
||||
queue.maxDiskSpace="256m" queue.saveOnShutdown="on"
|
||||
action.resumeRetryCount="-1")
|
||||
```
|
||||
3. Bake `audit.fleet.lab` (the generic name) — DNS resolves it per environment.
|
||||
## Production hardening (remaining)
|
||||
|
||||
Once the sender is wired, flip STIG deviation #6 in `docs/stig-deviations.md`
|
||||
from "local-only" to "offloading to ${AUDIT_HOST}" and the
|
||||
`rsyslog_remote_loghost` rule should pass.
|
||||
|
||||
## Production hardening (lab simplifications to revisit)
|
||||
|
||||
The lab stack is **plaintext TCP**, mirroring the HTTP-boot/no-TLS posture. For
|
||||
a secured fleet:
|
||||
|
||||
- **TLS**: rsyslog `gtls` driver (mutual auth with the boot-infra CA), or
|
||||
**RELP + TLS** (`imrelp`/`omrelp`) for guaranteed, acknowledged delivery.
|
||||
- **Restrict senders** by subnet (`$AllowedSender`) or firewall :514 to the
|
||||
boot LAN.
|
||||
- **Disable the legacy :514 plaintext intake** once no pre-TLS image is in
|
||||
service (comment out the input in `rsyslog.conf`, rebuild, close the port).
|
||||
- **TLS the Wazuh relay hop** (collector → wazuh1) — see `serving/siem/` +
|
||||
docs/TODO.md.
|
||||
- **Restrict senders** by subnet (`$AllowedSender`) as defense-in-depth beyond
|
||||
the firewall scope.
|
||||
- Add log rotation/retention (logrotate sidecar or rsyslog `outchannel`) and
|
||||
ship to long-term storage / SIEM.
|
||||
long-term archival per ISM event-logging retention.
|
||||
|
||||
Executable
+59
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env bash
|
||||
# issue-cert.sh — issue the audit collector's syslog-over-TLS server cert from
|
||||
# the fleet CA (the dot1x-bench step-ca) into ./tls/, where docker-compose
|
||||
# mounts it read-only at /etc/rsyslog-tls for the :6514 gtls listener.
|
||||
#
|
||||
# The cert's SAN must be the name senders verify (x509/name): ${AUDIT_HOST},
|
||||
# the same host-generic DNS name baked into the image's 2f forward and resolved
|
||||
# by the serving dnsmasq. Hosts validate it against the fleet CA root already
|
||||
# baked at /etc/dot1x/radius-ca.pem — so this MUST be the same CA whose root
|
||||
# the image was built with (var.fleet_ca).
|
||||
#
|
||||
# Same in-container issuance pattern as dot1x-bench/step-ca/bootstrap.sh: `step`
|
||||
# runs inside the step-ca container; /export = dot1x-bench/pki on the host.
|
||||
#
|
||||
# Certs expire (default 720h — see bootstrap.sh CA claims): re-run this script
|
||||
# and `docker compose restart audit` to renew. If the collector starts logging
|
||||
# TLS handshake errors and senders spool, check expiry first:
|
||||
# openssl x509 -in tls/server.crt -noout -enddate
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")"
|
||||
COLLECTOR_DIR=$PWD
|
||||
BENCH_DIR=../dot1x-bench
|
||||
|
||||
# AUDIT_HOST from the serving stack's .env (the name dnsmasq resolves for hosts).
|
||||
[ -f ../.env ] && . ../.env
|
||||
AUDIT_HOST=${AUDIT_HOST:-audit.fleet.lab}
|
||||
|
||||
cd "$BENCH_DIR"
|
||||
[ -f .env ] && . ./.env
|
||||
CA_PORT=${CA_PORT:-8443}
|
||||
CA_CONNECT=${CA_CONNECT:-localhost} # see bootstrap.sh: SAN-valid AND resolvable
|
||||
CA_URL="https://${CA_CONNECT}:${CA_PORT}"
|
||||
CONTAINER_PW="/home/step/password" # bootstrap.sh: pki/stepca/password in-container
|
||||
|
||||
step() { docker compose exec -T step-ca step "$@"; }
|
||||
incca() { docker compose exec -T step-ca "$@"; }
|
||||
|
||||
echo "==> issuing audit-collector cert (CN/SAN: $AUDIT_HOST) from the fleet CA"
|
||||
incca mkdir -p /export/audit
|
||||
step ca certificate "$AUDIT_HOST" \
|
||||
/export/audit/server.crt /export/audit/server.key \
|
||||
--ca-url "$CA_URL" --root /home/step/certs/root_ca.crt \
|
||||
--provisioner enrol --provisioner-password-file "$CONTAINER_PW" \
|
||||
--not-after "${AUDIT_CERT_DUR:-720h}" --force
|
||||
incca cp /home/step/certs/root_ca.crt /export/audit/ca.crt
|
||||
|
||||
# Stage into ./tls/ (compose bind-mounts it). pki/ may be uid-1000/root-owned
|
||||
# (see bootstrap.sh perm notes) — copy via the container's /export if needed.
|
||||
mkdir -p "$COLLECTOR_DIR/tls" 2>/dev/null || { echo "==> fixing tls/ perms with sudo (lab)"; sudo mkdir -p "$COLLECTOR_DIR/tls" && sudo chown "$(id -u)" "$COLLECTOR_DIR/tls"; }
|
||||
for f in server.crt server.key ca.crt; do
|
||||
cp pki/audit/"$f" "$COLLECTOR_DIR/tls/$f" 2>/dev/null \
|
||||
|| { echo "==> pki/audit unreadable by this user; fixing with sudo (lab)"; sudo cp pki/audit/"$f" "$COLLECTOR_DIR/tls/$f" && sudo chown "$(id -u)" "$COLLECTOR_DIR/tls/$f"; }
|
||||
done
|
||||
chmod 0600 "$COLLECTOR_DIR/tls/server.key"
|
||||
chmod 0644 "$COLLECTOR_DIR/tls/server.crt" "$COLLECTOR_DIR/tls/ca.crt"
|
||||
|
||||
echo "==> done: $(openssl x509 -in "$COLLECTOR_DIR/tls/server.crt" -noout -subject -enddate 2>/dev/null || echo 'tls/server.crt staged')"
|
||||
echo " apply: (cd .. && docker compose up -d audit) # or restart if already up"
|
||||
@@ -1,19 +1,39 @@
|
||||
# Remote audit / syslog collector for the diskless fleet (LAB).
|
||||
# Remote audit / syslog collector for the diskless fleet.
|
||||
#
|
||||
# Receives logs forwarded by booted diskless hosts over plain TCP/514 and writes
|
||||
# them per-source-host. Diskless hosts have an immutable, RAM-only overlay root,
|
||||
# Receives logs forwarded by booted diskless hosts and writes them
|
||||
# per-source-host. Diskless hosts have an immutable, RAM-only overlay root,
|
||||
# so their local audit log dies on reboot — this gives auditd somewhere durable
|
||||
# to offload to (STIG remote-audit-offload, deviation #6 in docs/stig-deviations.md).
|
||||
#
|
||||
# Lab posture: PLAINTEXT TCP, mirroring the HTTP-boot/no-TLS lab stack. For
|
||||
# production switch to TLS (gtls driver) or RELP+TLS and restrict senders by
|
||||
# subnet — see README.md.
|
||||
# Primary intake is syslog-over-TLS on 6514 (gtls; server cert issued by the
|
||||
# fleet step-ca — see issue-cert.sh + README.md). Hosts authenticate the
|
||||
# collector via the fleet CA baked in the image (x509/name); the collector does
|
||||
# NOT require client certs (authMode "anon"): hosts are fleet-generic (no
|
||||
# per-host state, invariant #1) and a baked fleet-wide client key would be
|
||||
# shared secret material in the image. Sender access is instead constrained by
|
||||
# the isolated boot LAN + the serving-host firewall scoping :6514 to it.
|
||||
|
||||
global(workDirectory="/var/spool/rsyslog")
|
||||
global(
|
||||
workDirectory="/var/spool/rsyslog"
|
||||
# TLS material for the 6514 listener, mounted from ./tls (issue-cert.sh).
|
||||
# NOTE: deliberately NOT setting DefaultNetstreamDriver="gtls" here — that
|
||||
# would flip every plain `protocol="tcp"` omfwd (the Wazuh relay below) to
|
||||
# TLS and break it. TLS is enabled per-listener instead.
|
||||
DefaultNetstreamDriverCAFile="/etc/rsyslog-tls/ca.crt"
|
||||
DefaultNetstreamDriverCertFile="/etc/rsyslog-tls/server.crt"
|
||||
DefaultNetstreamDriverKeyFile="/etc/rsyslog-tls/server.key"
|
||||
)
|
||||
|
||||
# --- intake: plain TCP syslog ---
|
||||
# --- intake: syslog-over-TLS (the hardened image's 2f sender targets this) ---
|
||||
module(load="imtcp" maxSessions="1000")
|
||||
input(type="imtcp" port="514")
|
||||
input(type="imtcp" port="6514"
|
||||
streamDriver.name="gtls" streamDriver.mode="1" streamDriver.authMode="anon")
|
||||
|
||||
# --- intake: plain TCP syslog (LEGACY/migration) ---
|
||||
# Kept for already-released images that still forward plaintext to :514.
|
||||
# Disable (comment out + close the firewall port) once the fleet is on
|
||||
# TLS-sending images — production must not accept plaintext audit records.
|
||||
input(type="imtcp" port="514" streamDriver.name="ptcp")
|
||||
# Some senders only speak UDP; uncomment to also accept it (lossy, no queue).
|
||||
#module(load="imudp")
|
||||
#input(type="imudp" port="514")
|
||||
|
||||
@@ -245,19 +245,25 @@ services:
|
||||
# --- Remote audit / log collector (RECEIVE side) ---------------------------
|
||||
# rsyslog receiver: booted diskless hosts (whose RAM-only overlay root loses
|
||||
# local logs on reboot) forward auditd via audisp-syslog + syslog here over
|
||||
# TCP/514. Records are written per-source-host under
|
||||
# syslog-TLS/6514 (gtls; legacy plaintext :514 kept for pre-TLS images during
|
||||
# migration). Records are written per-source-host under
|
||||
# ./audit-collector/logs/<sender-ip>/{audit,messages}.log. Host-networked so
|
||||
# hosts can reach it at ${AUDIT_HOST} (= this host) on the boot LAN.
|
||||
# NOTE: this is the receiver only — the IMAGE must be configured to send
|
||||
# (audisp-syslog + an rsyslog forward rule targeting ${AUDIT_HOST}); see
|
||||
# audit-collector/README.md. Until then the collector simply waits, idle.
|
||||
# BEFORE first start: issue the TLS server cert into ./audit-collector/tls/
|
||||
# with audit-collector/issue-cert.sh (fleet step-ca; SAN = ${AUDIT_HOST} — the
|
||||
# image's rsyslog verifies it x509/name against the baked fleet CA, so it MUST
|
||||
# be the same CA as the build's var.fleet_ca).
|
||||
# FIREWALL (host networking — same trap as chrony's 123/udp above): the
|
||||
# serving host's own firewall governs 6514/tcp (+ legacy 514/tcp); scope them
|
||||
# to the boot LAN: `ufw allow in on <boot-nic> to any port 6514 proto tcp`.
|
||||
audit:
|
||||
build: ./audit-collector
|
||||
network_mode: host # receive from booted hosts on the boot LAN (:514)
|
||||
network_mode: host # receive from booted hosts on the boot LAN (:6514)
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./audit-collector/logs:/var/log/remote # durable, inspectable on host
|
||||
- ./audit-collector/spool:/var/spool/rsyslog # disk-assisted queue
|
||||
- ./audit-collector/tls:/etc/rsyslog-tls:ro # server cert/key + fleet CA (issue-cert.sh)
|
||||
|
||||
# --- NVMe/TCP target (serves rootfs.erofs, read-only) ----------------------
|
||||
nvmeof:
|
||||
|
||||
Reference in New Issue
Block a user