Files
diskless-workstation/serving/audit-collector
2026-06-27 08:52:17 +00:00
..
2026-06-18 12:03:45 +00:00
2026-06-18 12:03:45 +00:00
2026-06-18 12:03:45 +00:00
2026-06-19 04:00:47 +00:00
2026-06-20 00:05:19 +00:00
2026-06-27 08:52:17 +00:00

Audit / log collector (receive side)

An rsyslog container that receives logs forwarded by booted diskless hosts and writes them to durable, per-source-host files. Diskless hosts run on an immutable erofs root with a RAM-only volatile overlay, so their local /var/log/audit/audit.log is lost on every reboot — this gives auditd somewhere permanent to offload to. It implements the remote-audit-offload that STIG deviation #6 (docs/stig-deviations.md) lists as future hardening.

booted host                              serving host
┌───────────────────────────┐           ┌──────────────────────────────┐
│ auditd → audisp-syslog     │  TCP/514  │ audit (this container)        │
│      → rsyslog (forward)   ├──────────►│ rsyslogd → /var/log/remote/   │
│         to ${AUDIT_HOST}   │  syslog   │   <sender-ip>/audit.log       │
└───────────────────────────┘           │   <sender-ip>/messages.log    │
                                         └──────────────────────────────┘

What this provides (the receiver)

  • Listens on TCP/514 (plain syslog). Disk-assisted queue so bursts / brief stalls don't drop records.
  • Writes per sender IP (not the claimed hostname): audit records → audit.log, everything else → messages.log, under ./logs/<ip>/ (bind-mounted, inspectable on the host).
  • Host-networked, so hosts reach it at ${AUDIT_HOST} (resolved to the serving host by the stack's dnsmasq), matching the boot/NVMe DNS pattern.

Run it with the rest of the stack:

cd serving
docker compose build audit
docker compose up -d audit
docker compose logs -f audit          # startup + connection messages
tail -F audit-collector/logs/*/audit.log

Quick self-test (send a line as if from a host):

logger -n 127.0.0.1 -P 514 -T -t audispd "test audit record"
ls audit-collector/logs/                 # a 127.0.0.1/ dir appears

Serving-host firewall — open 514/tcp (easy to miss)

This container uses host networking, so the serving host's own firewall governs port 514 — Docker does not manage iptables for host-net containers (the usual "Docker bypasses the host firewall" behaviour only applies to published bridge ports, which insert rules into the DOCKER chain ahead of the host firewall). The symptom when it's closed is deceptive: the container is healthy, ss -tlnp shows 0.0.0.0:514 listening, DNS/DHCP from the same host work, but booted hosts sit in SYN-SENT and rsyslog logs cannot connect to audit.fleet.lab:514 ... Connection timed out (timed out = dropped, not refused). The logs aren't lost — the sender's action.resumeRetryCount="-1" buffers them and drains once the port opens.

Open 514/tcp scoped to the boot LAN (don't expose it globally):

# ufw (Debian/Ubuntu serving host):
sudo ufw allow from 192.168.50.0/24 to any port 514 proto tcp
sudo ufw reload && sudo ufw status numbered

# firewalld (RHEL/Rocky serving host):
sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=192.168.50.0/24 port port=514 protocol=tcp accept'
sudo firewall-cmd --reload

Verify SYNs now complete (SYN → SYN-ACK) and a sender dir appears:

sudo tcpdump -ni any 'tcp port 514 and host <sender-ip>'   # SYN-ACK now returns
ls -R audit-collector/logs/                                 # <sender-ip>/ appears within ~1 min

If you ever move the collector from host networking to published ports (514:514), this firewall rule stops applying — published Docker ports bypass ufw/firewalld and must be controlled via Docker (DOCKER-USER chain) instead.

What's still needed (the sender — in the image)

This container only receives. To make hosts actually send, the hardened image needs (not yet wired — keep no per-host state, use the generic ${AUDIT_HOST} name, never a per-host address):

  1. auditd → syslog: enable the audisp-syslog plugin (/etc/audit/plugins.d/syslog.conf: active = yes), so audit events also go to the local syslog socket.

  2. rsyslog forward rule: a drop-in (e.g. /etc/rsyslog.d/60-fleet-remote.conf) forwarding to the collector with a disk-assisted action queue, so a network blip doesn't drop records:

    *.* action(type="omfwd" target="audit.fleet.lab" port="514" protocol="tcp"
               queue.type="LinkedList" queue.fileName="fwdq"
               queue.maxDiskSpace="256m" queue.saveOnShutdown="on"
               action.resumeRetryCount="-1")
    
  3. Bake audit.fleet.lab (the generic name) — DNS resolves it per environment.

Once the sender is wired, flip STIG deviation #6 in docs/stig-deviations.md from "local-only" to "offloading to ${AUDIT_HOST}" and the rsyslog_remote_loghost rule should pass.

Production hardening (lab simplifications to revisit)

The lab stack is plaintext TCP, mirroring the HTTP-boot/no-TLS posture. For a secured fleet:

  • TLS: rsyslog gtls driver (mutual auth with the boot-infra CA), or RELP + TLS (imrelp/omrelp) for guaranteed, acknowledged delivery.
  • Restrict senders by subnet ($AllowedSender) or firewall :514 to the boot LAN.
  • Add log rotation/retention (logrotate sidecar or rsyslog outchannel) and ship to long-term storage / SIEM.