diskless-workstation/docs/oqe-collection.md

OQE Collection Plan — Automated Objective Quality Evidence for the IRAP Assessment

Protective marking: STARFLEET RESTRICTED — CLEARANCE LEVEL 2 — BRIDGE CREW / SECURITY DIVISION (marking scheme per the SAR banner.)

Status: working plan (v0.1, 2026-07-11) — recorded from assessor guidance; not yet implemented. Companion to the SAR and POA&M; feeds the continuous-monitoring plan (POA&M P-08 M2) and the delta-assessment witness points (SAR §7.2).

1. Purpose

Define how Objective Quality Evidence (OQE) for the assessment is captured automatically, so that control effectiveness is demonstrated by machine-collected, attributable artifacts rather than screenshots and prose assertions — and so the delta assessment (and later, ongoing authorisation) can sample and re-verify evidence instead of regenerating it by hand.

This system is unusually well-positioned for automated OQE because it already has the three prerequisites:

1. A digest that uniquely names the thing being assessed (one release = one OCI digest = one manifest commit).
2. An authenticated channel from every host (fleet-agent over TPM-mTLS — evidence is cryptographically attributable to a specific hardware host).
3. A CI pipeline that produces the security-relevant state (signing, STIG scan, verity coupling, content assertions).

2. What makes an artifact OQE

Every evidence artifact must answer four questions without a human interpreting it:

Question	Requirement
What does it demonstrate?	Mapped to a control/claim (ISM control ID + SSP/Annex A row + POA&M milestone).
Which system produced it?	Exact attribution: OCI digest for builds; host serial (TPM cert serial) for hosts — never "the test box".
When was it captured?	Timestamp from the authenticated time source (NTS, post-P-18).
How was it produced?	The collection method is in-repo and inspectable — the assessor reads the collector before trusting its output.

Evidence strength ordering (prefer higher): signed attestation > raw command output with attribution > screenshot/prose assertion. Automation removes the human from the capture path — but only if the automation itself is auditable.

3. The four collection planes

3.1 Build-time evidence — attached to the OCI artifact

CI already produces most of it. Formalise into a per-build evidence bundle, emitted as a CI artifact and (target state) attached to the OCI image as a signed attestation (cosign/in-toto via the registry referrers API), so OQE is keyed to the digest: assessing release sha256:abc… means pulling its evidence chain from the registry.

Evidence	Source	Exists today?
STIG ARF + HTML report + deviations register (0 unjustified REVIEW)	CI artifacts (stig-report.html, stig-arf.xml, stig-deviations.md)	Yes
roothash coupling (UKI cmdline ↔ verity.json)	build-oci.sh hard gate result	Yes (gate; log it)
Initramfs content assertions (drivers, module, nvmf unit, NM hook)	lsinitrd assertions in the template	Yes (assert; capture listing)
SELinux labels survived export	getfattr build assertion	Yes (assert; capture output)
UKI + iPXE signature verification	sbverify output	Add
Extracted (immutable) cmdline + its hash	extract-artifacts.sh dump	Add
RPM manifest (+ size manifest)	rpm -qa dump (size manifest exists)	Partial
Reproducibility proof	second build → identical verity.json root hash	Add (CI job)
Production-profile assertion (which toggles/gates were active)	CI Production build gate output	Partial (gate exists — emit a machine-readable record)
Signed provenance over the whole bundle	cosign/in-toto attestation on the OCI image	Add (target state)

Supports: SAR §7.2 items 2 and 7; POA&M P-03 M2 (profile-tagged artifacts), P-04 M2, P-05.

3.2 Bench evidence — the harness emits structured results

Bench gates are witnessed live, but the harness should capture as it runs: a JSON results file per run — each pass criterion with its raw output, stamped with host serial, UKI digest, and timestamps. The assessor verifies the harness once, watches one live run, and keeps the bundle.

Bench gate	Captured criteria (raw outputs)	POA&M
Secure path (802.1X/MACsec)	wpa_cli status = Authorized; ip macsec show; macsec0 DHCP + renewal journal lines; nvme list-subsys (targeted NQN); supplicant PID alive post-switch-root	P-01
Application control	novel non-RPM binary in /tmp → fapolicyd FAN_DENY audit record	SAR §7.2(7)
Device control	USB mass-storage block event (LinuxAudit) + console HID still functional	SAR §7.2(7)
SSH host certificates	ssh -v host-cert offer; @cert-authority validation; broker signing log line	P-17 M3
Authenticated time	chronyc authdata (NTS cookies present); tcp/4460 handshake; cold-clock nocerttimecheck path	P-18 M3
Agent actuators inert	desired-state/command received → log-only, no mutation	P-23 / SAR §7.2(7)

3.3 Runtime/continuous evidence — the fleet-agent as evidence pipeline

The agent already streams SELinux state, chrony health, image version, and cert serial over TPM-mTLS — continuous, hardware-attributed OQE for those controls. Extending it per POA&M P-21 (SAR F-07) makes the InfluxDB store a rolling posture record: every host, hourly, attributed to its TPM.

Facts to add (P-21): Secure Boot state (SecureBoot EFI var), fapolicyd enforcement mode, kernel cmdline hash, USBGuard status.

This time-series is the substance of the continuous-monitoring plan (P-08 M2) and the ongoing-authorisation story; its retention follows the P-08/P-09 log-retention policy so the evidence itself survives. It also provides machine-checkable closure for other items — e.g. P-05 M2 closes when telemetry shows mokutil --sb-state = enabled for 100% of enrolled hosts.

3.4 Detect-pillar evidence — synthetic control tests

Exercise the detection path on a schedule and record the result: a probe runs sudo on a canary host, then confirms the USER_CMD event (a) landed in the collector's per-host file and (b) fired the Wazuh fleet_audit rule, within N seconds. Log each probe result. A control tested weekly with recorded outcomes is far stronger OQE than "the rule existed at assessment time."

Supports: Annex A §2.6 rows; POA&M P-09.

4. The mapping layer (where automation usually fails)

Evidence without a control map is a folder of files. Maintain a machine-readable matrix in-repo — docs/oqe-matrix.yaml (to be scaffolded) — one row per claim:

- id: OQE-013
  control: ISM-1542            # key against the ACSC OSCAL ISM edition
  claim: "Only fleet-signed UKIs boot under Secure Boot"   # SSP/Annex A row
  plane: build                  # build | bench | runtime | detect
  collector: "sbverify --cert fleet-db.pem fleet-<ver>.efi"
  frequency: per-build
  closes: P-05                  # POA&M milestone this evidences

Two decisions recorded with it:

1. Key controls against the ACSC OSCAL edition of the ISM. The ISM is published in OSCAL; stable machine-readable identifiers per release make the May-2026 renumbering reconciliation (P-10) a computable diff instead of a manual crosswalk.
2. Define closure evidence per POA&M milestone. Each M-milestone states its machine-checkable "done" signal in the matrix, so POA&M status stops being self-asserted.

5. Integrity and chain of custody

Evidence about a security system is itself a target. Proportionate measures at this classification:

• Build evidence: covered by the CI attestation signature (target state); until then, SHA256 manifests of each bundle committed to git — the one-commit-per-release convention makes git history the tamper-evident ledger.
• Runtime evidence: inherently attributed by the agent's mTLS session + TPM-bound identity.
• Bench bundles: hashed into a committed manifest per run.
• Timestamps: ride the NTS-verified clock (P-18).
• Storage: append-only files on the collector + committed manifests. An elaborate WORM store is not proportionate at STARFLEET RESTRICTED.

6. What the assessor still does in person

Automation changes the character of the SAR §7.2 witness points, it does not replace them. At the delta assessment the assessor will:

1. Read the collectors — confirm they measure what they claim.
2. Watch one live run of each critical bench (P-01 above all).
3. Independently re-run a sample of collection commands and diff against the automated bundles.

If the automated evidence survives that sampling, everything it produced between assessments is credible — which is what makes ongoing authorisation cheap instead of an annual re-assessment.

7. Implementation backlog (feeds TODO.md / POA&M)

#	Task	Plane	Pairs with
O-1	CI evidence job: bundle existing artifacts + add sbverify/cmdline/rpm-manifest dumps; emit machine-readable production-gate record	Build	P-03 M2
O-2	Reproducibility CI job (double build, compare root hash)	Build	SAR §7.2(7)
O-3	Signed attestation on the OCI image (cosign/in-toto, referrers API)	Build	P-03 M2
O-4	Bench harness structured JSON output (serving/dot1x-bench/ first)	Bench	P-01
O-5	Agent posture facts: SB state, fapolicyd mode, cmdline hash, USBGuard	Runtime	P-21
O-6	Synthetic detect probe (canary sudo → collector file + Wazuh alert, scheduled)	Detect	P-09
O-7	Scaffold docs/oqe-matrix.yaml from current POA&M rows; adopt OSCAL ISM keys	Mapping	P-10, P-08 M2
O-8	Define closure-evidence signal for every open POA&M milestone in the matrix	Mapping	all

---

End of OQE collection plan. Review alongside the POA&M at each monthly review; promote O-items into TODO.md as they are scheduled.