Files

T

Audiobuzz d2b2156a22 Fixed the secure boot enrollment script

2026-06-20 12:16:37 +00:00

13 KiB

Raw Blame History

Secure Boot key generation & enrollment — diskless fleet

The fleet boots a signed UKI directly via systemd-stub (no shim, no GRUB), so UEFI Secure Boot verifies the UKI against keys you enrol in firmware. This doc generates the three Secure Boot keys (the "trio": PK, KEK, db), enrols them, and wires the db key into the build.

Invariant #7: signing keys never enter the build VM or this repo. The commands below produce key material that lives offline / in an HSM. Only the db key signs UKIs routinely, and the build receives it as a path (sb_key/sb_cert) on the CI/HSM host — never committed, never echoed in logs. Treat every *.key file here as secret; never check one into git.

The trio — roles and where each lives

Key	Role	Used by	Where it lives
PK (Platform Key)	Root of trust; controls who may update the KEK. Setting it locks Secure Boot into User Mode.	Firmware enrolment only	Offline / HSM, used once (and for KEK rotation)
KEK (Key Exchange Key)	Signs updates to `db`/`dbx`.	Firmware enrolment only	Offline / HSM
db (Signature Database)	Signs the bootable UKI.	The fleet build — `sb_key`/`sb_cert` (../scripts/build-oci.sh → ukify/sbsign)	HSM/CI signing host

Trust chain at enrolment: PK signs KEK → KEK signs db → db signs the UKI.

Only db is online (it signs every release; the UKI cmdline is immutable once signed — invariant #6 — so re-signing is part of each build). PK and KEK are only needed to rotate keys later, so they stay offline.

Tooling

# Fedora / RHEL:   dnf install openssl efitools sbsigntools
# Debian / Ubuntu: apt install openssl efitools sbsigntool

1. Generate the three key/cert pairs

GUID=$(uuidgen); echo "$GUID" > GUID.txt    # owner GUID for the EFI variables

# Platform Key
openssl req -newkey rsa:2048 -nodes -keyout PK.key -new -x509 -sha256 -days 3650 \
  -subj "/CN=Fleet Platform Key/" -out PK.crt
# Key Exchange Key
openssl req -newkey rsa:2048 -nodes -keyout KEK.key -new -x509 -sha256 -days 3650 \
  -subj "/CN=Fleet KEK/" -out KEK.crt
# Signature DB key  <-- this pair becomes sb_key / sb_cert
openssl req -newkey rsa:2048 -nodes -keyout db.key -new -x509 -sha256 -days 3650 \
  -subj "/CN=Fleet db/" -out db.crt

rsa:2048 is the maximally firmware-compatible size for the enrolled variables; 4096 works on most modern UEFI but some implementations reject it for PK/KEK/db. The PEM db.key/db.crt are exactly what sbsign/ukify consume.

2. Convert to signed EFI variable updates (`.auth`)

for v in PK KEK db; do cert-to-efi-sig-list -g "$GUID" $v.crt $v.esl; done

sign-efi-sig-list -g "$GUID" -k PK.key  -c PK.crt  PK  PK.esl  PK.auth   # PK self-signs
sign-efi-sig-list -g "$GUID" -k PK.key  -c PK.crt  KEK KEK.esl KEK.auth  # PK signs KEK
sign-efi-sig-list -g "$GUID" -k KEK.key -c KEK.crt db  db.esl  db.auth   # KEK signs db

3. Enrol into firmware

Put the firmware in Setup Mode first (clear the existing keys in the UEFI setup screen). Then enrol db and KEK before PK — setting PK last is what flips the firmware out of Setup Mode and turns Secure Boot on:

efi-updatevar -f db.auth  db
efi-updatevar -f KEK.auth KEK
efi-updatevar -f PK.auth  PK      # do this LAST

Alternatively, enrol the .auth/.esl files via the firmware's own "enrol key from file" UI or KeyTool.efi — useful for headless fleet hardware where this is scripted into provisioning.

The fleet uses a UKI with no shim and no Microsoft keys, so no MOK is needed. If a host must also boot vendor-signed media, add your db alongside Microsoft's UEFI CA instead of clearing it.

On a fleet workstation: read state over SSH, enrol via firmware

efitools (efi-readvar/efi-updatevar/KeyTool) is NOT packaged for EL9 (confirmed on Rocky 9, 2026-06-20). The image therefore ships only efivar/mokutil/efibootmgr (base repos). That's enough to read Secure Boot state over SSH, but enrolment of custom PK/KEK/db is done in the firmware setup UI, not from the OS.

Read Secure Boot state over SSH (mokutil is the efi-readvar replacement; acts on efivarfs, works on the read-only root, RPM-installed so fapolicyd-trusted):

mokutil --sb-state     # enabled / setup mode
mokutil --pk           # Platform Key
mokutil --kek          # KEK
mokutil --db           # signature db — your enrolled fleet db cert appears here
mokutil --dbx          # revocations

Enrol — two paths:

(a) Baked helper over SSH (fleet-sb-enroll) — the OS-side, efitools-free path. The build bakes /usr/local/bin/fleet-sb-enroll plus the public PK/KEK/db.auth at /usr/share/fleet-secureboot/ (supplied via files/secureboot/keys/ or the SB_*_AUTH_B64 CI secrets). On a workstation in Setup Mode:

sudo fleet-sb-enroll --check     # show SB state + which keys are baked
sudo fleet-sb-enroll             # enrol db -> KEK -> PK (PK last; exits Setup Mode)
mokutil --db ; mokutil --sb-state   # verify

It writes the signed .auth updates to efivarfs directly (the EL9-native equivalent of efi-updatevar), refuses to run outside Setup Mode without --force, and is added to fapolicyd's trust DB at build (it's a non-RPM exec). ⚠ This is a deliberate trust-bootstrap action — run it only on a host you control, on a trusted/isolated network (see TODO #10).

(b) Firmware UI — no OS tooling, the most robust for one-off laptop bring-up:

Put the public db.crt (and KEK.crt/PK.crt, or the .esl/.auth) on a FAT USB stick (or the ESP).
Firmware setup → Secure Boot → Custom, Clear/Reset keys (Setup Mode), then Enrol / Load key from file for db, then KEK, then PK. On HP EliteBooks: F10 → Security → Secure Boot Configuration.
Save & exit; re-check over SSH with mokutil --db.

Only the public certs/.esl/.auth reach the workstation — never copy PK.key/KEK.key/db.key onto a host.

Troubleshooting `fleet-sb-enroll` (bench, 2026-06-20)

dd: ... Invalid argument (EINVAL) on the db write = the firmware's SetVariable rejected the payload (EFI_INVALID_PARAMETER), not an auth/Setup Mode failure (that surfaces as EACCES / "Permission denied"). Diagnose top-down:

Is the .auth actually a signed authenticated variable? First 16 bytes are an EFI_TIME: sudo od -An -tx1 -N16 db.auth → a valid one starts with the year little-endian (2026 = ea 07). If it starts with a1 59 c0 a5 … it's a bare ESL (the sign-efi-sig-list step was skipped); 30 82 … is a raw DER cert. Re-create the .auth (step 2) and re-bake.
The 4-byte attribute prefix must contain its NUL bytes. A bash variable cannot hold NULs — ATTR=$'\x27\x00\x00\x00' silently truncates to the single byte 0x27, so efivarfs reads a corrupt attribute u32 and the kernel returns EINVAL before the firmware is even called. This was a real bug in an early fleet-sb-enroll; the fix emits the prefix with printf '\x27\x00\x00\x00' at write time (never via a variable). If you hand-roll an enrolment, do the same.
Proxmox / OVMF red herring: VM EFI disks default to pre-enrolled-keys=1, so mokutil --db shows the Microsoft cert (Windows Production PCA 2011, owner 77fa9abd-…) even in Setup Mode. This looks like a "db not empty → rejected" problem but isn't — the write fails identically on a clean pre-enrolled-keys=0 disk if the NUL bug is present. For a clean OVMF test bed: qm set <vmid> --efidisk0 <storage>:1,efitype=4m,pre-enrolled-keys=0.

Verified: with the fix, a Proxmox VM enrols db→KEK→PK from the baked .auth files and boots the CI-signed UKI under enforcing Secure Boot over the HTTP boot path (mokutil --sb-state = enabled). The PXE/iPXE front-end additionally needs snponly.efi signed with db — see the iPXE section below.

4. Wire `db` into the build

packer build -var image_version=X.Y.Z \
  -var sb_key=/path/to/db.key \
  -var sb_cert=/path/to/db.crt \
  -var oci_image=<registry>/fleet-boot .

ukify signs fleet-<ver>.efi with db; the enrolled db cert lets firmware verify it. Without these vars the build stays unsigned (lab-only, with the loud warning). In CI, sb_key/sb_cert are secret paths on the HSM/signing host.

Signing in CI (Gitea Actions)

sb_key/sb_cert can be Gitea repo secrets, with one wrinkle: ukify consumes them as file paths, not values, and the signing post-processor runs shell-local on the runner (not in the build VM — consistent with invariant #7). So the job materializes the secrets to runner-temp files, passes the paths, and shreds the key afterward. This is wired in ../.gitea/workflows/gitea-main-ci.yaml (steps: Materialize Secure Boot signing material → packer build → Shred Secure Boot signing key). Set repo secrets SB_KEY (the db private key, PEM) and SB_CERT (the db cert, PEM); leave them unset for an unsigned lab build.

Caveats:

Only db belongs in CI. PK and KEK never leave the offline/HSM store.
SB_CERT is a public cert (not secret); kept as a secret only for convenience.
A CI secret puts the private key in plaintext on the runner FS for the duration of the build — only as safe as the runner. Use a trusted, ephemeral/self-hosted runner, ensure secrets are not exposed to fork/PR builds, and rely on RUNNER_TEMP wipe + the shred step. Gitea masks secret values in logs, and build-oci.sh passes the path (not content) to ukify, so the key won't leak into logs.
Stronger (production target): keep db.key in an HSM/PKCS#11 token or KMS and sign through it, so the raw key never touches the runner disk. The CI-secret flow is a reasonable interim; the HSM flow is the invariant-#7 ideal (TODO #8).

PXE-only hosts: iPXE chainload and Secure Boot

Firmware without UEFI HTTP boot (e.g. HP EliteBook 820 G4) falls back to PXE; the serving stack hands those hosts iPXE over TFTP (serving/tftp/ipxe.efi), which then chainloads the UKI over HTTP. dnsmasq selects per client automatically (HTTP-capable → UKI direct; PXE-only → iPXE → UKI), so no per-host config.

Secure Boot across that chain needs only the db key — the one already enrolled for the UKI. No extra keys, no Microsoft CA, no shim (the fleet owns its keys):

firmware --(verify snponly.efi vs db)--> iPXE --(chain -> firmware LoadImage
                                                  verifies UKI vs db)--> UKI

The firmware verifies the TFTP-downloaded NBP (ipxe.efi) against db → the iPXE binary must be signed by db (sbsign, same key/step as the UKI). Stock/unsigned iPXE is rejected under Secure Boot.
iPXE must load the UKI as an EFI image (imgfetch + boot, or chain) so the firmware's LoadImage() re-verifies it against db — do not use iPXE's kernel/boot on a raw vmlinuz, which bypasses SB.
cmdline gotcha (Secure Boot OFF only): when iPXE executes the UKI it sets the EFI load options to the image's cmdline, or — if empty — to the image name. With SB off systemd-stub uses the load options as the kernel cmdline, replacing the UKI's embedded cmdline (roothash/nvmf/...) → dracut emergency shell. Clearing the args does NOT help (empty → iPXE falls back to the name). The serving stack's rendered boot.ipxe therefore feeds iPXE the real cmdline, which the dnsmasq entrypoint extracts from the UKI's .cmdline PE section (objcopy, artifact mounted read-only) — see serving/tftp/README.md. Secure Boot ON is immune — the stub ignores any loader-supplied cmdline — so this is purely a lab/SB-off concern.

Lab posture (now): iPXE signing OFF → run the iPXE path with Secure Boot OFF. serving/tftp/ipxe.efi is unsigned. Production upgrade: sign it with db and Secure Boot stays enforced end-to-end (tracked in TODO #8). The HTTP-boot path (firmware → UKI) is unaffected and remains SB-clean regardless.

Key hygiene & lifecycle

PK.key and KEK.key go offline / into an HSM and stay there. They are only needed to rotate keys; routine signing uses db.key only.
Production ideal: keep db.key in an HSM/PKCS#11 token and have sbsign/ukify sign through it rather than reading a file from disk.
Validity: -days 3650 is 10 years; pick a rotation policy. Rotating db = sign new db.esl with KEK, push the db variable update, re-sign UKIs.
dbx (the revocation list) is a fourth, optional variable — manage it to blacklist a compromised db cert.
Lab shortcut: sbctl (sbctl create-keys → sbctl enroll-keys) generates and enrols the whole trio and tracks signed files — convenient for a dev box, but it keeps keys as local files, so for the fleet's HSM/offline-PK model use the manual flow above for PK/KEK.

Invariants #6 (UKI cmdline immutable once signed), #7 (keys never in repo/VM), #5 (roothash↔UKI hard gate) — ../CLAUDE.md.
Signing is the production-posture step in TODO.md #8.
FIPS is currently off and is to be revisited alongside signing/Secure Boot (stig-deviations.md #5).

13 KiB Raw Blame History