Update pxe boot flow
This commit is contained in:
Audiobuzz
@@ -0,0 +1,118 @@
|
||||
# TFTP root — iPXE chainload for PXE-only firmware
|
||||
|
||||
This directory is mounted read-only at `/srv/tftp` in the dnsmasq container and
|
||||
serves the iPXE network bootloader over TFTP to firmware that **lacks UEFI HTTP
|
||||
boot** (e.g. HP EliteBook 820 G4). HTTP-boot-capable firmware never touches this
|
||||
path — it gets the UKI straight over HTTP (see the `httpboot` tag in
|
||||
`docker-compose.yml`). The fallback is automatic, driven by what each client
|
||||
advertises (DHCP vendor-class / client-arch).
|
||||
|
||||
Flow for a PXE-only host:
|
||||
|
||||
```
|
||||
firmware PXE (PXEClient) --TFTP--> snponly.efi (iPXE)
|
||||
iPXE re-DHCPs (user-class "iPXE") --> handed boot.ipxe (TFTP)
|
||||
boot.ipxe: imgfetch UKI over HTTP -> imgargs <real cmdline> -> boot
|
||||
```
|
||||
|
||||
`boot.ipxe` is **rendered at container start** by the dnsmasq entrypoint; only the
|
||||
iPXE binary (`snponly.efi`) is supplied here. The script exists because of a sharp
|
||||
edge:
|
||||
|
||||
> **Why not chainload the UKI URL directly?** When iPXE executes the UKI it sets
|
||||
> the EFI load options to the image's command line — or, if that's empty, to the
|
||||
> image **name**. With **Secure Boot OFF**, systemd-stub uses the load options *as
|
||||
> the kernel cmdline*, **replacing** the UKI's embedded cmdline
|
||||
> (`roothash`/`rd.nvmf.discover`/`rd.dot1x=0`/`systemd.volatile=overlay`). So
|
||||
> booting the URL directly — *or clearing the args* — makes the kernel come up
|
||||
> with just `fleet-<ver>.efi` as its cmdline → **dracut emergency shell**.
|
||||
>
|
||||
> The fix: feed iPXE the **real cmdline**. On the serving host that cmdline only
|
||||
> lives *inside the UKI* (it's not in the OCI image — invariant #10 — nor shipped
|
||||
> as a file), so the entrypoint **extracts the UKI's `.cmdline` PE section with
|
||||
> `objcopy`** (the artifact image is mounted read-only at `/artifacts`) and bakes
|
||||
> it into `boot.ipxe` as `imgargs <uki> <cmdline>`. Needs `binutils` in the image
|
||||
> (added) and the artifact mount.
|
||||
>
|
||||
> **Secure Boot ON makes all of this moot** — the stub ignores any loader-supplied
|
||||
> cmdline and uses the embedded one — so this is purely a lab/SB-OFF concern; the
|
||||
> rendered script is harmless once SB is on.
|
||||
|
||||
## What to drop here
|
||||
|
||||
A UEFI x86-64 iPXE binary named **`snponly.efi`** (the name the dnsmasq config
|
||||
serves). `snponly` rides the **firmware's** NIC driver via the UEFI SNP protocol
|
||||
rather than iPXE's own, so it works on virtio VMs *and* real hardware where the
|
||||
native driver won't bind.
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian package (simplest — confirmed present):
|
||||
sudo apt-get install -y ipxe # then copy whatever snponly it ships, e.g.:
|
||||
# dpkg -L ipxe | grep -i snponly
|
||||
cp /usr/lib/ipxe/snponly.efi serving/tftp/snponly.efi
|
||||
# Fedora/EL: dnf install ipxe-bootimgs -> /usr/share/ipxe/ipxe-snponly-x86_64.efi
|
||||
# or build: git clone https://github.com/ipxe/ipxe && cd ipxe/src
|
||||
# make bin-x86_64-efi/snponly.efi
|
||||
```
|
||||
|
||||
> **Why `snponly`, not the full `ipxe.efi`?** iPXE's own NIC drivers don't bind on
|
||||
> some hardware — the **EliteBook 820 G4 (Intel I219-LM)** fails `ipxe.efi` with
|
||||
> *"net0: No configurable methods"*. `snponly` inherits the firmware's working
|
||||
> driver (the one that just loaded iPXE over PXE), so it's the robust default.
|
||||
> `boot.ipxe.org` only publishes the full `ipxe.efi`, not `snponly` — get `snponly`
|
||||
> from a distro package or build it.
|
||||
|
||||
> **Updating the binary requires a dnsmasq restart** — the entrypoint copies it
|
||||
> from this (read-only) mount into the container's TFTP root at startup:
|
||||
> `docker compose restart dnsmasq`.
|
||||
|
||||
(BIOS/legacy hosts would need `undionly.kpxe` + client-arch matching; this fleet
|
||||
is UEFI x64.)
|
||||
|
||||
## Secure Boot
|
||||
|
||||
**Lab (now): Secure Boot OFF for the iPXE path.** `snponly.efi` here is
|
||||
**unsigned**, so a Secure-Boot-enforcing machine would reject it.
|
||||
|
||||
**Production upgrade:** sign `snponly.efi` with the fleet **db** key (the same key
|
||||
that signs the UKI), then Secure Boot stays enforced end-to-end
|
||||
(firmware → verifies iPXE vs db → iPXE `chain`s the UKI → firmware LoadImage
|
||||
verifies UKI vs db). Only `db` needs enrolling — no Microsoft keys / shim, since
|
||||
the fleet owns its keys. See [../../docs/secure-boot-keys.md](../../docs/secure-boot-keys.md).
|
||||
|
||||
```bash
|
||||
# production (where db.key lives — CI/HSM):
|
||||
sbsign --key db.key --cert db.crt --output snponly.efi snponly.efi
|
||||
```
|
||||
|
||||
## Bench notes / gotchas (EliteBook 820 G4 / Intel I219-LM, 2026-06-20)
|
||||
|
||||
Hard-won findings from bringing up the first PXE-only physical host:
|
||||
|
||||
- **Use `snponly.efi`, not the full `ipxe.efi`.** The 820 G4's NIC is an Intel
|
||||
**I219-LM** (PCI `00:1f.6`). iPXE's native `intel` driver in the prebuilt full
|
||||
build doesn't bind it (`net0: No configurable methods` / no `net0`). `snponly`
|
||||
rides the firmware's driver via NII/SNP, so `net0` appears. Ubuntu's `ipxe`
|
||||
package ships a usable `snponly.efi` (no build needed).
|
||||
- **PortFast/edge on the access port is REQUIRED** (this was the real blocker).
|
||||
iPXE's `ifopen` resets the PHY → the link bounces → a managed switch restarts
|
||||
its STP forwarding delay (~30 s) and silently drops packets → iPXE's DHCP never
|
||||
reaches the server (`ifstat` shows `link:down` / `Link status: Unknown`; symptom
|
||||
is `No configurable methods` or a dracut shell). The firmware's own PXE works
|
||||
only because it keeps the link up continuously from POST so STP already
|
||||
converged. **Fix: enable PortFast/edge (or disable STP) on the workstation
|
||||
port** — VERIFIED to fix it 2026-06-20. (Cisco `spanning-tree portfast`;
|
||||
Aruba/HP `admin-edge-port`; UniFi "Fast"; or a dumb unmanaged switch.) The
|
||||
`autoexec.ipxe` retry loop (below) is a fallback that outlasts STP but adds
|
||||
~30 s to every boot — PortFast removes the delay entirely.
|
||||
- **iPXE aborts the script on any unguarded command failure** (→ iPXE exits → the
|
||||
firmware restarts the whole PXE process, looking like "it doesn't retry"). Every
|
||||
fallible step in `boot.ipxe`/`autoexec.ipxe` is therefore `||`-guarded.
|
||||
- **`autoexec.ipxe`** (rendered by the dnsmasq entrypoint) is fetched + run at
|
||||
iPXE startup, *replacing* the default autoboot; that's where the `ifopen` +
|
||||
DHCP-retry + `imgargs <real cmdline>` chain lives.
|
||||
|
||||
## Note
|
||||
|
||||
`*.efi` here is **gitignored** (binary, and the signed one is environment-specific
|
||||
/ produced from db). Keep this README + `.gitkeep`.
|
||||
Reference in New Issue
Block a user