From 4913079a02195901ac5a2e3219c992d26e56a972 Mon Sep 17 00:00:00 2001 From: Jeff Larson Date: Sat, 8 Aug 2026 12:48:39 -0700 Subject: [PATCH] feat(agent): eBPF load-time BTF preflight, retire manual offset re-verification (ADR-0014 amendment) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a load-time BTF preflight to the userspace loader: before attaching any probe, it re-verifies every kernel struct field offset the eBPF crate bakes in (plus the LOADING_MODULE enum value) against the node's live BTF. - Single source of truth: a const (struct, field, expected-offset) table plus the LOADING_MODULE value in agent/common/src/offsets.rs. The eBPF crate's offset_of! guard (vmlinux.rs) now asserts bindings == table at compile time; the loader's preflight asserts table == node-BTF at load time. - A small self-contained BTF binary parser (agent/protector-agent/src/ preflight/btf.rs) — neither aya nor aya-obj's public API exposes struct- member offsets or enum values (both are pub(crate) upstream), so this parses the raw type section directly, recursing into anonymous unions/ structs (inode.i_nlink) with a bounded recursion depth against a malformed blob. Off-fleet testable against hand-built fixture BTF blobs (no live kernel required). - Fail-closed on struct-reading probes (file_open, file_write, mmap_file, fix_setuid, bprm_check), fail-open on struct-free probes (connect, ptrace_access_check, kernel_load_data). Every divergent field is logged expected-vs-actual; the LOADING_MODULE enum mismatch is logged but never gates a probe (not verifier-checked either way). Degrades gracefully, never crash-loops. - Resolves the ON-NODE-PENDING markers in vmlinux.rs (the preflight is now their continuous verification, not a one-time manual task). - Corrects the false "CO-RE-relocated against node BTF at load" claims in agent/Dockerfile, .github/workflows/agent.yml, and docs/ebpf-testing-on-nodes.md — the object bakes offsets and the loader checks them at load; it does not relocate. No PROTECTOR_*_ENABLE toggle: this is a correctness guard, not a feature. Closes JEF-328 Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP --- .github/workflows/agent.yml | 12 +- agent/Dockerfile | 10 +- agent/common/src/lib.rs | 8 + agent/common/src/offsets.rs | 126 ++++++ agent/protector-agent-ebpf/src/main.rs | 16 +- agent/protector-agent-ebpf/src/vmlinux.rs | 96 ++-- agent/protector-agent/Cargo.toml | 14 +- agent/protector-agent/src/main.rs | 7 + agent/protector-agent/src/observer.rs | 25 ++ .../src/observer/ebpf/preflight_gate.rs | 100 +++++ agent/protector-agent/src/preflight/btf.rs | 421 ++++++++++++++++++ .../src/preflight/btf_tests.rs | 145 ++++++ .../protector-agent/src/preflight/fixture.rs | 137 ++++++ agent/protector-agent/src/preflight/mod.rs | 152 +++++++ agent/protector-agent/src/preflight/tests.rs | 170 +++++++ docs/ebpf-testing-on-nodes.md | 17 +- 16 files changed, 1394 insertions(+), 62 deletions(-) create mode 100644 agent/common/src/offsets.rs create mode 100644 agent/protector-agent/src/observer/ebpf/preflight_gate.rs create mode 100644 agent/protector-agent/src/preflight/btf.rs create mode 100644 agent/protector-agent/src/preflight/btf_tests.rs create mode 100644 agent/protector-agent/src/preflight/fixture.rs create mode 100644 agent/protector-agent/src/preflight/mod.rs create mode 100644 agent/protector-agent/src/preflight/tests.rs diff --git a/.github/workflows/agent.yml b/.github/workflows/agent.yml index 28b1bf38..5fd2dcc3 100644 --- a/.github/workflows/agent.yml +++ b/.github/workflows/agent.yml @@ -26,8 +26,10 @@ jobs: # Compile the BPF programs to bytecode on the self-hosted runners — their image # (../runner) bakes in nightly + rust-src + a prebuilt bpf-linker (ADR-0014), so the # job is just the compile (the tiny eBPF crate; the LLVM-heavy bpf-linker build was - # done once at image-build time). BPF is architecture-neutral bytecode, CO-RE- - # relocated against the node's BTF at load. + # done once at image-build time). BPF is architecture-neutral bytecode; its kernel + # struct offsets are hand-verified and baked in at compile time (no CO-RE field + # relocation — rustc emits none), then re-checked against each node's live BTF at + # load by the userspace loader's preflight (ADR-0014's amendment). ebpf: runs-on: protector-runners # Self-hosted + persistent: never execute forked-PR code (mirrors the docker job). @@ -98,8 +100,10 @@ jobs: # The image is built WITH the eBPF probe: agent/Dockerfile runs # `cargo build --release -p protector-agent --features ebpf`, and its build.rs compiles # the sibling protector-agent-ebpf crate to a BPF object (bpf-linker, nightly) embedded in - # the binary and CO-RE-relocated against the node's BTF at load (ADR-0014). The default - # (no-feature) build — a no-op observer — exists only for toolchain-free local dev/test. + # the binary, its kernel struct offsets baked in at compile time and re-verified against + # each node's live BTF at load by the userspace loader's preflight — NOT CO-RE-relocated + # (ADR-0014's amendment: rustc emits no BTF field relocations). The default (no-feature) + # build — a no-op observer — exists only for toolchain-free local dev/test. - name: Build and push agent image id: build-and-push uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7 diff --git a/agent/Dockerfile b/agent/Dockerfile index 118cdb19..c782720c 100644 --- a/agent/Dockerfile +++ b/agent/Dockerfile @@ -5,9 +5,13 @@ # Builds the agent WITH the eBPF probe (`--features ebpf`): the builder stage carries # the bpf toolchain (prebuilt bpf-linker + nightly + rust-src), and the userspace # build.rs compiles the sibling protector-agent-ebpf crate to a BPF object and embeds -# it in the loader. The BPF object is architecture-neutral (CO-RE-relocated against the -# node's BTF at load); the elevated caps it needs are granted at RUNTIME by the -# DaemonSet's securityContext, never baked into the image. +# it in the loader. The BPF object bakes hand-verified kernel struct offsets rather than +# CO-RE-relocating them — rustc emits no BTF field relocations (ADR-0014's amendment on +# the load-time BTF preflight) — so it is the SAME object on every node/arch; the +# userspace loader re-verifies those baked offsets against each node's live BTF before +# attach (a check, not a relocation) and degrades gracefully on a mismatch. The elevated +# caps it needs are granted at RUNTIME by the DaemonSet's securityContext, never baked +# into the image. # bookworm-based to match the bookworm-slim runtime's glibc 2.36. # Via mirror.gcr.io (Docker Hub pull-through) to dodge the anonymous 429. diff --git a/agent/common/src/lib.rs b/agent/common/src/lib.rs index 61895fb6..2bb148d2 100644 --- a/agent/common/src/lib.rs +++ b/agent/common/src/lib.rs @@ -10,6 +10,14 @@ #![no_std] +/// The single source of truth for every kernel struct field offset and BTF-visible enum +/// value the eBPF probes bake in (ADR-0014 amendment, load-time BTF preflight). The eBPF +/// crate's `offset_of!` guard (`vmlinux.rs`) asserts `bindings == table` at compile time; +/// the userspace loader's preflight (`agent/protector-agent/src/preflight`) asserts +/// `table == node-BTF` at load time — transitively `bindings == kernel`, with the number +/// living in exactly one place. +pub mod offsets; + /// Event-kind discriminators. Stable wire values; never renumber an existing one. pub const KIND_CONNECT: u32 = 1; /// A tmpfs file was opened (fentry on `security_file_open`). Carries the container path; diff --git a/agent/common/src/offsets.rs b/agent/common/src/offsets.rs new file mode 100644 index 00000000..0b7545dc --- /dev/null +++ b/agent/common/src/offsets.rs @@ -0,0 +1,126 @@ +//! The offset/enum table both the eBPF crate's compile-time guard and the userspace +//! loader's load-time BTF preflight read (ADR-0014 amendment). See the module doc in +//! `lib.rs` for why this exists: one number, checked twice (compile time against the +//! hand-laid bindings, load time against the running kernel's live BTF), never hand-kept +//! in sync between the two. + +/// One `(struct, field, expected byte offset)` entry — a field the eBPF probes read via a +/// baked offset (`agent/protector-agent-ebpf/src/vmlinux.rs`). `kernel_struct` is the +/// struct's name as it appears in kernel BTF (e.g. `"file"`, not a Rust type path). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct FieldOffset { + pub kernel_struct: &'static str, + pub field: &'static str, + /// Verified byte offset on the fleet's kernel (7.0.0 — see `vmlinux.rs`'s module doc + /// for the derivation of each field below). + pub offset: u32, +} + +impl FieldOffset { + const fn new(kernel_struct: &'static str, field: &'static str, offset: u32) -> Self { + Self { + kernel_struct, + field, + offset, + } + } +} + +/// Every field offset a probe bakes in. Order mirrors `vmlinux.rs`'s struct declarations +/// (`file` → `path` → `dentry` → `qstr` → `inode` → `super_block` → `cred`/`kuid_t` → +/// `linux_binprm`), so a diff against that file's `offset_of!` block reads in the same +/// order. +pub const FIELD_OFFSETS: &[FieldOffset] = &[ + FieldOffset::new("file", "f_inode", 32), + FieldOffset::new("file", "f_flags", 40), + FieldOffset::new("file", "f_path", 64), + FieldOffset::new("path", "dentry", 8), + FieldOffset::new("dentry", "d_name", 32), + FieldOffset::new("qstr", "name", 8), + FieldOffset::new("inode", "i_sb", 40), + FieldOffset::new("inode", "i_ino", 64), + // Lives in an anonymous union immediately after i_ino (`union { const unsigned int + // i_nlink; unsigned int __i_nlink; }`) — both the compile-time `offset_of!` (a plain + // Rust field access through the flattened binding) and the load-time BTF walk (which + // must recurse into the anonymous union to find it) land on the SAME byte offset, +72. + FieldOffset::new("inode", "i_nlink", 72), + FieldOffset::new("super_block", "s_magic", 96), + FieldOffset::new("cred", "uid", 8), + FieldOffset::new("kuid_t", "val", 0), + FieldOffset::new("linux_binprm", "file", 64), + FieldOffset::new("linux_binprm", "filename", 96), +]; + +/// The BTF enum the module-load probe's `LOADING_MODULE` constant must match +/// (`agent/protector-agent-ebpf/src/main.rs`; `include/linux/kernel_read_file.h`'s `enum +/// kernel_load_data_id`). Unlike a struct offset this is never verifier-checked — a wrong +/// value is a plain integer compare that misclassifies silently rather than failing loud +/// — which is why the preflight checks it explicitly (ADR-0014 amendment). +pub const LOADING_MODULE_ENUM: &str = "kernel_load_data_id"; +pub const LOADING_MODULE_VARIANT: &str = "LOADING_MODULE"; +pub const LOADING_MODULE_VALUE: u32 = 2; + +/// Look up `kernel_struct.field`'s expected byte offset in [`FIELD_OFFSETS`]. `const fn` +/// so the eBPF crate's `offset_of!` guard can assert `bindings == table` at compile time — +/// the identical lookup the userspace preflight performs against live BTF at load time. +/// Panics (a compile error in the `const` context it's used from) if the pair isn't in the +/// table — a coding mistake to fix by adding the entry, not a runtime condition. +pub const fn offset_of_table(kernel_struct: &str, field: &str) -> u32 { + let mut i = 0; + while i < FIELD_OFFSETS.len() { + let entry = FIELD_OFFSETS[i]; + if str_eq(entry.kernel_struct, kernel_struct) && str_eq(entry.field, field) { + return entry.offset; + } + i += 1; + } + panic!("offset_of_table: no FIELD_OFFSETS entry for this (struct, field) — add it there first") +} + +/// `const fn` byte-wise string equality — `&str`'s `PartialEq` isn't `const`, so +/// [`offset_of_table`] (evaluated at compile time by the eBPF crate's `offset_of!` guard) +/// needs its own. +const fn str_eq(a: &str, b: &str) -> bool { + let a = a.as_bytes(); + let b = b.as_bytes(); + if a.len() != b.len() { + return false; + } + let mut i = 0; + while i < a.len() { + if a[i] != b[i] { + return false; + } + i += 1; + } + true +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn offset_of_table_matches_the_declared_entries() { + for entry in FIELD_OFFSETS { + assert_eq!( + offset_of_table(entry.kernel_struct, entry.field), + entry.offset + ); + } + } + + #[test] + #[should_panic(expected = "no FIELD_OFFSETS entry")] + fn offset_of_table_panics_on_an_unknown_pair() { + offset_of_table("file", "not_a_real_field"); + } + + #[test] + fn str_eq_distinguishes_length_and_content() { + assert!(str_eq("file", "file")); + assert!(!str_eq("file", "files")); + assert!(!str_eq("file", "path")); + assert!(str_eq("", "")); + } +} diff --git a/agent/protector-agent-ebpf/src/main.rs b/agent/protector-agent-ebpf/src/main.rs index 731b067b..e763bad4 100644 --- a/agent/protector-agent-ebpf/src/main.rs +++ b/agent/protector-agent-ebpf/src/main.rs @@ -717,13 +717,15 @@ fn try_ptrace_access_check(ctx: &FEntryContext) -> Result<(), i64> { /// the enum is a stable, list-ordered generator macro — `LOADING_UNKNOWN`(0), /// `LOADING_FIRMWARE`(1), `LOADING_MODULE`(2), `LOADING_KEXEC_IMAGE`(3), /// `LOADING_KEXEC_INITRAMFS`(4), `LOADING_POLICY`(5), `LOADING_X509_CERTIFICATE`(6), -/// `LOADING_MAX_ID`(7). **ON-NODE BTF VERIFICATION PENDING:** confirm against -/// `bpftool btf dump … format c | grep -A8 'enum kernel_load_data_id'` on BOTH fleet arches -/// before this ships past a spike deploy (docs/ebpf-testing-on-nodes.md). Unlike a struct -/// offset, a wrong value here is NOT verifier-checked — it's a plain integer compare, so a -/// reorder (unlikely; this list has been stable since its 5.x introduction, but unconfirmed -/// on THIS fleet kernel) would misclassify silently rather than fail loud. -const LOADING_MODULE: u32 = 2; +/// `LOADING_MAX_ID`(7). Sourced from the SHARED table in `protector-agent-common` +/// (ADR-0014 amendment) rather than a bare literal, so the userspace loader's load-time +/// BTF preflight (`agent/protector-agent/src/preflight`) checks the SAME value against +/// each node's live BTF at every agent start. Unlike a struct offset, a wrong value here +/// is NOT verifier-checked — it's a plain integer compare, so a reorder (unlikely; this +/// list has been stable since its 5.x introduction) would misclassify silently rather than +/// fail loud, which is exactly why the preflight checks it explicitly and logs +/// expected-vs-actual on a mismatch rather than relying on this compile-time value alone. +const LOADING_MODULE: u32 = protector_agent_common::offsets::LOADING_MODULE_VALUE; /// fentry on `security_kernel_load_data(enum kernel_load_data_id id, bool contents)` — the /// kernel-module-load probe (Retire-Falco G2). Falco fires critical on diff --git a/agent/protector-agent-ebpf/src/vmlinux.rs b/agent/protector-agent-ebpf/src/vmlinux.rs index 85ffc3f6..921ab6b8 100644 --- a/agent/protector-agent-ebpf/src/vmlinux.rs +++ b/agent/protector-agent-ebpf/src/vmlinux.rs @@ -31,18 +31,21 @@ //! degraded the two `bpf_d_path` probes (secret-read `file_open` + `file_write`) to //! loaded=4/6 fleet-wide. Regenerate (re-verify the offsets) on any kernel struct change. //! -//! # `linux_binprm.file` / `inode.i_nlink` — ON-NODE BTF VERIFICATION PENDING +//! # `linux_binprm.file` / `inode.i_nlink` — verified continuously, not just once //! //! Two fields added for the fileless-exec (anon-inode) probe were derived from kernel //! *source* layout, not dumped from live BTF like everything else above: `linux_binprm.file` //! (+64) and `inode.i_nlink` (+72). Each carries its own derivation in its struct's doc -//! comment. Both must be confirmed against `bpftool btf dump` on BOTH fleet arches — the -//! same process that produced the offsets above — before this probe ships past a spike -//! deploy (docs/ebpf-testing-on-nodes.md). A wrong offset here fails the SAME way a wrong -//! `f_path` offset would have: either a verifier rejection (probe degrades, -//! loud in the heartbeat) or, worse, a silently wrong bool if the misread pointer happens -//! to still verify — which is why this module keeps every derivation reasoning explicit -//! rather than asserting a bare number. +//! comment. Unlike the offsets above (dumped once from live BTF on both fleet arches), +//! these two were never manually confirmed against `bpftool btf dump` — but they don't +//! need to be, ONE-TIME, by hand: the userspace loader's load-time BTF preflight +//! (`agent/protector-agent/src/preflight`, ADR-0014 amendment) re-derives every field in +//! this module — these two included — against each node's live BTF before any probe +//! attaches, every time the agent starts. A wrong offset here fails the SAME way a wrong +//! `f_path` offset would have: either a verifier rejection (probe degrades, loud in the +//! heartbeat) or, worse, a silently wrong bool if the misread pointer happens to still +//! verify — which is why the preflight disables the struct-reading probes on a mismatch +//! (fail-closed) rather than trusting a compile-time assertion alone. // Padding fields (and `mnt`, present only to place `dentry` at +8) are never read — they // exist solely to position the fields the probes DO read at the right byte offset. @@ -93,14 +96,14 @@ pub struct qstr { /// the anon-inode discriminator — `0` for an unlinked inode (a memfd, or any file `rm`'d /// while still executing), non-zero for a normal directory-linked file. /// -/// **ON-NODE BTF VERIFICATION PENDING for `i_nlink`:** derived from kernel -/// source, not dumped from live BTF like the fields above it. `i_nlink` is the first field -/// of an anonymous union (`union { const unsigned int i_nlink; unsigned int __i_nlink; }`) -/// immediately after `i_ino` in `struct inode` — no padding needed since `i_ino` (an -/// 8-byte `unsigned long`) already leaves the next field 8-aligned. +72 = +64 (`i_ino`'s -/// offset) + 8 (`i_ino`'s size). Must be confirmed against BOTH fleet arches' live BTF -/// (`bpftool btf dump … format c`) before this ships past a spike deploy — see -/// docs/ebpf-testing-on-nodes.md. +/// `i_nlink`'s offset was derived from kernel source, not dumped from live BTF like the +/// fields above it. `i_nlink` is the first field of an anonymous union (`union { const +/// unsigned int i_nlink; unsigned int __i_nlink; }`) immediately after `i_ino` in `struct +/// inode` — no padding needed since `i_ino` (an 8-byte `unsigned long`) already leaves the +/// next field 8-aligned. +72 = +64 (`i_ino`'s offset) + 8 (`i_ino`'s size). The load-time +/// BTF preflight re-derives this offset on every node at every agent start — including +/// recursing into the anonymous union — so a derivation error here is caught continuously, +/// not just once (see the module doc above and `agent/protector-agent/src/preflight`). #[repr(C)] #[derive(Copy, Clone)] pub struct inode { @@ -108,7 +111,7 @@ pub struct inode { pub i_sb: *mut super_block, // +40 _pad1: [u8; 16], pub i_ino: u64, // +64 unsigned long - pub i_nlink: u32, // +72 ON-NODE BTF VERIFICATION PENDING (see doc above) + pub i_nlink: u32, // +72 — verified at every load, not just once (see doc above) } /// `struct super_block` — prefix through `s_magic` (+96), the tmpfs filter's discriminator. @@ -150,38 +153,49 @@ pub struct cred { /// `interpreter`(+56) + `file`(+64) + `cred`(+72) + `unsafe`(+80) + `per_clear`(+84) + /// `argc`(+88) + `envc`(+92) = `filename` at +96 — which matches the INDEPENDENTLY /// on-node-verified `filename` offset below exactly, a strong (but not certain) signal -/// this derivation tracks the real fleet layout. Must still be confirmed against BOTH -/// fleet arches' live BTF (`bpftool btf dump … format c`) before this ships past a spike -/// deploy — see docs/ebpf-testing-on-nodes.md. +/// this derivation tracks the real fleet layout. Like `inode.i_nlink` above, the +/// load-time BTF preflight re-derives this offset from live BTF on every node at every +/// agent start, so this derivation is checked continuously rather than trusted once — see +/// the module doc above and `agent/protector-agent/src/preflight`. #[repr(C)] #[derive(Copy, Clone)] pub struct linux_binprm { _pad0: [u8; 64], - pub file: *mut file, // +64 ON-NODE BTF VERIFICATION PENDING (see doc above) + pub file: *mut file, // +64 — verified at every load, not just once (see doc above) _pad1: [u8; 24], pub filename: *const c_char, // +96 } -// Compile-time guard: pin every read field to its verified 7.0.0 byte offset (see the -// module header). These are the offsets the compiler bakes into `bpf_d_path` and the -// `bpf_probe_read_kernel` chases; if a future edit (padding slip, a reverted binding, a -// kernel struct change) moves one, the eBPF crate fails to BUILD here — loud at CI time -// rather than a silent misread or a verifier rejection only visible on a live node. -// `offset_of!` is const, so this costs nothing at runtime. +// Compile-time guard: pin every read field to the SHARED offset table in +// `protector-agent-common` (ADR-0014 amendment) — `bindings == table`, asserted here, plus +// the userspace loader's load-time preflight asserting `table == node-BTF` +// (agent/protector-agent/src/preflight), transitively proves `bindings == kernel` on every +// node at every start, not just at this compile. These are the offsets the compiler bakes +// into `bpf_d_path` and the `bpf_probe_read_kernel` chases; if a future edit (padding +// slip, a reverted binding, a kernel struct change) moves one without updating the shared +// table too, the eBPF crate fails to BUILD here — loud at CI time rather than a silent +// misread or a verifier rejection only visible on a live node. `offset_of!` and +// `offset_of_table` are both const, so this costs nothing at runtime. const _: () = { use core::mem::offset_of; - assert!(offset_of!(file, f_inode) == 32); - assert!(offset_of!(file, f_flags) == 40); - assert!(offset_of!(file, f_path) == 64); - assert!(offset_of!(path, dentry) == 8); - assert!(offset_of!(dentry, d_name) == 32); - assert!(offset_of!(qstr, name) == 8); - assert!(offset_of!(inode, i_sb) == 40); - assert!(offset_of!(inode, i_ino) == 64); - assert!(offset_of!(inode, i_nlink) == 72); // ON-NODE PENDING - assert!(offset_of!(super_block, s_magic) == 96); - assert!(offset_of!(cred, uid) == 8); - assert!(offset_of!(kuid_t, val) == 0); - assert!(offset_of!(linux_binprm, file) == 64); // ON-NODE PENDING - assert!(offset_of!(linux_binprm, filename) == 96); + // `offset_of!` yields `usize`; the shared table stores `u32` (plenty for any struct + // offset in these bindings) so it can be `no_std`-friendly without pulling in a + // pointer-width-specific type — cast at the comparison, not in the table. + const fn tbl(kernel_struct: &str, field: &str) -> usize { + protector_agent_common::offsets::offset_of_table(kernel_struct, field) as usize + } + assert!(offset_of!(file, f_inode) == tbl("file", "f_inode")); + assert!(offset_of!(file, f_flags) == tbl("file", "f_flags")); + assert!(offset_of!(file, f_path) == tbl("file", "f_path")); + assert!(offset_of!(path, dentry) == tbl("path", "dentry")); + assert!(offset_of!(dentry, d_name) == tbl("dentry", "d_name")); + assert!(offset_of!(qstr, name) == tbl("qstr", "name")); + assert!(offset_of!(inode, i_sb) == tbl("inode", "i_sb")); + assert!(offset_of!(inode, i_ino) == tbl("inode", "i_ino")); + assert!(offset_of!(inode, i_nlink) == tbl("inode", "i_nlink")); + assert!(offset_of!(super_block, s_magic) == tbl("super_block", "s_magic")); + assert!(offset_of!(cred, uid) == tbl("cred", "uid")); + assert!(offset_of!(kuid_t, val) == tbl("kuid_t", "val")); + assert!(offset_of!(linux_binprm, file) == tbl("linux_binprm", "file")); + assert!(offset_of!(linux_binprm, filename) == tbl("linux_binprm", "filename")); }; diff --git a/agent/protector-agent/Cargo.toml b/agent/protector-agent/Cargo.toml index a4426264..788a8f5a 100644 --- a/agent/protector-agent/Cargo.toml +++ b/agent/protector-agent/Cargo.toml @@ -14,14 +14,18 @@ reqwest = { version = "0.12", default-features = false, features = ["json", "rus anyhow = "1" tracing = "0.1" tracing-subscriber = { version = "0.3", features = ["env-filter"] } +# The shared repr(C) event layouts AND the FIELD_OFFSETS/LOADING_MODULE table +# (`agent/common/src/offsets.rs`, ADR-0014 amendment). NOT feature-gated: the load-time +# BTF preflight (`src/preflight`) that reads this table is plain userspace Rust with no +# bpf-toolchain dependency, so its fixture-based unit tests run in the default (no +# `--features ebpf`) build too — see `src/preflight/mod.rs`'s module doc. +protector-agent-common = { version = "0.1.0", path = "../common" } -# The eBPF loader + the shared repr(C) event layouts. Feature-gated so the default build -# (and its unit tests) compile without the bpf toolchain; enable `--features ebpf` on a -# node to load real probes. +# The eBPF loader itself. Feature-gated so the default build (and its unit tests) compile +# without the bpf toolchain; enable `--features ebpf` on a node to load real probes. aya = { version = "0.13", optional = true } aya-log = { version = "0.2", optional = true } -protector-agent-common = { path = "../common", optional = true } [features] default = [] -ebpf = ["dep:aya", "dep:aya-log", "dep:protector-agent-common"] +ebpf = ["dep:aya", "dep:aya-log"] diff --git a/agent/protector-agent/src/main.rs b/agent/protector-agent/src/main.rs index 402b5f58..114133f5 100644 --- a/agent/protector-agent/src/main.rs +++ b/agent/protector-agent/src/main.rs @@ -15,6 +15,13 @@ mod linkage; mod observer; #[cfg(any(feature = "ebpf", test))] mod pod; +// The load-time BTF preflight (ADR-0014 amendment): re-verifies every baked kernel-struct +// field offset against a node's live BTF before any probe attaches. Only called from the +// `ebpf`-gated observer, but is itself plain userspace Rust with no bpf-toolchain +// dependency — gated like `pod`/`linkage` so its fixture-based unit tests run in the +// default (no `ebpf` feature) build too. +#[cfg(any(feature = "ebpf", test))] +mod preflight; mod reporter; use std::io::IsTerminal; diff --git a/agent/protector-agent/src/observer.rs b/agent/protector-agent/src/observer.rs index 14e2d134..078996bf 100644 --- a/agent/protector-agent/src/observer.rs +++ b/agent/protector-agent/src/observer.rs @@ -85,6 +85,11 @@ mod ebpf { }; use protector_behavior::{Attribution, Behavior}; + // The load-time BTF preflight's probe-gating table + wiring (ADR-0014 amendment) — + // split into its own file to keep this one under the repo's 1,000-line file cap. + mod preflight_gate; + use preflight_gate::{load_preflight, struct_deps}; + /// This sensor's identity, carried into each observation's provenance so the engine /// can tell one sensor's signals from another's (ADR-0003 corroboration). const SOURCE: &str = "protector-agent"; @@ -599,6 +604,15 @@ mod ebpf { /// BTF, so it's separate from the kprobe table; the BTF is loaded once. Returns /// `(attached, attempted)` so the caller can publish the probe-attach status the /// per-node liveness beacon reads — a partial load reads degraded. + /// + /// Before attaching anything, runs the load-time BTF preflight (ADR-0014 + /// amendment, `crate::preflight`): re-verifies every field offset the eBPF crate + /// bakes in against THIS node's live BTF. A struct-reading probe whose struct(s) + /// diverged is skipped (fail-closed — a stale `bpf_probe_read_kernel` offset + /// reads garbage silently, unlike `bpf_d_path`'s verifier check); the struct-free + /// probes (`ptrace_access_check`, `kernel_load_data`; `connect` is a separate + /// kprobe, unaffected) always attach regardless (fail-open — see + /// [`preflight_gate::struct_deps`]). fn attach_fentry(ebpf: &mut Ebpf) -> (u32, u32) { const FENTRY_PROBES: &[(&str, &str)] = &[ ("file_open", "security_file_open"), @@ -617,8 +631,19 @@ mod ebpf { return (0, attempted); } }; + let preflight = load_preflight(); let mut attached = 0u32; for (name, func) in FENTRY_PROBES { + if let Some(bad_struct) = struct_deps(name).iter().find(|s| !preflight.struct_ok(s)) + { + tracing::warn!( + probe = *name, + kernel_struct = *bad_struct, + "BTF preflight: disabling struct-reading probe (offset mismatch — see \ + the preceding per-field mismatch line for expected-vs-actual)" + ); + continue; + } match Self::attach_one_fentry(ebpf, &btf, name, func) { Ok(()) => { attached += 1; diff --git a/agent/protector-agent/src/observer/ebpf/preflight_gate.rs b/agent/protector-agent/src/observer/ebpf/preflight_gate.rs new file mode 100644 index 00000000..fa037daf --- /dev/null +++ b/agent/protector-agent/src/observer/ebpf/preflight_gate.rs @@ -0,0 +1,100 @@ +//! Which kernel structs each struct-reading fentry probe depends on, and the load-time +//! BTF preflight wiring that turns that dependency table into an attach/skip decision +//! (ADR-0014 amendment). Split out of `observer.rs` to keep it under the repo's 1,000-line +//! file cap. `crate::preflight` does the actual BTF walk this module consumes; this file +//! is only the probe-classification policy and its logging. + +use crate::preflight::PreflightReport; + +/// Which kernel structs (by `protector_agent_common::offsets::FIELD_OFFSETS` struct name) +/// each fentry probe's kernel-side code (`protector-agent-ebpf/src/main.rs`) reads +/// through. The preflight disables a probe here if ANY field of ANY struct it depends on +/// diverged from live BTF — fail-closed, at struct granularity: a moved field can shift +/// every other pointer chase through that same struct, so a probe reading `inode` at all +/// is untrusted the moment ONE `inode` field diverges, not just the one that moved. A +/// probe absent from this table reads no vmlinux struct offset at all (only scalar +/// arguments and the shared `EventHeader`) and always attaches regardless of the +/// preflight. +const STRUCT_DEPS: &[(&str, &[&str])] = &[ + // file_open: is_tmpfs (file→f_inode→inode→i_sb→super_block) + the sensitive- + // credential-basename check (file→f_path→path→dentry→d_name→qstr) + emit_file_path + // (file.f_path itself, for bpf_d_path). + ( + "file_open", + &["file", "inode", "super_block", "path", "dentry", "qstr"], + ), + // file_write: the write-intent filter (file.f_flags) + inode_ino (file→f_inode→ + // inode.i_ino, the dedup key) + emit_file_path (file.f_path). + ("file_write", &["file", "inode", "path"]), + // mmap_file: emit_lib_name (file→f_path→path→dentry→d_name→qstr). + ("mmap_file", &["file", "path", "dentry", "qstr"]), + // fix_setuid: cred→uid (kuid_t.val). + ("fix_setuid", &["cred", "kuid_t"]), + // bprm_check: linux_binprm.file/.filename, plus exe_is_anon_inode's + // file→f_inode→inode→i_sb→super_block chase. + ( + "bprm_check", + &["linux_binprm", "file", "inode", "super_block"], + ), + // ptrace_access_check / kernel_load_data read only scalar arguments (`mode`, `id`) + // plus the shared EventHeader — no vmlinux struct offset — so they're absent here + // and always attach (an enum-value mismatch on kernel_load_data is still logged, but + // fail-OPENS: see `load_preflight` below). +]; + +/// The kernel structs `probe` depends on, per [`STRUCT_DEPS`] — `&[]` (always attaches) +/// if the probe isn't listed. +pub(super) fn struct_deps(probe: &str) -> &'static [&'static str] { + STRUCT_DEPS + .iter() + .find(|(name, _)| *name == probe) + .map(|(_, deps)| *deps) + .unwrap_or(&[]) +} + +/// Read and parse `/sys/kernel/btf/vmlinux` (independently of the `aya::Btf` the observer +/// also loads for fentry attach — `aya`/`aya-obj`'s public API can't answer offset/enum +/// questions, see `preflight/btf.rs`'s module doc) and check it against the shared +/// offset/enum table, logging every divergence found — the expected-vs-actual data an +/// operator needs to regenerate `vmlinux.rs`'s bindings. A read failure (missing BTF — an +/// older kernel, or the DaemonSet's `/sys/kernel/btf` mount missing) fails closed the same +/// way a parse failure does ([`PreflightReport::fail_closed`]). +pub(super) fn load_preflight() -> PreflightReport { + let report = match std::fs::read("/sys/kernel/btf/vmlinux") { + Ok(bytes) => crate::preflight::check_bytes(&bytes), + Err(error) => { + tracing::warn!( + %error, + "BTF preflight: could not read node BTF; every struct-reading probe disabled" + ); + PreflightReport::fail_closed() + } + }; + for mismatch in &report.field_mismatches { + tracing::warn!( + kernel_struct = mismatch.kernel_struct, + field = mismatch.field, + expected = mismatch.expected, + actual = ?mismatch.actual, + "BTF preflight: field offset mismatch — update agent/common's FIELD_OFFSETS \ + (and vmlinux.rs) from this line" + ); + } + if let Some(mismatch) = &report.enum_mismatch { + tracing::warn!( + enum_name = mismatch.enum_name, + variant = mismatch.variant, + expected = mismatch.expected, + actual = ?mismatch.actual, + "BTF preflight: LOADING_MODULE enum value mismatch — the module-load probe \ + may misclassify (still attached: not verifier-checked, fail-open)" + ); + } + if report.is_clean() { + tracing::info!( + "BTF preflight: every baked field offset and the LOADING_MODULE enum value \ + match this node's live BTF" + ); + } + report +} diff --git a/agent/protector-agent/src/preflight/btf.rs b/agent/protector-agent/src/preflight/btf.rs new file mode 100644 index 00000000..7de1a1b3 --- /dev/null +++ b/agent/protector-agent/src/preflight/btf.rs @@ -0,0 +1,421 @@ +//! A small, self-contained parser for the raw BTF binary format +//! (`/sys/kernel/btf/vmlinux`), used only to answer one question: "what byte offset does +//! `struct.field` have, and what integer value does `enum::VARIANT` have, on THIS node's +//! kernel?" (ADR-0014 amendment, load-time BTF preflight.) +//! +//! Neither `aya::Btf` nor `aya-obj::Btf`'s public API answers that: `aya-obj` 0.2.1's +//! `BtfType`/`BtfMember`/`Struct::members`/`Union::members` are all `pub(crate)`, and +//! `Btf::type_by_id` / `Btf::types()` are `pub(crate)` too (aya only needs to hand a `Btf` +//! to the kernel verifier, not answer field-offset questions from Rust — its one public +//! lookup, `id_by_type_name_kind`, returns a type id with no public way to read what that +//! id's members are). So this module parses the BTF type section directly, per the format +//! documented at . It only decodes +//! the handful of kinds the preflight table needs (STRUCT, UNION, ENUM, ENUM64) plus the +//! "see-through" kinds (TYPEDEF/CONST/VOLATILE/RESTRICT) needed to resolve an anonymous +//! member's type to the struct/union it wraps — every other kind is still walked (its +//! encoded length must be computed to find the next type) but its payload is discarded. + +use std::fmt; + +const BTF_MAGIC: u16 = 0xeb9f; +const HEADER_LEN: usize = 24; + +const KIND_STRUCT: u8 = 4; +const KIND_UNION: u8 = 5; +const KIND_ENUM: u8 = 6; +const KIND_TYPEDEF: u8 = 8; +const KIND_VOLATILE: u8 = 9; +const KIND_CONST: u8 = 10; +const KIND_RESTRICT: u8 = 11; +const KIND_FUNC_PROTO: u8 = 13; +const KIND_VAR: u8 = 14; +const KIND_DATASEC: u8 = 15; +const KIND_DECL_TAG: u8 = 17; +const KIND_ENUM64: u8 = 19; +// PTR(2), ARRAY(3, handled by its own 12-byte extra below), FWD(7), FUNC(12), FLOAT(16), +// TYPE_TAG(18), and the unnamed KIND_INT(1) all fall through to the generic 0/4/12-byte +// "opaque" arms below — see `parse_types`. +const KIND_INT: u8 = 1; +const KIND_ARRAY: u8 = 3; + +/// Why parsing a BTF blob failed. Any of these is treated identically by the caller +/// ([`super::check_bytes`]): fail closed on every struct-reading probe (ADR-0014's +/// degrade-gracefully — never crash, never trust an offset we couldn't independently +/// re-derive). +#[derive(Debug)] +pub enum BtfParseError { + /// Shorter than a BTF header. + TooShort, + /// The first two bytes aren't the BTF magic in either byte order. + BadMagic, + /// A header offset/length points outside the blob, or a type's declared member/ + /// variant count runs past the type section's end. + Truncated, +} + +impl fmt::Display for BtfParseError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::TooShort => write!(f, "shorter than a BTF header"), + Self::BadMagic => write!(f, "not a BTF blob (bad magic)"), + Self::Truncated => write!(f, "truncated or self-inconsistent BTF type section"), + } + } +} + +impl std::error::Error for BtfParseError {} + +#[derive(Clone, Copy)] +enum Endian { + Little, + Big, +} + +impl Endian { + fn u32(self, b: &[u8]) -> u32 { + let a: [u8; 4] = b.try_into().expect("4-byte slice"); + match self { + Self::Little => u32::from_le_bytes(a), + Self::Big => u32::from_be_bytes(a), + } + } + + fn i32(self, b: &[u8]) -> i32 { + self.u32(b) as i32 + } +} + +/// A decoded struct/union member: its name (empty ⇒ anonymous), the type id of its own +/// type, and its byte offset within the containing struct/union. BTF's bitfield-size +/// encoding is discarded — nothing this preflight ever reads is a bitfield, so only the +/// (always byte-aligned, in practice) offset matters. +struct Member { + name: String, + type_id: u32, + byte_offset: u32, +} + +/// A decoded enum variant: its name and signed value — covers both the 32-bit +/// `BTF_KIND_ENUM` and the 64-bit `BTF_KIND_ENUM64`. +struct EnumVariant { + name: String, + value: i64, +} + +/// What's kept about one parsed BTF type — only what the preflight ever asks for. +enum Decoded { + Struct(Vec), + Union(Vec), + Enum(Vec), + /// A "see-through" type (typedef/const/volatile/restrict) forwarding to `type_id` — + /// the only kinds an anonymous member's type is resolved through when the walk is + /// looking for the struct/union underneath. + Forward(u32), + /// Every other kind: still correctly skipped during parsing, payload discarded. + Opaque, +} + +struct RawType { + name: String, + kind: Decoded, +} + +/// A parsed BTF blob, queried by struct-field and enum-variant name. Built once from raw +/// bytes ([`RawBtf::parse`]) at agent startup, before any probe attaches. +pub struct RawBtf { + /// `types[i]` is BTF type id `i + 1` (id `0` is the implicit `void`, never stored). + types: Vec, +} + +impl RawBtf { + /// Parse a raw BTF blob (the bytes of `/sys/kernel/btf/vmlinux`, or a test fixture in + /// the same format). Detects endianness from the magic bytes rather than assuming the + /// host's — the same blob is valid on either fleet arch. + pub fn parse(data: &[u8]) -> Result { + if data.len() < HEADER_LEN { + return Err(BtfParseError::TooShort); + } + let endian = if u16::from_le_bytes([data[0], data[1]]) == BTF_MAGIC { + Endian::Little + } else if u16::from_be_bytes([data[0], data[1]]) == BTF_MAGIC { + Endian::Big + } else { + return Err(BtfParseError::BadMagic); + }; + let hdr_len = endian.u32(&data[4..8]) as usize; + let type_off = endian.u32(&data[8..12]) as usize; + let type_len = endian.u32(&data[12..16]) as usize; + let str_off = endian.u32(&data[16..20]) as usize; + let str_len = endian.u32(&data[20..24]) as usize; + + let type_start = hdr_len + .checked_add(type_off) + .ok_or(BtfParseError::Truncated)?; + let type_end = type_start + .checked_add(type_len) + .ok_or(BtfParseError::Truncated)?; + let str_start = hdr_len + .checked_add(str_off) + .ok_or(BtfParseError::Truncated)?; + let str_end = str_start + .checked_add(str_len) + .ok_or(BtfParseError::Truncated)?; + if type_end > data.len() || str_end > data.len() { + return Err(BtfParseError::Truncated); + } + + let strings = &data[str_start..str_end]; + let types = Self::parse_types(&data[type_start..type_end], strings, endian)?; + Ok(Self { types }) + } + + /// Read a NUL-terminated string at `offset` into `strings`. An out-of-range or + /// unterminated offset yields `""` rather than an error — a cosmetic-only failure + /// (a name that can't be read just never matches anything the preflight looks up). + fn string_at(strings: &[u8], offset: u32) -> String { + let start = offset as usize; + let Some(rest) = strings.get(start..) else { + return String::new(); + }; + let end = rest.iter().position(|&b| b == 0).unwrap_or(rest.len()); + String::from_utf8_lossy(&rest[..end]).into_owned() + } + + fn parse_types( + mut buf: &[u8], + strings: &[u8], + endian: Endian, + ) -> Result, BtfParseError> { + let mut out = Vec::new(); + while !buf.is_empty() { + if buf.len() < 12 { + return Err(BtfParseError::Truncated); + } + let name_off = endian.u32(&buf[0..4]); + let info = endian.u32(&buf[4..8]); + let extra = endian.u32(&buf[8..12]); + let kind = ((info >> 24) & 0x1f) as u8; + let kind_flag = (info >> 31) & 1 == 1; + let vlen = (info & 0xffff) as usize; + let mut consumed = 12usize; + + let decoded = match kind { + KIND_STRUCT | KIND_UNION => { + let need = vlen.checked_mul(12).ok_or(BtfParseError::Truncated)?; + if buf.len() < consumed + need { + return Err(BtfParseError::Truncated); + } + let mut members = Vec::with_capacity(vlen); + for i in 0..vlen { + let base = consumed + i * 12; + let m_name = endian.u32(&buf[base..base + 4]); + let m_type = endian.u32(&buf[base + 4..base + 8]); + let m_off = endian.u32(&buf[base + 8..base + 12]); + // kind_flag set ⇒ the low 24 bits are the bit-offset (high 8 the + // bitfield size, discarded); unset ⇒ the whole word is the plain + // bit-offset. Either way this crate's targets are never + // bitfields, so only the offset survives. + let bit_offset = if kind_flag { + m_off & 0x00ff_ffff + } else { + m_off + }; + members.push(Member { + name: Self::string_at(strings, m_name), + type_id: m_type, + byte_offset: bit_offset / 8, + }); + } + consumed += need; + if kind == KIND_STRUCT { + Decoded::Struct(members) + } else { + Decoded::Union(members) + } + } + KIND_ENUM => { + let need = vlen.checked_mul(8).ok_or(BtfParseError::Truncated)?; + if buf.len() < consumed + need { + return Err(BtfParseError::Truncated); + } + let mut variants = Vec::with_capacity(vlen); + for i in 0..vlen { + let base = consumed + i * 8; + let v_name = endian.u32(&buf[base..base + 4]); + let v_val = endian.i32(&buf[base + 4..base + 8]); + variants.push(EnumVariant { + name: Self::string_at(strings, v_name), + value: v_val as i64, + }); + } + consumed += need; + Decoded::Enum(variants) + } + KIND_ENUM64 => { + let need = vlen.checked_mul(12).ok_or(BtfParseError::Truncated)?; + if buf.len() < consumed + need { + return Err(BtfParseError::Truncated); + } + let mut variants = Vec::with_capacity(vlen); + for i in 0..vlen { + let base = consumed + i * 12; + let v_name = endian.u32(&buf[base..base + 4]); + let lo = endian.u32(&buf[base + 4..base + 8]) as u64; + let hi = endian.u32(&buf[base + 8..base + 12]) as u64; + variants.push(EnumVariant { + name: Self::string_at(strings, v_name), + value: ((hi << 32) | lo) as i64, + }); + } + consumed += need; + Decoded::Enum(variants) + } + KIND_TYPEDEF | KIND_CONST | KIND_RESTRICT | KIND_VOLATILE => { + Decoded::Forward(extra) + } + KIND_INT => { + consumed += 4; // one extra word: encoding/offset/bits, unused here + Decoded::Opaque + } + KIND_ARRAY => { + consumed += 12; // btf_array: element type, index type, nelems + Decoded::Opaque + } + KIND_FUNC_PROTO => { + let need = vlen.checked_mul(8).ok_or(BtfParseError::Truncated)?; // btf_param + if buf.len() < consumed + need { + return Err(BtfParseError::Truncated); + } + consumed += need; + Decoded::Opaque + } + KIND_DATASEC => { + let need = vlen.checked_mul(12).ok_or(BtfParseError::Truncated)?; // btf_var_secinfo + if buf.len() < consumed + need { + return Err(BtfParseError::Truncated); + } + consumed += need; + Decoded::Opaque + } + KIND_VAR => { + consumed += 4; // linkage + Decoded::Opaque + } + KIND_DECL_TAG => { + consumed += 4; // component_idx + Decoded::Opaque + } + // PTR/FWD/FUNC/FLOAT/TYPE_TAG (and BTF_KIND_UNKN(0), any future kind) + // have no trailing payload beyond the 12-byte fixed prefix already read. + _ => Decoded::Opaque, + }; + + if buf.len() < consumed { + return Err(BtfParseError::Truncated); + } + out.push(RawType { + name: Self::string_at(strings, name_off), + kind: decoded, + }); + buf = &buf[consumed..]; + } + Ok(out) + } + + fn decoded(&self, type_id: u32) -> Option<&Decoded> { + if type_id == 0 { + return None; // the implicit `void` type — never a struct/union/enum + } + self.types.get((type_id - 1) as usize).map(|t| &t.kind) + } + + /// Find the first `KIND_STRUCT` type named `name` and return its 1-based type id. + /// `HashMap`-free linear scan: this runs once at agent startup against a + /// (large but bounded) live-kernel BTF, not per-event. + fn struct_id(&self, name: &str) -> Option { + self.types.iter().enumerate().find_map(|(i, t)| { + (t.name == name && matches!(t.kind, Decoded::Struct(_))).then_some((i + 1) as u32) + }) + } + + fn enum_id(&self, name: &str) -> Option { + self.types.iter().enumerate().find_map(|(i, t)| { + (t.name == name && matches!(t.kind, Decoded::Enum(_))).then_some((i + 1) as u32) + }) + } + + /// Follow `Forward` (typedef/const/volatile/restrict) links from `type_id` to the + /// struct/union underneath, or `None` if it never resolves to one. Bounded so a + /// malformed/cyclic blob can't loop forever. + fn resolve_to_aggregate(&self, mut type_id: u32) -> Option { + for _ in 0..16 { + match self.decoded(type_id)? { + Decoded::Struct(_) | Decoded::Union(_) => return Some(type_id), + Decoded::Forward(to) => type_id = *to, + _ => return None, + } + } + None + } + + /// How many anonymous-member levels [`member_offset`] will recurse into. Every + /// binding this crate has ever needed is at most one level deep (`inode.i_nlink`'s + /// anonymous union), but a malformed or adversarial BTF blob could otherwise encode a + /// self-referential chain of anonymous members and recurse without bound — a stack + /// overflow, which would crash the agent outright rather than degrade gracefully + /// (ADR-0014). This cap turns that into an ordinary "field not found" instead. `/sys/ + /// kernel/btf/vmlinux` is kernel-generated and root-owned, not attacker-controlled in + /// the normal threat model, but the parser doesn't rely on that to stay safe. + const MAX_MEMBER_RECURSION: u8 = 16; + + /// The byte offset of `field` within the struct/union at `type_id`, recursing into + /// any anonymous (nameless) member that itself resolves to a struct/union — + /// `inode.i_nlink`'s anonymous union is exactly this shape. Returns the FIRST match + /// found in declaration order. Bounded by [`Self::MAX_MEMBER_RECURSION`]. + fn member_offset(&self, type_id: u32, field: &str) -> Option { + self.member_offset_bounded(type_id, field, Self::MAX_MEMBER_RECURSION) + } + + fn member_offset_bounded(&self, type_id: u32, field: &str, budget: u8) -> Option { + let budget = budget.checked_sub(1)?; + let members = match self.decoded(type_id)? { + Decoded::Struct(m) | Decoded::Union(m) => m, + _ => return None, + }; + for m in members { + if m.name == field { + return Some(m.byte_offset); + } + if m.name.is_empty() + && let Some(inner_id) = self.resolve_to_aggregate(m.type_id) + && let Some(inner_off) = self.member_offset_bounded(inner_id, field, budget) + { + return Some(m.byte_offset + inner_off); + } + } + None + } + + /// The byte offset of `struct_name.field` in this BTF, or `None` if the struct isn't + /// present or has no member (directly, or via anonymous-union/struct recursion) named + /// `field`. + pub fn struct_field_offset(&self, struct_name: &str, field: &str) -> Option { + let id = self.struct_id(struct_name)?; + self.member_offset(id, field) + } + + /// The value of `enum_name::variant` in this BTF (either `BTF_KIND_ENUM` or the + /// 64-bit `BTF_KIND_ENUM64` — both decode into the same [`Decoded::Enum`]), or `None` + /// if the enum or variant isn't present. + pub fn enum_value(&self, enum_name: &str, variant: &str) -> Option { + let id = self.enum_id(enum_name)?; + match self.decoded(id)? { + Decoded::Enum(variants) => variants.iter().find(|v| v.name == variant).map(|v| v.value), + _ => None, + } + } +} + +#[cfg(test)] +#[path = "btf_tests.rs"] +mod tests; diff --git a/agent/protector-agent/src/preflight/btf_tests.rs b/agent/protector-agent/src/preflight/btf_tests.rs new file mode 100644 index 00000000..04d72988 --- /dev/null +++ b/agent/protector-agent/src/preflight/btf_tests.rs @@ -0,0 +1,145 @@ +use super::*; +use crate::preflight::fixture::BtfBuilder; + +#[test] +fn rejects_data_shorter_than_a_header() { + assert!(matches!( + RawBtf::parse(&[0u8; 4]), + Err(BtfParseError::TooShort) + )); +} + +#[test] +fn rejects_bad_magic() { + let mut blob = vec![0u8; 24]; + blob[0] = 0xaa; + blob[1] = 0xbb; + assert!(matches!(RawBtf::parse(&blob), Err(BtfParseError::BadMagic))); +} + +#[test] +fn rejects_a_type_section_that_overruns_the_blob() { + // A well-formed header claiming a type_len far larger than the actual data. + let mut blob = vec![0u8; 24]; + blob[0..2].copy_from_slice(&0xeb9fu16.to_le_bytes()); + blob[4..8].copy_from_slice(&24u32.to_le_bytes()); // hdr_len + blob[12..16].copy_from_slice(&999u32.to_le_bytes()); // type_len — way past EOF + assert!(matches!( + RawBtf::parse(&blob), + Err(BtfParseError::Truncated) + )); +} + +#[test] +fn finds_a_plain_struct_field_offset() { + let mut b = BtfBuilder::new(); + b.add_struct("demo_file", false, &[("f_flags", 0, 40), ("f_path", 0, 64)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("demo_file", "f_flags"), Some(40)); + assert_eq!(btf.struct_field_offset("demo_file", "f_path"), Some(64)); +} + +#[test] +fn unknown_struct_or_field_is_none() { + let mut b = BtfBuilder::new(); + b.add_struct("demo_file", false, &[("f_flags", 0, 40)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("no_such_struct", "f_flags"), None); + assert_eq!(btf.struct_field_offset("demo_file", "no_such_field"), None); +} + +#[test] +fn recurses_into_an_anonymous_union_member() { + // Mirrors `struct inode`: i_ino at +64, then an anonymous union (itself starting at + // +72) whose first member is i_nlink — the parser must add the union member's own + // offset (+72) to i_nlink's offset WITHIN the union (0) to land on +72. + let mut b = BtfBuilder::new(); + let union_id = b.add_struct("", true, &[("i_nlink", 0, 0), ("__i_nlink", 0, 0)]); + b.add_struct("demo_inode", false, &[("i_ino", 0, 64), ("", union_id, 72)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("demo_inode", "i_ino"), Some(64)); + assert_eq!(btf.struct_field_offset("demo_inode", "i_nlink"), Some(72)); + assert_eq!(btf.struct_field_offset("demo_inode", "__i_nlink"), Some(72)); +} + +#[test] +fn recurses_through_a_typedef_wrapping_the_anonymous_members_type() { + // A pathological but legal shape: the anonymous member's type is a typedef of the + // union, not the union directly. The walk must see through it. + let mut b = BtfBuilder::new(); + let union_id = b.add_struct("", true, &[("val", 0, 0)]); + let typedef_id = b.add_typedef("kuid_t", union_id); + b.add_struct("demo_cred", false, &[("", typedef_id, 8)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("demo_cred", "val"), Some(8)); +} + +#[test] +fn finds_an_enum_variant_value() { + let mut b = BtfBuilder::new(); + b.add_enum( + "demo_kernel_load_data_id", + &[ + ("LOADING_UNKNOWN", 0), + ("LOADING_FIRMWARE", 1), + ("LOADING_MODULE", 2), + ], + ); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!( + btf.enum_value("demo_kernel_load_data_id", "LOADING_MODULE"), + Some(2) + ); + assert_eq!( + btf.enum_value("demo_kernel_load_data_id", "LOADING_FIRMWARE"), + Some(1) + ); + assert_eq!( + btf.enum_value("demo_kernel_load_data_id", "NOT_A_VARIANT"), + None + ); +} + +#[test] +fn anonymous_member_recursion_terminates_on_a_self_referential_type() { + // A malformed/adversarial BTF blob could encode a struct whose anonymous member + // points back at itself (or a longer cycle) — the walk must terminate (a bounded + // "not found", never a stack overflow that would crash the agent outright rather + // than degrade gracefully, ADR-0014). + let mut b = BtfBuilder::new(); + let self_id = b.next_id(); + b.add_struct("cyclic", false, &[("", self_id, 0)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("cyclic", "anything"), None); +} + +#[test] +fn a_second_struct_of_the_same_name_does_not_confuse_the_first_lookup() { + // BTF is append-only per compilation unit; a real vmlinux blob has thousands of + // types. Confirm the scan returns the FIRST struct match, not a later unrelated one. + let mut b = BtfBuilder::new(); + b.add_struct("file", false, &[("f_flags", 0, 40)]); + b.add_struct("other", false, &[("f_flags", 0, 999)]); + let btf = RawBtf::parse(&b.build()).unwrap(); + assert_eq!(btf.struct_field_offset("file", "f_flags"), Some(40)); +} + +#[test] +fn parses_a_big_endian_header() { + // A minimal, valid BTF header + one empty string table, every multi-byte field + // encoded BIG-endian — confirms `RawBtf::parse` detects endianness from the magic + // bytes rather than assuming the host's (the fixture builder above only ever + // produces little-endian blobs, matching a real `bpfel`-target node). + let mut blob = Vec::new(); + blob.extend_from_slice(&0xeb9fu16.to_be_bytes()); // magic + blob.push(1); // version + blob.push(0); // flags + blob.extend_from_slice(&24u32.to_be_bytes()); // hdr_len + blob.extend_from_slice(&0u32.to_be_bytes()); // type_off + blob.extend_from_slice(&0u32.to_be_bytes()); // type_len (no types) + blob.extend_from_slice(&0u32.to_be_bytes()); // str_off + blob.extend_from_slice(&1u32.to_be_bytes()); // str_len (just the leading NUL) + blob.push(0); // the string table's leading NUL byte + let btf = RawBtf::parse(&blob).unwrap(); + assert_eq!(btf.struct_field_offset("anything", "anything"), None); +} diff --git a/agent/protector-agent/src/preflight/fixture.rs b/agent/protector-agent/src/preflight/fixture.rs new file mode 100644 index 00000000..511bcceb --- /dev/null +++ b/agent/protector-agent/src/preflight/fixture.rs @@ -0,0 +1,137 @@ +//! Test-only: a tiny builder for synthetic BTF blobs, encoded in the same binary format +//! [`super::btf::RawBtf::parse`] reads (little-endian, per +//! ). Lets the preflight's unit +//! tests exercise the real parser and lookup logic against a hand-built "kernel" — no +//! live node, no `bpftool`, no root — so the offset-self-verification the module exists +//! to provide is itself off-fleet testable. + +/// KIND numbers duplicated from `btf.rs` rather than `pub(crate)`-exposing them there — +/// this file is the only other place that needs to encode (not decode) them, and keeping +/// the encoder self-contained means a fixture test can never accidentally pass because a +/// shared constant was wrong on both sides. +mod kind { + pub const STRUCT: u32 = 4; + pub const UNION: u32 = 5; + pub const ENUM: u32 = 6; + pub const TYPEDEF: u32 = 8; +} + +/// Builds one synthetic BTF blob. Types are appended in declaration order and get +/// sequential 1-based ids, matching real BTF's id assignment. +#[derive(Default)] +pub(crate) struct BtfBuilder { + strings: Vec, + types: Vec, + next_id: u32, +} + +impl BtfBuilder { + pub(crate) fn new() -> Self { + // Offset 0 in the string table is always the empty string (BTF convention; + // `name_off == 0` means "anonymous"). + Self { + strings: vec![0], + ..Default::default() + } + } + + /// The id the NEXT `add_*` call will assign — lets a test build a self-referential or + /// cyclic type (a member pointing at a type not created yet) by predicting its id + /// ahead of time. + pub(crate) fn next_id(&self) -> u32 { + self.next_id + 1 + } + + fn add_string(&mut self, s: &str) -> u32 { + if s.is_empty() { + return 0; + } + let off = self.strings.len() as u32; + self.strings.extend_from_slice(s.as_bytes()); + self.strings.push(0); + off + } + + fn alloc_id(&mut self) -> u32 { + self.next_id += 1; + self.next_id + } + + /// Append a `BTF_KIND_STRUCT` or `BTF_KIND_UNION` type. `members`: `(name, type_id, + /// byte_offset)` — `type_id` only matters for an ANONYMOUS member the test wants the + /// parser to recurse into (a named member's type is never chased); `byte_offset` is + /// encoded as a plain bit-offset (`kind_flag` unset — no bitfields, matching every + /// real binding this preflight ever checks). Returns the new type's 1-based id. + pub(crate) fn add_struct( + &mut self, + name: &str, + is_union: bool, + members: &[(&str, u32, u32)], + ) -> u32 { + let id = self.alloc_id(); + let name_off = self.add_string(name); + let kind = if is_union { kind::UNION } else { kind::STRUCT }; + let info = (kind << 24) | (members.len() as u32 & 0xffff); + self.types.extend_from_slice(&name_off.to_le_bytes()); + self.types.extend_from_slice(&info.to_le_bytes()); + self.types.extend_from_slice(&0u32.to_le_bytes()); // size — unused by the preflight + for (m_name, m_type, m_byte_off) in members { + let m_name_off = self.add_string(m_name); + self.types.extend_from_slice(&m_name_off.to_le_bytes()); + self.types.extend_from_slice(&m_type.to_le_bytes()); + self.types + .extend_from_slice(&(m_byte_off * 8).to_le_bytes()); + } + id + } + + /// Append a `BTF_KIND_TYPEDEF` forwarding to `to` — the "see-through" kind the parser + /// resolves an anonymous member's type through. + pub(crate) fn add_typedef(&mut self, name: &str, to: u32) -> u32 { + let id = self.alloc_id(); + let name_off = self.add_string(name); + let info = kind::TYPEDEF << 24; + self.types.extend_from_slice(&name_off.to_le_bytes()); + self.types.extend_from_slice(&info.to_le_bytes()); + self.types.extend_from_slice(&to.to_le_bytes()); + id + } + + /// Append a `BTF_KIND_ENUM` type with `(name, value)` variants. + pub(crate) fn add_enum(&mut self, name: &str, variants: &[(&str, i32)]) -> u32 { + let id = self.alloc_id(); + let name_off = self.add_string(name); + let info = (kind::ENUM << 24) | (variants.len() as u32 & 0xffff); + self.types.extend_from_slice(&name_off.to_le_bytes()); + self.types.extend_from_slice(&info.to_le_bytes()); + self.types.extend_from_slice(&4u32.to_le_bytes()); // size: 4-byte enum + for (v_name, v_val) in variants { + let v_name_off = self.add_string(v_name); + self.types.extend_from_slice(&v_name_off.to_le_bytes()); + self.types.extend_from_slice(&v_val.to_le_bytes()); + } + id + } + + /// Encode the finished blob as a full little-endian BTF binary (header, then the type + /// section, then the string section) — what [`super::btf::RawBtf::parse`] (and a real + /// `/sys/kernel/btf/vmlinux`) expects. + pub(crate) fn build(self) -> Vec { + const HEADER_LEN: u32 = 24; + let type_len = self.types.len() as u32; + let str_len = self.strings.len() as u32; + let mut out = + Vec::with_capacity(HEADER_LEN as usize + self.types.len() + self.strings.len()); + out.extend_from_slice(&0xeb9fu16.to_le_bytes()); // magic + out.push(1); // version + out.push(0); // flags + out.extend_from_slice(&HEADER_LEN.to_le_bytes()); + out.extend_from_slice(&0u32.to_le_bytes()); // type_off: types start right after the header + out.extend_from_slice(&type_len.to_le_bytes()); + out.extend_from_slice(&type_len.to_le_bytes()); // str_off: strings start right after types + out.extend_from_slice(&str_len.to_le_bytes()); + out.extend_from_slice(&self.types); + out.extend_from_slice(&self.strings); + out + } +} diff --git a/agent/protector-agent/src/preflight/mod.rs b/agent/protector-agent/src/preflight/mod.rs new file mode 100644 index 00000000..07d183ad --- /dev/null +++ b/agent/protector-agent/src/preflight/mod.rs @@ -0,0 +1,152 @@ +//! Load-time BTF preflight (ADR-0014 amendment): before any probe attaches, re-verify +//! every field offset the eBPF crate bakes in (`agent/protector-agent-ebpf/src/ +//! vmlinux.rs`, guarded at compile time against the shared table in +//! `protector-agent-common`) against THIS node's live BTF, plus the `LOADING_MODULE` +//! enum value the module-load probe compares against. +//! +//! The eBPF crate has no CO-RE field relocation (rustc emits no BTF field relocations — +//! see `vmlinux.rs`'s module doc): every `(*ptr).field` is baked in as a constant offset +//! at compile time, verified once against the fleet kernel that existed at the time. A +//! kernel upgrade can silently move a field; the compile-time `offset_of!` guard can't +//! catch that (it only proves the bindings are internally consistent with themselves, not +//! with a NEW kernel). This module is the guard that DOES catch it, on every node, at +//! every agent start — the observer (`crate::observer`) calls [`check_bytes`] before +//! attaching any probe and disables the probes whose reads it can no longer trust. +//! +//! This module is deliberately independent of the `ebpf` feature (unlike the observer +//! that calls it): it's plain, off-fleet-testable userspace Rust — see `preflight/ +//! btf_tests.rs` and `preflight/tests.rs`, both exercised against hand-built fixture BTF +//! blobs (`preflight/fixture.rs`), no live kernel or bpf toolchain required. + +mod btf; +#[cfg(test)] +mod fixture; +#[cfg(test)] +#[path = "tests.rs"] +mod tests; + +use protector_agent_common::offsets::{ + FIELD_OFFSETS, LOADING_MODULE_ENUM, LOADING_MODULE_VALUE, LOADING_MODULE_VARIANT, +}; + +pub use btf::RawBtf; + +/// One field whose live-BTF offset diverges from what `agent/common`'s table (and the +/// eBPF crate's `offset_of!` guard) expect — the regeneration data an operator needs: +/// which field, what the bindings expect, what the kernel actually has. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FieldMismatch { + pub kernel_struct: &'static str, + pub field: &'static str, + pub expected: u32, + /// `None` when the struct/field isn't found in BTF at all (a bigger divergence than a + /// moved offset — a renamed/removed field, or BTF that couldn't be read/parsed). + pub actual: Option, +} + +/// The `LOADING_MODULE` enum-value mismatch, if any. Unlike a struct offset this is NOT +/// verifier-checked — `try_kernel_load_data` compares it as a plain integer — so a wrong +/// value would silently misclassify rather than fail loud, which is why the preflight +/// checks it explicitly rather than trusting the compile-time constant alone. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct EnumMismatch { + pub enum_name: &'static str, + pub variant: &'static str, + pub expected: i64, + pub actual: Option, +} + +/// The result of walking a node's live BTF against [`FIELD_OFFSETS`] and +/// [`LOADING_MODULE_ENUM`]. Built once at agent startup, before any probe attaches. +#[derive(Debug, Default, Clone, PartialEq, Eq)] +pub struct PreflightReport { + pub field_mismatches: Vec, + pub enum_mismatch: Option, +} + +impl PreflightReport { + /// Whether every [`FIELD_OFFSETS`] entry for this `kernel_struct` name matched live + /// BTF. Fail-closed at struct granularity, not field granularity: if any field of a + /// struct a probe reads diverged, every other pointer chase through that same struct + /// is untrusted too (a moved field can shift everything after it) — see the + /// `STRUCT_DEPS` table in `observer.rs` that decides which probes this gates. + pub fn struct_ok(&self, kernel_struct: &str) -> bool { + !self + .field_mismatches + .iter() + .any(|m| m.kernel_struct == kernel_struct) + } + + /// True if nothing diverged and the enum matched — the common, healthy case. + pub fn is_clean(&self) -> bool { + self.field_mismatches.is_empty() && self.enum_mismatch.is_none() + } + + /// Every table entry marked mismatched (`actual` unknown) — the fail-closed report + /// for when BTF can't even be read (e.g. `/sys/kernel/btf/vmlinux` missing, an older + /// kernel with no exported BTF). Every struct-reading probe is then disabled via + /// [`Self::struct_ok`]; the struct-free probes (connect, ptrace-attach, module-load) + /// still attach — degrade-gracefully, never crash-loop (ADR-0014). + pub fn fail_closed() -> Self { + let field_mismatches = FIELD_OFFSETS + .iter() + .map(|entry| FieldMismatch { + kernel_struct: entry.kernel_struct, + field: entry.field, + expected: entry.offset, + actual: None, + }) + .collect(); + Self { + field_mismatches, + enum_mismatch: Some(EnumMismatch { + enum_name: LOADING_MODULE_ENUM, + variant: LOADING_MODULE_VARIANT, + expected: LOADING_MODULE_VALUE as i64, + actual: None, + }), + } + } +} + +/// Walk `btf` against the shared offset/enum table, producing every divergence found. +/// Pure (no I/O, no logging) so it's directly unit-testable; the caller (`observer.rs`) +/// owns turning the result into log lines and probe-attach decisions. +pub fn check(btf: &RawBtf) -> PreflightReport { + let field_mismatches = FIELD_OFFSETS + .iter() + .filter_map(|entry| { + let actual = btf.struct_field_offset(entry.kernel_struct, entry.field); + (actual != Some(entry.offset)).then_some(FieldMismatch { + kernel_struct: entry.kernel_struct, + field: entry.field, + expected: entry.offset, + actual, + }) + }) + .collect(); + + let actual_enum = btf.enum_value(LOADING_MODULE_ENUM, LOADING_MODULE_VARIANT); + let enum_mismatch = + (actual_enum != Some(LOADING_MODULE_VALUE as i64)).then_some(EnumMismatch { + enum_name: LOADING_MODULE_ENUM, + variant: LOADING_MODULE_VARIANT, + expected: LOADING_MODULE_VALUE as i64, + actual: actual_enum, + }); + + PreflightReport { + field_mismatches, + enum_mismatch, + } +} + +/// Parse `bytes` as BTF and run [`check`] against it. A parse failure (missing/corrupt +/// `/sys/kernel/btf/vmlinux`) folds into [`PreflightReport::fail_closed`] — the caller has +/// one fail-closed path to handle, not a separate parse-error case. +pub fn check_bytes(bytes: &[u8]) -> PreflightReport { + match RawBtf::parse(bytes) { + Ok(btf) => check(&btf), + Err(_) => PreflightReport::fail_closed(), + } +} diff --git a/agent/protector-agent/src/preflight/tests.rs b/agent/protector-agent/src/preflight/tests.rs new file mode 100644 index 00000000..8239b0ac --- /dev/null +++ b/agent/protector-agent/src/preflight/tests.rs @@ -0,0 +1,170 @@ +//! End-to-end preflight tests: [`super::check`]/[`super::check_bytes`] against a fixture +//! BTF blob ([`crate::preflight::fixture::BtfBuilder`]) that mirrors the real +//! `FIELD_OFFSETS` table field-for-field, so these exercise the exact same lookups the +//! agent runs against a node's live BTF — just against a hand-built "kernel" instead. + +use super::fixture::BtfBuilder; +use super::*; + +/// Build a fixture whose struct layout matches every [`FIELD_OFFSETS`] entry. Each +/// parameter lets a caller deliberately vary the one field a test is targeting while +/// every other field stays correct — the shared shape both `golden()` (all correct) and +/// every mismatch test build from, so a fixture's correct fields never drift out of sync +/// between tests. +/// +/// `f_path_offset`: `file.f_path`'s offset (varied to simulate a kernel-upgrade-moved +/// offset). `inode_union_offset`: the anonymous union holding `i_nlink`/`__i_nlink`'s own +/// offset within `inode` (varied to prove the anon-union recursion actually adds it in, +/// rather than ignoring it). `loading_module_value`: the `LOADING_MODULE` enum variant's +/// value. `include_linux_binprm`: omit the struct entirely to simulate a field BTF has no +/// record of at all (`actual: None`, not just a moved offset). +fn build_fixture( + f_path_offset: u32, + inode_union_offset: u32, + loading_module_value: i32, + include_linux_binprm: bool, +) -> Vec { + let mut b = BtfBuilder::new(); + b.add_struct( + "file", + false, + &[ + ("f_inode", 0, 32), + ("f_flags", 0, 40), + ("f_path", 0, f_path_offset), + ], + ); + b.add_struct("path", false, &[("dentry", 0, 8)]); + b.add_struct("dentry", false, &[("d_name", 0, 32)]); + b.add_struct("qstr", false, &[("name", 0, 8)]); + // inode.i_nlink lives in an anonymous union immediately after i_ino — the same shape + // the real kernel struct has (see vmlinux.rs's module doc). + let union_id = b.add_struct("", true, &[("i_nlink", 0, 0), ("__i_nlink", 0, 0)]); + b.add_struct( + "inode", + false, + &[ + ("i_sb", 0, 40), + ("i_ino", 0, 64), + ("", union_id, inode_union_offset), + ], + ); + b.add_struct("super_block", false, &[("s_magic", 0, 96)]); + b.add_struct("cred", false, &[("uid", 0, 8)]); + b.add_struct("kuid_t", false, &[("val", 0, 0)]); + if include_linux_binprm { + b.add_struct( + "linux_binprm", + false, + &[("file", 0, 64), ("filename", 0, 96)], + ); + } + b.add_enum( + "kernel_load_data_id", + &[ + ("LOADING_UNKNOWN", 0), + ("LOADING_FIRMWARE", 1), + ("LOADING_MODULE", loading_module_value), + ], + ); + b.build() +} + +fn golden() -> Vec { + build_fixture(64, 72, 2, true) +} + +#[test] +fn correct_offsets_and_enum_pass_clean() { + let report = check_bytes(&golden()); + assert!(report.is_clean(), "{report:?}"); + assert!(report.field_mismatches.is_empty()); + assert!(report.enum_mismatch.is_none()); +} + +#[test] +fn a_moved_offset_is_flagged_with_expected_and_actual() { + // Simulate the 6.8→6.11 struct file reorg the module doc describes: f_path moves + // from +64 to +72. + let report = check_bytes(&build_fixture(72, 72, 2, true)); + assert_eq!( + report.field_mismatches, + vec![FieldMismatch { + kernel_struct: "file", + field: "f_path", + expected: 64, + actual: Some(72), + }] + ); + assert!(!report.struct_ok("file")); + // Every OTHER struct's fields were untouched — only `file` fails. + assert!(report.struct_ok("inode")); + assert!(report.struct_ok("linux_binprm")); + assert!(report.enum_mismatch.is_none()); +} + +#[test] +fn anon_union_recursion_resolves_i_nlink_correctly() { + // A dedicated, independent check that the anon-union recursion the module doc calls + // out (`inode.i_nlink`) is exactly the field the full-table check relies on: placing + // the union itself at the WRONG offset (+80 instead of +72) must surface as exactly + // one `inode.i_nlink` mismatch (72 expected, 80 actual), proving the recursion + // actually adds the anon member's own offset in rather than ignoring it. + let report = check_bytes(&build_fixture(64, 80, 2, true)); + assert_eq!( + report.field_mismatches, + vec![FieldMismatch { + kernel_struct: "inode", + field: "i_nlink", + expected: 72, + actual: Some(80), + }] + ); + assert!(!report.struct_ok("inode")); +} + +#[test] +fn enum_value_mismatch_is_flagged_independently_of_field_offsets() { + let report = check_bytes(&build_fixture(64, 72, 5, true)); + assert!(report.field_mismatches.is_empty()); + assert_eq!( + report.enum_mismatch, + Some(EnumMismatch { + enum_name: "kernel_load_data_id", + variant: "LOADING_MODULE", + expected: 2, + actual: Some(5), + }) + ); +} + +#[test] +fn a_struct_missing_from_btf_entirely_flags_every_field_with_no_actual() { + let report = check_bytes(&build_fixture(64, 72, 2, false)); + let mut binprm: Vec<_> = report + .field_mismatches + .iter() + .filter(|m| m.kernel_struct == "linux_binprm") + .collect(); + binprm.sort_by_key(|m| m.field); + assert_eq!(binprm.len(), 2); + assert_eq!(binprm[0].field, "file"); + assert_eq!(binprm[0].actual, None); + assert_eq!(binprm[1].field, "filename"); + assert_eq!(binprm[1].actual, None); + assert!(!report.struct_ok("linux_binprm")); + assert!(report.struct_ok("file")); // unrelated structs stay clean +} + +#[test] +fn unparseable_btf_fails_closed_on_every_table_entry() { + let report = check_bytes(b"not a btf blob"); + assert_eq!(report, PreflightReport::fail_closed()); + assert_eq!(report.field_mismatches.len(), FIELD_OFFSETS.len()); + assert!( + FIELD_OFFSETS + .iter() + .all(|e| !report.struct_ok(e.kernel_struct)) + ); + assert!(report.enum_mismatch.is_some()); +} diff --git a/docs/ebpf-testing-on-nodes.md b/docs/ebpf-testing-on-nodes.md index e36428f5..fdef1f2d 100644 --- a/docs/ebpf-testing-on-nodes.md +++ b/docs/ebpf-testing-on-nodes.md @@ -31,8 +31,9 @@ first for this kernel: 1. **fentry on `security_file_open(struct file *file)` + `bpf_d_path(&file->f_path,…)`** — BTF-aware, clean. Needs vmlinux struct bindings for `struct file` so the probe can - take the address of `f_path` (CO-RE-relocated). Does **not** require BPF-LSM to be in - the active `lsm=` list, which is why it's preferred over option 2. + take the address of `f_path` (a baked, hand-verified offset — see the "no CO-RE field + relocation" note below). Does **not** require BPF-LSM to be in the active `lsm=` list, + which is why it's preferred over option 2. 2. **`lsm/file_open`** — simplest code, but requires `CONFIG_BPF_LSM=y` **and** `bpf` in the kernel's active LSM list (`/sys/kernel/security/lsm`). Ubuntu ships the config but whether `bpf` is in the active list is unconfirmed (couldn't read it without node @@ -51,6 +52,18 @@ btf` pod, then `bpftool btf dump … format c`, or parse the raw BTF) on **every and confirming the read fields share one offset. As of 2026-07-05 the fleet is `7.0.0` (arm64 raspi + amd64 generic) and all read fields align across both arches. +**Load-time BTF preflight:** re-verifying by hand on every kernel bump doesn't scale, and a +stale `bpf_probe_read_kernel` offset (unlike `bpf_d_path`) reads garbage silently rather +than failing the verifier. The userspace loader (`agent/protector-agent/src/observer.rs`) +therefore re-checks every baked offset — plus the `LOADING_MODULE` enum value — against each +node's own live BTF before attaching anything (`agent/protector-agent/src/preflight`, ADR-0014's +amendment). A struct-reading probe whose offsets diverged is disabled (fail-closed); the +struct-free probes (connect, ptrace-attach, module-load) still attach. Every divergence is +logged expected-vs-actual — that log line is the regeneration data for `vmlinux.rs`, replacing +the manual `bpftool btf dump` re-verification above as the *safety net*; doing it by hand +after a known kernel change is still the fastest way to fix the bindings once the preflight +has told you which field moved. + ## The validation loop (no exec, no SSH) 1. **Write the probe self-validating.** On attach, log `info` (`attached fentry …`). For