feat: granite 4.2 - kernels

kernels/CONVENTION.md

@@ -0,0 +1,88 @@
+# A convention for NPU kernel source in this repository
+
+This document proposes how contributed AIE kernel *source* lives in
+FastFlowLM. It is deliberately separable from the kernels that come with it:
+you can accept or reject the convention on its own terms, and `kernels/granite`
+is just the first worked example.
+
+## What this is, and what it is not
+
+`kernels/` holds **source**. `src/xclbins/` continues to hold the shipped
+binaries, and nothing here changes, moves, re-derives or replaces any of them.
+
+**No compiled artefact is ever committed under `kernels/`.** Artefacts are build
+outputs. They land in the build directory and are installed from there, exactly
+as the checked-in ones are.
+
+## Why source and not a binary
+
+An `.xclbin` is only valid for the toolchain that built it. A committed binary
+carries no record of what produced it and cannot be rebuilt when the toolchain
+moves; it rots silently, and the rot is invisible until a user hits it. Source
+plus a recorded toolchain fingerprint is the only representation that survives
+a version bump.
+
+That is the whole argument for this directory. The granite kernels are the
+occasion, not the point.
+
+## Layout
+
+One directory per model family, named to match `src/xclbins/<Model>/`:
+
+```
+kernels/
+├── CONVENTION.md          this file
+├── README.md              how to build
+├── LICENSE                MIT
+├── requirements.txt       pinned toolchain packages
+├── build_kernels.py       the only entry point
+├── common/                shared host-side helpers
+└── <family>/
+    ├── README.md          geometry, measured numbers, what was rejected
+    ├── geometry.json      model dimensions -- so the build needs no weights
+    ├── iron/              IRON Python: placement and data movement
+    └── aie/               C++ that runs on the AIE cores
+```
+
+**`iron/` and `aie/` are split because they are two different review surfaces.**
+`aie/` is device C++: it is what actually executes, and it is the surface that
+matters for provenance and for correctness. `iron/` is host Python that only
+describes where things are placed and how data moves. **Read `aie/` first.**
+
+## Rules
+
+**Never a build dependency.** No family may become a dependency of `flm`. With
+`FLM_BUILD_KERNELS=OFF` — the default — the build must be byte-identical to one
+in which `kernels/` does not exist.
+
+**The build must not need model weights**, network access, or any path outside
+the repository other than the toolchain itself. Shapes come from
+`geometry.json`. This is what makes the option safe to enable in a container,
+and it is why `geometry.json` exists at all.
+
+**No generated file is committed.** Generators live in `iron/`; their output
+goes to the build directory.
+
+**The toolchain is declared, not assumed.** Every build writes a
+`manifest.json` recording the exact package versions, compiler flags and a
+sha256 per artefact. A consumer that finds a fingerprint mismatch must refuse
+rather than dispatch a mismatched pair to the NPU.
+
+**Licence.** Everything under `kernels/` is MIT, matching
+`LICENSE_RUNTIME.txt`, with `SPDX-License-Identifier: MIT` in every file.
+
+## Provenance
+
+The kernels here were written from the public MLIR-AIE/IRON examples and
+published AIE2P documentation. **No shipped `.xclbin` was disassembled and no
+closed component was reverse-engineered.** The q4nx container layout was
+derived by inverting a published packer and cross-checked against published
+model files. (The companion granite PR documents the same derivation in
+`src/include/models/granite/q4nx_host.hpp`, if it has landed; this PR does not
+depend on it.)
+
+## What a new family must ship
+
+A `README.md` with measured numbers against a host reference, a `geometry.json`,
+and a `validate` path that can be run on hardware. A kernel with no reference
+comparison is not a contribution; it is a claim.

kernels/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 the FastFlowLM kernel contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

kernels/README.md

@@ -0,0 +1,81 @@
+# NPU kernel source
+
+Source for AIE kernels, built on request. See [CONVENTION.md](CONVENTION.md)
+for what this directory is and the rules it follows.
+
+**Nothing here is built by default.** `FLM_BUILD_KERNELS` is `OFF`, and with it
+off the `flm` build is identical to one in which this directory does not exist.
+
+## State
+
+| | |
+|---|---|
+| kernel source in tree | yes |
+| builds from source | yes, `-DFLM_BUILD_KERNELS=ON` |
+| validated against a host reference on NPU2 | yes — see `granite/README.md` |
+| **dispatched by `flm`** | **not yet** — see *The gap* below |
+
+## Prerequisites
+
+The [mlir-aie](https://github.com/Xilinx/mlir-aie) / IRON toolchain, which
+supplies the `aie` Python package, the `aiecc` driver and the Peano (LLVM-AIE)
+backend. Versions are pinned in [requirements.txt](requirements.txt) and
+recorded again in every build's `manifest.json`.
+
+Activate that environment first — the build does not try to activate it for
+you, because activation does not survive a subprocess.
+
+## Build
+
+```shell
+# with the IRON environment active
+python kernels/build_kernels.py --check-toolchain
+python kernels/build_kernels.py --family granite --out build/kernels
+```
+
+or through CMake:
+
+```shell
+cmake -S src -B src/build -DFLM_BUILD_KERNELS=ON
+cmake --build src/build --config Release
+```
+
+If the toolchain is not usable, configuring fails with one sentence naming both
+fixes, rather than an `aiecc` traceback halfway through a build. Pass
+`-DFLM_KERNELS_PYTHON=<path>` to point at a specific interpreter.
+
+Output mirrors `src/xclbins/<Model>/` so it installs through the rule that
+already exists:
+
+```
+build/kernels/Granite-4.2-3B-NPU2/
+    norm_qkv_rope.xclbin   norm_qkv_rope.insts.bin   norm_qkv_rope.json
+    attn_o.xclbin          ...
+    manifest.json          toolchain versions + sha256 per artefact
+```
+
+## Validate
+
+```shell
+python kernels/build_kernels.py --family granite --validate
+```
+
+Runs each design against a host reference built from the same bytes. **This is
+a hardware test**: it needs an NPU and the model weights, and CI is not asked to
+run it.
+
+## The gap
+
+These artefacts are complete and numerically validated, but `flm` cannot
+dispatch them yet, and the reason is specific.
+
+`src/include/npu_utils/npu_utils_xrt.hpp` builds its ELF from a control
+sequence assembled **on the host** by `npu_sequence`
+(`src/include/npu_utils/npu_instr_utils.hpp`). IRON emits that same control code
+as a prebuilt `insts.bin` at build time. Bridging the two is roughly one
+function: load `insts.bin` and hand it to `aiebu_assembler_get_elf` in place of
+the assembled sequence.
+
+That is deliberately not here. **This is a build and convention change, not a
+runtime change** — the two are worth deciding separately, and the runtime side
+follows only if the convention is wanted.