[Linux] Weight bos are MAP_LOCKED — models >~18GB cannot load on 22.6GB RAM machines and some request

[Cero-Xspr]

This issue was largely written by an AI, so there may be quite a few inaccuracies.

On Linux (amdxdna 0.7 / XRT 2.25.37, firmware 1.1.2.64), loading qwen3.6-moe:35b-a3b fails with:

Failed to allocate xrt::ext::bo: mmap(len=536870912, prot=3,
flags=0x2011 (MAP_SHARED|MAP_FIXED|MAP_LOCKED), ...) failed (err=-12): Cannot allocate memory

strace shows every weight chunk is allocated as a 512MB MAP_LOCKED device-window overlay on
/dev/accel/accel0. With LimitMEMLOCK=infinity the mapping succeeds, but the total lock
requirement (21.07 GiB language + 0.94 GiB vision + runtime buffers ≈ 22.5–23 GB) exceeds
physical RAM on a 22.6 GB machine — even fully headless. MAP_LOCKED pages cannot be swapped,
so this is a hard capacity wall, not a tunable.

Qwen3.6-35B-A3B activates only ~3B params/token. On Windows, engines run
this class of MoE on 16 GB machines by letting cold expert pages stream. The Linux runtime's
global MAP_LOCKED forecloses that: hot experts need only ~11–13 GB resident, the cold tail
could live on NVMe with router-coupled prefetching.

Request

1. An allocation mode (env var / xrt.ini / API flag) that skips MAP_LOCKED for weight bos on
   Linux, letting MoE expert pages fault/stream — analogous to how the Windows build pages on
   16 GB machines.
2. Consider supporting community pruned variants (e.g. REAP-pruned Qwen3.6-35B at 26–27B):
   pruned expert count needs recompiled graphs, but the reduced footprint (~15 GB) + reduced
   active params (~2.1B/token) would fit and run well on 22–24 GB Linux machines.

Environment: Lemonade 11.8.1 + flm:npu v1.0.2, Ubuntu 24.04 base,
kernel 7.0.0-30-generic, Ryzen AI 9 H365 (Krackan Point, aie2p 6x8).