Update HRX binaries to FLM 1.0.5 - #714
Open
Abhishek-Varma wants to merge 11 commits into
Open
Conversation
Rebuild the manylinux_2_28 Boost-free HRX .so set from IRON flm_use_hrx_flag, including gemma4-12b and hy-mt2, and pin the public package to flm-hrx-amdxdna-v2026.09.02. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
IRON engines keep BOs alive with shared_ptr; unique_ptr copies left data_ null across the .so boundary and SIGSEGV'd in sample_greedy. Leave the XRT unique_ptr path unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
hy-mt2 alone set log_raw_output=false, so its streamed answers carried no "Model RAW Output:" anchor. Qualify and the numerical-match harness parse that marker, so every hy-mt2 single-prompt/multi-turn/1k-bench read as 0 responses despite correct generation. Restore the default to match every other model. Co-authored-by: Cursor <cursoragent@cursor.com>
Restore the hunyuan translator's quiet interactive default (it streams one short line per turn, so echoing the raw dump only doubles the log) and add AutoModel::env_forces_raw_output() so diagnostics can opt back in. The qualification harness and numerical-match anchor per-turn parsing on the "Model RAW Output:" marker, so they set FLM_LOG_RAW_OUTPUT=1 to force it on without changing the interactive UX default. Co-authored-by: Cursor <cursoragent@cursor.com>
…ase onto main Co-authored-by: Cursor <cursoragent@cursor.com>
Abhishek-Varma
force-pushed
the
update_hrx_105
branch
from
September 9, 2026 06:43
2c93762 to
1bb7f28
Compare
Rebuilt the Linux HRX engine .so from FastFlowLM_IRON
fix/hrx-prefill-use-flm-rt @ 8d05a97 ("use flm_rt:: instead of xrt:: in
prefill dequant contexts") in the TheRock manylinux_2_28 container
(g++-13 13.3.1, HRX public package v2026.09.02, FLM_USE_HRX=1).
- Boost-free; max GLIBC 2.22 / GLIBCXX 3.4.22 (<= manylinux_2_28); HRX
engines link libhrx.so.0 via $ORIGIN; no XRT/aiebu linkage.
- 22 of 24 engines changed; libmha.so and liblm_head.so are byte-identical
(pure CPU kernels unaffected by the prefill path).
- libdequant_new.so is not produced by build.sh on this branch (no source
dir) and is left untouched.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rebuild 24 HRX dlls from IRON fix/hrx-prefill-use-flm-rt against the effecac public package. Import libs were byte-identical and left unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerated all 24 Linux HRX engine .so from FastFlowLM_IRON fix/hrx-prefill-use-flm-rt @ 6f22f9f (which bumps the public HRX package to flm-hrx-amdxdna-v2026.09.09 / effecac command-cache HAL). Built in the TheRock manylinux_2_28 container: engines link libhrx.so.0 via $ORIGIN, Boost-free, no XRT/aiebu linkage, GLIBC <= 2.28. libdequant_new.so is not produced by build.sh on this branch and is left untouched. Co-authored-by: Cursor <cursoragent@cursor.com>
…nload Sync the HRX context-exhaustion fix from FastFlowLM_IRON and refresh the prebuilt Windows HRX engine binaries. Every HRX executable is created with HRX_AMDXDNA_CONTEXT_MODE_CREATE and is backed by its own NPU hardware context. The executable cache was a process-wide map that was never pruned, so a `flm serve` sweep across many models leaked every model's executables and eventually exhausted the driver's context pool (D3DKMTCreateContextVirtual failed with 0xc01e0009 on the Windows 314 driver; BO allocation errno 11 / EAGAIN on the Linux amdxdna path). Reference-count cached executables: each npu_app that dispatches a given (xclbin, ctrl_seq) holds one reference and releases it in its destructor, so an executable (and its hardware context) is freed once its owning model is unloaded. npu_app is made move-only so references are never double-released. - src/include/hrx_cpp/hrx_cpp.hpp: refcounted executable cache - src/include/npu_utils/npu_utils_hrx.hpp: npu_app owns/releases its refs - src/lib/hrx/*.dll,*.lib: rebuilt Windows engines against pinned HRX v2026.09.09
…es fix
Rebuild the Linux HRX engine .so from FastFlowLM_IRON fix/hrx-prefill-use-flm-rt
@ e450f646 ("reference-count executables to free NPU contexts on model unload"),
against the pinned public HRX package flm-hrx-amdxdna-v2026.09.09 (effecac).
Built in the TheRock manylinux_2_28 container: engines link libhrx.so.0 via
$ORIGIN, Boost-free, no XRT/aiebu linkage, GLIBC <= 2.28. 21/24 engines changed;
libdequant.so, libgemm.so and libmha.so are pure compute kernels unaffected by
the refcount headers and stay byte-identical. libdequant_new.so is not produced
by build.sh on this branch and is left untouched. Matches the src/include
hrx_cpp/npu_utils refcount changes already landed for the Windows engines at
7c99700.
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refresh the bundled HRX (amdxdna) engines to
flm-hrx-amdxdna-v2026.09.02andfix two correctness issues surfaced by downstream qualification: an engine/app
buffer-ownership ABI mismatch, and the hy-mt2 (hunyuan) translator silently
dropping its per-turn
Model RAW Output:marker.Changes
shared_ptrownership in the app buffer path so the ABI matches the IRON-builtHRX engines (the XRT
unique_ptrpath is unchanged).FLM_LOG_RAW_OUTPUT- the translatorstays quiet by default (one short line per turn, no doubled log), but diagnostics
can opt back in via
FLM_LOG_RAW_OUTPUT=1throughAutoModel::env_forces_raw_output(). This keeps author intent while lettingqualification / numerical-match anchor per-turn parsing on the marker.