[MLAS] Fix fp16 overflow/NaN bugs in ARM64 NEON Gelu/Erf and add extreme-value test coverage

[kjg0724 (kjg0724)]

Summary

No test in the repository directly exercises MlasNeonGeluFP16Kernel or MlasNeonErfFP16Kernel (test_fp16_activation.cpp covers Tanh/Logistic/Clip but not Gelu/Erf). Adding coverage surfaced two lane-position-dependent bugs in the existing kernels, fixed here alongside the new tests.

Gelu overflow for large positive inputs (gelu_neon_fp16.cpp)

The vector path computed 0.5 * (x * (1 + t)), multiplying x by (1 + t) in fp16 before scaling by 0.5. For x >= 32768 this intermediate product overflows the fp16 range and rounds to +Inf, even though the true result is finite and close to x (since |0.5*(1+t)| never exceeds 1 for either the erf or tanh GELU approximation). The scalar tail computes the equivalent expression in fp32 with a single rounding and does not overflow, so the same logical input produces +Inf or the correct finite value depending only on which SIMD lane processes it.

Reassociating to x * (0.5 * (1 + t)) removes the overflow structurally: |0.5*(1+t)| is bounded by 1, so the product with a finite x cannot exceed x in magnitude.

Verified against all 65536 fp16 inputs on Neoverse-N1 (Oracle Cloud A1, dotprod, no SVE):

Input range	Old (buggy)	New	fp32 reference
[32768, 65504] (1024 values)	+Inf	correct finite value	finite
all other finite inputs	unaffected	unaffected	—

Overall accuracy statistics (max abs/rel error) are identical before and after outside the overflow range.

Erf NaN propagation (erf_neon_fp16.cpp)

The vector path compares a NaN input against 0 and against the |x| < 4 saturation threshold; both comparisons evaluate to false for NaN under IEEE 754 semantics, so the kernel selects the positive-saturation branch and returns +1.0 instead of propagating NaN. The scalar tail and the fp32 MlasComputeErf reference both propagate NaN as expected, so only the vector lanes disagree, and only for NaN inputs.

Fix: compare each lane against itself (vceqq_f16(x, x), false only for NaN) and select the original NaN bits on mismatch, via a new MlasCompareEqualFloat16 helper alongside the existing MlasCompareLessThanFloat16.

Verified against all 65536 fp16 inputs: only the 2046 NaN encodings change output (from +1.0 to NaN); no other values are affected.

New tests (test_gelu_erf_neon_fp16.cpp)

Following the test_hgemm_neon.cpp pattern (MlasTestBase, FloatEqual with explicit Inf/NaN handling, random + fixed-value + single-lane extreme-value cases): random-input coverage plus extreme-value cases for the erf saturation boundary (|x| == 4, both signs), the fp16 overflow boundary around x == 32768, subnormal/min-normal magnitudes, and Inf/NaN, for both GELU approximations (erf and tanh).

The single-lane cases place one extreme value at every offset within an 8-wide vector chunk (0-7), the first scalar-tail lane (8), and the last lane of the buffer, so a lane-dependent regression cannot hide behind an otherwise-benign input array — this is exactly how both kernel bugs above were found.

RefGeluTanh's reference round-trips tanh(clamp(inner, -5, 5)) through fp16 before combining with x, matching what the real kernel does on both the vector and scalar paths (MlasComputeTanh<MLAS_FP16>); without this, extreme inputs like -65504 and -Inf diverge from the kernel because tanh(±5) ≈ ∓0.9999092 in float32 is not exactly ∓1.0, but rounds to exactly ∓1.0 in fp16.

SVE (not touched in this PR)

sve/elementwise_sve_fp16.cpp has the same two bug patterns (svmul(OneHalf, svmul(x, ...)) overflow at elementwise_sve_fp16.cpp:437, and the same NaN-saturates-to-+1 issue in the SVE Erf implementation). This machine has no SVE hardware to verify against, and the production SVE kernel is linked from a pre-generated assembly file (aarch64/elementwise_sve_asm.S) rather than built from the .cpp directly, so a real fix needs gen_sve_asm.py regeneration. Flagging here for mirounga rather than guessing at a fix blind.

Testing

Neoverse-N1 (Oracle Cloud A1, Ubuntu):

• *NeonErfFP16* / *NeonGeluFP16*: new tests pass
• Full onnxruntime_mlas_test suite: 37959/37959 non-skipped tests passed, no regressions

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

The added test has critical non-ARM compilation and Windows ARM64 linkage issues.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes ARM64 NEON FP16 GELU overflow and Erf NaN propagation, adding boundary and lane-position tests.

Changes:

• Prevents GELU intermediate FP16 overflow.
• Preserves NaN values in vectorized Erf.
• Adds comprehensive extreme-value coverage.

File summaries

File	Summary	Review Notes
onnxruntime/test/mlas/unittest/test_gelu_erf_neon_fp16.cpp	Adds GELU/Erf tests.	Critical: headers break non-ARM builds; Windows ARM64 linkage is unresolved (2 votes each).
onnxruntime/core/mlas/lib/gelu_neon_fp16.cpp	Prevents intermediate GELU overflow.	No findings.
onnxruntime/core/mlas/lib/fp16_common.h	Adds FP16 equality comparison helper.	No findings.
onnxruntime/core/mlas/lib/erf_neon_fp16.cpp	Restores NaN propagation.	No findings.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approval recommended

The fixes are narrowly scoped, logically correct, and covered by targeted boundary and lane-position tests.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Balanced