Skip to content

platform: posix: run fuzz teardown in thread context - #11168

Open
tmleman wants to merge 1 commit into
thesofproject:mainfrom
tmleman:topic/upstream/pr/posix/ipc_fuzzer/fix/native_sim_timer
Open

platform: posix: run fuzz teardown in thread context#11168
tmleman wants to merge 1 commit into
thesofproject:mainfrom
tmleman:topic/upstream/pr/posix/ipc_fuzzer/fix/native_sim_timer

Conversation

@tmleman

@tmleman tmleman commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

The IPC fuzzer aborts partway through a run with:

programming error: nsif_cpu0_irq_raised_from_sw called from a HW
model thread

The between-testcase topology teardown (posix_ipc_teardown(), added in commit d53a762 "platform: posix: tear down IPC topology between fuzz testcases") was invoked from posix_fuzz_case_begin(), which executes on the libFuzzer driver thread. On native_sim that thread is a "HW model" context: the simulated CPU is halted (posix_is_cpu_running() == false) whenever execution is outside nsi_exec_for(). The teardown frees pipelines and cancels scheduler tasks, taking Zephyr spinlocks (and the LL scheduler's k_mutex). Releasing a spinlock reaches arch_irq_unlock() -> hw_irq_ctrl_change_lock(); if a HW interrupt is pending there it is vectored synchronously via nsif_cpu0_irq_raised_from_sw(), which aborts because the CPU is not running.

gdb confirms the pending interrupt is the system tick (irq_status == 0x1, IRQ 0 = TIMER_TICK_IRQ), delivered from pipeline_posn_unlock() -> k_spin_unlock() inside pipeline_free().

Running the teardown on the driver thread was always unsafe, but only became reproducible after a Zephyr update that converted the native_sim system timer from a periodic tick to a one-shot, fully tickless model (drivers/timer/native_sim_timer.c "use the generic timer core", plus the native_simulator hwtimer_set_tick_one_shot() addition). hwtimer_enable() previously armed a periodic tick on a fixed grid and no tick happened to be pending at the between-testcase boundary; the one-shot core now arms the tick at the exact next timeout deadline, which lands at/after an nsi_exec_for() quantum boundary. nsi_exec_for() stops on its time budget, so it returns after the tick fires but before the CPU services it, leaving TIMER_TICK_IRQ pending exactly when the teardown runs.

Fix this by running the teardown where SOF frees pipelines during normal operation: the EDF workqueue thread. posix_fuzz_case_begin() now only sets a flag; ipc_platform_do_cmd() consumes it and runs posix_ipc_teardown() before this testcase's first command. There the CPU is running (a pending tick is delivered legitimately) and blocking primitives such as k_mutex are valid.

Clearing the pending interrupt on the driver thread was rejected as an alternative: dropping the tick leaves the one-shot timer with no armed deadline (next_timer_time == NSI_NEVER), so the simulator exits via nsi_exit() and libFuzzer reports "fuzz target exited".

Validated with the IPC4 seed corpus (342142 runs) and the IPC3 seed corpus (111135 runs); both complete cleanly with no crash artifacts.

Copilot AI lite review requested due to automatic review settings September 4, 2026 10:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped to libFuzzer builds, matches the described root cause, and moves teardown into the intended thread context without altering IPC command semantics.

Pull request overview

This PR fixes a native_sim/libFuzzer crash by ensuring the between-testcase IPC topology teardown runs in a proper Zephyr thread context (with the simulated CPU running), instead of on the libFuzzer driver (“HW model”) thread where releasing spinlocks/mutexes can synchronously vector a pending tick IRQ and abort the simulator.

Changes:

  • Defer posix_ipc_teardown() by setting a pending flag in posix_fuzz_case_begin() instead of running teardown immediately.
  • Consume that pending flag in ipc_platform_do_cmd() (EDF workqueue thread) and run posix_ipc_teardown() before the first command of each testcase (fuzzer builds only).
File summaries
File Description
src/platform/posix/ipc.c Defers fuzz testcase teardown to the IPC command execution thread to avoid native_sim aborts from driver-thread teardown.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

The IPC fuzzer aborts partway through a run with:

  programming error: nsif_cpu0_irq_raised_from_sw called from a HW
  model thread

The between-testcase topology teardown (posix_ipc_teardown(), added in
commit d53a762 "platform: posix: tear down IPC topology between fuzz
testcases") was invoked from posix_fuzz_case_begin(), which executes on
the libFuzzer driver thread. On native_sim that thread is a "HW model"
context: the simulated CPU is halted (posix_is_cpu_running() == false)
whenever execution is outside nsi_exec_for(). The teardown frees
pipelines and cancels scheduler tasks, taking Zephyr spinlocks (and the
LL scheduler's k_mutex). Releasing a spinlock reaches arch_irq_unlock()
-> hw_irq_ctrl_change_lock(); if a HW interrupt is pending there it is
vectored synchronously via nsif_cpu0_irq_raised_from_sw(), which aborts
because the CPU is not running.

gdb confirms the pending interrupt is the system tick (irq_status == 0x1,
IRQ 0 = TIMER_TICK_IRQ), delivered from pipeline_posn_unlock() ->
k_spin_unlock() inside pipeline_free().

Running the teardown on the driver thread was always unsafe, but only
became reproducible after a Zephyr update that converted the native_sim
system timer from a periodic tick to a one-shot, fully tickless model
(drivers/timer/native_sim_timer.c "use the generic timer core", plus the
native_simulator hwtimer_set_tick_one_shot() addition). hwtimer_enable()
previously armed a periodic tick on a fixed grid and no tick happened to
be pending at the between-testcase boundary; the one-shot core now arms
the tick at the exact next timeout deadline, which lands at/after an
nsi_exec_for() quantum boundary. nsi_exec_for() stops on its time budget,
so it returns after the tick fires but before the CPU services it,
leaving TIMER_TICK_IRQ pending exactly when the teardown runs.

Fix this by running the teardown where SOF frees pipelines during normal
operation: the EDF workqueue thread. posix_fuzz_case_begin() now only
sets a flag; ipc_platform_do_cmd() consumes it and runs
posix_ipc_teardown() before this testcase's first command. There the CPU
is running (a pending tick is delivered legitimately) and blocking
primitives such as k_mutex are valid.

Clearing the pending interrupt on the driver thread was rejected as an
alternative: dropping the tick leaves the one-shot timer with no armed
deadline (next_timer_time == NSI_NEVER), so the simulator exits via
nsi_exit() and libFuzzer reports "fuzz target exited".

Validated with the IPC4 seed corpus (342142 runs) and the IPC3 seed
corpus (111135 runs); both complete cleanly with no crash artifacts.

Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
@intel-sofci

Copy link
Copy Markdown

PR 11168: test results

Run date: 2026-09-04 12:54 UTC

Tested commit: 22077a5f49e79071cddcb4d1ee5cfe49a7c6bc0e

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG][CI] fuzzer tests started failing after Sep 1st update of Zephyr

3 participants