Skip to content

Fix: close profiler drain and mapping races - #2145

Open
zhusy54 wants to merge 1 commit into
hw-native-sys:mainfrom
zhusy54:fix/profiler-quiesce-race
Open

Fix: close profiler drain and mapping races#2145
zhusy54 wants to merge 1 commit into
hw-native-sys:mainfrom
zhusy54:fix/profiler-quiesce-race

Conversation

@zhusy54

@zhusy54 zhusy54 commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Problem

Runtime replenishment can append to dev_to_host_ and block_ranges_
while drain shards call resolve_host_ptr(). The writer held
mapping_mutex_, but the reader relied on an invalid immutability assumption,
so container rehash or reallocation could race with lookup.

The drain loop also loaded the requested quiesce epoch after scanning its
queues. A sweep that began before the request could therefore acknowledge the
new epoch and let export begin before records published during that sweep were
drained.

Follow-up to #2127.

Changes

  • Protect exact and block-range resolution with a shared mapping lock while
    keeping runtime mutation under the exclusive lock.
  • Observe the requested quiesce epoch before each full drain sweep and only
    acknowledge that epoch after the sweep reports no work.
  • Add concurrent exact/range mapping coverage and a gated regression test for
    the stale-sweep acknowledgement ordering.
  • Make the test publisher follow device entry-before-tail ordering, document
    the mapping ownership contract, and apply the repository's #pragma once
    convention to the touched headers.

Reviewer guide

The main correctness boundaries are the lifetime of the shared mapping lock in
resolve_host_ptr() and the placement of the acquire-load before the complete
queue sweep in mgmt_drain_loop().

Testing

  • Full no-hardware C++ unit suite: 135/135 passed
  • test_profiler_base: 10/10 passed
  • New quiesce regression: old implementation failed; fixed implementation
    passed 20 consecutive runs
  • Concurrent mapping test passed under ThreadSanitizer
  • Concurrent mapping test passed 50 consecutive runs

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: acd2a8ad-5668-425b-9bf3-9fabfc8ed04b

📥 Commits

Reviewing files that changed from the base of the PR and between 5d046b1 and 843af9e.

📒 Files selected for processing (4)
  • src/common/platform/include/host/buffer_pool_manager.h
  • src/common/platform/include/host/profiler_base.h
  • tests/ut/cpp/common/test_buffer_pool_manager.cpp
  • tests/ut/cpp/common/test_profiler_base.cpp

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The changes add reader-writer locking to host pointer mappings and align profiler quiescence acknowledgements with queue sweeps. Unit tests cover concurrent mapping resolution and explicit buffer publication ordering.

Changes

Host mapping synchronization

Layer / File(s) Summary
Mapping locking and concurrent resolution
src/common/platform/include/host/buffer_pool_manager.h, tests/ut/cpp/common/test_buffer_pool_manager.cpp
The buffer pool manager uses exclusive locks for mapping updates and shared locks for pointer lookups. A concurrent test resolves exact and inner-range pointers while another thread registers 2048 blocks.

Profiler drain epoch ordering

Layer / File(s) Summary
Drain epoch ordering and queue publication
src/common/platform/include/host/profiler_base.h, tests/ut/cpp/common/test_profiler_base.cpp
The drain worker samples the quiescence epoch before sweeping queues. Test helpers now register buffers separately and issue a write barrier before advancing queue tails.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 843af

This change synchronizes concurrent host-mapping access and makes profiler drain acknowledgements correspond to post-epoch sweeps, with updated coverage for concurrent resolution and publication ordering. No concrete current-head merge-blocking risk remains.

Poem

A rabbit guards the mapping gate
Shared locks let readers congregate
The drain marks time before its sweep
Queue tails wake from orderly sleep
Blocks resolve through thicket and stream
Safe little pointers hop in a dream

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.16% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 19 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and clearly summarizes the two main fixes: profiler drain races and mapping races.
Description check ✅ Passed The description directly explains the mapping and profiler drain races, the implemented fixes, test coverage, and reviewer focus.
  • Fix all pre-merge checks with AI

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Synchronize dynamic mapping updates with drain-time pointer resolution
using shared reader and exclusive writer locking. Cover exact and range
lookups under concurrent block registration.

Observe each quiesce epoch before its drain sweep so an acknowledgement
cannot use stale work. Add a gated regression test that reproduces the
stale-sweep acknowledgement ordering.

Keep the profiling concurrency contract aligned with the implementation
and use the repository's pragma-once convention in touched headers.
@zhusy54
zhusy54 force-pushed the fix/profiler-quiesce-race branch from 843af9e to b73cd64 Compare September 7, 2026 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant