From 769fe6816737509047d3a9efd8993fa0ab934ef3 Mon Sep 17 00:00:00 2001 From: Simon Birkholz Date: Wed, 19 Aug 2026 19:41:38 +0200 Subject: [PATCH 1/4] poc: real awaitables, and the GIL trap on the way there Not a merge candidate. cuvis.python exposes async/await, but nothing underneath waits on an event: Async.__await__ polls at 10 ms, Worker.get_next_result_async at 100 ms, register_worker_callback spins at 1 ms. The SDK is not the limit; it has blocking waits with timeouts. The limit was that no wrapped function released the GIL, so a blocking wait froze the interpreter for its whole timeout. Async.py grows one _wait_off_the_loop helper and both __await__ methods collapse into it. Worker runs the SDK's blocking wait in an executor, and the 1 ms spin is gone. A private _wait_for_result reads the status directly, because SDKException logs unconditionally and an idle wait would otherwise write a traceback every second. Paired runs: awaited worker results 108.6 ms to 28.0 ms, 3.20x the blocking floor down to 1.21x, idle callback 0.8 % of a core to 0.0 %. POC_REAL_AWAITABLE.md carries the write-up, including why %module(threads="1") is the wrong instrument, and what the C SDK would need for this to be finished without a thread per outstanding wait. --- POC_REAL_AWAITABLE.md | 176 ++++++++++++++++++++++++++++++ cuvis/Async.py | 47 ++++---- cuvis/Worker.py | 73 ++++++++----- scripts/bench_worker.py | 90 +++++++++++++++ scripts/seh_under_released_gil.py | 50 +++++++++ 5 files changed, 383 insertions(+), 53 deletions(-) create mode 100644 POC_REAL_AWAITABLE.md create mode 100644 scripts/bench_worker.py create mode 100644 scripts/seh_under_released_gil.py diff --git a/POC_REAL_AWAITABLE.md b/POC_REAL_AWAITABLE.md new file mode 100644 index 0000000..30c1766 --- /dev/null +++ b/POC_REAL_AWAITABLE.md @@ -0,0 +1,176 @@ +# POC: real awaitables for the cuvis Python SDK (ALL-1671) + +Not a merge candidate. +This branch exists to answer one question: can `await` on a cuvis object resume when the SDK is done, instead of when the next poll happens to come round? + +It can, and the change needed is smaller than expected. +It also uncovered a crash that the obvious implementation walks straight into, which is the main reason this is written down rather than merged. + +Companion branch: `poc/gil-release` in `cuvis.swig`. + +## The problem + +`cuvis.python` already exposes `async`/`await`, so ALL-1671 looked done. +It is not. +Every path underneath is a sleep loop: + +| Site | Interval | +| --- | --- | +| `cuvis/Async.py` `AsyncMesu.__await__` | 10 ms | +| `cuvis/Async.py` `Async.__await__` | 10 ms | +| `cuvis/Worker.py` `Worker.get_next_result_async` | 100 ms | +| `cuvis/Worker.py` `Worker.register_worker_callback` | 1 ms | + +None of these is an awaitable in any meaningful sense. +They are timers that check a flag, and the interval sets a floor under the latency that no amount of Python-side work can lift. + +The SDK is not the limitation. +It already offers blocking waits with timeouts, and documents them: + +- `cuvis_async_capture_get(handle, timeout_ms, out)` - `cuvis.h:1912`, "Give 0 to wait for ever" +- `cuvis_async_call_get(handle, timeout_ms)` - `cuvis.h:1839` +- `cuvis_worker_get_next_result(worker, mesu, view, timeout_ms)` - `cuvis.h:3066`, "-1 to wait indefinitely" + +## Why the wrapper could not use them + +`cuvis.swig/src/cuvis_il.i` carried no threading directive and `cuvis.pyil/CMakeLists.txt` passes SWIG only `-doxygen`. +**No wrapped function released the GIL.** +A blocking `cuvis_async_capture_get(h, 5000)` therefore froze the entire interpreter for its full timeout, which makes it unusable from an event loop. +Polling was not a design choice; it was the only thing available. + +## The trap: `%module(threads="1")` crashes + +SWIG has no per-function opt in for GIL release. +`%threadallow` is defined in `swigwin/Lib/python/pyuserdir.swg` as `%feature("nothreadallow","0")` - there is only a module-wide switch and a per-function opt *out*. +So the obvious change is `%module(threads="1") cuvis_il`, and it generates exactly what you want: + +```cpp +CUVIS_GUARD({ + SWIG_PYTHON_THREAD_BEGIN_ALLOW; + result = cuvis_async_capture_get(arg1,arg2,arg3); + SWIG_PYTHON_THREAD_END_ALLOW; + }) +``` + +It also segfaults. + +`SWIG_PYTHON_THREAD_BEGIN_ALLOW` is an RAII object (`SWIG_Python_Thread_Allow`) whose destructor reacquires the GIL. +It sits *inside* `CUVIS_GUARD`, the MSVC structured-exception guard that turns a delay-load failure into a C++ throw so a missing symbol becomes a Python exception rather than a dead process. +The extension is built `/EHsc` (`ExceptionHandling=Sync`), under which **MSVC does not run C++ destructors while unwinding an SEH exception**. + +So when the delay-load stub fires, the guard never reacquires the GIL, and `SWIG_exception` in the `%exception` handler calls into the Python C API without it. + +Measured, not reasoned about. +Built against the staged SDK, run against the installed one, which does not export the eight CUDA entry points, calling one on the unshadowed extension: + +| Build | Result | +| --- | --- | +| `threads="1"` | `0xc06d007f`, then access violation, exit 139 | +| no threading (control) | `RuntimeError`, interpreter healthy, exit 0 | +| this branch's fix | `RuntimeError`, interpreter healthy, exit 0 | + +This matters because that path is not exotic: it is the exact scenario `cuvis.binding` was built to report, an installed SDK older than the binding. + +## The fix + +Drop `threads="1"` and manage the GIL explicitly in the `%exception` block that is already there, so correctness does not depend on unwinding running a destructor: + +```c +%exception { + PyThreadState *_cuvis_thread = PyEval_SaveThread(); + try { CUVIS_GUARD($action) } + catch (std::invalid_argument const& e) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_ValueError, e.what()); } + catch (std::exception const& e) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_RuntimeError, e.what()); } + catch (...) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_UnknownError, "unknown C++ exception from cuvis"); } + CUVIS_REGAIN_GIL; +} +``` + +`CUVIS_REGAIN_GIL` is idempotent, so the same statement serves the catch blocks and the success path. +This covers 325 wrapped call sites. + +There are no SWIG directors and no Python callables handed to the SDK, so nothing re-enters the interpreter from a thread that has given up the GIL. + +## What changed in cuvis.python + +`cuvis/Async.py` gains one helper and the two `__await__` methods collapse into it: + +```python +async def _wait_off_the_loop(blocking_get): + loop = a.get_running_loop() + return await loop.run_in_executor(None, blocking_get, _WAIT_FOREVER) +``` + +`cuvis/Worker.py`: + +- `get_next_result_async` runs the SDK's blocking wait in the executor instead of polling. It now raises `SDKException` on timeout rather than returning a `WorkerResult` built from handles the SDK never filled in, which is what the old version did. +- `register_worker_callback` awaits results in a loop; the 1 ms spin is gone. +- A private `_wait_for_result` reads the status directly and returns `None` on a timeout. `SDKException.__init__` calls `logging.exception` unconditionally, so a wait that expects to come back empty cannot go through the public method without writing a traceback to the log every second. + +## Measurements + +Simulated camera, one frame ingested per round, 12 rounds, same machine, runs taken back to back. +`scripts/bench_worker.py` in this branch reproduces it. + +| | before (develop) | after (this branch) | +| --- | --- | --- | +| worker result, blocking | 33.9 ms | 23.1 ms | +| worker result, awaited | **108.6 ms** | **28.0 ms** | +| awaited / blocking | **3.20x** | **1.21x** | +| registered callback, idle | 0.8 % of a core | 0.0 % of a core | + +The ratio is the number to read: absolute timings drift with machine load, but the awaited path went from three times the cost of the blocking floor to within a fifth of it. +The 108 ms figure is not a coincidence - it is the 100 ms poll interval, and it was there whether or not the result was ready. + +Full suite: 123 passed against the rebuilt extension on Python 3.12. + +## What the SDK would need for this to be finished + +The executor approach is a real improvement but it is not free, and both costs are the SDK's to remove: + +1. **One pooled thread per outstanding wait.** Fine for a handful of captures, not for hundreds. +2. **A wait cannot be cancelled.** A thread blocked in C is not interruptible, so `reset_worker_callback` cannot take effect until the current window expires. That is why `_RESULT_WAIT_MS` is 1000 rather than "wait for ever", and it is a workaround, not a design. + +Both go away if the SDK hands out something an event loop can watch directly. +The proposal, for a cuvis.c ticket: + +```c +/** @brief A waitable OS handle that is signalled when the async capture completes. + * + * On Windows a Win32 event HANDLE, suitable for IocpProactor.wait_for_handle. + * On Linux an eventfd, suitable for loop.add_reader. + * The handle is owned by the SDK and stays valid until the async result is freed. + */ +SDK_CAPI CUVIS_STATUS SDK_CCALL cuvis_async_capture_get_event( + CUVIS_ASYNC_CAPTURE_RESULT i_asyncResult, CUVIS_EVENT_HANDLE* o_pEvent); +``` + +with the same shape for `cuvis_async_call_*` and `cuvis_worker_*`. +With that, the awaitables need no threads at all and cancellation is ordinary asyncio. + +A completion callback routed through `loop.call_soon_threadsafe` would also work. +The SDK has callback plumbing already (`log_callback` at `cuvis.h:1595`, `external_event_callback` at `:1637`), but neither is tied to an async result. + +A second, smaller ask: `cuvis_worker_get_next_result` reports "nothing arrived yet" through the same error channel as real failures, which is why `_wait_for_result` has to exist. +A distinct status for an expired timeout would remove it. + +## Reproducing + +```powershell +# 1. the SWIG side +cd C:\dev\cuvis_sdk\cuvis.pyil\cuvis.swig +git checkout poc/gil-release +powershell -ExecutionPolicy Bypass -File C:\dev\cuvis_sdk\cuvis.pyil\build_pyil.ps1 + +# 2. the measurements +$env:PYTHONPATH = "C:\dev\cuvis_sdk\cuvis.pyil;C:\dev\cuvis_sdk\cuvis.python-await" +& C:\dev\cuvis_sdk\.venv-pyil312\Scripts\python.exe scripts\bench_worker.py + +# 3. the crash test: build against the staged SDK, run against the installed one +Remove-Item -Recurse -Force C:\dev\cuvis_sdk\cuvis.pyil\build +powershell -ExecutionPolicy Bypass -File C:\dev\cuvis_sdk\cuvis.pyil\build_pyil.ps1 ` + -CuvisRoot "C:\dev\cuvis_sdk\.cuvis-new" -CuvisLib "C:\dev\cuvis_sdk\.cuvis-new\sdk\cuvis_c\cuvis.lib" +& C:\dev\cuvis_sdk\.venv-pyil312\Scripts\python.exe scripts\seh_under_released_gil.py +``` + +The build must be launched from a directory holding no stray `cuvis.dll`: the configure-time probe finds one through the Windows current-directory rule, before PATH, and bakes the wrong build hash into the binding. diff --git a/cuvis/Async.py b/cuvis/Async.py index e9ad752..95ca8e7 100644 --- a/cuvis/Async.py +++ b/cuvis/Async.py @@ -17,6 +17,25 @@ def _to_ms(value: int | timedelta) -> int: raise SDKException("Unknown type for converting to ms") +# The SDK's own waits take a timeout in ms and treat 0 as "wait for ever". +_WAIT_FOREVER = 0 + + +async def _wait_off_the_loop(blocking_get): + """Run one of the SDK's blocking waits in a worker thread. + + The wrappers release the GIL for the duration of the call, so the thread parks in the + SDK and the event loop keeps running. This is what makes the awaitables below real: + they resume when the SDK is done, not when the next poll happens to come round. + + The cost is one pooled thread per outstanding wait, and a wait that never completes + cannot be cancelled, because a thread blocked in C is not interruptible. Both go away + only if the SDK hands out something the event loop can watch directly. + """ + loop = a.get_running_loop() + return await loop.run_in_executor(None, blocking_get, _WAIT_FOREVER) + + class AsyncMesu(object): def __init__(self, handle): self._handle = handle @@ -46,17 +65,8 @@ def get( def __await__(self) -> Measurement | None: async def _wait_for_return(): - _status_ptr = cuvis_il.new_p_cuvis_status_t() - while True: - if cuvis_il.status_ok != cuvis_il.cuvis_async_capture_status( - self._handle, _status_ptr - ): - raise SDKException() - status = cuvis_il.p_cuvis_status_t_value(_status_ptr) - if status == cuvis_il.status_ok: - return self.get(0)[0] - else: - await a.sleep(10.0 / 1000) + mesu, _ = await _wait_off_the_loop(self.get) + return mesu return _wait_for_return().__await__() @@ -101,20 +111,7 @@ def get(self, timeout_ms: int | timedelta) -> AsyncResult: # Python Magic Methods def __await__(self) -> AsyncResult: - async def _wait_for_return(): - _status_ptr = cuvis_il.new_p_cuvis_status_t() - while True: - if cuvis_il.status_ok != cuvis_il.cuvis_async_call_status( - self._handle, _status_ptr - ): - raise SDKException() - status = cuvis_il.p_cuvis_status_t_value(_status_ptr) - if status == cuvis_il.status_ok: - return self.get(0) - else: - await a.sleep(10.0 / 1000) - - return _wait_for_return().__await__() + return _wait_off_the_loop(self.get).__await__() def __del__(self): if self._handle is None: diff --git a/cuvis/Worker.py b/cuvis/Worker.py index 12e982e..05b0068 100644 --- a/cuvis/Worker.py +++ b/cuvis/Worker.py @@ -14,6 +14,11 @@ from dataclasses import dataclass from collections.abc import Callable, Awaitable +# How long each wait for a worker result blocks before looping round. Long enough that a +# quiet worker costs nothing, short enough that reset_worker_callback is not held up by +# more than one window. +_RESULT_WAIT_MS = 1000 + @dataclass class WorkerResult: @@ -152,29 +157,40 @@ def get_next_result(self, timeout) -> WorkerResult: view = None return WorkerResult(mesu, view) - async def get_next_result_async(self, timeout: int) -> WorkerResult: - poll_intervall = 100 + def _wait_for_result(self, timeout_ms: int): + """The SDK's own blocking wait, returning None instead of raising on a timeout. + + `get_next_result` reports "nothing arrived" as an error, and `SDKException` logs + every instance it is constructed with. A wait that expects to come back empty + cannot go through it without filling the log with tracebacks, so the status is + read here directly. + """ ptr_mesu = cuvis_il.new_p_int() ptr_view = cuvis_il.new_p_int() + if cuvis_il.status_ok != cuvis_il.cuvis_worker_get_next_result( + self._handle, ptr_mesu, ptr_view, timeout_ms + ): + return None + view = ( + Viewer._create_view_data(None, cuvis_il.p_int_value(ptr_view)) + if self._viewer_set + else None + ) + return WorkerResult(Measurement(cuvis_il.p_int_value(ptr_mesu)), view) - tries = 0 - while tries * poll_intervall < timeout: - if self.has_next_result(): - await a.sleep(0) - if cuvis_il.status_ok != cuvis_il.cuvis_worker_get_next_result( - self._handle, ptr_mesu, ptr_view, 100 - ): - raise SDKException() - break - else: - tries += 1 - await a.sleep(poll_intervall / 1000) - mesu = Measurement(cuvis_il.p_int_value(ptr_mesu)) - if self._viewer_set: - view = Viewer._create_view_data(None, cuvis_il.p_int_value(ptr_view)) - else: - view = None - return WorkerResult(mesu, view) + async def get_next_result_async(self, timeout: int) -> WorkerResult: + """Wait for the next result without polling for it. + + The SDK's own wait is blocking and releases the GIL, so it runs in a worker + thread while the event loop carries on. Unlike the polling version this raises + `SDKException` when the timeout passes rather than returning a result built from + handles the SDK never filled in. + """ + loop = a.get_running_loop() + result = await loop.run_in_executor(None, self._wait_for_result, timeout) + if result is None: + raise SDKException("Worker produced no result within {} ms".format(timeout)) + return result @property @copydoc(cuvis_il.cuvis_worker_get_input_queue_limit) @@ -311,17 +327,18 @@ def register_worker_callback( self, callback: Callable[[WorkerResult], Awaitable[None]] ) -> None: self.reset_worker_callback() - poll_time = 0.001 async def _internal_worker_loop(): + loop = a.get_running_loop() while True: - if self.has_next_result(): - workerContainer = await self.get_next_result_async(1000) - a.create_task(callback(workerContainer)) - - # TODO limit number of created task objects like in the cpp wrapper - else: - await a.sleep(poll_time) + result = await loop.run_in_executor( + None, self._wait_for_result, _RESULT_WAIT_MS + ) + if result is None: + continue # nothing arrived inside the window; wait again + + a.create_task(callback(result)) + # TODO limit number of created task objects like in the cpp wrapper self._worker_poll_task = a.create_task(_internal_worker_loop()) diff --git a/scripts/bench_worker.py b/scripts/bench_worker.py new file mode 100644 index 0000000..631b5f9 --- /dev/null +++ b/scripts/bench_worker.py @@ -0,0 +1,90 @@ +"""What the polling costs on the worker path: per-result latency and idle CPU. + +The worker is the clearest case. `Worker.get_next_result_async` sleeps 100 ms between +checks, so a result that arrives just after a check waits out the rest of the interval; +`register_worker_callback` spins at 1 ms, which is pure CPU with nothing to show for it. +""" + +import asyncio +import statistics +import time + +import cuvis + +cuvis.init(".") +session = cuvis.SessionFile("tests/test_data/test_mesu.cu3s") + + +def built_worker(): + pc = cuvis.ProcessingContext(session) + pc.processing_mode = cuvis.ProcessingMode.Raw + worker = cuvis.Worker(cuvis.WorkerSettings(output_queue_size=8)) + worker.set_processing_context(pc) + worker.start_processing() + return worker + + +def report(label, samples): + print( + "{:36} median {:7.2f} ms mean {:7.2f} ms max {:7.2f} ms".format( + label, statistics.median(samples), statistics.mean(samples), max(samples) + ) + ) + + +async def result_latency(rounds=12): + """Time from ingesting one frame to the awaited result coming back.""" + worker = built_worker() + samples = [] + try: + for _ in range(rounds): + start = time.perf_counter() + worker.ingest_session_file(session, frame_selection="0") + await worker.get_next_result_async(10000) + samples.append((time.perf_counter() - start) * 1000) + finally: + worker.stop_processing() + worker.drop_all_queued() + worker.reset_worker_callback() + return samples + + +def blocking_latency(rounds=12): + """The same, straight through the SDK's own blocking wait.""" + worker = built_worker() + samples = [] + try: + for _ in range(rounds): + start = time.perf_counter() + worker.ingest_session_file(session, frame_selection="0") + worker.get_next_result(10000) + samples.append((time.perf_counter() - start) * 1000) + finally: + worker.stop_processing() + worker.drop_all_queued() + return samples + + +async def idle_cpu(seconds=8.0): + """CPU spent by an event loop with one registered callback and nothing to do.""" + worker = built_worker() + worker.register_worker_callback(lambda result: asyncio.sleep(0)) + cpu, wall = time.process_time(), time.perf_counter() + await asyncio.sleep(seconds) + used = 100.0 * (time.process_time() - cpu) / (time.perf_counter() - wall) + worker.reset_worker_callback() + worker.stop_processing() + return used + + +async def main(): + report("worker result, blocking", blocking_latency()) + report("worker result, awaited", await result_latency()) + print( + "{:36} {:.1f} % of one core".format( + "registered callback, idle", await idle_cpu() + ) + ) + + +asyncio.run(main()) diff --git a/scripts/seh_under_released_gil.py b/scripts/seh_under_released_gil.py new file mode 100644 index 0000000..a287b88 --- /dev/null +++ b/scripts/seh_under_released_gil.py @@ -0,0 +1,50 @@ +"""Does the delay-load SEH path hand back a clean exception once the GIL is released? + +Built against the staged SDK, run against the installed one, which does not export the +CUDA entry points. The extension is imported as a top-level module so the package's +__init__ never runs and never replaces the missing symbols with Python stubs: the call +reaches the real SWIG wrapper, the delay-load stub raises a Win32 SEH exception, +cuvis_seh_call turns it into a C++ throw, and %exception turns that into a Python error. + +With threads="1" the GIL is released around the call by an RAII guard. The build is +/EHsc, under which MSVC does not run C++ destructors while unwinding an SEH exception, so +the question is whether the GIL has been reacquired by the time SWIG_exception touches +the Python C API. +""" + +import ctypes +import gc +import os +import sys + +CUVIS_BIN = r"C:\Program Files\Cuvis\bin" +os.add_dll_directory(CUVIS_BIN) +for sub in ("bin", os.path.join("bin", "x64")): + cuda = os.path.join(os.environ.get("CUDA_PATH", ""), sub) + if os.path.isdir(cuda): + os.add_dll_directory(cuda) + +ctypes.WinDLL( + os.path.join(CUVIS_BIN, "cuvis.dll") +) # pin the library the stub will bind +import numpy # noqa: F401,E402 + +sys.path.insert(0, r"C:\dev\cuvis_sdk\cuvis.pyil\cuvis_il") +import _cuvis_pyil as raw # noqa: E402 + +print("raw extension:", raw.__file__) +print("shadowed? :", "no") + +name = "cuvis_cuda_mem_free" +print("calling raw {} ... a crash here means the GIL was not reacquired".format(name)) +sys.stdout.flush() +try: + getattr(raw, name)(0) +except Exception as exc: + print("clean exception:", type(exc).__name__, "|", str(exc)[:110]) +else: + print("returned without raising") + +sys.stdout.flush() +gc.collect() +print("interpreter still healthy:", sum(len(str(i)) for i in range(1000))) From 13dd2ab031f9eeb9b2e6ef0ec3463dc640dd16dd Mon Sep 17 00:00:00 2001 From: Simon Birkholz Date: Wed, 19 Aug 2026 20:01:31 +0200 Subject: [PATCH 2/4] poc: record the Linux results Built and run in cubertgmbh/cuvis_pyil:3.5.3-ubuntu24.04. Needs no CMake change; SWIG emits the GIL release at 311 call sites there. Awaited worker results 101.5 ms to 22.6 ms, 4.70x the blocking floor down to 1.05x, idle callback 2.9 % of a core to 0.1 %. The blocking floor is identical across both runs, so only the awaited number moved. 134 tests pass. The 3.4.1 image segfaults in cuvis_proc_cont_create_from_session_file, but a control binding with %exception restored to its develop form crashes at the same line with the same exit code, so that is the SDK version mismatch and not this change. --- POC_REAL_AWAITABLE.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/POC_REAL_AWAITABLE.md b/POC_REAL_AWAITABLE.md index 30c1766..4bf2be0 100644 --- a/POC_REAL_AWAITABLE.md +++ b/POC_REAL_AWAITABLE.md @@ -124,6 +124,37 @@ The 108 ms figure is not a coincidence - it is the 100 ms poll interval, and it Full suite: 123 passed against the rebuilt extension on Python 3.12. +### Linux + +The same thing, built and run inside `cubertgmbh/cuvis_pyil:3.5.3-ubuntu24.04` on Python +3.12, against the SDK the image ships. The build needs no CMake change; SWIG emits the +GIL release at 311 call sites there against 325 on Windows, the difference being the +entry points the two platforms wrap. + +| | before | after | +| --- | --- | --- | +| worker result, blocking | 21.6 ms | 21.6 ms | +| worker result, awaited | **101.5 ms** | **22.6 ms** | +| awaited / blocking | 4.70x | 1.05x | +| registered callback, idle | 2.9 % of a core | 0.1 % of a core | + +The blocking floor is identical across the two runs, which is what makes this pair +trustworthy: only the awaited number moved. Linux lands closer to the floor than Windows +does, and its idle cost was the higher of the two to begin with. + +Full suite: 134 passed. + +The mismatch case behaves as it does on Windows. A binding built in the 3.5.3 image and +run against `cubertgmbh/cuvis_pyil:3.4.1-ubuntu24.04` reports the three functions 3.4.1 +does not export, warns once, and turns a call to one of them into a `RuntimeError` rather +than a crash, with the GIL released throughout. + +That image pairing does segfault, in `cuvis_proc_cont_create_from_session_file`, but the +fault is not this branch's: a control binding built from the same tree with `%exception` +restored to its `develop` form crashes at the same line with the same exit code. It is +the SDK version mismatch itself, present before any of this work, and worth its own +ticket rather than a mention here. + ## What the SDK would need for this to be finished The executor approach is a real improvement but it is not free, and both costs are the SDK's to remove: From 285fe9ec73615aa509221e6d2af5101e93e563a4 Mon Sep 17 00:00:00 2001 From: Simon Birkholz Date: Thu, 20 Aug 2026 09:29:26 +0200 Subject: [PATCH 3/4] Drop the POC write-up and benchmark scripts from the branch They were review material, not library code. Kept locally at C:\dev\cuvis_sdk\.poc-archive\real-awaitable, which is outside any git repo. The PR now carries only the Async.py and Worker.py change. --- POC_REAL_AWAITABLE.md | 207 ------------------------------ scripts/bench_worker.py | 90 ------------- scripts/seh_under_released_gil.py | 50 -------- 3 files changed, 347 deletions(-) delete mode 100644 POC_REAL_AWAITABLE.md delete mode 100644 scripts/bench_worker.py delete mode 100644 scripts/seh_under_released_gil.py diff --git a/POC_REAL_AWAITABLE.md b/POC_REAL_AWAITABLE.md deleted file mode 100644 index 4bf2be0..0000000 --- a/POC_REAL_AWAITABLE.md +++ /dev/null @@ -1,207 +0,0 @@ -# POC: real awaitables for the cuvis Python SDK (ALL-1671) - -Not a merge candidate. -This branch exists to answer one question: can `await` on a cuvis object resume when the SDK is done, instead of when the next poll happens to come round? - -It can, and the change needed is smaller than expected. -It also uncovered a crash that the obvious implementation walks straight into, which is the main reason this is written down rather than merged. - -Companion branch: `poc/gil-release` in `cuvis.swig`. - -## The problem - -`cuvis.python` already exposes `async`/`await`, so ALL-1671 looked done. -It is not. -Every path underneath is a sleep loop: - -| Site | Interval | -| --- | --- | -| `cuvis/Async.py` `AsyncMesu.__await__` | 10 ms | -| `cuvis/Async.py` `Async.__await__` | 10 ms | -| `cuvis/Worker.py` `Worker.get_next_result_async` | 100 ms | -| `cuvis/Worker.py` `Worker.register_worker_callback` | 1 ms | - -None of these is an awaitable in any meaningful sense. -They are timers that check a flag, and the interval sets a floor under the latency that no amount of Python-side work can lift. - -The SDK is not the limitation. -It already offers blocking waits with timeouts, and documents them: - -- `cuvis_async_capture_get(handle, timeout_ms, out)` - `cuvis.h:1912`, "Give 0 to wait for ever" -- `cuvis_async_call_get(handle, timeout_ms)` - `cuvis.h:1839` -- `cuvis_worker_get_next_result(worker, mesu, view, timeout_ms)` - `cuvis.h:3066`, "-1 to wait indefinitely" - -## Why the wrapper could not use them - -`cuvis.swig/src/cuvis_il.i` carried no threading directive and `cuvis.pyil/CMakeLists.txt` passes SWIG only `-doxygen`. -**No wrapped function released the GIL.** -A blocking `cuvis_async_capture_get(h, 5000)` therefore froze the entire interpreter for its full timeout, which makes it unusable from an event loop. -Polling was not a design choice; it was the only thing available. - -## The trap: `%module(threads="1")` crashes - -SWIG has no per-function opt in for GIL release. -`%threadallow` is defined in `swigwin/Lib/python/pyuserdir.swg` as `%feature("nothreadallow","0")` - there is only a module-wide switch and a per-function opt *out*. -So the obvious change is `%module(threads="1") cuvis_il`, and it generates exactly what you want: - -```cpp -CUVIS_GUARD({ - SWIG_PYTHON_THREAD_BEGIN_ALLOW; - result = cuvis_async_capture_get(arg1,arg2,arg3); - SWIG_PYTHON_THREAD_END_ALLOW; - }) -``` - -It also segfaults. - -`SWIG_PYTHON_THREAD_BEGIN_ALLOW` is an RAII object (`SWIG_Python_Thread_Allow`) whose destructor reacquires the GIL. -It sits *inside* `CUVIS_GUARD`, the MSVC structured-exception guard that turns a delay-load failure into a C++ throw so a missing symbol becomes a Python exception rather than a dead process. -The extension is built `/EHsc` (`ExceptionHandling=Sync`), under which **MSVC does not run C++ destructors while unwinding an SEH exception**. - -So when the delay-load stub fires, the guard never reacquires the GIL, and `SWIG_exception` in the `%exception` handler calls into the Python C API without it. - -Measured, not reasoned about. -Built against the staged SDK, run against the installed one, which does not export the eight CUDA entry points, calling one on the unshadowed extension: - -| Build | Result | -| --- | --- | -| `threads="1"` | `0xc06d007f`, then access violation, exit 139 | -| no threading (control) | `RuntimeError`, interpreter healthy, exit 0 | -| this branch's fix | `RuntimeError`, interpreter healthy, exit 0 | - -This matters because that path is not exotic: it is the exact scenario `cuvis.binding` was built to report, an installed SDK older than the binding. - -## The fix - -Drop `threads="1"` and manage the GIL explicitly in the `%exception` block that is already there, so correctness does not depend on unwinding running a destructor: - -```c -%exception { - PyThreadState *_cuvis_thread = PyEval_SaveThread(); - try { CUVIS_GUARD($action) } - catch (std::invalid_argument const& e) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_ValueError, e.what()); } - catch (std::exception const& e) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_RuntimeError, e.what()); } - catch (...) { CUVIS_REGAIN_GIL; SWIG_exception(SWIG_UnknownError, "unknown C++ exception from cuvis"); } - CUVIS_REGAIN_GIL; -} -``` - -`CUVIS_REGAIN_GIL` is idempotent, so the same statement serves the catch blocks and the success path. -This covers 325 wrapped call sites. - -There are no SWIG directors and no Python callables handed to the SDK, so nothing re-enters the interpreter from a thread that has given up the GIL. - -## What changed in cuvis.python - -`cuvis/Async.py` gains one helper and the two `__await__` methods collapse into it: - -```python -async def _wait_off_the_loop(blocking_get): - loop = a.get_running_loop() - return await loop.run_in_executor(None, blocking_get, _WAIT_FOREVER) -``` - -`cuvis/Worker.py`: - -- `get_next_result_async` runs the SDK's blocking wait in the executor instead of polling. It now raises `SDKException` on timeout rather than returning a `WorkerResult` built from handles the SDK never filled in, which is what the old version did. -- `register_worker_callback` awaits results in a loop; the 1 ms spin is gone. -- A private `_wait_for_result` reads the status directly and returns `None` on a timeout. `SDKException.__init__` calls `logging.exception` unconditionally, so a wait that expects to come back empty cannot go through the public method without writing a traceback to the log every second. - -## Measurements - -Simulated camera, one frame ingested per round, 12 rounds, same machine, runs taken back to back. -`scripts/bench_worker.py` in this branch reproduces it. - -| | before (develop) | after (this branch) | -| --- | --- | --- | -| worker result, blocking | 33.9 ms | 23.1 ms | -| worker result, awaited | **108.6 ms** | **28.0 ms** | -| awaited / blocking | **3.20x** | **1.21x** | -| registered callback, idle | 0.8 % of a core | 0.0 % of a core | - -The ratio is the number to read: absolute timings drift with machine load, but the awaited path went from three times the cost of the blocking floor to within a fifth of it. -The 108 ms figure is not a coincidence - it is the 100 ms poll interval, and it was there whether or not the result was ready. - -Full suite: 123 passed against the rebuilt extension on Python 3.12. - -### Linux - -The same thing, built and run inside `cubertgmbh/cuvis_pyil:3.5.3-ubuntu24.04` on Python -3.12, against the SDK the image ships. The build needs no CMake change; SWIG emits the -GIL release at 311 call sites there against 325 on Windows, the difference being the -entry points the two platforms wrap. - -| | before | after | -| --- | --- | --- | -| worker result, blocking | 21.6 ms | 21.6 ms | -| worker result, awaited | **101.5 ms** | **22.6 ms** | -| awaited / blocking | 4.70x | 1.05x | -| registered callback, idle | 2.9 % of a core | 0.1 % of a core | - -The blocking floor is identical across the two runs, which is what makes this pair -trustworthy: only the awaited number moved. Linux lands closer to the floor than Windows -does, and its idle cost was the higher of the two to begin with. - -Full suite: 134 passed. - -The mismatch case behaves as it does on Windows. A binding built in the 3.5.3 image and -run against `cubertgmbh/cuvis_pyil:3.4.1-ubuntu24.04` reports the three functions 3.4.1 -does not export, warns once, and turns a call to one of them into a `RuntimeError` rather -than a crash, with the GIL released throughout. - -That image pairing does segfault, in `cuvis_proc_cont_create_from_session_file`, but the -fault is not this branch's: a control binding built from the same tree with `%exception` -restored to its `develop` form crashes at the same line with the same exit code. It is -the SDK version mismatch itself, present before any of this work, and worth its own -ticket rather than a mention here. - -## What the SDK would need for this to be finished - -The executor approach is a real improvement but it is not free, and both costs are the SDK's to remove: - -1. **One pooled thread per outstanding wait.** Fine for a handful of captures, not for hundreds. -2. **A wait cannot be cancelled.** A thread blocked in C is not interruptible, so `reset_worker_callback` cannot take effect until the current window expires. That is why `_RESULT_WAIT_MS` is 1000 rather than "wait for ever", and it is a workaround, not a design. - -Both go away if the SDK hands out something an event loop can watch directly. -The proposal, for a cuvis.c ticket: - -```c -/** @brief A waitable OS handle that is signalled when the async capture completes. - * - * On Windows a Win32 event HANDLE, suitable for IocpProactor.wait_for_handle. - * On Linux an eventfd, suitable for loop.add_reader. - * The handle is owned by the SDK and stays valid until the async result is freed. - */ -SDK_CAPI CUVIS_STATUS SDK_CCALL cuvis_async_capture_get_event( - CUVIS_ASYNC_CAPTURE_RESULT i_asyncResult, CUVIS_EVENT_HANDLE* o_pEvent); -``` - -with the same shape for `cuvis_async_call_*` and `cuvis_worker_*`. -With that, the awaitables need no threads at all and cancellation is ordinary asyncio. - -A completion callback routed through `loop.call_soon_threadsafe` would also work. -The SDK has callback plumbing already (`log_callback` at `cuvis.h:1595`, `external_event_callback` at `:1637`), but neither is tied to an async result. - -A second, smaller ask: `cuvis_worker_get_next_result` reports "nothing arrived yet" through the same error channel as real failures, which is why `_wait_for_result` has to exist. -A distinct status for an expired timeout would remove it. - -## Reproducing - -```powershell -# 1. the SWIG side -cd C:\dev\cuvis_sdk\cuvis.pyil\cuvis.swig -git checkout poc/gil-release -powershell -ExecutionPolicy Bypass -File C:\dev\cuvis_sdk\cuvis.pyil\build_pyil.ps1 - -# 2. the measurements -$env:PYTHONPATH = "C:\dev\cuvis_sdk\cuvis.pyil;C:\dev\cuvis_sdk\cuvis.python-await" -& C:\dev\cuvis_sdk\.venv-pyil312\Scripts\python.exe scripts\bench_worker.py - -# 3. the crash test: build against the staged SDK, run against the installed one -Remove-Item -Recurse -Force C:\dev\cuvis_sdk\cuvis.pyil\build -powershell -ExecutionPolicy Bypass -File C:\dev\cuvis_sdk\cuvis.pyil\build_pyil.ps1 ` - -CuvisRoot "C:\dev\cuvis_sdk\.cuvis-new" -CuvisLib "C:\dev\cuvis_sdk\.cuvis-new\sdk\cuvis_c\cuvis.lib" -& C:\dev\cuvis_sdk\.venv-pyil312\Scripts\python.exe scripts\seh_under_released_gil.py -``` - -The build must be launched from a directory holding no stray `cuvis.dll`: the configure-time probe finds one through the Windows current-directory rule, before PATH, and bakes the wrong build hash into the binding. diff --git a/scripts/bench_worker.py b/scripts/bench_worker.py deleted file mode 100644 index 631b5f9..0000000 --- a/scripts/bench_worker.py +++ /dev/null @@ -1,90 +0,0 @@ -"""What the polling costs on the worker path: per-result latency and idle CPU. - -The worker is the clearest case. `Worker.get_next_result_async` sleeps 100 ms between -checks, so a result that arrives just after a check waits out the rest of the interval; -`register_worker_callback` spins at 1 ms, which is pure CPU with nothing to show for it. -""" - -import asyncio -import statistics -import time - -import cuvis - -cuvis.init(".") -session = cuvis.SessionFile("tests/test_data/test_mesu.cu3s") - - -def built_worker(): - pc = cuvis.ProcessingContext(session) - pc.processing_mode = cuvis.ProcessingMode.Raw - worker = cuvis.Worker(cuvis.WorkerSettings(output_queue_size=8)) - worker.set_processing_context(pc) - worker.start_processing() - return worker - - -def report(label, samples): - print( - "{:36} median {:7.2f} ms mean {:7.2f} ms max {:7.2f} ms".format( - label, statistics.median(samples), statistics.mean(samples), max(samples) - ) - ) - - -async def result_latency(rounds=12): - """Time from ingesting one frame to the awaited result coming back.""" - worker = built_worker() - samples = [] - try: - for _ in range(rounds): - start = time.perf_counter() - worker.ingest_session_file(session, frame_selection="0") - await worker.get_next_result_async(10000) - samples.append((time.perf_counter() - start) * 1000) - finally: - worker.stop_processing() - worker.drop_all_queued() - worker.reset_worker_callback() - return samples - - -def blocking_latency(rounds=12): - """The same, straight through the SDK's own blocking wait.""" - worker = built_worker() - samples = [] - try: - for _ in range(rounds): - start = time.perf_counter() - worker.ingest_session_file(session, frame_selection="0") - worker.get_next_result(10000) - samples.append((time.perf_counter() - start) * 1000) - finally: - worker.stop_processing() - worker.drop_all_queued() - return samples - - -async def idle_cpu(seconds=8.0): - """CPU spent by an event loop with one registered callback and nothing to do.""" - worker = built_worker() - worker.register_worker_callback(lambda result: asyncio.sleep(0)) - cpu, wall = time.process_time(), time.perf_counter() - await asyncio.sleep(seconds) - used = 100.0 * (time.process_time() - cpu) / (time.perf_counter() - wall) - worker.reset_worker_callback() - worker.stop_processing() - return used - - -async def main(): - report("worker result, blocking", blocking_latency()) - report("worker result, awaited", await result_latency()) - print( - "{:36} {:.1f} % of one core".format( - "registered callback, idle", await idle_cpu() - ) - ) - - -asyncio.run(main()) diff --git a/scripts/seh_under_released_gil.py b/scripts/seh_under_released_gil.py deleted file mode 100644 index a287b88..0000000 --- a/scripts/seh_under_released_gil.py +++ /dev/null @@ -1,50 +0,0 @@ -"""Does the delay-load SEH path hand back a clean exception once the GIL is released? - -Built against the staged SDK, run against the installed one, which does not export the -CUDA entry points. The extension is imported as a top-level module so the package's -__init__ never runs and never replaces the missing symbols with Python stubs: the call -reaches the real SWIG wrapper, the delay-load stub raises a Win32 SEH exception, -cuvis_seh_call turns it into a C++ throw, and %exception turns that into a Python error. - -With threads="1" the GIL is released around the call by an RAII guard. The build is -/EHsc, under which MSVC does not run C++ destructors while unwinding an SEH exception, so -the question is whether the GIL has been reacquired by the time SWIG_exception touches -the Python C API. -""" - -import ctypes -import gc -import os -import sys - -CUVIS_BIN = r"C:\Program Files\Cuvis\bin" -os.add_dll_directory(CUVIS_BIN) -for sub in ("bin", os.path.join("bin", "x64")): - cuda = os.path.join(os.environ.get("CUDA_PATH", ""), sub) - if os.path.isdir(cuda): - os.add_dll_directory(cuda) - -ctypes.WinDLL( - os.path.join(CUVIS_BIN, "cuvis.dll") -) # pin the library the stub will bind -import numpy # noqa: F401,E402 - -sys.path.insert(0, r"C:\dev\cuvis_sdk\cuvis.pyil\cuvis_il") -import _cuvis_pyil as raw # noqa: E402 - -print("raw extension:", raw.__file__) -print("shadowed? :", "no") - -name = "cuvis_cuda_mem_free" -print("calling raw {} ... a crash here means the GIL was not reacquired".format(name)) -sys.stdout.flush() -try: - getattr(raw, name)(0) -except Exception as exc: - print("clean exception:", type(exc).__name__, "|", str(exc)[:110]) -else: - print("returned without raising") - -sys.stdout.flush() -gc.collect() -print("interpreter still healthy:", sum(len(str(i)) for i in range(1000))) From 7db060d9674c4aa521aad0e497eb2b21b393b9e8 Mon Sep 17 00:00:00 2001 From: Simon Birkholz Date: Thu, 20 Aug 2026 09:46:25 +0200 Subject: [PATCH 4/4] Record the awaitable change in the changelog --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f440f47..b83127d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -51,6 +51,10 @@ Pre-releases (`b*`, `rc*`) are not listed. - Annotations throughout `cuvis` restated in the forms Python 3.10 provides: `Union[A, B]` and `Optional[A]` became `A | B` and `A | None`, and `Tuple`, `FrozenSet`, `Sequence`, `Callable` and `Awaitable` now come from `builtins` and `collections.abc` rather than `typing`. Every signature denotes what it denoted before; `cuvis.cube_utils.ImageData.__getitem__` keeps `Union`, because its member list contains a forward reference and `|` cannot join a type to a string at runtime. - `cuvis.AcquisitionContext.capture` - parameter `to_interal` renamed to `to_internal`. +- `cuvis.Async.AsyncMesu.__await__`, `cuvis.Async.Async.__await__`, `cuvis.Worker.get_next_result_async` - await the SDK's own blocking wait in a worker thread instead of polling it every 10 ms or 100 ms, so they resume when the SDK finishes rather than at the next poll. + Each outstanding wait occupies one pooled thread, and it cannot be cancelled, because a thread blocked in C is not interruptible. +- `cuvis.Worker.register_worker_callback` - awaits each result instead of re-checking every 1 ms, so an idle registered callback no longer occupies a core. + `cuvis.Worker.reset_worker_callback` therefore takes effect only once the current one-second wait expires. ### Removed @@ -69,6 +73,7 @@ Pre-releases (`b*`, `rc*`) are not listed. - `cuvis.AcquisitionContext.register_ready_callback` - parameter `callback` was annotated `Callable[None, Awaitable[None]]`, which is not a valid `Callable` form; it is now `Callable[[], Awaitable[None]]`, matching the no-argument call the implementation makes. - `cuvis.AcquisitionContext.capture` - `to_internal=True` raised `TypeError` instead of queueing the measurement, because it passed a Python `0` where SWIG requires a null pointer. - `cuvis.Calibration`, `cuvis.AcquisitionContext`, `cuvis.ProcessingContext`, `cuvis.SessionFile`, `cuvis.Measurement`, `cuvis.Viewer`, `cuvis.Worker`, `cuvis.CubeExporter`, `cuvis.EnviExporter`, `cuvis.TiffExporter`, `cuvis.ViewExporter` - `__del__` raised `TypeError` after a failed construction, because it freed a handle that was still `None`. +- `cuvis.Worker.get_next_result_async` - a timeout returned a `cuvis.Worker.WorkerResult` built from handles the SDK never filled in; it now raises `cuvis.cuvis_aux.SDKException`. ## [3.5.3.2] - 2026-08-19