Skip to content

Every process creation strands an EPROCESS object in nonpaged pool (Proc tag); dev machines with high process churn become unusable within hours #137

Description

@dwoodford-ita

STATUS, updated 2026-08-16. This does not reproduce on other machines
running the same Windows build. A control machine (ASUS Vivobook, same build)
shows 0.07 leaked objects per launch against 1.9-2.0 here, at comparable
uptime, and it runs the same ASUS drivers and services. Treat this as
machine-specific until a trigger is identified.
Three root-cause
attributions originally published here have been retracted; see the corrected
mechanism section below. My earlier speculation about Windows Server assumed a
platform-wide defect and should be discounted entirely.

Windows Build Number

10.0.26200.0 (25H2, build 26200.9168)

Processor Architecture

AMD64

Memory

31 GB usable (2 x 16 GB DDR5-4800)

Storage Type, free / capacity

C: NVMe SSD 576 GB free / 924 GB

Relevant apps installed

Podman Desktop (containers on WSL2)
WSL2 (VirtualMachinePlatform enabled; full Hyper-V role not installed)
Visual Studio Code
Git for Windows
Amazon Corretto JDK 8, Eclipse Temurin JDK 21
.NET SDK
Python 3.12
Microsoft Defender (WdFilter.sys active)
NVIDIA drivers + NVIDIA App

Note: ASUS Armoury Crate was fully removed with the official uninstall tool
before these measurements. The leak continued unchanged afterwards.

Traces collected via Feedback Hub

https://aka.ms/AA1324dy

Trace captured with WPR:
wpr -start Pool -start Handle -start VirtualAllocation -filemode
wpr -stop proc-leak.etl
Result: 5.55 GB, 43,499,201 events, 0 lost.

Also available on request: raw !obtrace output for six leaked process objects
(12 MB), showing the full reference/dereference history of each.

Isssue description

Process objects are never freed on this machine. Every process that exits leaves
its EPROCESS block allocated and still linked into the active process list.
Nonpaged pool climbs continuously until the machine becomes unresponsive, and
only a reboot clears it.

This belongs to the developer inner loop because the leak scales with process
creation rate. Container tooling, WSL, Git and build tools spawn short-lived
processes constantly. This system creates roughly 25,000 processes per hour,
about two orders of magnitude above a typical desktop, so a defect that is
invisible elsewhere makes a dev machine unusable within a single working day.

MECHANISM (corrected 2026-08-16 - see retractions below)

I enabled object reference tracing for the Proc tag (gflags /ko /t Proc), took a
kernel mirror dump with LiveKd, and ran !obtrace offline.

What is established:

After process exit the EPROCESS reaches HandleCount 0 / PointerCount 1.
One untagged Dflt reference survives indefinitely, preventing final object
deletion. THE CURRENT EVIDENCE DOES NOT IDENTIFY ITS OWNER.

Confirmed with a probe that creates bait via CreateProcessW and closes its own
handles before measuring: no handle is held on the leaked objects at all, by any
process, from the moment the creating handle closes, polled for 10 minutes. The
retention is a raw reference, not a handle.

RETRACTED ATTRIBUTIONS

Three separate attributions were published in this issue and all three are
withdrawn. Each failed the same way: a balance of one outstanding reference under
a shared tag does not assign to any particular acquisition.

  1. win32kbase!ForegroundBoost::LogProcessInformation. WITHDRAWN. Derived by
    taking the last unmatched acquisition under the Dflt tag. That method cannot
    identify which reference is outstanding.

  2. NtOpenProcess / PsOpenProcess. WITHDRAWN. Post-exit NtOpenProcess activity is
    balanced: it adds a lookup reference and a handle, then releases both.

  3. WdFilter / PsLookupProcessByProcessId. WITHDRAWN. The trace does show a
    +32768 Dflt reference-count cache refill through WdFilter during process
    creation, and a -32687 drain during rundown leaving the tag count at one. But
    the trace also contains WdFilter dereferences, and Dflt is the common tag for
    untagged references, so the surviving unit is not assignable to the refill.
    Per Microsoft's guidance, attribution requires matching reference and
    dereference calls under a DISTINCT tag; these all share Dflt.

WHAT I COULD NOT ESTABLISH

  • The owner of the surviving reference. There is no supported user-mode
    enumeration of who holds a raw object reference.
  • Sample size. Ten traced objects, all bait processes created seconds apart.
    Attempts to widen this ended in hard system locks before producing output.
  • The trigger. I could not determine which component repeatedly enables
    providers. Active ETW sessions were stable at 29 with no churn over several
    minutes, so the enable calls happen to providers inside sessions that already
    exist.

The leak itself is not in doubt. The precise culprit within it carries the
caveat above.

RELATED REPORTS

Users are reporting this symptom without reaching a cause:

Note the hardware spread: the first report is an Intel i7-13700K on a Z790-E
desktop, mine is an AMD Ryzen 9 7845HX laptop on a later build. Not CPU-vendor
specific, not chipset specific, and it survives into 25H2.

Steps to reproduce

TO OBSERVE THE LEAK

  1. Run a workload that creates short-lived processes continuously. Container
    tooling on WSL2 is sufficient; so is any build or test loop that shells out.

  2. Sample the Proc pool tag and the live process count periodically. poolmon
    works, or NtQuerySystemInformation(SystemPoolTagInformation) via P/Invoke.

  3. Compare Proc outstanding allocations against the live process count. The gap
    is the leak, and it only grows.

A DETERMINISTIC TEST

copy C:\Windows\System32\hostname.exe C:\temp\leakbait.exe
(launch 24 instances, all exit within 10 seconds)

Then against a kernel dump:

!process 0 0 leakbait.exe

All 24 EPROCESS blocks were still resident. They appear with
ObjectTable: 00000000 and HandleCount: 0, but remain linked into the active
process list.

TO REPRODUCE THE DIAGNOSIS

Several steps cost me significant time and are worth stating.

  • Setting ObTracePoolTags, ObTraceProcessName and ObTracePermanent in the
    registry does NOT enable object reference tracing. After a reboot with all
    three set, !obtrace still reported "Object Reference tracing not enabled".
    Use gflags /ko /t Proc, which takes effect immediately and needs no reboot.

  • Do NOT add /p (permanent traces). Trace data is then retained beyond the
    object's life, and on a system where every process object leaks, that grows
    without bound.

  • Analyse a mirror dump (livekd64 -m -o kernel.dmp, 2.5 GB in about 3 seconds),
    not a live view. LiveKd's live view resolves every read through its driver
    against running memory and is punishingly slow.

  • Do NOT run a bare !process 0 0. It hard-locked this machine three times, under
    both LiveKd and offline against the dump, because the active process list holds
    thousands of dead entries. The filtered form (!process 0 0 ) completed
    in 18 seconds every time.

  • Symbols:
    set _NT_SYMBOL_PATH=srvC:\symbolshttps://msdl.microsoft.com/download/symbols

Expected Behavior

Proc outstanding allocations track the live process count. When a process exits,
its EPROCESS object is dereferenced by every component that referenced it, the
object is freed, and it is unlinked from the active process list.

Nonpaged pool usage should therefore be roughly flat over time for a steady
workload, regardless of how many processes have been created and destroyed.

Actual Behavior

Proc outstanding grows without bound and never returns. Exited processes remain
in the active process list indefinitely.

MEASUREMENTS

96 seconds after a clean boot, nothing unusual running:

Proc outstanding : 3,109
live processes   :   245

23 minutes after boot:

Proc outstanding : 11,749
live processes   :   287

Growth is roughly 25,000/hour on this workload. Thre tracks it, since threads are
pinned by the processes holding them, and Job grows alongside.

CONFIRMED TWO INDEPENDENT WAYS

ETW: a two-minute capture with the Pool, Handle and VirtualAllocation profiles
recorded 855 process creations and 853 exits, with the live process count flat
throughout. All 855 process objects were still outstanding at the end of the
trace. A 100% leak rate on processes created during the window.

Controlled test: 24 of 24 deliberately launched processes leaked, as described
under Steps to reproduce.

RETAINED OBJECTS IN THE ACTIVE PROCESS LIST

On one boot, 569 of the first 774 entries (73.5%) were exited processes still
being held, including an smss.exe from startup. Top images among them, which is a
fair picture of a dev machine's process churn:

conhost.exe    169
wsl.exe        118
podman.exe      45
logman.exe      31
wslhost.exe     22

IMPACT

  • The machine becomes unresponsive after roughly 15 hours of normal dev work,
    with free RAM in the tens of megabytes.
  • A reboot is the only remedy.
  • Measurable mitigation: quitting Podman Desktop cut Proc growth from about
    22,300/hour to 8,229/hour (a 63% reduction) and took Job growth from roughly
    102,000/hour to zero. That is a workaround by not doing the work, not a fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions