Skip to content

Process limit leaks slots for children reaped with WNOHANG, eventually refusing all forks #229

Description

@mnowotnik

With -P 50, the reproduction below fails on fork 50 with:

BlockingIOError: [Errno 11] Resource temporarily unavailable

Changing os.waitpid(pid, os.WNOHANG) to a blocking os.waitpid(pid, 0) allows all 80 children to complete.

Reproduction

Save as /tmp/repro.py:

import os
import time

for i in range(80):
    try:
        pid = os.fork()
    except OSError as exc:
        print(f"fork {i + 1} failed: {exc}", flush=True)
        raise

    if pid == 0:
        os._exit(0)

    while os.waitpid(pid, os.WNOHANG)[0] == 0:
        time.sleep(0.005)

    print(f"child {i + 1} reaped", flush=True)

Run:

sandlock run \
  --allow-degraded fs-ioctl-dev \
  --allow-degraded signal-scope \
  --allow-degraded abstract-unix-socket-scope \
  -r / \
  -w /dev/null \
  -w /dev/zero \
  -w /dev/full \
  -w /tmp \
  --clean-env \
  -P 50 \
  --env PATH=/usr/local/bin:/usr/bin:/bin \
  --env HOME=/tmp \
  --env TMPDIR=/tmp \
  -- python3 /tmp/repro.py

Tested with the official Linux ARM64 release binaries for 0.8.6 and 0.8.8, inside a python:3.13-slim-trixie Docker container on kernel 6.12.76-linuxkit. Docker’s seccomp filter was disabled with --security-opt seccomp=unconfined because it otherwise blocked Sandlock’s
pidfd_getfd call.

Expected behavior

All 80 iterations complete. There is only one outstanding child at a time, so the concurrent process limit should not be exhausted.

Observed behavior

Children 1–49 are successfully reaped. Fork 50 fails with EAGAIN. The blocking-wait control completes all 80 iterations.

Suspected cause

In resource.rs, handle_fork() increments proc_count, while handle_wait() decrements it only for blocking waits.

The seccomp filter allows WNOHANG waits without notifying the supervisor. These waits can successfully reap children, but their process slots remain charged. The PID exit cleanup also appears not to release those slots.

This makes the concurrent process allowance behave like a cumulative fork allowance for this workload.

Impact

We investigated this after a long-lived sandboxed coding agent lost command execution following pytest. Subsequent shell commands failed while the agent itself remained alive. The reproduction establishes the accounting defect independently; we have not traced the
original incident’s internal counter.

Please correct accounting for nonblocking reaping and add a regression test that runs more sequential children than the configured process limit. An explicit option to disable process limiting while retaining filesystem and network confinement would also be useful.

Possible fix

Account for actual child lifetimes rather than decrementing the counter before blocking wait() calls. Reserve a slot before allowing process creation, release it if creation fails, and release it exactly once when the child exits, regardless of how its parent waits.
The existing pidfd exit watchers may provide a foundation, but registration must also cover short-lived children that exit without issuing a supervised syscall. Remove the wait-based decrement to prevent double counting, and add regression coverage for blocking waits,
WNOHANG reaping, failed forks, and concurrent short-lived children.

Workaround

Add flag to disable the process limit.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions