fix: JRuby hangs joining leftover server/browser threads at teardown - #248
Conversation
|
Warning Review limit reached
Next review available in: 25 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
JRuby joins every live Ruby thread at interpreter teardown (Ruby.tearDown -> ThreadService.teardown -> Thread#join) where MRI just kills them. Puma's reactor thread parks in a native KQueue.poll/epoll_wait inside nio4r that no interrupt can wake, so a Capybara-booted Puma keeps a JRuby process alive forever after an otherwise green run (#244). The process that actually hangs is usually not the suite's own: it is one of the Open3.capture2e subprocesses run by the RSpec fixtures, which load support/setup_capybara, boot Puma and Chrome, finish their example and never exit -- the parent then blocks in capture2e with no timeout, which is the reported "stalls mid-run" shape. Capybara's stock :puma block builds its Puma::Server into a block-local and joins it, so nothing keeps a handle and Capybara::Server has no #stop. Register an equivalent block that keeps the handle and stop the server for real once the framework is done. Puma::Server#stop closes the reactor's input queue and wakes the selector, which is the only thing that gets that thread out of the native poll. Ferrum is not implicated: its threads park on Queue#pop, which JRuby interrupts cleanly, so no browser quit is needed.
Reviewer's GuideIntroduces a custom Capybara server configuration that keeps references to Puma::Server instances so they can be cleanly stopped at test teardown, preventing JRuby from hanging while joining the Puma reactor thread; integrates this shutdown with both Minitest and standalone RSpec fixtures. File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
6f2d0a3 to
7592b14
Compare
Closes #244.
The defect
JRuby joins every live Ruby thread at interpreter teardown
(
Ruby.tearDown->ThreadService.teardown->Thread#join) where MRI justkills them. Puma's reactor thread parks in a native
KQueue.poll/epoll_waitinside nio4r that no interrupt can wake, so a Capybara-bootedPuma keeps a JRuby process alive forever after an otherwise green run.
Two refinements to the diagnosis in #244:
1. The hang is usually not the suite's own process. The process that
hangs is one of the
Open3.capture2esubprocesses — the RSpec fixturesdriven by
test/integration/rspec_pending_masking_test.rbandrspec_after_hook_order_masking_test.rb. Those fixtures loadsupport/setup_capybara+setup_capybara_drivers, boot Puma and Chrome,finish their example, and then never exit. The parent blocks in
capture2e, which has no timeout. That is exactly the reported "reached~230 of 600 tests in 15 minutes" shape: the suite stalls mid-run, not
after it. (The suite process itself has the same leak and would hang at the
end too.)
2. Ferrum is not implicated. Its threads park on
Queue#pop, whichJRuby interrupts cleanly at teardown — verified with a standalone probe, and
confirmed by the fix below: the fixture now exits in 7s with its ferrum
threads never quit. The single blocker is the Puma reactor.
jstack evidence (before the fix)
Hung child process,
Full thread dump OpenJDK 26.0.2.1:mainstayed in thatThread#joinfor over two minutes with the suitealready green. The
java.lang.Threadit is blocked on (0x…bd20) sits inthe same allocation neighbourhood as the reactor's own selector monitors.
Puma's
select_loopadditionally wraps the whole loop inrescue StandardError => e; retry, so even an interruptible exceptionwould be swallowed and the thread would resume polling.
The fix
Capybara's stock
:pumablock builds itsPuma::Serverinto a block-localand calls
.run.join— nothing keeps a handle, andCapybara::Serverhas no#stop, so nothing anywhere can shut that server down. The harness nowregisters an equivalent server block that keeps the handle, and stops the
server for real once the test framework is done.
Puma::Server#stop(true)closes the reactor's input queue and wakes theselector, which is the only thing that gets that thread out of the native
poll.
Why not the alternatives
Capybara.reset_sessions!in anat_exit(the direction guessed in theissue): does not help. It calls
session.reset!->driver.reset!; itnever touches the server, and Capybara exposes no server shutdown at all.
ferrum's threads are
Queue#pop-parked and JRuby kills them fine. Addingit would grow the diff without changing the outcome.
it, and is not reachable anyway — the reactor thread is created inside
Puma.
correct on every engine; MRI just never punished us for skipping it.
Hook placement
minitest/autorunowns the process exit through its ownat_exit,registered before
support/setup_capybaraloads — so a plainat_exitthere would fire before the suite runs. The shutdown goes on
Minitest.after_runwhen minitest is present, and onat_exitfor thestandalone RSpec fixture subprocesses, which deliberately avoid
minitest/autorun.Minitest.after_runhooks run in reverse registrationorder, so the gem's own
Reporting.finalize!still runs before the servergoes away.
Before / after
Deterministic A/B on the fixture that was hanging. JRuby 10.0.6.0,
SCREENSHOT_DRIVER=vips,JRUBY_OPTS=--dev -J-Djruby.thread.pool.enabled=true,macOS/kqueue, single change being
Capybara.server:Capybara.server:puma(stock)timeout 120had to kill it — 2:00.33:stoppable_puma(this PR)Full
bin/rake teston JRuby, four seeds including the one from the issue(measured at 600 tests, before rebasing onto #246/#247):
Every process exited on its own; none needed a kill, and all four are
comfortably inside the 15-minute per-attempt budget from #243. Before the
fix, seed 29927 stalled at test 294 and
jstackshowed the teardown joinabove.
MRI is unaffected — full
bin/rake teston ruby 4.0.6, same machine, onlyCapybara.serverdiffering:Capybara.serverFinished in:puma(stock):stoppable_puma(this PR)After rebasing onto current master (613 tests), seed 29927 again:
613 runs, 1747 assertions, 0 failures, 0 errors, 1 skipin 7m13s onJRuby, and
613 runs, 0 failures, 0 errors, 1 skipin 85.37s on MRI.One earlier post-rebase JRuby run produced 21
Ferrum::DeadBrowserErrorsin
BrowserScreenshotTest— Chrome died under load on a busy laptop. It didnot reproduce (the file alone: 21 runs, 0 errors; the full suite at the same
seed: 0 errors) and it is unrelated to this change, which only touches
process exit. Flagging it rather than hiding it.
standardrb: 158 files, no offenses.Honest verdict
Eliminated for the Puma reactor, which is the thread that was actually
blocking teardown — and that half is now deterministic rather than
probabilistic: the server is stopped explicitly, so there is no leftover
poll to join. What is not claimed: that no third-party gem can ever leave
another uninterruptible thread behind. Ferrum's threads are still not
quit — they were measured as harmless, not fixed. If a future dependency
parks a thread in native code the same way, the same class of hang can
return.