[pull] master from ruby:master - #1288
Merged
Merged
Conversation
Give each Ractor its own objspace so an ordinary local GC runs without the VM lock or a barrier. A stop-the-world global GC does the cross-objspace mark/sweep/compaction and drains the weak tables. Includes objspace creation, absorption of a terminated Ractor's heap, zombie objspace handling, the containment checks that keep a Ractor from holding another objspace's unshareable object, and RGENGC_CHECK_MODE support for the new invariants. Co-authored-by: Claude <noreply@anthropic.com>
Implement send(move: true) with an off-heap courier that carries the payload between objspaces, neutralizing the source into a RactorMovedObject shell. Handle absorbing a terminated Ractor's heap, keeping a zombie owner alive while another Ractor still reads its value, and per-Ractor ownership of the VM tables. Co-authored-by: Claude <noreply@anthropic.com>
Keep the generic-ivar host-to-fields_obj mapping in one global table, serializing mutators with a dedicated lock. A local GC marks only the values living in its own objspace; the global GC marks the values of live keys in the weak pass that runs after marking. This removes the table-ownership transitions that used to happen on shareable promotion, on send, and on absorb.
Order thread teardown against GC-time frees, route Fiber and Thread frees from a local GC through the VM lock, and stop touching VM-global fiber-pool and zombie-thread state without synchronization. Co-authored-by: Claude <noreply@anthropic.com>
Make the VM-global GC roots per-Ractor so the main Ractor's ordinary local GC runs lock-free. Add the shareability and objspace-locality checks to the write barrier, and split the VM-wide tables behind finer-grained locks. Co-authored-by: Claude <noreply@anthropic.com>
A single-objspace implementation has no pinning or shareable-reference bits, no zombie objspaces, and no per-Ractor local GC: liveness is decided by marking alone. Give plain marking paths to everything the per-Ractor GC otherwise keeps alive through C-struct root scans and pinning (every Ractor's local roots, in-flight payloads, a terminated Ractor's rb_gc_register_mark_object entries, and the main Ractor before it joins vm->ractor.set), and connect during_global_gc_p to the mmtk world-stopped state. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adjust bootstraptest and test-all for the per-Ractor local GC, and add regression tests for the Ractor lifecycle and cross-objspace cases. Co-authored-by: Claude <noreply@anthropic.com>
…tor-local roots
Skipping test_enable leaves GC enabled for the rest of the suite, which
exposes two more tests:
(1) test_stress_and_stress= runs the whole assert_send_type type check
with GC.stress on and does not finish within the CI budget (the
leaked GC-off used to make it a no-op). Once the rbs fix ships,
upstream CI hits the same path.
(2) test_reachable_objects_from_root has assert_send_type eagerly build
trace.inspect of the entire roots hash even on success. Exposing
per-Ractor roots makes that inspect huge and pushes peak memory into
the gigabytes (NoMemoryError on the Windows CI).
Also skip GCSingletonTest#test_enable itself; the rbs-side fix is merged
but not yet released.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ractor#value used to be repeatable: the successor could call it again and get the same object back, because the value stayed reachable from the dead Ractor's C struct (sync.legacy). Keeping that slot valid after the Ractor is gone is what forced the successor to root and pin every Ractor it absorbed, through a VM-wide list guarded by its own lock, with add, unlink and scan coming from three different threads. Take the value out of the struct on the first call and raise Ractor::Error afterwards. The value is then rooted by the caller like any other return value, so the list, its lock and the pinning go away. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
RACTOR_CHECK_MODE stamped every object with the id of the Ractor that allocated it (a 4 byte suffix on every slot), and re-stamped whole object graphs whenever a value crossed a Ractor boundary. Per-Ractor objspaces already record the owner: an object's page knows its objspace, so rb_gc_obj_foreign_p() answers "is this another Ractor's object" with one load, and the id, its suffix and the re-stamping traversals all go away. Deriving it is also more accurate. The stamped id was never rewritten when a zombie objspace was absorbed, which is what ractor_reset_belonging() had to paper over by re-stamping received and inherited values. Page ownership moves with the absorb, so nothing is left stale. thread.c stamped the child's main Thread wrapper with the child's id because upstream allocates that wrapper in the parent's objspace. This branch already allocates it in the child's objspace, so the site goes too. With ractor_reset_belonging() gone, no rb_obj_traverse() caller has an enter function that descends into arbitrary types: make_shareable raises on T_FILE and shareable_p stops there, so obj_traverse_i() can no longer reach T_FILE and its case goes as well. GC impls that keep a single objspace (mmtk) cannot tell owners apart and report every object as their own, so the check degrades to a no-op there; in exchange, dropping the suffix re-enables mmtk's ZJIT allocation fastpath. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A move courier carries an IO by taking over its rb_io, so the source IO is already a RactorMovedObject and nobody else can close the descriptor. When the courier is freed without being delivered -- Ractor#send raising because the port is closed is enough -- the branch for MOVE_KIND_IO did nothing and the fd was leaked. Sending 20 IOs to a closed Ractor leaked 20 descriptors. Finalize the rb_io instead, which closes the fd and frees the buffers. A delivered IO already left fptr == NULL, so it is untouched. Found by an automated review of the branch. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rewriting the atfork block to reinitialize the GC's locks dropped the two lines that reinitialize vm->once_lock and vm->once_cond. They are only initialized in Init_Thread_Mutex now, so a child of a fork that happened while another thread held once_lock inherits it locked, and the first vm_once_wait_no_gvl in the child waits forever. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adding a generic-fields entry takes the VM-wide generic_fields_lock and then calls st_insert, which mallocs when the table grows. With this Ractor's GC disabled the allocation cannot fall back to a collection, so it can raise NoMemoryError -- and the raise skipped both gf_unlock() and the matching rb_gc_local_enable(). A leaked generic_fields_lock hangs every later access to generic fields, including the marking of any Ractor's local GC, so an otherwise recoverable NoMemoryError turned into a deadlock. Unwind through a tag instead. While here, drop three things that no longer match the code: a basket payload comment describing a generic-ivar map field that was removed, the unused st_table placeholder left in ractor_basket_new, and two comments in variable.c still describing the per-Ractor generic-fields tables that the single global table replaced. This commit also carries the mechanical translation of the Japanese comments in these two files, which was in progress in the same working tree. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verified from an independent review's findings: - Ractor#value: the legacy_taken check ran only before the wait for ractor_terminated, which yields the GVL, so a second thread of the same Ractor calling #value returned Qnil instead of raising "The value was already taken". Re-check after the wait. - basket_new: allocating the basket after building a move courier meant a NoMemoryError from the alloc orphaned the courier (sources already destroyed, registry entry alive forever). Allocate the basket first and free it if the courier build raises. - Ractor creation: rb_thread_alloc can raise NoMemoryError inside RB_VM_LOCKING, whose unlock a longjmp skips -- and it would also leave cr->objspace pointing at the child. Catch inside the lock, restore, and re-raise outside. - thread_start: rb_ractor_setup_default_port ran outside the tag that cancels creation, so a raise left an unstartable Ractor in the set and terminate_all waited forever. Move it inside. - thread_start: native_thread_create failure for a child Ractor called rb_ractor_living_threads_remove, which asserts the current Ractor and runs the Ractor-exit protocol from the wrong thread; cancel the creation instead, like the send-failure path. btest 2054 PASS, test_ractor.rb 38 tests / 364 assertions PASS; the #value race is covered by a new deterministic check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
After a native copy succeeds, ractor_basket_alloc can still raise NoMemoryError before cr->pin_capture is handed to the basket. The stale list was then re-pinned by every global GC's root pass (rb_ractor_repin_in_flight walks it unconditionally), keeping the failed send's snapshot alive until the next copy send replaced the list. Free the list on that path; the nodes stay shref-pinned only until the next global GC clears the bits. Found by an independent review of the previous raise-path fixes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A monitor entry holds a port of the monitoring Ractor and the exit token is sent through it, so the monitoring Ractor's struct has to outlive the monitored one: its wrapper is the reference that keeps it alive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tegers. When this code was refactored in 9c4db31, the mantissa_digits captured only the length of the mantissa and the negative sign was captured separately. Additionally, the paths diverged between signed and unsigned values. However, MAX_FAST_INTEGER_SIZE stayed at 18 which limited the range of uint64_t and int64_t that took the fast path. This change expands the range of uint64_t and int64_t's that can be returned by the fast paths without falling back to json_decode_large_number. ruby/json@0238abcaf3
RubyVM::InstructionSequence#to_binary raises "should not compile with coverage" while coverage measurement is enabled. Most to_binary tests in test_iseq.rb go through the iseq_to_binary helper, which omits the test in that case, but these two call to_binary directly. This makes `make check COVERAGE=true` fail deterministically, which has kept the ruby/actions coverage workflow red on every scheduled run. Use the helper in test_serialize_anonymous_outer_variables, and guard test_rb_iseq_load_from_binary with the same idiom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
heap_alloc_slot tested is_incremental_marking() and bumped a counter on
every allocation, to bring the mutator out to newobj_refill every
INCREMENTAL_MARK_STEP_ALLOCATIONS slots. Test it in the branch that has
already run out of the region instead, so that the fast path is back to
the single comparison it makes anyway.
A free region holds about twenty slots, which is far more often than the
step size, so newobj_refill charges the region it takes to
incremental_mark_step_allocated_slots and steps once the regions add up
to INCREMENTAL_MARK_STEP_ALLOCATIONS.
Allocating 40M empty arrays in one Ractor, instructions retired:
before 14.784G
after 14.170G
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Isolate console modes and version information from `IO` into a common namespace. ruby/io-console@62890333ac
Returns the AST node that the proc or method was compiled from, by re-parsing the source ([Feature #21795]). Returns nil if the node cannot be retrieved reliably: the stored source hash guarantees that the re-parsed source is likely the same code before the node is looked up by its node id. For a proc defined by a block, the outer node that owns the block (such as the method call with the block) is returned rather than the block node itself. The source is re-parsed by the same parser that compiled it. For prism, the default gem prism is expected, because it is the same parser as the one built into the interpreter; when another prism gem is loaded, a warning is emitted in verbose mode since it may parse the source differently. `version: "current"` is passed so that the source is parsed with the grammar of the running Ruby. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Returns the AST node at the location: the node of the surrounding method or block is retrieved and validated in the same way as Proc#syntax_tree, and then the node at the program counter is searched within it ([Feature #21795]). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )