test/grpo_multi_student_fresh_process_test.exs ("fresh BEAM resumes only the active student and preserves distinct portable results") failed once in the stranger.check lane on PR #127 (commit 45edc41), which changes no code the test reads:
assert first =~ "grpo_step_outcome_unknown"
left: "{:error, {:reinforcement_callback_timeout, :final_model_artifact, 100}}\n"
The test runs a child BEAM with {:hang_after_step, "fresh-base-b"} and a 100 ms reinforcement callback timeout, then expects the child's stdout to contain both grpo_step_outcome_unknown and reinforcement_callback_timeout. On a loaded runner the timeout fired before the step outcome was recorded, so only the timeout error printed. The same lane passes on main and on PR #126, so this is a timing race in the test, not a regression.
To fix: read run_fresh/2, Imp.Test.FileMultiStudentGRPOTrainer, and the Imp.Clients.TrainingJob path that emits both messages, and decide whether the expected order is guaranteed by code or only by timing. Make it deterministic (have the fixture trainer record the step outcome synchronously before hanging, or assert on the checkpoint and events on disk that the test already checks, or use a timeout that cannot race). Do not weaken it to "either message". Show it no longer races by running the file repeatedly under nice -n 19 with a CPU hog.
test/grpo_multi_student_fresh_process_test.exs("fresh BEAM resumes only the active student and preserves distinct portable results") failed once in thestranger.checklane on PR #127 (commit 45edc41), which changes no code the test reads:The test runs a child BEAM with
{:hang_after_step, "fresh-base-b"}and a 100 ms reinforcement callback timeout, then expects the child's stdout to contain bothgrpo_step_outcome_unknownandreinforcement_callback_timeout. On a loaded runner the timeout fired before the step outcome was recorded, so only the timeout error printed. The same lane passes onmainand on PR #126, so this is a timing race in the test, not a regression.To fix: read
run_fresh/2,Imp.Test.FileMultiStudentGRPOTrainer, and theImp.Clients.TrainingJobpath that emits both messages, and decide whether the expected order is guaranteed by code or only by timing. Make it deterministic (have the fixture trainer record the step outcome synchronously before hanging, or assert on the checkpoint and events on disk that the test already checks, or use a timeout that cannot race). Do not weaken it to "either message". Show it no longer races by running the file repeatedly undernice -n 19with a CPU hog.