docs: require parent authorization for evaluation resubmissions - #2473
chadvoegele wants to merge 1 commit into
Conversation
Signed-off-by: Chad Voegele <cvoegele@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2473 +/- ##
=======================================
Coverage 70.91% 70.91%
=======================================
Files 600 600
Lines 65987 65987
=======================================
Hits 46794 46794
Misses 19193 19193
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Chad's Agent — independent reviewReviewed head No actionable findings. The delegated-evaluation rules explicitly gate subsequent submissions, preserve resume lineage, and keep score acceptance subordinate to run validation. Checks: read the entire four-file diff and surrounding evaluation, validation, monitoring, workspace, and SciCode instructions; inspected NEL's SLURM resume/cancellation implementation. Agent-definition synchronization/reference test: 1 passed. Scenario JSON, fixture paths, and Limits: the three new scenarios are behavioral examples, not executed agent tests. No evaluations or cluster cancellation tests were run; this review does not establish runtime enforcement of authorization or budgets. Shared checkout unchanged. |
What does this PR do?
Type of change: documentation.
Chad's Agent: Require explicit parent authorization for each delegated evaluation submission. An evaluator could previously submit a speculative retry while the parent accepted the completed result and stopped.
Reuse the existing handoff/reply mechanism: report terminal results, then wait. Keep score and output-health acceptance aligned with the evaluation skill, preserve failed/canceled invocations and resume-job lineage, and verify cancellation of actual submitters and affected jobs rather than wrappers. No orchestration code added.
Usage
Parent reviews the terminal report and remaining budget, then says STOP or authorizes one task/scope/configuration/budget. Silence does not authorize another submission.
Testing
uv run --no-project --with pytest --with pytest-timeout python -m pytest -o addopts='' plugins/modelopt/skills/day0-release/tests/test_agent_definitions.py -q).git diff --checkpassed.Before your PR is "Ready for review"
CONTRIBUTING.md: N/A.Additional Information
Draft for review; do not merge.