Skip to content

Harden controlled agent study evaluation - #632

Merged
aryamthecodebreaker merged 1 commit into
mainfrom
codex/agent-study-harness-rigor
Aug 20, 2026
Merged

Harden controlled agent study evaluation#632
aryamthecodebreaker merged 1 commit into
mainfrom
codex/agent-study-harness-rigor

Conversation

@aryamthecodebreaker

Copy link
Copy Markdown
Owner

Summary

  • freeze a 12-task machine-readable manifest for the controlled agent study
  • require the complete task-by-arm cross product, global model/FixMap pins, and verified task/transcript hashes
  • retain failed and timed-out runs, keep unsupported provider token counters null, and add rejection coverage
  • align AGENT_STUDY and protocol documentation with the validator

Validation

  • npm ci (0 vulnerabilities)
  • npm run study:agent:test (18 passed)
  • npm run study:agent:check
  • npm run ci

No controlled model runs or paid API experiments were executed.

@aryamthecodebreaker
aryamthecodebreaker force-pushed the codex/agent-study-harness-rigor branch from b827687 to 58e628d Compare August 20, 2026 12:39
@aryamthecodebreaker
aryamthecodebreaker force-pushed the codex/agent-study-harness-rigor branch from 58e628d to 0750fa8 Compare August 20, 2026 12:58
@aryamthecodebreaker
aryamthecodebreaker merged commit 8450791 into main Aug 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant