Skip to content

fix(bench): preserve skipped workload results - #2496

Open
chubes4 wants to merge 2 commits into
mainfrom
fix/2495-benchmark-skip-results
Open

fix(bench): preserve skipped workload results#2496
chubes4 wants to merge 2 commits into
mainfrom
fix/2495-benchmark-skip-results

Conversation

@chubes4

@chubes4 chubes4 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • preserve explicit skipped workload status and reason through the generated benchmark dispatcher
  • exclude skipped workload calls from timing and metric samples while reporting required-scenario completeness
  • make benchmark matrix aggregation report incomplete cells separately from passed and failed cells

Verification

  • /Users/chubes/Developer/wp-codebox/node_modules/.bin/tsx tests/bench-skip-results.test.ts
  • NODE_PATH=/Users/chubes/Developer/wp-codebox/node_modules /Users/chubes/Developer/wp-codebox/node_modules/typescript/bin/tsc -b packages/runtime-core packages/runtime-playground packages/cli --force

Fixes #2495.

AI Assistance

GPT-5.6 Terra through OpenCode (openai/gpt-5.6-terra) implemented and verified this repair under Chris Huber's direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark aggregation reports skipped workloads as successful timing samples

1 participant