Skip to content

report each compared node's trace in the graph matrix, on request (ADR-0031) - #134

Merged
kikashy merged 4 commits into
mainfrom
graph-traces
Aug 24, 2026
Merged

report each compared node's trace in the graph matrix, on request (ADR-0031)#134
kikashy merged 4 commits into
mainfrom
graph-traces

Conversation

@kikashy

@kikashy kikashy commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Closes #127.

ADR-0031: experimental_test_graphs accepts an optional boolean include_traces (decoded exactly as ADR-0028's rehearsal boolean — exact spelling, JSON boolean only); experimental graph test grows --include-traces on both forms. Asked, each reported comparison of an evaluated node carries the evaluation's own ADR-0027 trace, by pointer so presence tracks the request (asked is present, [] at minimum; not asked is absent — off is the previous payload byte for byte). An undeclared node was never evaluated and carries none; a row that fails before node comparisons carries none. Traces ride inside each row's marshaled report, so the existing ADR-0026 budget charges them with no new machinery.

ADR-0027 §0's matrix-results-carry-no-traces determination is partially superseded for graph matrix node comparisons via the index annotation; its text is untouched, per the ADR README's partial-supersession rule.

Tests assert the wire: presence on every compared node across MCP/CLI/graph layers, absence (not emptiness) when unasked with declined-equals-omitted byte identity, trace retention on a mismatching comparison, no trace on an undeclared node, byte equality between the comparison's trace and the evaluate composite's for the same node and inputs, and case-fold/type refusals for the new member. Six mutations each fail a test; the one safeguard that cannot discriminate is the nil-floor guard in the attach (unreachable while the evaluator honors ADR-0027 §1's never-nil floor), labeled here as such.

Material-decision impact: public-surface; review: round to be posted on this PR

🤖 Generated with Claude Code

kikashy and others added 4 commits August 24, 2026 09:14
…R-0031, closes #127)

The walk already holds every node evaluation's ADR-0027 trace and drops it;
a wire client showing why one node went unknown had nothing to render.

- experimental_test_graphs accepts an optional boolean include_traces,
  decoded exactly as the rehearsal boolean; experimental graph test grows
  --include-traces on both forms. Off is the previous payload, byte for byte.
- Each reported comparison of an evaluated node carries the evaluation's own
  trace, by pointer so presence tracks the request: asked is present, [] at
  minimum (the contract's floor, held at the one assignment), not asked is
  absent. An undeclared node was never evaluated and has no trace.
- Traces ride inside each row's marshaled report, so the existing budget
  charges them with no new machinery.
- ADR-0027 §0's matrix-results-carry-no-traces determination is partially
  superseded for graph matrix node comparisons via the index annotation;
  its text is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
The presence tests cannot tell the real trace from an empty substitute (the
mutation check said so); this one compares the comparison's trace byte for
byte against what the evaluate composite reports for the same node and
inputs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
…iew round 1)

- Budget differentials on both paths: the bare run fits its own bytes
  exactly, the traced run overruns that boundary and is refused by
  JPS-GRAPH-REPORT-BUDGET — no magic numbers, the reconcile makes the
  bare spend equal the envelope's bytes.
- The CLI project walk is asserted like the path-named form; deleting its
  wiring now fails a test.
- include_traces refusals are pinned to their full messages, rehearsal-style,
  including number/object/array values and both case-folded spellings; the
  tools-list pins the closed schema's optional boolean like rehearsal's.
- The declined-equals-omitted equality compares literal wire bytes, not a
  remarshal that would hide an unmodelled member.
- Two-node row pins each comparison to its own node's trace; a headline
  mismatch is pinned to zero comparisons and zero traces.
- Help, Long, ADR, and changelog say JSON report; the human rendering is
  deliberately unchanged. Two stale two-member comments now say three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
@kikashy

kikashy commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Review round 1 — disposition

Reviewer: Codex (codex-cli 0.145.0, OpenAI) · Reviewed SHA: d2cd53b9315d1747e1b9c48218a556bd3c6aa959 · Fix commit: 74bbbed (covered by the dispositioned-finding exception; 34e2e93 is the status flip only)

# Severity Finding Disposition
1 MEDIUM Traced budget behavior not mutation-tested: no test where a bare run fits and a traced run is refused. Accepted, fixed. Differential boundaries on both paths with no magic numbers: the reconcile makes the bare spend exactly the envelope's (direct) / entry's (walk) own bytes, so the bare run passes at that exact budget and the traced run is refused with JPS-GRAPH-REPORT-BUDGET. Mutation-verified: disabling all three charge sites in rows.go fails the test.
2 MEDIUM Public-interface gaps: CLI walk wiring untested, MCP refusals asserted by isError only, tools-list doesn't pin the new schema member. Accepted, fixed. Walk form asserted like the path form (deleting its wiring fails); a rehearsal-style full-message test pins every refusal verbatim including number/object/array values and both case-folded spellings; the tools-list pins the closed schema's optional boolean include_traces with no required members (type drift to string fails it).
3 LOW Pointer/shape edges unpinned: two-node row, empty-trace-as-[], headline-mismatch-with-expectedNodes. Accepted with one declined part. The two-node row now pins each comparison to its own node's trace (byte equality against a direct evaluate, plus an inequality guard; the borrow-another-node's-trace mutation fails it), and a headline mismatch is pinned to zero comparisons and zero traces. The empty-trace transport case is declined: no fixture pack this evaluator admits produces an empty trace, so the case would require inventing an unreachable shape — the nil-floor guard covering it is defensive, cannot discriminate, and is labeled as such in the PR body per the repo's mutation-check practice.
4 LOW --include-traces invisible in human format while help implies otherwise. Accepted, fixed (intended branch). Human rendering is deliberately unchanged; flag help, Long text, the ADR (new Neutral consequence), and the changelog now all say the JSON report carries it.
5 LOW Two stale two-member comments. Accepted, fixed. testGraphsArguments doc says all three members; the refusal-table comment names the optional boolean.

Also adopted from the verification notes: the declined-equals-omitted equality now compares literal wire bytes (toolText), not a remarshal that could hide an unmodelled member.

Reviewer's sandbox could not start go test (read-only FS); the full gate ran author-side at 74bbbed: gofmt/vet clean, full suite green, corpus 20/20. Nine hard-verification items (off-output byte identity, pointer aliasing, budget escape, supersession mechanics, decode parity, CLI wiring, mutation table, roster/claims, ADR conventions) recorded in the review log; no functional runtime defect was confirmed.

ADR-0031 flipped to accepted (34e2e93).

@kikashy
kikashy merged commit f6881e0 into main Aug 24, 2026
6 checks passed
@github-actions

Copy link
Copy Markdown

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

@kikashy kikashy mentioned this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report each node's trace in the graph matrix payload over MCP

1 participant