Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ All notable changes to tagged releases are documented here.

## Unreleased

- **Graph matrix runs and validations bind themselves to the document they loaded** (ADR-0030,
closes #132): the matrix suite entry, its validation twin, and the direct single-graph test
and validation envelopes carry `graphSha256` — the bare-hex digest of the exact bytes the run
decoded, read off the one load. Equality with `experimental_get_graph`'s own `sha256` proves
rows fetched in one call and the document fetched in another describe one revision; the member
is absent exactly when the document did not load, beside the detail or diagnostics that say
why. Additive under VERSIONING.md's MINOR rule; the evaluator's conformance claim is
unaffected and stated, in full and only, in `CONFORMANCE.md`.

- **Graph rows members are held to their exact spelling** — `encoding/json` case-folds member
names, so a rows document carrying `"Cases"` or `"ID"` bound past the strict decoder and was
silently read as the members it is not. The hold the pack matrix has carried since ADR-0025,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
status: accepted
date: 2026-08-24
deciders: maintainer
---

# Bind graph matrix runs and validations to the loaded document

## Context and problem statement

A wire-only client can fetch a configured graph document (`experimental_get_graph`,
[ADR-0029](0029-serve-graphs-and-their-inventory.md), digest included) and run its matrix
(`experimental_test_graphs`, [ADR-0026](0026-run-the-declared-graph-matrix-over-mcp.md)) — two
calls, two reads, one file that may be edited between them. The matrix payload carries no digest
of the graph document its run loaded, so a client joining the two — nodes by name, edge witnesses
by coverage index — cannot prove the rows and the structure describe one revision, and a review
of exactly such a client (issue #132) found the join silently combining revision-A rows with
revision-B arrows across an edit. The client's interim mitigation is connection-epoch gating,
which bounds staleness without ever proving sameness.

The value already exists at exactly the right moment: `graph.Load` computes the digest of the
exact bytes it decoded, the lock ([ADR-0019](0019-reviewed-set-lock.md)) already pins graph
digests per configured id, and the walk holds the loaded document in hand when it builds each
entry. Only the wire omits it.

## Decision drivers

- The echo rule every graph payload already follows: report what this run actually read, off the
one read, never off a second one that could name a different revision.
- The payload digest convention settled in ADR-0029's rounds: a member named for its algorithm
carries bare hex; the `sha256:`-prefixed spelling belongs to the lock and audit records.
- Absence must be honest: a document that did not load has no bytes to bind, and a digest of
nothing would be an invention beside the detail that says why.
- VERSIONING.md's MINOR rule: additive members move no `outputVersion`.

## Considered options

- **A. `graphSha256` on the four run payloads** — the suite entry, its validation twin, and the
direct single-graph test and validation envelopes — read off the loaded document's own digest.
- **B. Client-side binding only** (epoch gating, as the desk does today).
- **C. A digest member on the rows instead of the entry.**

## Decision outcome

Chosen option: **A**. Option B bounds staleness and proves nothing — it is the mitigation this
member exists to retire, not an answer. Option C repeats one fact per row on a surface whose
report budget was redesigned once already for exactly that multiplication (ADR-0026); the
document is loaded at most once per entry — a rowless entry skips before loading — and the
entry is where a per-load fact belongs.

Settled constraints:

1. **Member and format.** `graphSha256`, bare hex, on `GraphSuiteEntry`, `GraphValidationEntry`,
`GraphTest`, and `GraphValidation` — the digest `graph.Load` computed from the exact bytes
this run decoded, with the lock/audit `sha256:` prefix stripped at the payload boundary,
matching every other payload digest member.
2. **Present exactly when the document loaded.** An entry whose document could not be read or
loaded carries no digest, beside the detail or diagnostics that say why; a rows failure after
a successful load keeps it, because the bytes the binding names did load. The direct
envelopes exist only after their caller's load succeeded, so their member is required where
the walk entries' is `omitempty`.
3. **The binding it enables, stated for consumers:** equality with `experimental_get_graph`'s
`sha256` proves the served document and this run's results are about one revision; inequality
proves an edit happened between the calls. It is a binding of bytes, not a verdict about
either revision.
4. **Scope.** The matrix and validation surfaces, exactly. The graph *evaluate* composite
already binds differently — the audit record carries the graph digest per ADR-0018's record
design — and extending the envelope there is its own decision if a consumer ever needs it.

### Consequences

- Good, because the client's document/matrix join becomes provable instead of epoch-bounded, and
the desk's recorded mitigation can be retired for a real binding.
- Good, because the member costs one string per entry, charged by the existing report budget on
the budgeted matrix-test paths; the validation walk has no budget, and one string per entry
does not create the multiplication a budget exists for.
- Neutral, because human renderings are unchanged: the member exists for machine consumers doing
the join; a person reading the walk report is not comparing digests.
- Neutral, because `outputVersion` stays: additive members under the MINOR rule.
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,3 +126,4 @@ authority, and following it confers no conformance status on anything.
| [0027](0027-pin-the-evaluation-trace-contract.md) | Pin the evaluation trace: deterministic, complete, ordered, and still informative | accepted |
| [0028](0028-declare-an-evaluation-a-rehearsal.md) | Declare an evaluation a rehearsal, so exploration never writes a decision record | accepted |
| [0029](0029-serve-graphs-and-their-inventory.md) | Serve the configured graphs and their inventory over the wire, read-only | accepted |
| [0030](0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md) | Bind graph matrix runs and validations to the loaded document | accepted |
1 change: 1 addition & 0 deletions internal/cli/graph.go
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ func (a *App) graphValidateCommand() *cobra.Command {
GraphPath: graphPath,
GraphID: document.ID,
GraphVersion: document.Version,
GraphSHA256: strings.TrimPrefix(document.Digest, "sha256:"),
Diagnostics: document.Semantic(loaded),
}
if len(output.Diagnostics) > 0 {
Expand Down
9 changes: 9 additions & 0 deletions internal/cli/graph_test.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package cli

import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
Expand Down Expand Up @@ -85,6 +87,13 @@ func TestGraphValidateCommand(t *testing.T) {
if !found {
t.Fatalf("the cycle must be a diagnostic: %+v", validation.Diagnostics)
}
// The envelope binds the exact bytes this validation decoded (ADR-0030):
// bare hex on the wire, and present even when the document is invalid —
// it loaded, and these findings are about that revision.
sum := sha256.Sum256([]byte(looped))
if !strings.Contains(stdout, `"graphSha256":"`+hex.EncodeToString(sum[:])+`"`) {
t.Fatalf("the validation envelope binds the loaded bytes: %q", stdout)
}
}

func TestGraphEvaluateCommandComposesAndReportsJSON(t *testing.T) {
Expand Down
153 changes: 153 additions & 0 deletions internal/graph/digest_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
package graph

import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)

// rawJSON marshals a payload the way the wire does, so the assertions below
// discriminate the contract — the member's spelling and presence in bytes —
// not just the Go field a renderer might never emit.
func rawJSON(t *testing.T, payload any) string {
t.Helper()
data, err := json.Marshal(payload)
if err != nil {
t.Fatal(err)
}
return string(data)
}

func fixtureDigest(t *testing.T) string {
t.Helper()
data, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.graph.json"))
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}

// Every graph matrix run and validation binds itself to the exact document
// bytes it loaded (ADR-0030): the bare-hex digest a consumer holds against
// the served document's own sha256, so rows fetched in one call and structure
// fetched in another are provably about one revision. The digest is read off
// the one load, and it is absent exactly when the document did not load.
func TestGraphRunsCarryTheDocumentDigest(t *testing.T) {
want := fixtureDigest(t)
binding := `"graphSha256":"` + want + `"`

loaded := fixtureProject(t)
tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"})
if failure != nil {
t.Fatal(failure.Message)
}
if len(tested.Graphs) != 1 || !strings.Contains(rawJSON(t, tested.Graphs[0]), binding) {
t.Fatalf("the test entry binds the loaded bytes on the wire: %s", rawJSON(t, tested.Graphs[0]))
}

validated, failure := ValidateProject(loaded, "", "validate")
if failure != nil {
t.Fatal(failure.Message)
}
if len(validated.Graphs) != 1 || !strings.Contains(rawJSON(t, validated.Graphs[0]), binding) {
t.Fatalf("the validation entry binds the loaded bytes on the wire: %s", rawJSON(t, validated.Graphs[0]))
}
}

// The direct single-graph envelope echoes the digest of the document it was
// handed — the one load its caller performed — never a second read of the
// path. A digest planted on the document and impossible for the on-disk
// bytes proves which one the envelope reports.
func TestDirectTestEchoesTheHandedDocumentDigest(t *testing.T) {
loaded := fixtureProject(t)
graphBytes, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.graph.json"))
if err != nil {
t.Fatal(err)
}
document, loadFailure := Load(graphBytes, "onboarding.graph.json")
if loadFailure != nil {
t.Fatal(loadFailure.Message)
}
planted := strings.Repeat("ab", 32)
document.Digest = "sha256:" + planted
rowsBytes, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.rows.json"))
if err != nil {
t.Fatal(err)
}
rows, rowsFailure := LoadRows(rowsBytes, "onboarding.rows.json")
if rowsFailure != nil {
t.Fatal(rowsFailure.Message)
}
output, failure := Test(loaded, newEngine(t), document, "onboarding.graph.json", "onboarding.rows.json", rows, Options{Command: "test"})
if failure != nil {
t.Fatal(failure.Message)
}
if !strings.Contains(rawJSON(t, output), `"graphSha256":"`+planted+`"`) {
t.Fatalf("the envelope echoes the handed document's digest, bare hex: got %q", output.GraphSHA256)
}
}

// A rows failure after a successful document load keeps the digest: the
// document did load, those are the bytes the binding names, and the detail
// says what stopped the run.
func TestRowsFailureRetainsTheDocumentDigest(t *testing.T) {
files := map[string]string{}
for _, name := range []string{"jpack.json", "onboarding.graph.json", "sanctions-screening-0.1.0.pack.json", "vendor-onboarding-0.1.0.pack.json"} {
data, err := os.ReadFile(filepath.Join("testdata", "project", name))
if err != nil {
t.Fatal(err)
}
files[name] = string(data)
}
files["onboarding.rows.json"] = "{not json"
loaded := writeProject(t, files)
tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"})
if failure != nil {
t.Fatal(failure.Message)
}
entry := tested.Graphs[0]
if len(tested.Graphs) != 1 || entry.Status != "mismatch" || entry.Detail == "" ||
!strings.Contains(rawJSON(t, entry), `"graphSha256":"`+fixtureDigest(t)+`"`) {
t.Fatalf("loaded document, failed rows: digest stays beside the detail: %s", rawJSON(t, entry))
}
}

// A document that did not load has no bytes to bind: the member is absent
// from the wire — not empty — beside the detail or diagnostics that say why,
// on the test walk and the validation walk both.
func TestUnloadableGraphCarriesNoDigest(t *testing.T) {
files := map[string]string{}
for _, name := range []string{"jpack.json", "onboarding.rows.json", "sanctions-screening-0.1.0.pack.json", "vendor-onboarding-0.1.0.pack.json"} {
data, err := os.ReadFile(filepath.Join("testdata", "project", name))
if err != nil {
t.Fatal(err)
}
files[name] = string(data)
}
files["onboarding.graph.json"] = "{not json"
loaded := writeProject(t, files)

tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"})
if failure != nil {
t.Fatal(failure.Message)
}
if len(tested.Graphs) != 1 || tested.Graphs[0].Detail == "" ||
strings.Contains(rawJSON(t, tested.Graphs[0]), "graphSha256") {
t.Fatalf("no load, no digest member, a detail instead: %s", rawJSON(t, tested.Graphs[0]))
}

validated, failure := ValidateProject(loaded, "", "validate")
if failure != nil {
t.Fatal(failure.Message)
}
if len(validated.Graphs) != 1 || validated.Graphs[0].Status != "invalid" ||
len(validated.Graphs[0].Diagnostics) == 0 ||
strings.Contains(rawJSON(t, validated.Graphs[0]), "graphSha256") {
t.Fatalf("no load, no digest member, diagnostics instead: %s", rawJSON(t, validated.Graphs[0]))
}
}
1 change: 1 addition & 0 deletions internal/graph/rows.go
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,7 @@ func Test(loaded *project.Project, engine *evaluation.Engine, doc Document, grap
RowsPath: rowsPath,
GraphID: doc.ID,
GraphVersion: doc.Version,
GraphSHA256: strings.TrimPrefix(doc.Digest, "sha256:"),
Rows: make([]result.GraphTestRow, 0, len(rows.Cases)),
}
spent := 0
Expand Down
3 changes: 3 additions & 0 deletions internal/graph/suite.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"encoding/json"
"errors"
"fmt"
"strings"

"unicode/utf8"

Expand Down Expand Up @@ -125,6 +126,7 @@ func testEntry(loaded *project.Project, engine *evaluation.Engine, id string, en
return mismatch(detail)
}
report.GraphID, report.GraphVersion = document.ID, document.Version
report.GraphSHA256 = strings.TrimPrefix(document.Digest, "sha256:")
rowsBytes, err := loaded.ReadGraphRows(entry, MaxRowsBytes)
if err != nil {
if errors.Is(err, fssecure.ErrTooLarge) {
Expand Down Expand Up @@ -210,6 +212,7 @@ func validateEntry(loaded *project.Project, id string, entry project.Graph) resu
return report
}
report.GraphID, report.GraphVersion = document.ID, document.Version
report.GraphSHA256 = strings.TrimPrefix(document.Digest, "sha256:")
report.Diagnostics = append(report.Diagnostics, document.Semantic(loaded)...)
if len(report.Diagnostics) > 0 {
report.Status = "invalid"
Expand Down
Loading