From 4e92a5d7f6be241589249b9c6b2904d3ac33ae5a Mon Sep 17 00:00:00 2001 From: Brian Jin <35789537+kikashy@users.noreply.github.com> Date: Mon, 24 Aug 2026 08:43:57 -0400 Subject: [PATCH 1/3] bind every graph run to the document it loaded (ADR-0030, closes #132) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A client can fetch a configured graph document and run its matrix in two calls with an edit between them, and the matrix payload carried no digest of the document its run loaded — so a join of rows and structure could silently span revisions. The suite entry, its validation twin, and the direct test envelope now carry graphSha256, the bare-hex digest of the exact bytes the run decoded, read off the one load; equality with the served document's own sha256 proves one revision, and the member is absent exactly when the document did not load, beside the detail that says why. The value existed at the right moment all along — graph.Load computes it and the lock pins it — and only the wire omitted it. Co-authored-by: Claude Fable 5 Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com> --- CHANGELOG.md | 9 ++ ...-bind-graph-runs-to-the-loaded-document.md | 74 ++++++++++++++++ docs/adr/README.md | 1 + internal/graph/digest_test.go | 62 ++++++++++++++ internal/graph/rows.go | 1 + internal/graph/suite.go | 3 + internal/result/graph.go | 85 +++++++++++-------- 7 files changed, 200 insertions(+), 35 deletions(-) create mode 100644 docs/adr/0030-bind-graph-runs-to-the-loaded-document.md create mode 100644 internal/graph/digest_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 85d8952..6f15933 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,15 @@ All notable changes to tagged releases are documented here. ## Unreleased +- **Every graph run binds itself to the document it loaded** (ADR-0030, closes #132): the matrix + suite entry, its validation twin, and the direct single-graph test envelope carry + `graphSha256` — the bare-hex digest of the exact bytes the run decoded, read off the one load. + Equality with `experimental_get_graph`'s own `sha256` proves rows fetched in one call and the + document fetched in another describe one revision; the member is absent exactly when the + document did not load, beside the detail that says why. Additive under VERSIONING.md's MINOR + rule; the evaluator's conformance claim is unaffected and stated, in full and only, in + `CONFORMANCE.md`. + - **Graph rows members are held to their exact spelling** — `encoding/json` case-folds member names, so a rows document carrying `"Cases"` or `"ID"` bound past the strict decoder and was silently read as the members it is not. The hold the pack matrix has carried since ADR-0025, diff --git a/docs/adr/0030-bind-graph-runs-to-the-loaded-document.md b/docs/adr/0030-bind-graph-runs-to-the-loaded-document.md new file mode 100644 index 0000000..6dc51ad --- /dev/null +++ b/docs/adr/0030-bind-graph-runs-to-the-loaded-document.md @@ -0,0 +1,74 @@ +--- +status: proposed +date: 2026-08-24 +deciders: maintainer +--- + +# Bind every graph run to the document it loaded, on the wire + +## Context and problem statement + +A wire-only client can fetch a configured graph document (`experimental_get_graph`, +[ADR-0029](0029-serve-graphs-and-their-inventory.md), digest included) and run its matrix +(`experimental_test_graphs`, [ADR-0026](0026-run-the-declared-graph-matrix-over-mcp.md)) — two +calls, two reads, one file that may be edited between them. The matrix payload carries no digest +of the graph document its run loaded, so a client joining the two — nodes by name, edge witnesses +by coverage index — cannot prove the rows and the structure describe one revision, and a review +of exactly such a client (issue #132) found the join silently combining revision-A rows with +revision-B arrows across an edit. The client's interim mitigation is connection-epoch gating, +which bounds staleness without ever proving sameness. + +The value already exists at exactly the right moment: `graph.Load` computes the digest of the +exact bytes it decoded, the lock ([ADR-0019](0019-reviewed-set-lock.md)) already pins graph +digests per configured id, and the walk holds the loaded document in hand when it builds each +entry. Only the wire omits it. + +## Decision drivers + +- The echo rule every graph payload already follows: report what this run actually read, off the + one read, never off a second one that could name a different revision. +- The payload digest convention settled in ADR-0029's rounds: a member named for its algorithm + carries bare hex; the `sha256:`-prefixed spelling belongs to the lock and audit records. +- Absence must be honest: a document that did not load has no bytes to bind, and a digest of + nothing would be an invention beside the detail that says why. +- VERSIONING.md's MINOR rule: additive members move no `outputVersion`. + +## Considered options + +- **A. `graphSha256` on the three run payloads** — the suite entry, its validation twin, and the + direct single-graph test envelope — read off the loaded document's own digest. +- **B. Client-side binding only** (epoch gating, as the desk does today). +- **C. A digest member on the rows instead of the entry.** + +## Decision outcome + +Chosen option: **A**. Option B bounds staleness and proves nothing — it is the mitigation this +member exists to retire, not an answer. Option C repeats one fact per row on a surface whose +report budget was redesigned once already for exactly that multiplication (ADR-0026); the +document is loaded once per entry, and the entry is where a per-load fact belongs. + +Settled constraints: + +1. **Member and format.** `graphSha256`, bare hex, on `GraphSuiteEntry`, `GraphValidationEntry`, + and `GraphTest` — the digest `graph.Load` computed from the exact bytes this run decoded, + with the lock/audit `sha256:` prefix stripped at the payload boundary, matching every other + payload digest member. +2. **Present exactly when the document loaded.** An entry whose document could not be read or + loaded carries no digest, beside the detail or diagnostics that say why. +3. **The binding it enables, stated for consumers:** equality with `experimental_get_graph`'s + `sha256` proves the served document and this run's results are about one revision; inequality + proves an edit happened between the calls. It is a binding of bytes, not a verdict about + either revision. +4. **Scope.** The matrix and validation surfaces, exactly. The graph *evaluate* composite + already binds differently — the audit record carries the graph digest per ADR-0018's record + design — and extending the envelope there is its own decision if a consumer ever needs it. + +### Consequences + +- Good, because the client's document/matrix join becomes provable instead of epoch-bounded, and + the desk's recorded mitigation can be retired for a real binding. +- Good, because the member costs one string per entry, charged by the existing report budget like + every retained member. +- Neutral, because human renderings are unchanged: the member exists for machine consumers doing + the join; a person reading the walk report is not comparing digests. +- Neutral, because `outputVersion` stays: additive members under the MINOR rule. diff --git a/docs/adr/README.md b/docs/adr/README.md index d1102cc..98c600e 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -126,3 +126,4 @@ authority, and following it confers no conformance status on anything. | [0027](0027-pin-the-evaluation-trace-contract.md) | Pin the evaluation trace: deterministic, complete, ordered, and still informative | accepted | | [0028](0028-declare-an-evaluation-a-rehearsal.md) | Declare an evaluation a rehearsal, so exploration never writes a decision record | accepted | | [0029](0029-serve-graphs-and-their-inventory.md) | Serve the configured graphs and their inventory over the wire, read-only | accepted | +| [0030](0030-bind-graph-runs-to-the-loaded-document.md) | Bind every graph run to the document it loaded, on the wire | proposed | diff --git a/internal/graph/digest_test.go b/internal/graph/digest_test.go new file mode 100644 index 0000000..229af36 --- /dev/null +++ b/internal/graph/digest_test.go @@ -0,0 +1,62 @@ +package graph + +import ( + "crypto/sha256" + "encoding/hex" + "os" + "path/filepath" + "testing" +) + +// Every graph run and validation binds itself to the exact document bytes it +// loaded (ADR-0030): the bare-hex digest a consumer holds against the served +// document's own sha256, so rows fetched in one call and structure fetched in +// another are provably about one revision. The digest is read off the one +// load, and it is absent exactly when the document did not load. +func TestGraphRunsCarryTheDocumentDigest(t *testing.T) { + data, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.graph.json")) + if err != nil { + t.Fatal(err) + } + sum := sha256.Sum256(data) + want := hex.EncodeToString(sum[:]) + + loaded := fixtureProject(t) + tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"}) + if failure != nil { + t.Fatal(failure.Message) + } + if len(tested.Graphs) != 1 || tested.Graphs[0].GraphSHA256 != want { + t.Fatalf("the test entry binds the loaded bytes: got %q, want %q", tested.Graphs[0].GraphSHA256, want) + } + + validated, failure := ValidateProject(loaded, "", "validate") + if failure != nil { + t.Fatal(failure.Message) + } + if len(validated.Graphs) != 1 || validated.Graphs[0].GraphSHA256 != want { + t.Fatalf("the validation entry binds the loaded bytes: got %q, want %q", validated.Graphs[0].GraphSHA256, want) + } +} + +// A document that did not load has no bytes to bind: the member is absent +// beside the detail, never a digest of nothing. +func TestUnloadableGraphCarriesNoDigest(t *testing.T) { + files := map[string]string{} + for _, name := range []string{"jpack.json", "onboarding.rows.json", "sanctions-screening-0.1.0.pack.json", "vendor-onboarding-0.1.0.pack.json"} { + data, err := os.ReadFile(filepath.Join("testdata", "project", name)) + if err != nil { + t.Fatal(err) + } + files[name] = string(data) + } + files["onboarding.graph.json"] = "{not json" + loaded := writeProject(t, files) + tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"}) + if failure != nil { + t.Fatal(failure.Message) + } + if len(tested.Graphs) != 1 || tested.Graphs[0].GraphSHA256 != "" || tested.Graphs[0].Detail == "" { + t.Fatalf("no load, no digest, a detail instead: %+v", tested.Graphs[0]) + } +} diff --git a/internal/graph/rows.go b/internal/graph/rows.go index d4b882e..46bad6d 100644 --- a/internal/graph/rows.go +++ b/internal/graph/rows.go @@ -261,6 +261,7 @@ func Test(loaded *project.Project, engine *evaluation.Engine, doc Document, grap RowsPath: rowsPath, GraphID: doc.ID, GraphVersion: doc.Version, + GraphSHA256: strings.TrimPrefix(doc.Digest, "sha256:"), Rows: make([]result.GraphTestRow, 0, len(rows.Cases)), } spent := 0 diff --git a/internal/graph/suite.go b/internal/graph/suite.go index 176974b..ccc8bef 100644 --- a/internal/graph/suite.go +++ b/internal/graph/suite.go @@ -4,6 +4,7 @@ import ( "encoding/json" "errors" "fmt" + "strings" "unicode/utf8" @@ -125,6 +126,7 @@ func testEntry(loaded *project.Project, engine *evaluation.Engine, id string, en return mismatch(detail) } report.GraphID, report.GraphVersion = document.ID, document.Version + report.GraphSHA256 = strings.TrimPrefix(document.Digest, "sha256:") rowsBytes, err := loaded.ReadGraphRows(entry, MaxRowsBytes) if err != nil { if errors.Is(err, fssecure.ErrTooLarge) { @@ -210,6 +212,7 @@ func validateEntry(loaded *project.Project, id string, entry project.Graph) resu return report } report.GraphID, report.GraphVersion = document.ID, document.Version + report.GraphSHA256 = strings.TrimPrefix(document.Digest, "sha256:") report.Diagnostics = append(report.Diagnostics, document.Semantic(loaded)...) if len(report.Diagnostics) > 0 { report.Status = "invalid" diff --git a/internal/result/graph.go b/internal/result/graph.go index 5d5be74..5aafb43 100644 --- a/internal/result/graph.go +++ b/internal/result/graph.go @@ -245,24 +245,27 @@ type GraphTestRow struct { // additive, so outputVersion is unchanged by the VERSIONING.md machine-output // rules. type GraphTest struct { - OutputVersion string `json:"outputVersion"` - Tool Tool `json:"tool"` - Command string `json:"command"` - Status string `json:"status"` - Experimental bool `json:"experimental"` - ConformanceClaimReference string `json:"conformanceClaimReference"` - Label string `json:"label"` - Kind string `json:"kind"` - FormatVersion string `json:"formatVersion"` - EvaluatorSpecVersion string `json:"evaluatorSpecVersion"` - ConfigPath string `json:"configPath"` - GraphPath string `json:"graphPath"` - RowsPath string `json:"rowsPath"` - GraphID string `json:"graphId"` - GraphVersion string `json:"graphVersion"` - Summary SuiteSummary `json:"summary"` - Rows []GraphTestRow `json:"rows"` - Coverage []MatrixProbe `json:"coverage,omitempty"` + OutputVersion string `json:"outputVersion"` + Tool Tool `json:"tool"` + Command string `json:"command"` + Status string `json:"status"` + Experimental bool `json:"experimental"` + ConformanceClaimReference string `json:"conformanceClaimReference"` + Label string `json:"label"` + Kind string `json:"kind"` + FormatVersion string `json:"formatVersion"` + EvaluatorSpecVersion string `json:"evaluatorSpecVersion"` + ConfigPath string `json:"configPath"` + GraphPath string `json:"graphPath"` + RowsPath string `json:"rowsPath"` + GraphID string `json:"graphId"` + GraphVersion string `json:"graphVersion"` + // GraphSHA256 binds this run to the exact document bytes it loaded + // (ADR-0030), bare hex per the payload convention. + GraphSHA256 string `json:"graphSha256"` + Summary SuiteSummary `json:"summary"` + Rows []GraphTestRow `json:"rows"` + Coverage []MatrixProbe `json:"coverage,omitempty"` } // GraphSuiteEntry is one configured graph's matrix run inside the project walk @@ -273,16 +276,22 @@ type GraphTest struct { // which case the identity echoes are empty rather than guessed and Rows and // Coverage are absent. type GraphSuiteEntry struct { - ID string `json:"id"` - Path string `json:"path"` - RowsPath string `json:"rowsPath,omitempty"` - GraphID string `json:"graphId,omitempty"` - GraphVersion string `json:"graphVersion,omitempty"` - Status string `json:"status"` - Summary SuiteSummary `json:"summary"` - Rows []GraphTestRow `json:"rows,omitempty"` - Coverage []MatrixProbe `json:"coverage,omitempty"` - Detail string `json:"detail,omitempty"` + ID string `json:"id"` + Path string `json:"path"` + RowsPath string `json:"rowsPath,omitempty"` + GraphID string `json:"graphId,omitempty"` + GraphVersion string `json:"graphVersion,omitempty"` + // GraphSHA256 is the bare-hex digest of the exact bytes this entry's run + // loaded (ADR-0030), read off the one load: a consumer holding the served + // document from another call binds these rows to that revision by equality + // rather than by hope. Absent exactly when the document did not load, + // beside the detail that says why. + GraphSHA256 string `json:"graphSha256,omitempty"` + Status string `json:"status"` + Summary SuiteSummary `json:"summary"` + Rows []GraphTestRow `json:"rows,omitempty"` + Coverage []MatrixProbe `json:"coverage,omitempty"` + Detail string `json:"detail,omitempty"` } // GraphSuite is the project graph-matrix walk: every configured graph's rows, @@ -314,13 +323,19 @@ type GraphSuite struct { // applies — plus the declared rows document's containment when one is // declared. Diagnostics carry every finding rather than the first one. type GraphValidationEntry struct { - ID string `json:"id"` - Path string `json:"path"` - RowsPath string `json:"rowsPath,omitempty"` - GraphID string `json:"graphId,omitempty"` - GraphVersion string `json:"graphVersion,omitempty"` - Status string `json:"status"` - Diagnostics []Diagnostic `json:"diagnostics"` + ID string `json:"id"` + Path string `json:"path"` + RowsPath string `json:"rowsPath,omitempty"` + GraphID string `json:"graphId,omitempty"` + GraphVersion string `json:"graphVersion,omitempty"` + // GraphSHA256 is the bare-hex digest of the exact bytes this walk loaded + // (ADR-0030): the member that lets a consumer holding the document from + // another call know these results are about the same revision, read off + // the one load rather than a second one. Absent exactly when the document + // did not load, beside the diagnostics that say why. + GraphSHA256 string `json:"graphSha256,omitempty"` + Status string `json:"status"` + Diagnostics []Diagnostic `json:"diagnostics"` } // GraphValidationSuite is the project graph-validation walk, on From 7f6ed6b7300813e4adaa4d9dd4cbb89ab8ca6609 Mon Sep 17 00:00:00 2001 From: Brian Jin <35789537+kikashy@users.noreply.github.com> Date: Mon, 24 Aug 2026 09:03:55 -0400 Subject: [PATCH 2/3] carry the digest on the direct validation envelope; make every binding test discriminate (review round 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - GraphValidation (direct single-graph validate) gains required graphSha256, populated from the one load its command performed — the ADR promised the validation surface and the direct envelope was missed (round-1 MEDIUM). - Tests now assert the wire, not the Go field: raw-JSON presence on all four payloads, raw-JSON absence (not emptiness) on both failed walk entries, a planted impossible digest proving the direct test envelope echoes the handed document, and digest retention across a post-load rows failure. - ADR/changelog narrowed to what ships: matrix runs and validations (evaluate composite excluded as before), load at most once per entry, absence beside the detail or diagnostics, budget claim qualified to the budgeted paths. Co-Authored-By: Claude Fable 5 Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com> --- CHANGELOG.md | 16 +-- ...and-validations-to-the-loaded-document.md} | 25 ++-- docs/adr/README.md | 2 +- internal/cli/graph.go | 1 + internal/cli/graph_test.go | 9 ++ internal/graph/digest_test.go | 119 +++++++++++++++--- internal/result/graph.go | 4 + 7 files changed, 143 insertions(+), 33 deletions(-) rename docs/adr/{0030-bind-graph-runs-to-the-loaded-document.md => 0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md} (77%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f15933..09075b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,14 +4,14 @@ All notable changes to tagged releases are documented here. ## Unreleased -- **Every graph run binds itself to the document it loaded** (ADR-0030, closes #132): the matrix - suite entry, its validation twin, and the direct single-graph test envelope carry - `graphSha256` — the bare-hex digest of the exact bytes the run decoded, read off the one load. - Equality with `experimental_get_graph`'s own `sha256` proves rows fetched in one call and the - document fetched in another describe one revision; the member is absent exactly when the - document did not load, beside the detail that says why. Additive under VERSIONING.md's MINOR - rule; the evaluator's conformance claim is unaffected and stated, in full and only, in - `CONFORMANCE.md`. +- **Graph matrix runs and validations bind themselves to the document they loaded** (ADR-0030, + closes #132): the matrix suite entry, its validation twin, and the direct single-graph test + and validation envelopes carry `graphSha256` — the bare-hex digest of the exact bytes the run + decoded, read off the one load. Equality with `experimental_get_graph`'s own `sha256` proves + rows fetched in one call and the document fetched in another describe one revision; the member + is absent exactly when the document did not load, beside the detail or diagnostics that say + why. Additive under VERSIONING.md's MINOR rule; the evaluator's conformance claim is + unaffected and stated, in full and only, in `CONFORMANCE.md`. - **Graph rows members are held to their exact spelling** — `encoding/json` case-folds member names, so a rows document carrying `"Cases"` or `"ID"` bound past the strict decoder and was diff --git a/docs/adr/0030-bind-graph-runs-to-the-loaded-document.md b/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md similarity index 77% rename from docs/adr/0030-bind-graph-runs-to-the-loaded-document.md rename to docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md index 6dc51ad..b2daf93 100644 --- a/docs/adr/0030-bind-graph-runs-to-the-loaded-document.md +++ b/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md @@ -4,7 +4,7 @@ date: 2026-08-24 deciders: maintainer --- -# Bind every graph run to the document it loaded, on the wire +# Bind graph matrix runs and validations to the loaded document ## Context and problem statement @@ -35,8 +35,8 @@ entry. Only the wire omits it. ## Considered options -- **A. `graphSha256` on the three run payloads** — the suite entry, its validation twin, and the - direct single-graph test envelope — read off the loaded document's own digest. +- **A. `graphSha256` on the four run payloads** — the suite entry, its validation twin, and the + direct single-graph test and validation envelopes — read off the loaded document's own digest. - **B. Client-side binding only** (epoch gating, as the desk does today). - **C. A digest member on the rows instead of the entry.** @@ -45,16 +45,20 @@ entry. Only the wire omits it. Chosen option: **A**. Option B bounds staleness and proves nothing — it is the mitigation this member exists to retire, not an answer. Option C repeats one fact per row on a surface whose report budget was redesigned once already for exactly that multiplication (ADR-0026); the -document is loaded once per entry, and the entry is where a per-load fact belongs. +document is loaded at most once per entry — a rowless entry skips before loading — and the +entry is where a per-load fact belongs. Settled constraints: 1. **Member and format.** `graphSha256`, bare hex, on `GraphSuiteEntry`, `GraphValidationEntry`, - and `GraphTest` — the digest `graph.Load` computed from the exact bytes this run decoded, - with the lock/audit `sha256:` prefix stripped at the payload boundary, matching every other - payload digest member. + `GraphTest`, and `GraphValidation` — the digest `graph.Load` computed from the exact bytes + this run decoded, with the lock/audit `sha256:` prefix stripped at the payload boundary, + matching every other payload digest member. 2. **Present exactly when the document loaded.** An entry whose document could not be read or - loaded carries no digest, beside the detail or diagnostics that say why. + loaded carries no digest, beside the detail or diagnostics that say why; a rows failure after + a successful load keeps it, because the bytes the binding names did load. The direct + envelopes exist only after their caller's load succeeded, so their member is required where + the walk entries' is `omitempty`. 3. **The binding it enables, stated for consumers:** equality with `experimental_get_graph`'s `sha256` proves the served document and this run's results are about one revision; inequality proves an edit happened between the calls. It is a binding of bytes, not a verdict about @@ -67,8 +71,9 @@ Settled constraints: - Good, because the client's document/matrix join becomes provable instead of epoch-bounded, and the desk's recorded mitigation can be retired for a real binding. -- Good, because the member costs one string per entry, charged by the existing report budget like - every retained member. +- Good, because the member costs one string per entry, charged by the existing report budget on + the budgeted matrix-test paths; the validation walk has no budget, and one string per entry + does not create the multiplication a budget exists for. - Neutral, because human renderings are unchanged: the member exists for machine consumers doing the join; a person reading the walk report is not comparing digests. - Neutral, because `outputVersion` stays: additive members under the MINOR rule. diff --git a/docs/adr/README.md b/docs/adr/README.md index 98c600e..74b13d3 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -126,4 +126,4 @@ authority, and following it confers no conformance status on anything. | [0027](0027-pin-the-evaluation-trace-contract.md) | Pin the evaluation trace: deterministic, complete, ordered, and still informative | accepted | | [0028](0028-declare-an-evaluation-a-rehearsal.md) | Declare an evaluation a rehearsal, so exploration never writes a decision record | accepted | | [0029](0029-serve-graphs-and-their-inventory.md) | Serve the configured graphs and their inventory over the wire, read-only | accepted | -| [0030](0030-bind-graph-runs-to-the-loaded-document.md) | Bind every graph run to the document it loaded, on the wire | proposed | +| [0030](0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md) | Bind graph matrix runs and validations to the loaded document | proposed | diff --git a/internal/cli/graph.go b/internal/cli/graph.go index b1e8800..2066cea 100644 --- a/internal/cli/graph.go +++ b/internal/cli/graph.go @@ -158,6 +158,7 @@ func (a *App) graphValidateCommand() *cobra.Command { GraphPath: graphPath, GraphID: document.ID, GraphVersion: document.Version, + GraphSHA256: strings.TrimPrefix(document.Digest, "sha256:"), Diagnostics: document.Semantic(loaded), } if len(output.Diagnostics) > 0 { diff --git a/internal/cli/graph_test.go b/internal/cli/graph_test.go index 54d6a35..fdcf527 100644 --- a/internal/cli/graph_test.go +++ b/internal/cli/graph_test.go @@ -1,6 +1,8 @@ package cli import ( + "crypto/sha256" + "encoding/hex" "encoding/json" "os" "path/filepath" @@ -85,6 +87,13 @@ func TestGraphValidateCommand(t *testing.T) { if !found { t.Fatalf("the cycle must be a diagnostic: %+v", validation.Diagnostics) } + // The envelope binds the exact bytes this validation decoded (ADR-0030): + // bare hex on the wire, and present even when the document is invalid — + // it loaded, and these findings are about that revision. + sum := sha256.Sum256([]byte(looped)) + if !strings.Contains(stdout, `"graphSha256":"`+hex.EncodeToString(sum[:])+`"`) { + t.Fatalf("the validation envelope binds the loaded bytes: %q", stdout) + } } func TestGraphEvaluateCommandComposesAndReportsJSON(t *testing.T) { diff --git a/internal/graph/digest_test.go b/internal/graph/digest_test.go index 229af36..3ab6b17 100644 --- a/internal/graph/digest_test.go +++ b/internal/graph/digest_test.go @@ -3,44 +3,123 @@ package graph import ( "crypto/sha256" "encoding/hex" + "encoding/json" "os" "path/filepath" + "strings" "testing" ) -// Every graph run and validation binds itself to the exact document bytes it -// loaded (ADR-0030): the bare-hex digest a consumer holds against the served -// document's own sha256, so rows fetched in one call and structure fetched in -// another are provably about one revision. The digest is read off the one -// load, and it is absent exactly when the document did not load. -func TestGraphRunsCarryTheDocumentDigest(t *testing.T) { +// rawJSON marshals a payload the way the wire does, so the assertions below +// discriminate the contract — the member's spelling and presence in bytes — +// not just the Go field a renderer might never emit. +func rawJSON(t *testing.T, payload any) string { + t.Helper() + data, err := json.Marshal(payload) + if err != nil { + t.Fatal(err) + } + return string(data) +} + +func fixtureDigest(t *testing.T) string { + t.Helper() data, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.graph.json")) if err != nil { t.Fatal(err) } sum := sha256.Sum256(data) - want := hex.EncodeToString(sum[:]) + return hex.EncodeToString(sum[:]) +} + +// Every graph matrix run and validation binds itself to the exact document +// bytes it loaded (ADR-0030): the bare-hex digest a consumer holds against +// the served document's own sha256, so rows fetched in one call and structure +// fetched in another are provably about one revision. The digest is read off +// the one load, and it is absent exactly when the document did not load. +func TestGraphRunsCarryTheDocumentDigest(t *testing.T) { + want := fixtureDigest(t) + binding := `"graphSha256":"` + want + `"` loaded := fixtureProject(t) tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"}) if failure != nil { t.Fatal(failure.Message) } - if len(tested.Graphs) != 1 || tested.Graphs[0].GraphSHA256 != want { - t.Fatalf("the test entry binds the loaded bytes: got %q, want %q", tested.Graphs[0].GraphSHA256, want) + if len(tested.Graphs) != 1 || !strings.Contains(rawJSON(t, tested.Graphs[0]), binding) { + t.Fatalf("the test entry binds the loaded bytes on the wire: %s", rawJSON(t, tested.Graphs[0])) } validated, failure := ValidateProject(loaded, "", "validate") if failure != nil { t.Fatal(failure.Message) } - if len(validated.Graphs) != 1 || validated.Graphs[0].GraphSHA256 != want { - t.Fatalf("the validation entry binds the loaded bytes: got %q, want %q", validated.Graphs[0].GraphSHA256, want) + if len(validated.Graphs) != 1 || !strings.Contains(rawJSON(t, validated.Graphs[0]), binding) { + t.Fatalf("the validation entry binds the loaded bytes on the wire: %s", rawJSON(t, validated.Graphs[0])) + } +} + +// The direct single-graph envelope echoes the digest of the document it was +// handed — the one load its caller performed — never a second read of the +// path. A digest planted on the document and impossible for the on-disk +// bytes proves which one the envelope reports. +func TestDirectTestEchoesTheHandedDocumentDigest(t *testing.T) { + loaded := fixtureProject(t) + graphBytes, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.graph.json")) + if err != nil { + t.Fatal(err) + } + document, loadFailure := Load(graphBytes, "onboarding.graph.json") + if loadFailure != nil { + t.Fatal(loadFailure.Message) + } + planted := strings.Repeat("ab", 32) + document.Digest = "sha256:" + planted + rowsBytes, err := os.ReadFile(filepath.Join("testdata", "project", "onboarding.rows.json")) + if err != nil { + t.Fatal(err) + } + rows, rowsFailure := LoadRows(rowsBytes, "onboarding.rows.json") + if rowsFailure != nil { + t.Fatal(rowsFailure.Message) + } + output, failure := Test(loaded, newEngine(t), document, "onboarding.graph.json", "onboarding.rows.json", rows, Options{Command: "test"}) + if failure != nil { + t.Fatal(failure.Message) + } + if !strings.Contains(rawJSON(t, output), `"graphSha256":"`+planted+`"`) { + t.Fatalf("the envelope echoes the handed document's digest, bare hex: got %q", output.GraphSHA256) + } +} + +// A rows failure after a successful document load keeps the digest: the +// document did load, those are the bytes the binding names, and the detail +// says what stopped the run. +func TestRowsFailureRetainsTheDocumentDigest(t *testing.T) { + files := map[string]string{} + for _, name := range []string{"jpack.json", "onboarding.graph.json", "sanctions-screening-0.1.0.pack.json", "vendor-onboarding-0.1.0.pack.json"} { + data, err := os.ReadFile(filepath.Join("testdata", "project", name)) + if err != nil { + t.Fatal(err) + } + files[name] = string(data) + } + files["onboarding.rows.json"] = "{not json" + loaded := writeProject(t, files) + tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"}) + if failure != nil { + t.Fatal(failure.Message) + } + entry := tested.Graphs[0] + if len(tested.Graphs) != 1 || entry.Status != "mismatch" || entry.Detail == "" || + !strings.Contains(rawJSON(t, entry), `"graphSha256":"`+fixtureDigest(t)+`"`) { + t.Fatalf("loaded document, failed rows: digest stays beside the detail: %s", rawJSON(t, entry)) } } // A document that did not load has no bytes to bind: the member is absent -// beside the detail, never a digest of nothing. +// from the wire — not empty — beside the detail or diagnostics that say why, +// on the test walk and the validation walk both. func TestUnloadableGraphCarriesNoDigest(t *testing.T) { files := map[string]string{} for _, name := range []string{"jpack.json", "onboarding.rows.json", "sanctions-screening-0.1.0.pack.json", "vendor-onboarding-0.1.0.pack.json"} { @@ -52,11 +131,23 @@ func TestUnloadableGraphCarriesNoDigest(t *testing.T) { } files["onboarding.graph.json"] = "{not json" loaded := writeProject(t, files) + tested, failure := TestProject(loaded, newEngine(t), "", Options{Command: "test"}) if failure != nil { t.Fatal(failure.Message) } - if len(tested.Graphs) != 1 || tested.Graphs[0].GraphSHA256 != "" || tested.Graphs[0].Detail == "" { - t.Fatalf("no load, no digest, a detail instead: %+v", tested.Graphs[0]) + if len(tested.Graphs) != 1 || tested.Graphs[0].Detail == "" || + strings.Contains(rawJSON(t, tested.Graphs[0]), "graphSha256") { + t.Fatalf("no load, no digest member, a detail instead: %s", rawJSON(t, tested.Graphs[0])) + } + + validated, failure := ValidateProject(loaded, "", "validate") + if failure != nil { + t.Fatal(failure.Message) + } + if len(validated.Graphs) != 1 || validated.Graphs[0].Status != "invalid" || + len(validated.Graphs[0].Diagnostics) == 0 || + strings.Contains(rawJSON(t, validated.Graphs[0]), "graphSha256") { + t.Fatalf("no load, no digest member, diagnostics instead: %s", rawJSON(t, validated.Graphs[0])) } } diff --git a/internal/result/graph.go b/internal/result/graph.go index 5aafb43..6b5301f 100644 --- a/internal/result/graph.go +++ b/internal/result/graph.go @@ -175,6 +175,9 @@ type GraphSchema struct { // references against the project — every node resolves through the // configuration, every evidence feed names a requirement its target pack // declares. Diagnostics carry every finding rather than the first one. +// GraphSHA256 is the bare-hex digest of the exact bytes this validation +// decoded (ADR-0030); the payload exists only after the document loaded, so +// unlike the walk entries the member is required. type GraphValidation struct { OutputVersion string `json:"outputVersion"` Tool Tool `json:"tool"` @@ -186,6 +189,7 @@ type GraphValidation struct { GraphPath string `json:"graphPath"` GraphID string `json:"graphId,omitempty"` GraphVersion string `json:"graphVersion,omitempty"` + GraphSHA256 string `json:"graphSha256"` Diagnostics []Diagnostic `json:"diagnostics"` } From 0066ad6ddc08a2186079cbc043e800ea92017b08 Mon Sep 17 00:00:00 2001 From: Brian Jin <35789537+kikashy@users.noreply.github.com> Date: Mon, 24 Aug 2026 09:04:45 -0400 Subject: [PATCH 3/3] ADR-0030 accepted Co-Authored-By: Claude Fable 5 Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com> --- ...-graph-matrix-runs-and-validations-to-the-loaded-document.md | 2 +- docs/adr/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md b/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md index b2daf93..fee12c9 100644 --- a/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md +++ b/docs/adr/0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md @@ -1,5 +1,5 @@ --- -status: proposed +status: accepted date: 2026-08-24 deciders: maintainer --- diff --git a/docs/adr/README.md b/docs/adr/README.md index 74b13d3..b383eec 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -126,4 +126,4 @@ authority, and following it confers no conformance status on anything. | [0027](0027-pin-the-evaluation-trace-contract.md) | Pin the evaluation trace: deterministic, complete, ordered, and still informative | accepted | | [0028](0028-declare-an-evaluation-a-rehearsal.md) | Declare an evaluation a rehearsal, so exploration never writes a decision record | accepted | | [0029](0029-serve-graphs-and-their-inventory.md) | Serve the configured graphs and their inventory over the wire, read-only | accepted | -| [0030](0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md) | Bind graph matrix runs and validations to the loaded document | proposed | +| [0030](0030-bind-graph-matrix-runs-and-validations-to-the-loaded-document.md) | Bind graph matrix runs and validations to the loaded document | accepted |