Skip to content

feat(reporting): support per-metric source availability evidence - #1173

Open
bokelley wants to merge 2 commits into
mainfrom
conductor/reporting-metric-evidence-1170
Open

bokelley wants to merge 2 commits into
mainfrom
conductor/reporting-metric-evidence-1170

Conversation

@bokelley

@bokelley bokelley commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Inline reporting fetches can now report different availability for each requested constituent/metric cell. A single constituent can retain impressions, explicitly report zero clicks, mark viewability delayed, and explain unsupported completed views in one sealed manifest.

Closes #1170

  • Add immutable MetricEvidence, optional nested InlineFetchResult.cell_availability, and helpers for complete data, an unsupported metric everywhere, and a delayed metric through a watermark.
  • Validate requested keys, duplicate mapping entries, status evidence, watermarks, and contradictory present/zero row assertions before staging. Explicit cells override constituent defaults; coverage and eligible control totals follow the resolved cells.
  • Keep semantic-contract identity exclusively in the selected SDK offering. The manifest schema and fingerprint/replay algorithms remain unchanged.
  • Add sync/async adopter type fixtures, documentation, and table-driven conformance tests.

Compatibility

Bare rows, None, existing positional InlineFetchResult arguments, and omitted/empty evidence maps retain their input behavior and constituent defaults. An independent review compared 12 representative legacy sealed shapes byte-for-byte with origin/main; all remain identical. Legacy-derived incomplete cells and unmatched staged rows continue to participate in their existing control-total behavior. A metric total is withheld only when the adapter explicitly declares a cell of that metric unavailable.

Validation

  • Initial focused reporting tests, including PostgreSQL 16 ledger/lifecycle conformance: 458 passed.
  • Initial full local PostgreSQL gate: 8,531 passed, 34 skipped, 9 deselected, 1 expected failure; 84.90% coverage.
  • Independent review: 113 focused, 479 reporting/inline with 16 skipped, and 8,374 full-suite with 44 skipped, 9 deselected, 1 expected failure; strict typing, packaging hooks, and generated-code validation passed.
  • Compatibility probes: 12-shape byte parity against origin/main, a 1,920-case exhaustive sweep, and a 10,800-case randomized validation fuzz, with no crashes or validation failures.
  • GitHub CI passes Python 3.10–3.13, PostgreSQL conformance, schema validation, security, downstream imports, and JavaScript storyboard lanes.

Residual risks and limits

  • Omitted cells retain legacy derivation; adapters must explicitly declare exceptions. The existing v1 wire rule still disallows mixing available and unavailable cells in a zero-row batch.
  • Structural/data contradictions currently raise from execute() rather than becoming typed source failures. This widens a pre-existing class and should be handled separately at the worker boundary.
  • A cell watermark later than the batch watermark promotes the batch watermark; a later publication that drops that evidence can correctly fail revision-sequence regression checks.
  • Unavailable cells may retain raw staged values; their metric control total is withheld, and consumers must follow the explicit evidence.
  • Live GAM/FreeWheel reads were not exercised in this PR; the closing feat(reporting): provide an adapter-first ReliableReportingService and conformance kit #1172 interop/conformance slice covers adapter integration.

Open workspace in Conductor

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Sep 16, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — additive per-metric source availability evidence.

Checked:

  • Change confined to src/adcp/reporting/inline_source.py with correct feat: semver signal.
  • Appended dataclass field preserves positional compatibility (no public-API break, no required→optional flip).
  • Fail-closed validation runs before staging.
  • Comprehensive conformance (tests/conformance/reporting/test_inline_cell_availability.py) and type-check coverage added.
  • No type-system layering, generated-code, forward-compat, or credential concerns touched.

Decision path: no critical/high/medium findings; gated_paths false; high_risk false; no no-auto-approve team match; review_decision: REVIEW_REQUIRED does not gate absent gated_paths. Falls through to row 9 → approve.

Per-cell evidence withdrew a metric's control total whenever *any* cell
for it was unavailable, and dropped every total whenever a staged row
matched no requested constituent. Both conditions are reachable with no
`cell_availability` at all, so five documented legacy shapes -- a `None`
not-ready answer, `covered_constituent_ids`, `unavailable_constituents`,
a non-default `unavailable_status`, and bare rows with one unmatched row
-- silently changed their sealed bytes, `content_fingerprint`, ledger
`control_totals`, and `revision_content_sha256` for adopters who never
opted in. A full-coverage manifest whose every cell was `present` lost
its totals entirely because one extra row was staged.

A control total is a checksum over the staged rows: `reporting_inspection`
and `_reconcile` verify it by recomputing from the revision's rows, so
unmatched rows belong in it and a derived constituent status says nothing
about it. Withdraw a column only when the adapter explicitly declared a
cell of that metric unavailable -- the case where row values exist but the
adapter has said they are not a measurement.

Also stop the zero-row mixing guard from naming `cell_availability`: a
derived result reaches it too, and an adopter who never set that field
should not be sent looking for it.

Verified all twelve representative legacy shapes now seal byte-identical
to origin/main, and re-ran 10,800 randomized override/fallback
combinations through the execution conformance validator.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve

Subsequent pass to a previously-approved PR (prior head 2037b40, no findings). The delta narrows control-total withdrawal to explicitly declared incomplete cells so the total is a checksum over staged rows, restoring legacy sealed-bytes/fingerprint behavior for adopters supplying no cell_availability and closing a latent replay-fingerprint regression. Reviewer verified the Collection import, freshness-gate interaction, None-fetch path, and that no test was disabled — no public-surface or semver break.

No blocking (critical/high) findings and no medium findings.

Gate checks:

  • No no-auto-approve team match (row 7 n/a).
  • gated_paths: false → row 2 n/a (review_decision REVIEW_REQUIRED does not gate because paths are not gated).
  • high_risk: false → rows 3/5 n/a.
  • Prior decision was approve → row 6 n/a.
  • Zero medium findings → row 8 n/a.

Falls through to row 9: approve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(reporting): support per-constituent, per-metric source availability

1 participant