Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions docs/reporting-ledger-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Account-qualified reporting generations

The fix for [#1169](https://github.com/adcontextprotocol/adcp-client-python/issues/1169)
changes a reporting configuration generation's identity to
`(account_id, delivery_config_id, delivery_config_version)`. Two accounts can
each accept `daily@1`, with different immutable contents, in one ledger.

Use the frozen, hashable public value for maps and joins:

```python
from adcp.reporting.ledger import ReportingConfigurationGenerationKey

key = ReportingConfigurationGenerationKey(
account_id="account-a",
delivery_config_id="daily",
delivery_config_version=1,
)
generations[key] = configuration
```

`ReportingConfiguration.generation_key` now returns this value. Code that
unpacked or indexed the beta.15 two-tuple must use the named attributes instead.
`ReportingObligationRecord`, `ConsumerStatusRecord`, and `LeasedConfiguration`
also expose `generation_key`. The existing constructors and store method
arguments, including `find_obligation` and the lease/release calls, remain
compatible. Consumer-status chain tuples, retained hashes, and derived
obligation/issue identifiers keep their existing serialization.

The worker lease API still selects work across the store's accounts. Resolve
work using the returned lease's account and generation. Releases match its
account, config ID, version, worker ID, and expiry; a release of an expired
handle cannot clear a newer lease held under the same worker ID. Callers must
retain the returned expiry when persisting or reconstructing a lease handle.
Passing a configuration from a different account or generation to
`ReportingProducer.acquire_obligation` now raises
`CONFIGURATION_GENERATION_MISMATCH` before touching the source.

Seller-issued obligation IDs remain globally unique. A low-level write that
reuses an ID for a different logical period now raises
`OBLIGATION_IDENTITY_CONFLICT` in both stores; it cannot overwrite another
account's obligation in memory. When filtering consumer statements by
obligation IDs, the named obligations must exist in the requested account.

## Upgrading PostgreSQL from 8.0.0-beta.15

1. Stop and drain all older reporting workers and configuration writers that
use this ledger. Keep them stopped throughout the upgrade. Older code
still selects and releases generations without an account predicate, so
mixing old and new workers is unsafe once accounts reuse a config ID.
2. With the upgraded SDK, run `await store.create_schema()` before starting
reporting work. It creates missing tables and applies the bundled
`reporting_ledger_account_generations.sql` migration in one transaction.
3. Restart reporting work with the upgraded SDK on every instance.

For deployments managed by a migration tool, the standalone migration is
[`reporting_ledger_account_generations.sql`](../src/adcp/reporting/ledger/reporting_ledger_account_generations.sql).
It upgrades an existing beta.15 ledger by itself, including in autocommit mode.
For a combined bootstrap and upgrade, run both bundled files in one transaction:

```sh
psql "$REPORTING_DATABASE_URL" --set=ON_ERROR_STOP=1 --single-transaction \
-f src/adcp/reporting/ledger/reporting_ledger.sql \
-f src/adcp/reporting/ledger/reporting_ledger_account_generations.sql
```

Use the ledger's existing `search_path` and a role that owns its tables. Both
SQL files are also included as resources in the installed SDK's
`adcp.reporting.ledger` package. Running only `CREATE TABLE IF NOT EXISTS`
leaves the old primary key in place and does not perform this upgrade.

The migration inspects the primary-key columns under an exclusive table lock
and replaces the beta.15 global primary key with the account-qualified key.
It preserves the constraint's name, including a renamed beta.15 key. Existing
configuration rows, leases, obligations, revisions and frozen rows,
adjustments, consumer statements, issue lifecycle records, change-feed
sequences, hashes, and unrelated constraints/indexes remain intact.

Bootstrap and migration share a transaction-scoped advisory lock, so concurrent
upgraded processes serialize their DDL. Repeated runs recognize the new primary
key and leave its index intact. A failed upgrade rolls back; the migration
does not use `CASCADE` or discard records. An unexpected primary key or an
adopter-added foreign key referencing the old key requires an explicit
adopter migration. The released beta.15 schema has no foreign keys referencing
`reporting_configurations`.

The key's index rebuild holds an `ACCESS EXCLUSIVE` lock on
`reporting_configurations`, temporarily blocking its reads, writes, and leases.
Allow a maintenance window appropriate to the number of retained generations;
large obligation/revision tables are not rewritten. Configure deployment
lock/statement timeouts to match that window and retry a rolled-back migration
after resolving the blocking condition. These lock semantics follow
[PostgreSQL's ALTER TABLE documentation](https://www.postgresql.org/docs/16/sql-altertable.html).

A rollback to beta.15 is unsafe once multiple accounts share a config ID and
version. Do not recreate the global key or delete conflicting account rows to
make an old worker start. Preserve the ledger and roll forward with a corrected
account-qualified implementation.

The in-memory store needs no schema migration. Restart it with the upgraded
SDK and reload accepted configurations from the adopter's source of truth.
2 changes: 2 additions & 0 deletions src/adcp/reporting/ledger/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@
LedgerSnapshot,
ReportingAdjustmentRecord,
ReportingConfiguration,
ReportingConfigurationGenerationKey,
ReportingDefinitionBinding,
ReportingDeliveryEscalation,
ReportingFinality,
Expand Down Expand Up @@ -140,6 +141,7 @@
"ProducerOfferings",
"ReportingAdjustmentRecord",
"ReportingConfiguration",
"ReportingConfigurationGenerationKey",
"ReportingDefinitionBinding",
"ReportingDeliveryEscalation",
"ReportingFinality",
Expand Down
21 changes: 11 additions & 10 deletions src/adcp/reporting/ledger/consumer_status.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
from adcp.reporting.ledger.models import (
ConsumerStatusRecord,
ConsumerStatusValue,
ReportingConfigurationGenerationKey,
ReportingDeliveryEscalation,
ReportingHealth,
ReportingIssue,
Expand Down Expand Up @@ -283,11 +284,7 @@ async def _validate_against_configuration(self, record: ConsumerStatusRecord) ->
account_id=record.account_id, delivery_config_ids=[record.delivery_config_id]
)
generation = next(
(
item
for item in configurations
if item.delivery_config_version == record.delivery_config_version
),
(item for item in configurations if item.generation_key == record.generation_key),
None,
)
if generation is None:
Expand Down Expand Up @@ -330,8 +327,7 @@ async def _resolve_named_records(self, record: ConsumerStatusRecord) -> None:
"resolve for this caller and account",
)
if (
obligation.delivery_config_id != record.delivery_config_id
or obligation.delivery_config_version != record.delivery_config_version
obligation.generation_key != record.generation_key
or obligation.report_definition_id != record.report_definition_id
or _utc(obligation.period.start) != _utc(record.period_start)
or _utc(obligation.period.end) != _utc(record.period_end)
Expand Down Expand Up @@ -528,13 +524,18 @@ def consumer_mismatch_issue_key(
``obligation_missing`` attaches to the repaired obligation later, and the
spec requires that chain never be lost, forked, or reset.
"""
generation = ReportingConfigurationGenerationKey(
account_id=account_id,
delivery_config_id=delivery_config_id,
delivery_config_version=delivery_config_version,
)
payload = canonical_json_utf8_v1(
[
"core-consumer-status-mismatch-v1",
account_id,
generation.account_id,
consumer_id,
delivery_config_id,
delivery_config_version,
generation.delivery_config_id,
generation.delivery_config_version,
report_definition_id,
_utc(period_start).isoformat(),
_utc(period_end).isoformat(),
Expand Down
42 changes: 40 additions & 2 deletions src/adcp/reporting/ledger/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"LedgerSnapshot",
"ReportingAdjustmentRecord",
"ReportingConfiguration",
"ReportingConfigurationGenerationKey",
"ReportingFinality",
"ReportingHealth",
"ReportingDeliveryEscalation",
Expand Down Expand Up @@ -287,6 +288,20 @@ def first_ordinal_after(
ordinal += 1


@dataclass(frozen=True)
class ReportingConfigurationGenerationKey:
"""The account-qualified identity of one accepted configuration generation.

``delivery_config_id`` is caller-selected and may be reused by another
account. Use this value for lookups, joins, and leases rather than a tuple
that could omit the account. It is immutable and hashable for use in maps.
"""

account_id: str
delivery_config_id: str
delivery_config_version: int


@dataclass(frozen=True)
class ReportingConfiguration:
"""One accepted reporting configuration generation.
Expand Down Expand Up @@ -320,8 +335,12 @@ class ReportingConfiguration:
authoritative_party: Literal["seller", "consumer"] = "seller"

@property
def generation_key(self) -> tuple[str, int]:
return (self.delivery_config_id, self.delivery_config_version)
def generation_key(self) -> ReportingConfigurationGenerationKey:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST FIX: ReportingConfiguration.generation_key changes its public return type from tuple[str, int] to ReportingConfigurationGenerationKey. That is a breaking type-signature change to a public export in the adcp.* namespace — any adopter doing config_id, version = config.generation_key breaks. The commit is fix(reporting): scope configuration generations by account with no ! and no BREAKING CHANGE: footer. release-please reads the prefix, not the diff; the break ships without a major signal. Retag fix!: or add a BREAKING CHANGE: footer. Migration note is present (docs/reporting-ledger-migration.md), so only the semver signal is missing.

Context, not absolution: generation_key was introduced in #1153 within the 8.0.0-beta line and has never shipped in a stable release, so the practical blast radius is beta adopters only — the arbiter can weigh that.

return ReportingConfigurationGenerationKey(
account_id=self.account_id,
delivery_config_id=self.delivery_config_id,
delivery_config_version=self.delivery_config_version,
)


@dataclass(frozen=True)
Expand Down Expand Up @@ -352,6 +371,14 @@ class ReportingObligationRecord:
definition: ReportingDefinitionBinding | None = None
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))

@property
def generation_key(self) -> ReportingConfigurationGenerationKey:
return ReportingConfigurationGenerationKey(
account_id=self.account_id,
delivery_config_id=self.delivery_config_id,
delivery_config_version=self.delivery_config_version,
)

def __post_init__(self) -> None:
if _utc(self.scope_resolved_at) != _utc(self.period.end):
raise ValueError(
Expand Down Expand Up @@ -478,13 +505,24 @@ class ConsumerStatusRecord:
seller_ledger_as_of: datetime | None = None
superseded: bool = False

@property
def generation_key(self) -> ReportingConfigurationGenerationKey:
return ReportingConfigurationGenerationKey(
account_id=self.account_id,
delivery_config_id=self.delivery_config_id,
delivery_config_version=self.delivery_config_version,
)

@property
def chain_key(self) -> tuple[str, str, str, int, str, str, str]:
"""The logical chain this statement belongs to.

Deliberately keyed *without* the seller's obligation id. Requiring it
would make the first missing report invisible again, which is the exact
failure this loop exists to surface.

The flat shape is retained for compatibility with persisted statement
digests. Use ``generation_key`` when joining configuration generations.
"""
return (
self.account_id,
Expand Down
Loading
Loading