feat(models): modular domain structure and elimination of legacy postprocessors

[gsmith85]

Summary

Companion PR to Universal-Commerce-Protocol/ucp-schema#73 and Universal-Commerce-Protocol/ucp#817.

This PR transitions the Python SDK models to be generated directly from the canonical OpenAPI 3.1 specification (shopping.openapi.json) emitted by ucp-schema export-openapi. This change removes both the schema preprocessing script (preprocess_schemas.py) and the model postprocessing script (postprocess_models.py), replacing the legacy 227-file schema directory with a consolidated model engine and modular re-export packages.

Overall diff: +11,280 lines, -21,663 lines (net reduction of 10,383 lines of code).

---

Why the Preprocessor and Postprocessor Were Removed Concurrently

While splitting the removal of preprocess_schemas.py and postprocess_models.py into separate PRs was considered, the two tools were tightly coupled by design:

1. File-Structure Coupling: postprocess_models.py was implemented around AST traversal of individual per-schema files across src/ucp_sdk/models/schemas/{common,shopping}/. It looked for specific file paths (such as totals.py and time_interval.py) to inject custom @model_validator methods.
2. Circular Reference Invariants in OpenAPI: The canonical OpenAPI 3.1 schema includes mutual references (for example, ValueConstraint and ConstraintExpression). Generating separate files per schema with datamodel-code-generator triggers Pydantic v2 circular reference exceptions (PydanticUserError). Resolving these references reliably requires topological ordering within a consolidated module (models.py).
3. Prevention of Phantom Pipelines: Because postprocess_models.py could not operate on the consolidated models.py without a substantial rewrite of its AST parser, keeping it in an intermediate state would have required maintaining the 227 legacy files and running the postprocessor solely against unreferenced code.

Consequently, removing both scripts together and introducing lightweight modular packages avoids maintaining an artificial intermediate state while aligning with the strategy to decouple wire DTOs from full JSON Schema semantic evaluation.

---

Key Changes

1. Elimination of Legacy Scripts and Files:

   • Removed preprocess_schemas.py (-834 LOC): Schema composition, reference rewriting, $defs hoisting, and polymorphic lowering are handled upstream by ucp-schema.
   • Removed postprocess_models.py (-1,965 LOC): Manual AST modification and validator injection are eliminated.
   • Removed 227 legacy generated files (-15,530 LOC) previously located in src/ucp_sdk/models/schemas/{common,shopping}/.

2. Consolidated Topological Engine (models.py):

   • Models are generated directly from shopping.openapi.json (244 KB) into src/ucp_sdk/models/schemas/models.py.
   • Uses native datamodel-code-generator configuration (--use-one-literal-as-default, --use-type-alias, --use-default) with no regex post-filtering.

3. Modular Domain Packages:

   • Synthesizes modular domain modules (shopping/checkout.py, shopping/cart.py, shopping/types/fulfillment_destination.py, common/types/totals.py, etc.) that re-export classes from models.py with explicit __all__.
   • Preserves standard import paths and IDE autocompletion:

     from ucp_sdk.models.schemas.shopping.checkout import Checkout, CheckoutCreateRequest
     from ucp_sdk.models.schemas.shopping.types.fulfillment_destination import (
         FulfillmentDestination,
         ShippingDestination,
     )
     from ucp_sdk.models import Checkout as RootCheckout

4. Updated Test Suite (tests/test_models.py):

   • Replaced tests/test_codegen_pipeline.py (which tested internal helpers of the deleted preprocessor) with unit tests in tests/test_models.py that directly validate the generated models:
     • Discriminated union validation (FulfillmentDestination dynamically parsing ShippingDestination and LocationDestination).
     • Directional request models (CheckoutCreateRequest omitting read-only server fields).
     • Round-trip serialization and deserialization of Checkout, Cart, and Order.
     • Model validation behavior under invalid input.
     • Import parity between modular domain paths and top-level exports.

---

Verification

# Generate models and domain packages
./generate_models.sh

# Run unit tests (20 tests passed, 0 skipped, 0 failures)
uv run python -m unittest discover -s tests -v

# Code formatting and lint checks
uv run ruff check src tests
uv run ruff format --check src tests