Skip to content

test(validation): check context equality over every schema, not thirteen - #153

Merged
simontaurus merged 1 commit into
mainfrom
test/context-equality-whole-corpus
Sep 11, 2026
Merged

test(validation): check context equality over every schema, not thirteen#153
simontaurus merged 1 commit into
mainfrom
test/context-equality-whole-corpus

Conversation

@simontaurus

Copy link
Copy Markdown
Contributor

Follow-up to #150, which pinned the pyld equivalence on the committed examples only.

What was and was not covered

Corpus Schemas with a resolvable @context Was pinned
committed examples 13 yes
committed compliance fixtures 13 no
upstream main tracks, ~26 today no

The untested half is the half that matters. The compliance fixtures exist to hold the unusual constructs, so pinning the equivalence on the examples alone tested it against the documents least likely to break it.

Measured before wiring it in: all three corpora agree, 0 divergences, 0 errors.

Changes

  • the committed-slice test now sweeps tests/data/oold/compliance/ as well, 13 -> 26 schemas
  • a parity test runs the same sweep over OOLD_SCHEMA_DIR, so it tracks main instead of the snapshot the fixture slice was cut from
  • the fixture traversal moved to conftest.embedded_schemas so both use one copy

Why this belongs in parity too

context_resolution's docstring argues from what pyld does not return. A snapshot can only hold that against constructs upstream had when it was taken; a new one that breaks the claim appears on main first, and the parity job is the only thing that looks there.

Verification

  • pytest tests/test_validation: 498 passed, 7 skipped
  • parity with both checkouts: 7 passed
  • pre-commit: clean

The equivalence was pinned on the examples only, which is half the
corpus and the half least likely to break it. The compliance fixtures
carry as many schemas again and exist precisely to hold the unusual
constructs.

Adds the same sweep to the parity suite, so it runs against whatever
oold-schema main has now rather than against the snapshot taken when the
fixture slice was cut - that is where a construct which breaks the claim
would show up first.
@github-actions

Copy link
Copy Markdown
Contributor

Release preview

No version bump from the current commits (stays at v0.20.0). Use conventional commit types (feat, fix, ...) to trigger a release.

Changelog preview (truncated)

Preview via python-semantic-release and conventional commits.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0016s → 0.0016s (+0.2%)
  ➖ test_sqlite_document_store: 0.0017s → 0.0017s (+0.6%)
  ➖ test_local_sparql_store: 0.0367s → 0.0363s (-1.0%)
  ➖ test_oneof_subschema: 0.0554s → 0.0562s (+1.4%)
  ➖ test_enum_docstrings: 0.0471s → 0.0485s (+2.9%)
  ➖ test_subclass_inheritance: 0.0503s → 0.0535s (+6.4%)
  ➖ test_class_hierarchy: 0.0476s → 0.0481s (+1.1%)
  ➖ test_core[v1]: 0.0354s → 0.0362s (+2.3%)
  ➖ test_core[v2]: 0.0407s → 0.0420s (+3.1%)
  ➖ test_schema_generation[v1]: 0.0015s → 0.0015s (+1.8%)
  ➖ test_schema_generation[v2]: 0.0026s → 0.0026s (+1.0%)
  ➖ test_simple_json: 0.0006s → 0.0006s (+3.4%)
  ➖ test_complex_graph: 0.0014s → 0.0015s (+0.8%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@simontaurus
simontaurus merged commit d47f39e into main Sep 11, 2026
21 checks passed
@simontaurus
simontaurus deleted the test/context-equality-whole-corpus branch September 11, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant