feat(validation): add oold meta vendor command

docs/maintaining-meta-schemas.md

@@ -48,7 +48,7 @@ meta-schema file, so they are not ordinary source files:
 - they must be LF. A CRLF copy hashes differently, which passes on Windows and fails on Linux.
   This has happened; `test_the_vendored_files_are_stored_with_unix_line_endings` now guards it.
 
-That is why every extraction command below uses `git cat-file blob`, never `git show`: `show`
+That is why `oold meta vendor` (below) extracts with `git cat-file blob`, never `git show`: `show`
 applies the checkout's end-of-line conversion, so on Windows it writes CRLF, which changes every
 digest and fails only once it reaches Linux CI.
 
@@ -72,55 +72,43 @@ skip - drift here is exactly what this is meant to catch.
 
 When oold-schema cuts a release, from a checkout of it:
 
-### 1. Vendor the meta-schema files
-
 ```bash
-V=1.0.0
-mkdir -p src/oold/validation/meta/$V
-for f in oold-meta-schema oold-meta-schema-base oold-pattern-lint.schema oold-ui-meta-schema oold-rules oold-rules.schema; do
-  git -C ../oold-schema cat-file blob v$V:meta/$f.json > src/oold/validation/meta/$V/$f.json
-done
-sha256sum src/oold/validation/meta/$V/*.json
-git -C ../oold-schema rev-parse v$V
-git -C ../oold-schema log -1 --format=%cI v$V
+uv run oold meta vendor 1.0.0 --from ../oold-schema
 ```
 
-**Check what the release actually ships before running the loop.** The set has grown twice.
-`oold-rules.json`, the catalogue of normative statements, and `oold-rules.schema.json`, which
-describes it, arrived in 1.0.0-rc.1; `oold-meta-schema-base.json` arrived in 1.0.0-rc.2, when the
-dialect split into a wrapper and the body it `$ref`s. Drop from the loop whatever a given version
-predates, and name the set in that version's own `files` entry when it differs from the shared
-default. Listing only the three meta-schemas here once cost a vendoring the catalogue entirely,
-which is silent: findings simply stop citing rules and every `rule.*` check skips as though the
-version had stated nothing. Omitting the base is not silent, but it fails obscurely, as an
-unresolvable `$ref` rather than a missing file.
-
-Extract from the **tag**, not from the working tree. The two diverge: at the time 0.7.0 was added,
-`main` had already changed all three files, including the canonical `$id` domain.
-
-The catalogue is the one exception, and only while it is unreleased. `1.0.0-rc.1`'s copy comes from
-an oold-schema branch because no tag carries one yet; when that happens, record the branch and
-commit under `rules_source` so the provenance is still exact. Never do this for a meta-schema.
-
-Then add an entry to `index.json` with the tag, commit, commit date, the `$id` base in use for that
-release (see "Why `id_base` is recorded and not assumed" below), and the checksums.
-
-### 2. Refresh the fixture slice
-
-Refresh `tests/data/oold/` from the **same tag**, so fixtures and meta-schemas always come from
-one release, then record that tag as `fixtures.tag` in `index.json`:
-
-```bash
-V=$(uv run python -c "from oold.validation.meta_store import latest_version; print(latest_version())")
-DEST=tests/data/oold
-for f in $(git -C ../oold-schema ls-tree --name-only v$V examples/ | grep '\.json$'); do
-  git -C ../oold-schema cat-file blob "v$V:$f" > "$DEST/$(basename $f)"
-done
-for f in $(git -C ../oold-schema ls-tree --name-only v$V examples/compliance/); do
-  git -C ../oold-schema cat-file blob "v$V:$f" > "$DEST/compliance/$(basename $f)"
-done
-make validate
-```
+This resolves the tag `v1.0.0` in that checkout and does, in one call, what used to be two
+hand-run procedures:
+
+- reads what `meta/` actually contains **at that tag**, rather than a fixed list someone has to
+  remember to edit. The set has grown twice already - `oold-rules.json`, the catalogue of
+  normative statements, and `oold-rules.schema.json`, which describes it, arrived in 1.0.0-rc.1;
+  `oold-meta-schema-base.json` arrived in 1.0.0-rc.2, when the dialect split into a wrapper and the
+  body it `$ref`s - and a version that predates one of these is simply not made to load a file it
+  does not ship. Listing only the three meta-schemas here once cost a vendoring the catalogue
+  entirely, which is silent: findings stop citing rules and every `rule.*` check skips as though
+  the version had stated nothing. Omitting the base is not silent, but it fails obscurely, as an
+  unresolvable `$ref` rather than a missing file;
+- writes every file with `git cat-file blob`, never `git show`, so nothing here can pick up the
+  checkout's line-ending conversion (see "Byte-exactness" above);
+- records the tag, commit, commit date, the `$id` base declared in the vendored wrapper (see "Why
+  `id_base` is recorded and not assumed" below), and a sha256 of each file, in `index.json`;
+- refreshes `tests/data/oold/` from the **same tag** and sets `fixtures.tag` to it, so fixtures and
+  meta-schemas can never drift apart the way a separate, easy-to-skip second step once let them.
+  Keeping the two in step is not cosmetic: a compliance fixture asserts the lint rules of the
+  release that introduced them, so a newer fixture set combined with an older meta-schema fails in
+  ways that say nothing about the code. `test_the_fixture_slice_records_the_release_it_came_from`
+  is what would have caught the earlier miss - a README claiming a release the fixture slice had
+  already moved past.
+
+It refuses to overwrite a version already tracked; pass `--force` to replace one deliberately.
+Optional narrative fields on an entry - `notes`, `prerelease` - are not generated and can be added
+by hand afterward.
+
+The rule catalogue is the one thing this command does not vendor from an unreleased source.
+`1.0.0-rc.1`'s copy comes from an oold-schema branch because no tag carried one yet; when that
+happens, add the catalogue and a `rules_source` entry by hand, recording the branch and commit so
+the provenance stays exact. Never do this for a meta-schema - a document that has not reached a tag
+has not been released.
 
 Then confirm both refreshes still pass:
 
@@ -129,21 +117,13 @@ uv run oold validate tests/data/oold --offline --meta all
 make validate && uv run pytest tests/test_validation -q
 ```
 
-Keeping the two in step is not cosmetic. A compliance fixture asserts the lint rules of the release
-that introduced them, so a newer fixture set combined with an older meta-schema fails in ways that
-say nothing about the code. `fixtures.tag` is what makes the pairing checkable rather than a habit:
-`test_the_fixture_slice_records_the_release_it_came_from` compares it against the newest tracked
-version, because this step has been skipped before and prose describing the tag in a README did
-not notice - the sentence kept naming an old release after a vendoring had already moved the
-fixture files on. The tag is recorded only in `index.json` now, for exactly that reason.
-
 ## The fixture slice
 
 Only the top level and `compliance/` are the upstream snapshot; both come from `examples/` at the
 recorded tag. `broken/`, `remote_context/` and `x_oold_context/` are written here by hand, exist
-in no oold-schema release, and the refresh loop above never touches them. Upstream's `examples/`
-also has a `spec/` subdirectory, which is deliberately outside the slice - the loops above do not
-descend into it.
+in no oold-schema release, and `oold meta vendor` never touches them. Upstream's `examples/` also
+has a `spec/` subdirectory, which is deliberately outside the slice - the command does not descend
+into it.
 
 Upstream's current `main` is covered instead by the opt-in parity tests
 (`tests/test_validation/test_parity_live.py`), which validate against `--meta remote`.

src/oold/validation/cli.py

@@ -19,6 +19,7 @@
 import click
 
 from .meta_store import MetaSchemaError, describe_store, fetch_remote, load_index, resolve_selection
+from .meta_vendor import vendor_version
 from .pipeline import Options, run_compliance, validate_directory, validate_instance, validate_schema
 from .report import FAIL, OK, SKIP, WARN, Report
 
@@ -319,6 +320,33 @@ def meta_fetch(force: bool) -> None:
     click.echo(f"fetched into {target}")
 
 
+@meta_group.command("vendor")
+@click.argument("version")
+@click.option(
+    "--from",
+    "source",
+    required=True,
+    type=click.Path(exists=True, file_okay=False, path_type=Path),
+    help="A checkout of oold-schema to vendor the release from.",
+)
+@click.option("--force", is_flag=True, help="Overwrite an already-tracked version.")
+def meta_vendor(version: str, source: Path, force: bool) -> None:
+    """Vendor one released meta-schema version, and refresh the fixture slice from the same tag.
+
+    Reads the file set the tag actually ships rather than a fixed list, writes every file
+    byte-for-byte with no line-ending conversion, and records its sha256, tag, commit and commit
+    date in meta/index.json. See docs/maintaining-meta-schemas.md for what this replaces.
+    """
+    try:
+        result = vendor_version(version, source, force=force)
+    except MetaSchemaError as exc:
+        raise click.ClickException(str(exc)) from exc
+    click.echo(f"vendored {result.version} from {result.tag} ({result.commit})")
+    for name in result.files:
+        click.echo(f"  {name}")
+    click.echo(f"fixtures refreshed from {result.tag}: {len(result.fixture_files)} files")
+
+
 @click.group("rules")
 def rules_group() -> None:
     """Look up the normative rules the validator cites."""

src/oold/validation/meta_vendor.py

@@ -0,0 +1,219 @@
+"""Vendor a released meta-schema version from an oold-schema checkout.
+
+``meta_store.py`` only ever reads the tracked tree; this module is the one place that writes it,
+and only in response to an explicit ``oold meta vendor`` invocation - never at validation time.
+
+It exists to remove, by construction, three mistakes the hand-run procedure in
+``docs/maintaining-meta-schemas.md`` could only warn about:
+
+- ``git show`` instead of ``git cat-file blob``, which applies the checkout's autocrlf filter and
+  writes CRLF on Windows, changing every recorded digest and failing only once it reaches Linux
+  CI. Every file below is read with ``git cat-file blob`` and written with :meth:`Path.write_bytes`,
+  so nothing in the path ever re-encodes a line ending.
+- copying the wrong file set. The set has grown twice already (the rule catalogue in 1.0.0-rc.1,
+  the meta-schema base in 1.0.0-rc.2), so this reads what the tag's ``meta/`` directory actually
+  contains rather than a fixed list a human has to remember to edit.
+- updating the vendored files but not ``fixtures.tag``. The two writes happen in one call, from
+  one resolved tag, so they cannot drift apart the way a two-step manual procedure did.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import subprocess
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+from . import meta_store
+
+#: Meta-schema documents, in the order they get committed with. Not every version ships every one
+#: - the base split off in 1.0.0-rc.2 - so which of these a version records is decided by what its
+#: tag actually contains, never assumed.
+_CANDIDATE_DOCUMENTS = (
+    meta_store.META_SCHEMA_FILE,
+    meta_store.META_SCHEMA_BASE_FILE,
+    meta_store.PATTERN_LINT_FILE,
+    meta_store.UI_META_SCHEMA_FILE,
+)
+
+#: The rule catalogue and its schema, optional and never part of a version's ``files`` override:
+#: unlike the documents above they are not loaded through the registry, so meta_files() has no
+#: reason to know about them.
+_CANDIDATE_RULE_FILES = (meta_store.RULES_FILE, meta_store.RULES_SCHEMA_FILE)
+
+
+@dataclass
+class VendorResult:
+    """What one ``vendor_version`` call did, for the CLI to report."""
+
+    version: str
+    tag: str
+    commit: str
+    committed: str
+    files: list[str]
+    fixture_files: list[str]
+
+
+def _fixtures_dir() -> Path:
+    """``tests/data/oold/`` in this checkout, located from this file rather than the CWD.
+
+    A separate lookup from :func:`meta_store.meta_dir`, because this directory ships only in a
+    source checkout, never in the installed package - the same reason the fixture refresh half of
+    this module is a repository-maintenance operation rather than something ``oold`` needs at
+    runtime.
+    """
+    # src/oold/validation/meta_vendor.py -> repository root is three parents up.
+    return Path(__file__).resolve().parents[3] / "tests" / "data" / "oold"
+
+
+def _git(source: Path, *args: str) -> str:
+    """Run a git command in ``source`` and return its stdout as text, stripped."""
+    # S603/S607: "git" is not resolved from an untrusted PATH here - it is the same interpreter
+    # this whole toolchain already depends on, and args are fixed revision/path literals, never
+    # user-supplied shell text.
+    result = subprocess.run(["git", "-C", str(source), *args], capture_output=True)  # noqa: S603, S607
+    if result.returncode != 0:
+        raise meta_store.MetaSchemaError(
+            f"git {' '.join(args)} failed in {source}: {result.stderr.decode(errors='replace').strip()}"
+        )
+    return result.stdout.decode(errors="replace").strip()
+
+
+def _git_blob(source: Path, rev: str) -> bytes:
+    """The verbatim bytes of one blob, bypassing any working-tree line-ending conversion.
+
+    ``git cat-file blob``, never ``git show``: ``show`` applies the checkout's autocrlf filter,
+    which on Windows turns LF into CRLF and changes every digest computed from the result. The
+    bytes are captured directly from the subprocess pipe and written with :meth:`Path.write_bytes`
+    - no text-mode decoding happens anywhere between the object database and the file on disk.
+    """
+    # S603/S607: see _git above.
+    result = subprocess.run(["git", "-C", str(source), "cat-file", "blob", rev], capture_output=True)  # noqa: S603, S607
+    if result.returncode != 0:
+        raise meta_store.MetaSchemaError(
+            f"git cat-file blob {rev} failed in {source}: {result.stderr.decode(errors='replace').strip()}"
+        )
+    return result.stdout
+
+
+def _ls_tree(source: Path, rev: str, directory: str) -> list[str]:
+    """File names (not full paths) one level under ``directory`` at ``rev``."""
+    listing = _git(source, "ls-tree", "--name-only", rev, directory)
+    return [Path(line).name for line in listing.splitlines() if line]
+
+
+def _id_base(document: dict[str, Any], filename: str) -> str:
+    """The ``$id`` base this release publishes under, derived from its own wrapper document.
+
+    Recorded per version rather than assumed, because the canonical domain has already moved once
+    (see :func:`meta_store._build_registry`); reading it from the file itself means a future move
+    needs only a new vendored entry, never a code change here.
+    """
+    declared = document.get("$id")
+    if not isinstance(declared, str) or not declared.endswith(filename):
+        raise meta_store.MetaSchemaError(f"{filename}'s $id does not end with its own file name: {declared!r}")
+    return declared[: -len(filename)]
+
+
+def _refresh_fixtures(source: Path, tag: str) -> list[str]:
+    """Copy the fixture slice from ``examples/`` at ``tag``, mirroring the documented procedure.
+
+    Only the top level and ``compliance/`` are upstream; ``broken/``, ``remote_context/`` and
+    ``x_oold_context/`` are written here by hand and this never touches them, because it never
+    looks anywhere but those two source directories.
+    """
+    dest = _fixtures_dir()
+    written: list[str] = []
+
+    for name in _ls_tree(source, tag, "examples/"):
+        if not name.endswith(".json"):
+            continue
+        (dest / name).write_bytes(_git_blob(source, f"{tag}:examples/{name}"))
+        written.append(name)
+
+    dest_compliance = dest / "compliance"
+    for name in _ls_tree(source, tag, "examples/compliance/"):
+        (dest_compliance / name).write_bytes(_git_blob(source, f"{tag}:examples/compliance/{name}"))
+        written.append(f"compliance/{name}")
+
+    return written
+
+
+def vendor_version(version: str, source: Path, *, force: bool = False) -> VendorResult:
+    """Vendor one released meta-schema version from ``source``, and refresh the fixture slice.
+
+    ``source`` is a checkout of oold-schema; ``version`` names the release, whose tag is assumed
+    to be ``v<version>`` - the convention every tracked entry in ``index.json`` already follows.
+    Refuses to overwrite a version already present in the tracked folder or the index unless
+    ``force`` is set, so a typo'd version cannot silently discard a curated entry.
+    """
+    index = meta_store.load_index()
+    target_dir = meta_store.meta_dir() / version
+    already_tracked = version in index.get("versions", {}) or target_dir.is_dir()
+    if already_tracked and not force:
+        raise meta_store.MetaSchemaError(
+            f"meta-schema version {version!r} is already tracked; pass --force to overwrite it"
+        )
+
+    tag = f"v{version}"
+    commit = _git(source, "rev-parse", f"{tag}^{{commit}}")
+    committed = _git(source, "log", "-1", "--format=%cI", commit)
+
+    present = set(_ls_tree(source, tag, "meta/"))
+    documents = [name for name in _CANDIDATE_DOCUMENTS if name in present]
+    if meta_store.META_SCHEMA_FILE not in documents:
+        raise meta_store.MetaSchemaError(f"{tag} carries no {meta_store.META_SCHEMA_FILE} under meta/ in {source}")
+    rule_files = [name for name in _CANDIDATE_RULE_FILES if name in present]
+    all_files = documents + rule_files
+
+    target_dir.mkdir(parents=True, exist_ok=True)
+    sha256: dict[str, str] = {}
+    for name in all_files:
+        content = _git_blob(source, f"{tag}:meta/{name}")
+        (target_dir / name).write_bytes(content)
+        sha256[name] = hashlib.sha256(content).hexdigest()
+
+    wrapper = json.loads((target_dir / meta_store.META_SCHEMA_FILE).read_bytes())
+    id_base = _id_base(wrapper, meta_store.META_SCHEMA_FILE)
+
+    entry: dict[str, Any] = {
+        "tag": tag,
+        "commit": commit,
+        "committed": committed,
+        "added": datetime.now(timezone.utc).date().isoformat(),
+        "id_base": id_base,
+        "sha256": sha256,
+    }
+    # Mirrors the fallback in meta_store.meta_files(): the three files every version predating the
+    # 1.0.0-rc.2 split ships, and the default this version's own set is compared against below.
+    default_documents = index.get("files") or [
+        meta_store.META_SCHEMA_FILE,
+        meta_store.PATTERN_LINT_FILE,
+        meta_store.UI_META_SCHEMA_FILE,
+    ]
+    if documents != default_documents:
+        entry["files"] = documents
+
+    index.setdefault("versions", {})[version] = entry
+
+    fixture_files = _refresh_fixtures(source, tag)
+    index.setdefault("fixtures", {})["tag"] = tag
+
+    index_path = meta_store.meta_dir() / "index.json"
+    # newline="\n": write_text defaults to translating "\n" to os.linesep, which on Windows would
+    # author this file with CRLF - the same trap the vendored files avoid by being written with
+    # write_bytes, just reappearing in the one file this command writes as text.
+    index_path.write_text(json.dumps(index, indent=2) + "\n", encoding="utf-8", newline="\n")
+    meta_store.load_index.cache_clear()
+
+    return VendorResult(
+        version=version,
+        tag=tag,
+        commit=commit,
+        committed=committed,
+        files=all_files,
+        fixture_files=fixture_files,
+    )