Skip to content

githooks: validate-a2ml admits 0 of 222 files and validate-spdx rejects all 1,046 .adoc — neither was run against its own corpus #791

Description

@hyperpolymath

Summary

Two .githooks validators police populations whose file format they do not describe. Measured on main this session, validate-a2ml.sh admits 0 of 222 files and validate-spdx.sh rejects every one of the 1,046 tracked .adoc files, 231 of which carry a correct SPDX header.

The single root cause is not a typo in either script. It is that neither validator was ever run against the corpus it polices. Each encodes an imagined dialect: YAML key: fields, and a # comment prefix. The actual corpus is S-expression, TOML-ish and AsciiDoc.

The practical consequence: standards is a documentation repo that cannot commit documentation. .adoc is 1,046 of 2,610 tracked files, and the pre-commit hook rejects all of them.

Symptom 1 — validate-a2ml.sh admits nothing (0 of 222)

The validator asserts three predicates. Measured against all 222 tracked .a2ml:

Predicate Passes
^(agent-id|pedigree): 0 of 222
^version: 3 of 222
^# SPDX-License-Identifier: (first 5 lines) 178 of 222

The first two demand YAML. .a2ml is written in two other dialects, neither of them YAML:

  • S-expression; comments, (version "1.0"). 13 files carry ; SPDX. Example: toolchain-readiness-grades/TOOLCHAIN-READINESS-GRADES.a2ml.
  • TOML-ish# comments, version = "...". 134 files match ^\s*version\s*=.

So version is present and correct in most of the corpus; the validator simply looks for the wrong punctuation. agent-id/pedigree appear in no file in either dialect.

Note .claude/CLAUDE.md describes A2ML as "TOML-like format", which matches the 134 but not the 13, and matches the validator not at all.

Symptom 2 — validate-spdx.sh's two modes ask different questions

if [ -n "$STAGED_FILES" ]; then
  FILES_TO_CHECK=$STAGED_FILES          # no extension filter at all
else
  FILES_TO_CHECK=$(find ... -name '*.rs' -o ... )   # 17-extension allowlist
fi

Scan mode filters to 17 source extensions. Staged mode applies no filter, so it demands ^# SPDX on every staged path regardless of type. Only the staged branch is reachable in practice, because this validator is invoked from .githooks/pre-commit and .githooks/pre-push and from no workflow at all.

Result: the pre-commit hook rejects files the whole-repo scan would never examine. For .adoc, which comments with //:

  • 0 of 1,046 tracked .adoc pass
  • 231 of those carry a correct // SPDX-License-Identifier: header
  • staged-mode pass rate across all tracked files: 687 of 2,610

Already being fixed, independently. A concurrent session has a staged (uncommitted) change to .githooks/validate-spdx.sh in the main clone that hoists a single is_source_file() predicate used by both modes, with a comment diagnosing this exact asymmetry. Its reported casualty was .github/workflows/actions.lock (generated by gh actions-lock, "Do not edit by hand", restamped on every regeneration, so a hand-added header does not survive — cf. #746). This issue records the .adoc symptom of the same defect and should not be used to duplicate that work. Symptom 2 is theirs; symptoms 1 and 3 are not covered by it.

Symptom 3 — the scan-mode allowlist is itself partly unsatisfiable

Independent of the mode split. The allowlist names 17 extensions and the predicate is ^# SPDX-License-Identifier: — a # comment prefix. Measured per extension:

Verdict Extensions Files Pass
Cannot ever pass .rs .js .zig .ex .ml .adb .ads .json 128 0
Legitimately #-commented .sh .ncl .toml .yaml .yml 445 372

.rs/.js/.zig comment with //; .ex with # but the 6 present do not carry headers; .ml with (* *); .adb/.ads with --. .json admits no comments at all — 56 files that can never satisfy the predicate under any authoring discipline.

So roughly a fifth of the validator's own declared scope is unsatisfiable by construction. Fixing symptom 2 alone will not surface this, because the surviving filter keeps these extensions in scope.

Evidence that this has never gated anything

Since 42599e7 ("feat(hooks): comprehensive CI/CD hooks implementation"), 88 commits have landed on main15 touching .adoc and 7 touching .a2ml. The hooks have never once admitted a change of either kind.

This is not evidence of deliberate bypass. core.hooksPath is per-clone config and is never committed, so a clone that has not run .githooks/install.sh has no hooks installed at all.

CI is unaffected. git grep -n 'validate-spdx\|validate-a2ml' -- .github/ returns nothing. The only .github/ reference to .githooks is propagate-hooks.yml, which copies the directory to other repos and never executes a validator. So these are local-only gates, and repairing them cannot turn CI red — but note that propagate-hooks.yml means both defects propagate estate-wide.

Suggested repair

Each validator should be run against its own corpus as part of landing the fix, with the pass rate recorded — that is the check that was missing in the first place.

  1. validate-a2ml.sh — accept all three field dialects (key:, key = value, (key "value")) and all three comment prefixes (#, ;, //), or narrow the validator to whichever dialect is normative and say so. agent-id/pedigree should be dropped or made advisory until some file actually carries them.
  2. validate-spdx.sh — the concurrent session's is_source_file() unification lands symptom 2. Symptom 3 additionally needs a per-extension comment prefix (#, //, --, (* *)) and the removal of .json, which cannot carry a comment.
  3. Anti-vacuity control. A validator that admits nothing discriminates nothing, exactly as a validator that admits everything does. Whichever way these are repaired, the fix should ship with a recorded run against a known-bad input that must trip it and a known-good input that must not.

Not proposed here: any bulk SPDX header sweep. 815 of the 1,046 .adoc files carry no SPDX header. That is a genuine finding and it is flag-only under the licence-policy guardrail in .claude/CLAUDE.md ("NEVER sweep SPDX headers in bulk"; "NEVER generate licence-change PRs without prior owner approval"). Repairing the validator and populating headers are separate decisions, and the second is the owner's.

How this was found

While landing the Testing-Taxonomy Categories 17/18 ratification, which touches four .adoc files and one .a2ml. That commit required --no-verify, disclosed in its own message, having first run all eight validators individually to confirm only these two fail: validate-k9, validate-spdx-workflows, validate-sha-pins, validate-permissions, validate-codeql and validate-bot-directives all return 0 on the same file set.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationDocs, prose, diagrams, READMEs, ADRslicensingLicences, SPDX headers, REUSE compliance, attribution

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions