Skip to content

Pin-consumption census TSV has TWO extractor defects — 145 of 1,434 rows carry a non-SHA in pin_sha, and every background rate quoted off it is contaminated #792

Description

@hyperpolymath

Summary

The pin-consumption census table developer/.claude/checkpoints/22-pin-consumption.tsv (1,434 data rows, produced 2026-09-09 by 22-pin-consumption.sh) has a contaminated pin_sha column, and any background rate computed from the raw column is wrong. The extractor script was fixed, but the table on disk was never regenerated — and it is the table other work has been quoting.

Re-measured 2026-09-14T21:5xZ. There are TWO extractor defects, not one. The second has not been recorded anywhere before now.

Measured split of the 1,434 rows

Class Rows % Cause
CLEAN — 40-hex SHA and a hyperpolymath/standards reusable pin_path 1,289 89.9%
Third-party action captured instead of the reusable 75 5.2% grep -m1 -E '^\s*uses:' takes the first uses: in the caller, not the reusable's
SHA with its inline YAML comment still attached 65 4.5% second defect, previously unrecorded — the field was never split on #
Tag or branch name, not a SHA at all (v3.93.8, main, master, stable, v7.0.1, v2.4.3) 27 1.9% unpinned callers, correctly observed but indistinguishable from a SHA downstream
Sentinels (LOCAL for ../../ relative reusables, - for NO-FILE) 53 3.7% intended, not a defect

Defect 1 — wrong uses: captured. The 75 rows carry a pin_path that is not a standards reusable at all: dtolnay/rust-toolchain 35, trufflesecurity/trufflehog 28, actions/checkout 5, ossf/scorecard-action 4, webfactory/ssh-agent 3.

Defect 2 — the comment came along for the ride. 65 rows hold values like:

efa25f7f19611383d5b0ccf2d1c8914531636bf9 # stable
6c05c4a00b91aa542267d8e32a8254774799d68d # v3
571cc734cd69fb846032ec77a662aa8ee4fc32cd # main 2026-08-04 (lockfile-bearing ref: actions.lock required by caller-side enforcement, standards#570)

This one is nastier than defect 1 because the value looks correct to a human reading the table — the SHA is right there, and it is the right SHA. It only breaks machines: any downstream string-equality join on pin_sha silently misses these rows, and length($4)==40 rejects them. A join written against this column reports a smaller population and prints a confident total, which is this estate's recurring failure shape.

Independent verification that the clean rows are clean

All 31 distinct 40-char SHAs in the column were probed for membership in hyperpolymath/standards, every non-200 folded into FAIL:

145 rows carry a field that is not 40 characters at all.

Why this matters beyond tidiness

The numerator of the killer-pin finding is unaffected — all 58 killer rows carry a genuine hyperpolymath/standards/*-reusable.yml pin_path. What is contaminated is every denominator and background rate quoted off the raw table. Any "N% of rows are dead" figure computed without excluding the 145 anomalous rows is measuring a different population than it claims to.

Requested fix

  1. Regenerate the table with the corrected extractor and publish the new byte count and row count alongside it, so the stale copy cannot be mistaken for the new one.
  2. Split on # in the extractor (defect 2) and assert pin_sha matches ^[0-9a-f]{40}$ or a declared sentinel — anything else fails the row loudly rather than writing an ambiguous value.
  3. Generator-level rule: the extractor must select the uses: whose value matches the reusable being censused, not the first uses: in the file. Taking "the first match" is only ever correct when the file is known to hold one.
  4. Any consumer of this column should assert its clean row count before reporting a rate — asserting a non-zero population is already estate doctrine, and 89.9% is not 100%.

Tracked locally as task #44.


🤖 Generated with Claude Code

https://claude.ai/code/session_01VG5AnnA12E8GikbXZS7NbD

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions