feat(audit): upstream-authored rules are unscored, not just undated (0.1.0-alpha.18) - #48
Conversation
Answering the ask: exempt from rule-quality scoring too, but scoped per check rather than removed from the scoring surface — and the signal is stated, not dropped. postiz-app: CRITICAL -> NEEDS WORK. openclaw stays NEEDS WORK. arbor and every repo without an upstream remote are untouched, because the check never fires there. Two parts. Low-Yield now exempts upstream content alongside pointers. A finding about the quality of a vendor's documentation is not actionable: acting on it means rewriting someone else's docs to satisfy our audit. Same justification as the date exemption, so the same treatment. Scoped to Low-Yield DELIBERATELY, not to the whole scoring surface. Upstream's AGENTS.md really is loaded into our sessions, so it still counts toward Context Load Pressure, redundancy and conflict — postiz still reports 187 always-on lines. Dropping it from everything would understate load we actually pay, which is the mistake the pointer exclusion in #41 nearly made. The exclusion is named in the detail rather than silent — "0/0 scoring files miss Why or Examples (excludes 2 pointer/upstream docs)" — so a reader still learns that upstream ships rules we judged low-yield, without a blocking finding we have no standing to act on. That is the informational-but-unscored option, reached by reusing the mechanism already there. The rename is also fixed, and n=1 does not argue against it the way it argued against the basename discriminator. That check was a PROXY for renames and could collide; this one is content identity. Git's blob hash IS the content, so a file whose blob appears anywhere in the upstream tree contains zero bytes of ours no matter what it is called, and one edited character diverges the hash and drops the exemption. The bound comes from construction rather than sample size. Verified on the real case: HEAD:AGENTS.md and upstream/main:CLAUDE.md are both blob 2704017990140945f94d2ab68ef641f2a0fdb8a8. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 46 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (12)
Comment |
1 participant
Yes to excluding from rule-quality scoring — and the two options you offered turn out to be the same thing, because the mechanism already states its exclusions.
postiz-app: CRITICAL → NEEDS WORK. openclaw stays NEEDS WORK. arbor and every repo without an upstream remote are untouched.
The answer, with one correction to the framing
Low-Yield now exempts upstream content alongside pointers. A finding about the quality of a vendor's docs isn't actionable: acting on it means rewriting someone else's documentation to satisfy our audit.
But scoped to Low-Yield, not to the whole scoring surface. Upstream's
AGENTS.mdreally is loaded into our sessions, so it still counts toward Context Load Pressure, redundancy and conflict — postiz still reports 187 always-on lines. "Exclude from scoring" as a blanket principle would understate load we actually pay; that's the mistake the pointer exclusion in #41 nearly made, and it's the same per-check reasoning.On informational-vs-excluded: the detail string already names exclusions, so you get both —
0/0 scoring files miss Why or Examples (excludes 2 pointer/upstream docs). A reader still learns upstream ships rules we judged low-yield, without a blocking finding.The rename: n=1 doesn't argue against this one
Your filter was right to kill the basename discriminator, but that check was a proxy for renames and could collide. This is content identity.
Git's blob hash is the content. A file whose blob appears anywhere in the upstream tree contains zero bytes of ours, whatever it's called — and one edited character diverges the hash and drops the exemption. The bound comes from construction, not from sample size. That's the distinction, and it's why n=1 doesn't apply the same way.
Verified on the real case:
HEAD:AGENTS.mdandupstream/main:CLAUDE.mdare both blob2704017990140945f94d2ab68ef641f2a0fdb8a8.Five tests cover it, including the two that bound it: editing one character drops the exemption, and a file we authored is never exempt.
Verify
lint · format:check · release:pin --check · typecheck · 528 pass / 0 fail · verify:self-audit-proof.
🤖 Generated with Claude Code