scarecrow: make stale NSFW_CARD_IMAGE URL verdict cleanup reachable - #104

Open

Pitchfork-and-Torch wants to merge 4 commits into

xai-org:mainfrom

Pitchfork-and-Torch:nsfw-card-verdict-cleanup-upstream

Pitchfork-and-Torch commented Sep 4, 2026

Problem

Rule 7429 writes a 7-day NSFW_CARD_IMAGE verdict on near-perfect NSFW card images. Its cleanup branch was meant to delete that verdict when a later score is clean, but it could never run: contradictory guards, inverted age check, and wrong time units.

False-positive card URL labels therefore lasted the full TTL, interstitialing linked posts and dropping them from out-of-network recommendations with no automatic correction.

Change

Make the cleanup branch reachable and arithmetically correct. Leave the write path untouched.

Fork PR: Pitchfork-and-Torch#8

Pitchfork-and-Torch and others added 4 commits

August 14, 2026 07:43


          Meritocratic For You: rank by value, not audience size (#1)

1c231a4

* Add meritocratic author-size IPS to RankingScorer.

* Make For You rank by merit, not reach.

Add size-aware OON relief for small creators, origin-author diversity so
viral originals cannot flood via many retweeters, and mute/block symmetry
for quotes and reposts. Complements author-size IPS; docs in FEED_FAIRNESS.

Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>


          home-mixer: keep VF verdicts separate per safety level in VFCandidate…

34015e9

…Hydrator (#6)

VFCandidateHydrator asks visibility filtering twice per request: once at
TimelineHome for in-network candidates (plus repost sources) and once at
TimelineHomeRecommendations for out-of-network candidates (plus ancestors
and quoted posts). It then merged both answers into one HashMap keyed by
tweet id, with the recommendations map applied last.

A tweet id can be in both sets. The common case is a followed author's
own thread: the root post is an in-network candidate, and the reply in
the same thread lists the root as an ancestor. The same happens whenever
another selected candidate quotes or replies to an in-network post. In
every such case the recommendations verdict overwrote the in-network
verdict, so the in-network post was judged under the rules that are
meant to apply only to recommendations from accounts the viewer does not
follow (SpamHighRecall, NsfwHighRecall, DoNotAmplify, NsfwText,
FosnrAbuseInsults, the NSFW author/tweet flags, DMCA and geo-restricted
media, and the OON-only user labels in
visibility-filtering/rules/registry.rs). VFFilter then removed the post
from the viewer's For You feed even though the viewer follows the author
and README.md states that "the same post is allowed to a follower".

The same collision runs the other way for an out-of-network candidate
that is also the source of a followed account's repost: the merge order
decides which verdict wins, and neither order is right for both cases.

Keep the two result maps separate and route every lookup to the map
matching how the id was requested: a candidate's own verdict comes from
the map for its in_network flag; ancestors and quoted posts read the
recommendations map; repost sources read the in-network map. No VF rule
changes and no extra RPCs.

Tests cover both collision directions, the ancillary routing, tombstoned
ancestors, interstitials, error propagation, and an end-to-end hydrate()
run with a client that answers Allow at TimelineHome and Drop at
TimelineHomeRecommendations. The end-to-end test fails on the previous
code with the root post carrying the recommendations-only drop reason.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>


          abuse-enforcement-service: fail closed when the allowlist store errors (

b4a3553

#7)

The allowlist is an exemption check that runs before any rule can fire.
Until now a Manhattan GET failure during that check was swallowed inside
ManhattanAllowlist::get_entity and returned as None, which the fetch
helpers read as is_allowlisted: false. Enforcement then continued into the
rule pipeline for an account or post that may have been exempt, while the
neighbouring Gizmoduck and credibility fetches on the same path abort with
`?` and are retried.

Make the allowlist lookup behave like those fetches:

- ManhattanAllowlist::get / get_entity return anyhow::Result<Option<_>>.
  Ok(None) means the store confirmed the key is absent. A GET error or an
  undecodable stored entry is returned as Err instead of None.
- fetch_user_allowlist / fetch_entity_allowlist return Result and
  propagate the error. Only a confirmed absence maps to "not allowlisted".
- run_enforcement_inner uses `?` on the allowlist lookups, so a store
  error aborts the attempt and the score lands in the existing retry
  queue (backoff, then dropped without enforcing) rather than proceeding
  to rules.
- Admin handlers: GET /allowlist/{id} and GET /allowlist/{type}/{id}
  return 500 on a read error instead of 404; bulk upsert reports a failed
  pre-read as a per-row error; the DELETE audit snapshot stays best-effort.

Adds unit tests for the lookup-to-facts conversion, including one that
asserts a store error is not turned into is_allowlisted=false.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>


          scarecrow: make stale NSFW_CARD_IMAGE URL verdict cleanup reachable (#8)

866d4cf

Rule 7429 writes an NSFW_CARD_IMAGE verdict (7-day TTL) for a card URL
when its image scores near-perfect NSFW, and has a cleanup branch meant
to delete that verdict when a later score for the same card comes back
clean. The cleanup branch could never run:

- The rule condition required IsNearPerfectNsfw (precision >= 0.999),
  but the cleanup guard required !IsHighPrecisionNsfw (precision <
  0.95). Both cannot hold, so the branch was dead code.
- The age check computed creation - now, which is never positive.
- The threshold 60 * 60 * 1000 was compared against seconds, i.e. about
  41 days, longer than the verdict's own 7-day TTL.

Widen the condition to also admit clean card-image scores, compute the
age as now - creation in seconds, and use a one-hour threshold. Require
a present precision score on the cleanup path so a media update with no
NSFW score cannot clear a verdict. The write path is unchanged.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet