fix(scan,get): rank patches by severity, then merge state, then patch recency - #147
Merged
Merged
Conversation
When a package has more than one available patch, `select_patches` never
consulted severity. For an authorized user (`can_access_paid == true`) the
pick was "newest paid patch, else newest patch":
group.sort_by(|a, b| b.published_at.cmp(&a.published_at));
let choice = group.iter().find(|p| p.tier == "paid").or_else(|| group.first())
so a package whose newest patch fixed a `low` got that one and its `critical`
patch was silently dropped.
The date sort was broken outright besides. `publishedAt` is RFC 2822 /
HTTP-date on the wire (`Fri, 27 Mar 2026 19:12:42 GMT`, verified live across
npm, PyPI, cargo and gem), so a raw `String::cmp` orders by *day-of-week
name* — `Fri` < `Mon` < `Sat` < `Sun` < `Thu` < `Tue` < `Wed`. Every fixture
in this repo uses ISO-8601, which is why it was never caught.
A second, independent chooser — `detect_updates` — took `.first()` off the
raw batch response with no ordering at all, so `updates[].newUuid` could name
a different patch than `--apply` installed.
The new order, best first:
1. merged patches
2. severity: critical > high > medium/moderate > low > unknown
3. patch publish date, most recent first
4. tier (paid), then uuid — tiebreaks only, so the order is total and
output is reproducible
`tier` is now an access filter, not a ranking key: a free `critical` outranks
a paid `low`. This reverses `select_paid_user_prefers_paid_over_free_same_purl`,
which now holds only when everything above tier ties.
Rank 3 is the date *the patch* was published, never the upstream package's
release date. The two are unrelated — axios@1.6.0 shipped 2023-10-26 and
carries patches published 2026-03-27 and 2026-08-03.
Implementation notes:
* `api::ranking` is the one comparator; ordering is normalized at the API
client boundary so the table, `--json` arrays, `get`'s listing, the
interactive prompt and `detect_updates` all inherit it instead of each
re-deriving it.
* `utils::date` parses RFC 2822, RFC 3339 and bare dates to epoch seconds
with no new dependency (reuses the Hinnant calendar algorithm already in
`vex::time`).
* The three duplicated severity ladders now delegate to one.
* `merged` is not emitted by any endpoint yet. It deserializes defensively
(`de_truthy_flag` + aliases) so a `mergedAt` *string* payload cannot
hard-error an entire patch-list response. Confirm the real JSON key and
prune the aliases before this ships.
* Free/unauthorized callers keep the interactive picker and the
`selection_required` JSON error; only the presented order changed — which
is what fixes `scan` for them, since `select_one` auto-selects index 0 in
non-TTY and defaults to it in a TTY.
Known gap, documented in CLI_CONTRACT.md: the batch endpoint omits
`publishedAt` while by-package carries it, so scan's *listing* and apply's
*selection* can differ when top candidates tie on merged AND severity. Only
the reported order is affected, never what lands on disk.
`BatchPatchInfo.published_at` is already wired, so this closes server-side.
Tests: the date rung is mutation-checked — stubbing it out fails 8 tests
across core, cli and the wiremock e2e. Adds a live production canary
(`canary_published_at_is_a_patch_date_not_a_package_date`) that fails if the
API ever switches `publishedAt` to a package-level date, which would silently
disable recency ranking with no error anywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…anks it
Replaces the speculative `merged` API field with an inference, and reorders
the ranking so a merged patch can never shadow a higher-severity fix.
## Merge state is inferable today
The previous commit added a `merged: bool` deserialized from a guessed key
(`merged`/`isMerged`/`mergedAt`/`upstreamMerged`) behind a type-tolerant
deserializer, because no endpoint emits one. That guesswork is unnecessary:
a merged patch is by definition one that folds several fixes into a single
blob, so it NAMES several advisories — which every endpoint already returns.
Merge state is now the count of distinct advisories a patch remediates:
`vulnerabilities` map keys on by-package/view, `ghsaIds` on batch (falling
back to `cveIds` only when no GHSA is named). Advisories are counted, not
CVE ids: one advisory routinely carries several CVE aliases, and counting
those would inflate a single-fix patch into a phantom merged one.
Removes `merged`, `de_truthy_flag` and `is_false` — no speculative API
surface, nothing to confirm before merge, and no server change needed.
Surveyed production 2026-08-05: all 28 patches sampled across
npm/PyPI/gem/cargo cover exactly one advisory, so the rung is inert today
and ranking falls through to recency. Also confirmed the patches for a
package are built against a shared pristine baseline (identical
`beforeHash`) and conflict rather than chain — there is no merged patch to
find yet, not a detection failure.
## Severity now outranks merge state
Order was merged -> severity -> recency; it is now:
1. severity 2. merge state 3. patch recency 4. tier, uuid
The merged patch stays the general preference — breadth is what an operator
wants when only one patch per PURL can be applied — but it must not shadow
a worse vulnerability. Because a patch's severity is the worst advisory it
fixes, putting severity on top expresses that exactly:
| merged | rival | winner | why |
| high | critical | rival | higher severity available |
| critical | high | merged | merged covers the worst |
| high | high | merged | tie -> breadth decides |
## Testing
Both rungs are mutation-checked:
* coverage rung stubbed to a constant -> 9 tests fail across core, cli
and the wiremock e2e;
* coverage promoted above severity (losing the "unless higher severity"
rule) -> 3 tests fail, exactly the ones expressing the exception.
An earlier mutation run also exposed two recency tests passing vacuously
(their UUIDs agreed with the correct answer); they now use adversarial
UUIDs where the tiebreak points the wrong way.
Adds a live production canary,
`canary_patches_name_advisories_so_merge_state_is_inferable`, guarding the
inference signal itself: if production ever stopped populating
`vulnerabilities`, coverage would collapse to 0, the merge rung would go
permanently inert, and selection would fall through to recency with no
error anywhere. It asserts only that the signal exists (>= 1 advisory per
patch), never a count, so publishing a genuinely merged patch does not fail
it — that case is reported informationally instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wenxin Jiang (Wenxin-Jiang)
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Running
scanas a logged-in/authorized user picked the wrong patch when a package had more than one available.select_patches— the single selector bothgetand everyscanmode route through — never consulted severity at all:For
can_access_paid == truethat is "newest paid patch, else newest patch". A package whose newest patch fixed alowgot that one, and itscriticalpatch was silently dropped.Two more things surfaced while verifying against the live API, both hidden by this repo's fixtures:
publishedAtis RFC 2822 / HTTP-date on the wire —Fri, 27 Mar 2026 19:12:42 GMT, confirmed across npm, PyPI, cargo and gem. TheString::cmpabove therefore sorts by day-of-week name (Fri<Mon<Sat<Sun<Thu<Tue<Wed). Every in-repo fixture uses ISO-8601, which is why this was never caught. "Newest first" was not even true.detect_updatestook.first()off the raw batch response with no ordering. Soupdates[].newUuidcould name a different patch than--applyinstalled.The order now
Best first, one comparator (
socket_patch_core::api::ranking) for selection and every listing:tier(paid), thenuuid— tiebreaks only, so the order is total and output is reproducibleMerge state is inferred, not reported — no upstream change needed
There is no
mergedfield on the wire and none is required. A merged patch is by definition one that folds several fixes into a single blob, so it names several advisories, which every endpoint already returns. Merge state is the count of distinct advisories a patch remediates:vulnerabilitiesmap keys on by-package/view,ghsaIdson batch (falling back tocveIdsonly when no GHSA is named).Advisories are counted, not CVE ids — one advisory routinely carries several CVE aliases, and counting those would inflate a single-fix patch into a phantom merged one.
Production survey, 2026-08-05 (63 PURLs queried, 32 with patches, 28 patches total):
So the rung is inert today and ranking falls through to recency. I also confirmed the multiple patches for a package are built against a shared pristine baseline (identical
beforeHash) and conflict rather than chain — e.g. axioslib/adapters/http.jsgoes from5bbe4571…to two different outputs. There is no merged patch to find yet; this is not a detection failure. The moment one is published it is preferred automatically, with no client or server change.Why severity sits above merge state
The merged patch is the general preference — breadth is what an operator wants when only one patch per PURL can be applied — but it must never shadow a worse vulnerability. Because a patch's severity is the worst advisory it fixes, putting severity on the top rung expresses that exactly:
tieris an access filter, not a ranking keyA free
criticaloutranks a paidlow. This deliberately reversesselect_paid_user_prefers_paid_over_free_same_purl, which now holds only when everything above tier ties.Rank 3 is the patch's date
Never the upstream package's release date.
axios@1.6.0shipped to npm 2023-10-26 and carries patches published 2026-03-27 and 2026-08-03.Implementation
api/ranking.rs(new) — the single comparator. Ordering is normalized at the API-client boundary, so the table,--jsonarrays,get's listing, the interactive prompt anddetect_updatesall inherit it rather than each re-deriving it.utils/date.rs(new) — parses RFC 2822, RFC 3339 and bare dates to epoch seconds. No new dependency; reuses the Hinnant calendar algorithm already invex::time, and pins the inverse against it.selection_requiredJSON error. Only the presented order changed — which is what fixesscanfor them too, sinceselect_oneauto-selects index 0 in non-TTY and defaults to it in a TTY.Known gap
The batch endpoint omits
publishedAt; by-package carries it. Ranks 1, 2 and 4 agree across both, so they diverge only when top candidates tie on severity and merge state — there the batch side falls to the UUID tiebreak while apply correctly uses the date.Live:
pkg:npm/axios@1.6.0has two freeHIGHpatches;packages[0].patches[0]reports0bc312a6…(2026-03-27) while--applyinstalls the newer83f5a654…(2026-08-03), the correct choice. Only the reported ordering is affected — never which patch lands on disk. Documented inCLI_CONTRACT.md;BatchPatchInfo.published_atis already wired with#[serde(default)], so it closes once the batch endpoint emits it.Testing
3891 pass, 0 fail. CI clippy (
--workspace --all-features -D warnings) clean; changed files rustfmt-clean.Both new rungs are mutation-checked:
That exercise also caught two of my own recency tests passing vacuously — their UUIDs happened to agree with the correct answer, so the tiebreak was silently supplying the result. They now use adversarial UUIDs where the tiebreak points the wrong way.
Coverage highlights:
scanpicks with multiple candidates — every prior scan fixture mocked exactly one patch per package.merged_patch_wins_when_severities_tie/a_higher_severity_patch_beats_the_merged_one/merged_patch_wins_when_it_already_covers_the_worst_advisory— the three rows of the table above.coverage_counts_advisories_not_cve_aliases+ batch twin, andbatch_falls_back_to_cve_ids_when_no_ghsa_is_named.scan_apply_picks_critical_over_more_recent_low_for_paid_user, the free/non-TTY twin, andscan_apply_picks_the_more_recently_published_patch_when_severity_ties.published_at_is_per_patch_not_per_package— verbatim production payload: one package version, two patches, two dates.#[ignore], verified passing):canary_patches_name_advisories_so_merge_state_is_inferable— guards the inference signal itself. If production stopped populatingvulnerabilities, coverage would collapse to 0, the merge rung would go permanently inert, and selection would fall through to recency with no error anywhere. Asserts only that the signal exists, never a count, so publishing a genuinely merged patch does not fail it — that is reported informationally.canary_published_at_is_a_patch_date_not_a_package_date— fails if the API switchespublishedAtto a package-level date.utils::datesuite: all 12 RFC-2822 months, zone aliases and offsets, RFC-3339 variants, rejection cases, adversarial non-ASCII input, a monotonicity sweep, and an exhaustive cross-check of the calendar inverse over ~1265 years.Out of scope
The manifest still holds one patch record per PURL, so a package with two applicable patches still gets only its best one (
axios@1.6.0has two free HIGH patches; applying one reverts the other — they conflict onlib/adapters/http.js). Pre-existing and tracked separately.🤖 Generated with Claude Code