Harden PyPI install verification against stale cache - #102
Open
lelia wants to merge 2 commits into
Open
Conversation
lelia
marked this pull request as ready for review
August 6, 2026 00:31
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 07ae95f. Configure here.
Match the socket-python-cli release workflow hardening (PR #290 there): the 2026-08-05 propagation delay exceeded 10 minutes from the release runner's vantage point, so extend the retry budget to 30 minutes, and log when the JSON API already has the version but the Simple index does not, making CDN propagation delay distinguishable from a failed publish in the logs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
https://pypi.org/simple/indexpython -m pipconsistently for install/uninstallThe verify step is now line-for-line consistent with the same hardening in SocketDev/socket-python-cli#290.
Why
The 3.4.2 release uploaded both distributions successfully, and PyPI's JSON API/project page showed the release, but the post-publish
pip install socketdev==3.4.2loop repeatedly saw a stale Simple-index response whose version list ended at 3.3.0. It ultimately exhausted all 30 attempts and failed after 10m52s.pip caches HTTP responses by default. If the first install attempt races PyPI index propagation, the existing loop can reuse that initial stale response on every retry. It also performs its 30 checks at roughly 0–580 seconds and then sleeps once more before failing, so it never makes a post-expiry check beyond the Simple index's observed 10-minute cache lifetime.
Root cause findings (cross-repo)
This was not specific to this repo or release. socket-python-cli hit the identical failure the same day: socketsecurity v2.5.9 (uploaded 16:40 UTC) failed its release verify loop with the Simple index stuck at 2.5.8 for the full 10-minute budget, which also skipped its Docker publish. Two independent packages, seven hours apart, same signature: JSON API fresh immediately, CDN-cached Simple index stale for 10+ minutes.
Notably, while this repo's verify loop was failing (23:30–23:39 UTC), other CI runners were successfully installing
socketdev==3.4.2from PyPI — the staleness was specific to individual CDN cache nodes, i.e. a delayed/partial purge propagation on PyPI's side. PyPI's status page reported no incident, and pip was ruled out as a cause (the previous day's successful releases used the same pip 26.2.1). Both indexes recovered on their own.This is a latent race rather than a package compatibility failure. The same verification loop was present in the successful 3.3.0 release, where attempt 1 missed the new version and attempt 2 happened to find it 20 seconds later — the propagation lag has always existed; on 2026-08-05 it grew past the loop's budget.
With this change, every retry performs a fresh lookup and the retry budget extends well beyond the observed staleness window.
Validation
actionlint .github/workflows/release.yml.github/workflows/release.ymlwith Ruby's YAML parsergit diff --checkReferences
Prior reports of the same PyPI Simple-index staleness class: