fix(ci): restore committed catalogue over cache to break live-data deadlock - #809
Open
kylerankin wants to merge 1 commit into
Open
kylerankin wants to merge 1 commit into
kylerankin wants to merge 1 commit into
Conversation
…adlock The Update Live Data job fails every run at 'Report albums needing a manual ingest' even after an album is ingested and committed, because the cache restore above it clobbers the version-controlled public/experiences/catalogue.json with a stale copy. catalogue.json is only ever rebuilt by a local 'npm run update:back-catalogue' (yt-dlp), so CI can never hold a fresher catalogue than HEAD — the restore can only make it staler, and a stale catalogue fails this run hard (a missing album escalates to an issue) rather than merely shipping stale data. The pre-fix failed runs each saved an incomplete catalogue, so every run since has restored it and failed again: the recurring 'Failed to update live data from SBOM sources' issue (projectbluefin#808). Restore the committed catalogue over the cache before refreshMetadata runs, so a stale cache can never wedge the next ingest. The auto-generated feeds below the fold are still carried by the cache. Adds a workflow test asserting the committed catalogue is restored over the cache after the restore step. Signed-off-by: kylerankin <kylerankin@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The daily Update Live Data job fails every run at Report albums needing a manual ingest, even after the missing album is ingested and committed. That is what keeps re-firing the recurring Failed to update live data from SBOM sources issue (#808).
The root cause is the cache wiring, not the data:
public/experiences/catalogue.jsonis version-controlled, and only a localnpm run update:back-catalogue(yt-dlp) can build its track segments. CI therefore can never hold a fresher catalogue than HEAD.refreshMetadataran.Fix
Restore the committed catalogue over the cache in a dedicated step before
refreshMetadataruns, so a stale cache can never wedge the next ingest. The auto-generated feeds below the fold (stream/dakota/flickr/growth/sbom) are still carried by the cache.This makes the maintainer's existing workflow actually work: commit an ingest locally, and the next scheduled run goes green.
Tests
Adds a workflow test asserting the committed catalogue is restored over the cache after the restore step. Full suite: 1292 passing.
— hive: backend=pi model=lemonade/Ornith-1.5-35B-A3B-GGUF-Q6_K