Repair what the WordPress database dump found, and retire three cutover deadlines - #63
Merged
Merged
Conversation
Diffing the 2026-09-21 database dump against content/ found eight published
posts whose wp_posts.post_modified is newer than the lastmod here -- authors
still editing on WordPress after this repo had taken its copy. Seven were
re-fetched with transfer/Posts.java --url; boxlang-1-16-0-released is frozen,
so its four added categories were applied by hand.
What actually changed, beyond lastmod:
hibernate-orm-and-mongodb (both parts) the sample repo moved to
mongodb-jvm-showcase and the
per-article git tags were dropped
whats-new-...-azul-payara-release two sections reordered
foojay-podcast-75 + DuckDB
foojay-podcast-101 - OpenJDK
building-an-agentic-wms-part-1 + Java, + Mongo
boxlang-1-16-0-released + Developer Tools, Java, Java Core,
Release Notes
did-your-ai-agent-run-the-debugger title only -- see below
Three converter regressions the re-scrape introduced are corrected in the same
commit, because they are noise rather than author intent: a space eaten after
an inline link in each hibernate post, and `&` escaped to `\&` three times in
the Payara post.
did-your-ai-agent-run-the-debugger is the awkward one: the WordPress edit that
produced its new title ALSO mangled six links in the body -- schemes stripped
("github.com/obra/superpowers"), doubled ("http:https://...///"), one anchor
text spliced into the middle of another. The dump confirms the damage is in
post_content itself, so it is the live copy that is broken and a re-scrape
reproduces it faithfully. That post keeps the new title and the old body, and
is now frozen so the next run cannot re-import the damage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 2026-09-21 SQL dump is the first source for this migration that is not a
scrape, and diffing it against content/ turned up four gaps the scraper could
not have closed, because in each case the live HTML is not where the fact
lives.
1. keeping-your-fonts-in-embedded-svg had NO BODY at all -- frontmatter and
nothing else, live on the site that way. The WordPress page serves an empty
.article__main-content, so transfer/Posts.java rewrites it to empty on every
run and says so ("body is only 0 word(s) and REST offered nothing better").
wp_posts.post_content holds the real 90 KB article; it is rebuilt here
through the repo's own HtmlToMarkdown, with the two body images pulled into
the bundle. Frozen, or the next run blanks it again. The one departure from
what WordPress held -- an 81,661-character base64 font payload elided inside
the self-contained-SVG listing -- is argued in the frontmatter.
2. calling-microservices-in-java was the only post with a featured image in
WordPress and none here. Its og:image URL contains a NARROW NO-BREAK SPACE
(U+202F) that WordPress emits unencoded, which the scraper cannot fetch.
Pulled by hand with the space percent-encoded, renamed with a hyphen (no
other file under content/ has a space in its name). Frozen for the same
reason as above.
3. Eleven redirects that would have 404'd. Nine come from `_wp_old_slug`, the
postmeta WordPress writes when a post is renamed and then redirects from for
ever -- invisible to the Redirection plugin export the aliases were built
from, which lists only redirects somebody added. The other two are plugin
rules 106/107, added after that export was taken. Same blind spot as
CUTOVER's rules 4 and 5, and the CUTOVER note now says to re-run the query
against the final dump.
4. Five author social links, on two profiles. Stefan Richthofer's three the
scraper would pick up today; Maximillian Arruda's LinkedIn and YouTube it
never will -- he routes them through link.maxdearruda.com and the scraper
classifies a social link by host, so it writes "" over both on every run.
The usermeta keys in the dump name the network outright. His profile is
frozen; verified by re-running Authors.java over both.
Two more flagged by the diff are NOT changed here, on purpose. The `user_url`
of iryna-dohndorf and micksembwever is the same LinkedIn URL their profiles
already carry, so carrying it into `website:` would print one link twice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WordPress had alt="" on all five, so there was nothing to carry over, and the whole point of this article is what the pictures show: the same drawing with the handwritten Virgil font and with the browser's fallback. Without a description a screen-reader user gets the argument with the evidence removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 1 is built around scraping a site that is about to be switched off, and
three of its items say some version of "this cannot be re-run after cutover".
A phpMyAdmin dump of wp_foojay makes all three false, so the phase now opens by
saying to take the dump FIRST.
view counts wp_post_views WHERE type='4' is the per-post total, exactly,
in one query. Not merely tidier than the crawl: on
2026-09-21 the crawl-built data/legacy-views.json was 87,617
views light across 2,169 posts and missing 91 posts
outright. Nothing was over-counted, so every difference is
something the crawl did not see. Rebuild the file from the
dump before seeding.
comment archive wp_comments holds every comment. The 2026-09-21 diff found
content/posts/**/comments.json already complete -- 590
approved on 276 posts -- so this stops being a deadline.
CloudflareEmails obfuscation happens at Cloudflare's edge, so post_content is
clean; 233 posts carry a raw address there. Still worth
running the script while WordPress is up, but a missed
address is now recoverable rather than gone. One is not, and
the note says which and why: that placeholder is baked into
post_content too.
Two things the intro is explicit about, because both are easy to assume wrong:
the dump does NOT contain the media files, and it holds password hashes and
commenter IPs, so it lives outside this repo with its own README.
The redirect section gains the `_wp_old_slug` blind spot -- a third family the
plugin export never knew about, alongside rules 4 and 5 -- with the query to
re-run against the final dump, and the alias count moves 86/89 -> 88/92.
The hero-image and canonical-URL audits are untouched: those probe third-party
URLs and still need the network.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (23)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Validating the WordPress SQL dump (
Foojay/WordPress backup/sftp96438_wpengine_com.sql, taken 2026-09-21 08:34 UTC) againstcontent/turned up a short list of real gaps. This is that list, fixed.The migration itself came out clean. All 2,255 published posts are here, all 326 post authors have a profile, post dates match the URL paths exactly, no post is missing a description, and every approved comment is archived. What follows is the remainder.
What changed
Eight posts were edited on WordPress after this repo took its copy. Seven re-fetched with
transfer/Posts.java --url;boxlang-1-16-0-releasedis frozen so its four added categories went in by hand. Mostly categories, plus a moved sample repo in the two Hibernate/MongoDB posts and a section reorder in the Payara release post.Four things only the database has, because in each case the live HTML is not where the fact lives:
keeping-your-fonts-in-embedded-svg.article__main-content, so every scrape rewrites it empty and says so. Rebuilt frompost_content(90 KB) through the repo's ownHtmlToMarkdown, images pulled into the bundle, now frozen.calling-microservices-in-javaog:imageURL contains a narrow no-break space (U+202F) that WordPress emits unencoded and the scraper cannot fetch._wp_old_slug— the postmeta WordPress writes on a rename and redirects from for ever, invisible to the plugin export the aliases were built from — plus plugin rules 106/107, added after that export.link.maxdearruda.com, and the scraper classifies a social link by host, so it writes""over both on every run. His profile is frozen; the usermeta keys name the network outright.One post is frozen against its own source. The WordPress edit that gave
did-your-ai-agent-run-the-debuggera better title also mangled six links in the body — schemes stripped, doubled (http:https://...///), one anchor text spliced into another. The dump confirms the damage is inpost_content, so the live copy is the broken one and a re-scrape reproduces it faithfully. This keeps the new title and the old body. Worth fixing on WordPress too if the post is going to be edited there again.Also: three converter regressions the re-scrape introduced (two eaten spaces, three
&→\&), and alt text for the five images in the restored post, where WordPress had none and the images are the argument.What this changes about cutover day
CUTOVER.mdPhase 1 is built around scraping a site that is about to be switched off, and three of its items say some version of this cannot be re-run after cutover. A database dump makes all three false, so the phase now opens by saying to take the dump first.The view counts are the one to read closely.
wp_post_views WHERE type='4'is the per-post total, exactly, in one query — and on 2026-09-21 the crawl-builtdata/legacy-views.jsonwas 87,617 views light across 2,169 posts, and missing 91 posts outright. Nothing was over-counted, so every difference is something the crawl did not see. Rebuild that file from the final dump rather than crawling for it.The comment archive and
CloudflareEmailssimilarly stop being deadlines (wp_commentsis complete; obfuscation happens at Cloudflare's edge, sopost_contentis clean — 233 posts carry a raw address there). The hero-image and canonical audits are untouched: those probe third-party URLs and still need the network.Not in this PR
/today/<type>/…with no Hugo equivalent and no redirect family:/today/options/(41,672 — the JVM options explorer, now/jvm-options/),/today/fixes/(23,813),/today/event/(5,430), and smaller sets. Allindexinwpseo_titles, none carrying a recorded page view. Wants a decision — a rule, or a bulk 410 — before cutover; the full URL list is in the dump._default/term.htmlrenders only{{ .Title }}, so carrying them needs a layout change, not just content.website:fields the diff flagged: foriryna-dohndorfandmicksembweverthe WordPressuser_urlis the same LinkedIn URL their profile already shows, so adding it would print one link twice.stefan-richthofer'swebsite:isapidia.nethere andx.com/steworion the live page, so the next fullAuthors.javarun will flip it. Pre-existing, left alone.A
README.mdnow sits next to the dump on local disk with provenance, the table inventory, a loader script (no MySQL needed) and the queries behind every number above.Verification
hugobuilds clean,validate/Frontmatter.java --changed-since mainpasses,validate/BuiltSite.javareports only pre-existing dead links in untouched posts, and all eleven new aliases plus both repaired posts were checked inpublic/.🤖 Generated with Claude Code