fix(sitesearch): preserve the custom index alias across a crawl (#36983) - #37010
fix(sitesearch): preserve the custom index alias across a crawl (#36983)#37010fabrizzio-dotCMS wants to merge 4 commits into
Conversation
The Site Search job scheduler resolved index aliases through the content-index router (ESIndexAPI), which is not site-search .os-aware. In Phases 2/3 the physical index lives in OpenSearch tagged with .os, so the lookup missed and the index selector fell back to the raw internal index name — which was then saved as the job's `indexAlias`. From there a full crawl destroyed the alias: it deletes the old index (taking the real alias with it) and re-applies the job's stored string to the new index, so a dead index's NAME became the new index's alias. - site_search_job_schedule.jsp: resolve aliases via the phase-aware SiteSearchAPI#getAliasToIndexMap(), like the Indices tab already does. - SiteSearchJobImpl: when the stored value is a raw index name, recover that index's real alias (or none) instead of carrying the raw name forward. This also repairs jobs already saved with a raw name. - site_search.jsp: the scheduler's alias field caps at 255 chars (the engine limit) instead of 60 — a crawl-built name is 62 chars, so the old cap made those indices impossible to schedule at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @fabrizzio-dotCMS's task in 3m 7s —— View job PR ReviewI reviewed the full diff against The fix is coherent and the resolution rules are pinned by tests. A few observations below; none block merge. New Issues
Notes (non-blocking, no action required)
Test coverageSolid. Overall: no blocking issues. The one Medium is a diagnostic-message refinement, not a correctness defect. · |
…ness endpoint (#36983) An operator knows a site-search index by its alias, never by its sitesearch_<timestamp>_<uuid> name, so the readiness report was hard to act on. Each Site Search row now carries the alias each engine has attached to the index. Per engine on purpose: an index can hold its alias on one side and not the other (created before dual-write started, counterpart built later), and that asymmetry is exactly what has to be visible before promoting a phase. One alias lookup per engine covers the whole set. An alias that is itself shaped like an index name gets a NOTE appended to `recommendation`: that is the fingerprint of the crawl overwrite fixed in this same PR, which cannot be repaired retroactively — this is the only way to find the indices that still need their alias restored. It does not change `verdict`: the verdict measures data integrity, and a damaged alias costs no data, so it must not block a phase change. Content rows are unaffected — `alias` is null there and omitted from the JSON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iews (#36983) listIndices() is a UNION of both engines in the dual-write phases, while getAliasToIndexMap() resolves against a single engine (the read provider). Any index living only on the other engine therefore appears in the list with a blank alias — two mirror images of one defect: - Phase 2 + an index created in Phase 0 (Elasticsearch only). - Phase 1 + an index created in Phase 3 (OpenSearch only), which is what a tester hits after downgrading 3 -> 2 -> 1. Adds SiteSearchAPI#getAliasToIndexMapAllEngines(), resolved over the same provider set listIndices() uses, with the read provider applied last so it wins a mirror desync and the view never contradicts what a search would hit. The single-engine method stays as is: searching must resolve against the engine that serves the query. Switched to it: the Indices tab, the crawl index selector, the Search tab selector (which also stops showing raw index IDs there) and SiteSearchJobImpl — where an invisible alias made the crawl treat an existing index as new and drop its alias, the same loss this PR fixes for the raw-name case. Docs: the two alias views and when to use each; how to read the readiness report (including the admin + migration-role gate, 403 otherwise) with worked examples for the downgrade case and for activating a pre-migration backup content index, whose OpenSearch counterpart is never built and which only a full reindex repairs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…36983) The field table gave the sign convention but not the formula, and omitted +100.0 (original empty, mirror holds data) — which is the value the downgrade example prints, so a reader could not reconcile the two. Also names which verdict each sign blocks: negative blocks advance, positive blocks rollback. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Proposed Changes
Fixes Bug 1 of #36983 (QA-G17): the custom alias of a Site Search index is destroyed by a crawl in the OpenSearch read phases, and the index then becomes impossible to schedule again. Plus the diagnostic that lets an operator find the indices already damaged by it.
Root cause.
site_search_job_schedule.jspresolved index aliases through the content-index router (new ESIndexAPI()), which is not site-search.os-aware. In Phases 2/3 the physical index lives in OpenSearch tagged with.os, so the lookup misses and the index selector falls back to the raw internal index name — which is then saved as the job'sindexAlias. From there the crawl does the damage: it deletes the old index (taking the real alias with it) and re-applies the job's stored string as the new index's alias, so a dead index's NAME becomes the alias. That is thesitesearch-ph-3→sitesearch_20260810160529swap QA reported. #36797 rerouted the Indices tab and 6 other callers to the phase-aware API but missed this JSP.The fix
site_search_job_schedule.jsp— resolve aliases via the phase-aware,.os-awareSiteSearchAPI#getAliasToIndexMap(), exactly as the Indices tab already does. Dropped the now-unusedESIndexAPIimport.SiteSearchJobImpl— when the storedindexAliasis actually a raw index name, recover that index's real alias (or none) instead of carrying the raw name forward to the publisher. Server-side guard, so jobs already saved with a raw name are repaired on their next run rather than needing to be recreated by hand.site_search.jsp— the scheduler's alias field now accepts up to 255 chars (the engine's name limit) instead of 60. A crawl-built name (sitesearch_<timestamp>_<uuid>) is 62 chars, so the old cap rejected it with "Invalid Index alias" and left the index permanently un-schedulable.The diagnostic (second commit)
The fix stops new occurrences but cannot restore an alias already overwritten, so the migration-readiness endpoint (
GET /api/v1/index/migration/readiness) now reports the alias each engine has attached to every Site Search index:es.alias/os.aliason each Site Search row — per engine, because an index can hold its alias on one side and not the other (created before dual-write started, counterpart built later) and that asymmetry is what must be visible before promoting a phase. Also the only way the report is usable at all: an operator knows these indices by alias, never bysitesearch_<timestamp>_<uuid>.recommendation— the fingerprint of this defect, so the already-damaged indices can be found and re-crawled.verdict: the verdict measures data integrity (existence + counts); a damaged alias costs no data and must not block a phase change.aliasis null there and omitted from the JSON).The aggregated alias view (third commit)
The tester's downgrade run (3 → 2 → 1) surfaced the structural half of the same defect:
listIndices()is a union of both engines in the dual-write phases, whilegetAliasToIndexMap()resolves against a single engine (the read provider). Any index living only on the other engine appears in the list with a blank alias — and that is Bug 3, in both of its reported forms:Phase 2 + an index created in Phase 0 (Elasticsearch only) → alias invisible.
Phase 1 + an index created in Phase 3 (OpenSearch only, after a downgrade) → alias invisible.
SiteSearchAPI#getAliasToIndexMapAllEngines()— resolves over the same provider setlistIndices()uses, with the read provider applied last so it wins a mirror desync and the management view never contradicts what a search would hit. Single-provider phases (0 and 3) do not consult the idle engine.The single-engine
getAliasToIndexMap()stays as is — searching must resolve an alias against the engine that will serve the query. The split is documented so the two do not get merged later.Switched to the new view: the Indices tab, the crawl index selector, the Search tab selector (so Bug 2's symptom — raw index IDs in the dropdown — is gone too) and
SiteSearchJobImpl, where an invisible alias made the crawl treat an existing index as new and drop its alias.Docs (
OPENSEARCH_MIGRATION.md)OS_MIGRATION_INDEX_VISIBILITY_ROLE_KEYrole, defaultos_migration_qa— 403 otherwise, and@Hiddenso it is not discoverable by browsing), the order to read the fields in, a per-field table, and a worked example of the downgrade case.activateIndexrepoints the OpenSearch pointer by pure name transformation with no existence check and no create, so the counterpart never exists — silent in Phase 1, masked by the read fallback in Phase 2, customer-visible as lost content in Phase 3. Documents what readiness reports, that a backup is invisible until activated, thatsafeToAdvanceis forcedtruein Phase 3 and must not be read alone, and that only a full reindex repairs it.Checklist
@Hidden+ admin-plus-migration-role gate, and is absent from the OpenAPI schema so no regeneration applies)Additional Info
Tests
SiteSearchJobAliasResolutionTest(new, unit, container-free) — 4 cases pinning the alias-resolution rules: real alias carried through, raw index name → real alias, raw index name with no alias → no alias, unknown name → new-index alias. 4/4 green.SiteSearchRouterReconciliationTest(existing, unit) — 4 new cases on the aggregated view: it includes the engine the phase does not read from, the read provider wins a conflicting alias, Phase 0 never consults OpenSearch, and the single-engine method stays on the read provider. 17/17 green.SiteSearchMirrorReconcilerTest(new, unit, container-free) — 6 cases: alias per engine, alias missing on one engine only, no alias at all, index-name-shaped alias flagged without changing the verdict, a legitimatesitesearch-prefixed alias NOT flagged, and one alias lookup per engine. 6/6 green.SiteSearchJobImplTest(existing IT,MainSuite1a) — new end-to-end caseTest_Non_Incremental_Job_Stored_With_Raw_Index_Name_Expect_Custom_Alias_Preserved: after a full crawl the custom alias must follow onto the new index, and the replaced index's name must never become an alias.Validation run
dotcms-coreinstall: BUILD SUCCESS ·dotcms-integrationtest-compile: BUILD SUCCESSSiteSearchRouterReconciliationTest,ContentIndexMirrorReconcilerTest,MigrationReadinessServiceTest,MigrationReadinessResourceTest,OSSiteSearchAliasMapTestandESIndexHelperTest(no regression).MainSuite1a.Scope
Bug 1 in full. Bugs 2 and 3 turned out to be the same root cause seen from other screens, and the aggregated alias view fixes their symptoms as well (the Search tab dropdown, and the blank Alias column in both cross-phase directions). They are left open in the issue for QA to re-verify against a build rather than closed from here.
🤖 Generated with Claude Code
This PR fixes: #36983