Skip to content

fix(sitesearch): preserve the custom index alias across a crawl (#36983) - #37010

Open
fabrizzio-dotCMS wants to merge 4 commits into
mainfrom
issue-36983-sitesearch-alias-phase-aware
Open

fix(sitesearch): preserve the custom index alias across a crawl (#36983)#37010
fabrizzio-dotCMS wants to merge 4 commits into
mainfrom
issue-36983-sitesearch-alias-phase-aware

Conversation

@fabrizzio-dotCMS

@fabrizzio-dotCMS fabrizzio-dotCMS commented Aug 11, 2026

Copy link
Copy Markdown
Member

Proposed Changes

Fixes Bug 1 of #36983 (QA-G17): the custom alias of a Site Search index is destroyed by a crawl in the OpenSearch read phases, and the index then becomes impossible to schedule again. Plus the diagnostic that lets an operator find the indices already damaged by it.

Root cause. site_search_job_schedule.jsp resolved index aliases through the content-index router (new ESIndexAPI()), which is not site-search .os-aware. In Phases 2/3 the physical index lives in OpenSearch tagged with .os, so the lookup misses and the index selector falls back to the raw internal index name — which is then saved as the job's indexAlias. From there the crawl does the damage: it deletes the old index (taking the real alias with it) and re-applies the job's stored string as the new index's alias, so a dead index's NAME becomes the alias. That is the sitesearch-ph-3sitesearch_20260810160529 swap QA reported. #36797 rerouted the Indices tab and 6 other callers to the phase-aware API but missed this JSP.

The fix

  • site_search_job_schedule.jsp — resolve aliases via the phase-aware, .os-aware SiteSearchAPI#getAliasToIndexMap(), exactly as the Indices tab already does. Dropped the now-unused ESIndexAPI import.
  • SiteSearchJobImpl — when the stored indexAlias is actually a raw index name, recover that index's real alias (or none) instead of carrying the raw name forward to the publisher. Server-side guard, so jobs already saved with a raw name are repaired on their next run rather than needing to be recreated by hand.
  • site_search.jsp — the scheduler's alias field now accepts up to 255 chars (the engine's name limit) instead of 60. A crawl-built name (sitesearch_<timestamp>_<uuid>) is 62 chars, so the old cap rejected it with "Invalid Index alias" and left the index permanently un-schedulable.

The diagnostic (second commit)

The fix stops new occurrences but cannot restore an alias already overwritten, so the migration-readiness endpoint (GET /api/v1/index/migration/readiness) now reports the alias each engine has attached to every Site Search index:

  • es.alias / os.alias on each Site Search row — per engine, because an index can hold its alias on one side and not the other (created before dual-write started, counterpart built later) and that asymmetry is what must be visible before promoting a phase. Also the only way the report is usable at all: an operator knows these indices by alias, never by sitesearch_<timestamp>_<uuid>.
  • An alias that is itself shaped like an index name appends a NOTE to recommendation — the fingerprint of this defect, so the already-damaged indices can be found and re-crawled.
  • It does not touch verdict: the verdict measures data integrity (existence + counts); a damaged alias costs no data and must not block a phase change.
  • Cost is one alias lookup per engine for the whole set, not one per index. Content rows are unaffected (alias is null there and omitted from the JSON).

The aggregated alias view (third commit)

The tester's downgrade run (3 → 2 → 1) surfaced the structural half of the same defect: listIndices() is a union of both engines in the dual-write phases, while getAliasToIndexMap() resolves against a single engine (the read provider). Any index living only on the other engine appears in the list with a blank alias — and that is Bug 3, in both of its reported forms:

  • Phase 2 + an index created in Phase 0 (Elasticsearch only) → alias invisible.

  • Phase 1 + an index created in Phase 3 (OpenSearch only, after a downgrade) → alias invisible.

  • SiteSearchAPI#getAliasToIndexMapAllEngines() — resolves over the same provider set listIndices() uses, with the read provider applied last so it wins a mirror desync and the management view never contradicts what a search would hit. Single-provider phases (0 and 3) do not consult the idle engine.

  • The single-engine getAliasToIndexMap() stays as is — searching must resolve an alias against the engine that will serve the query. The split is documented so the two do not get merged later.

  • Switched to the new view: the Indices tab, the crawl index selector, the Search tab selector (so Bug 2's symptom — raw index IDs in the dropdown — is gone too) and SiteSearchJobImpl, where an invisible alias made the crawl treat an existing index as new and drop its alias.

Docs (OPENSEARCH_MIGRATION.md)

  • The two alias views and when to use each, plus the rule that a phase change never builds counterparts retroactively.
  • How to read the readiness report: the access gate (CMS admin and the OS_MIGRATION_INDEX_VISIBILITY_ROLE_KEY role, default os_migration_qa — 403 otherwise, and @Hidden so it is not discoverable by browsing), the order to read the fields in, a per-field table, and a worked example of the downgrade case.
  • A second worked example: activating a pre-migration backup content index. activateIndex repoints the OpenSearch pointer by pure name transformation with no existence check and no create, so the counterpart never exists — silent in Phase 1, masked by the read fallback in Phase 2, customer-visible as lost content in Phase 3. Documents what readiness reports, that a backup is invisible until activated, that safeToAdvance is forced true in Phase 3 and must not be read alone, and that only a full reindex repairs it.

Checklist

  • Tests
  • Translations
  • Security Implications Contemplated (none — alias resolution and reporting only; the readiness endpoint keeps its existing @Hidden + admin-plus-migration-role gate, and is absent from the OpenAPI schema so no regeneration applies)

Additional Info

Tests

  • SiteSearchJobAliasResolutionTest (new, unit, container-free) — 4 cases pinning the alias-resolution rules: real alias carried through, raw index name → real alias, raw index name with no alias → no alias, unknown name → new-index alias. 4/4 green.
  • SiteSearchRouterReconciliationTest (existing, unit) — 4 new cases on the aggregated view: it includes the engine the phase does not read from, the read provider wins a conflicting alias, Phase 0 never consults OpenSearch, and the single-engine method stays on the read provider. 17/17 green.
  • SiteSearchMirrorReconcilerTest (new, unit, container-free) — 6 cases: alias per engine, alias missing on one engine only, no alias at all, index-name-shaped alias flagged without changing the verdict, a legitimate sitesearch-prefixed alias NOT flagged, and one alias lookup per engine. 6/6 green.
  • SiteSearchJobImplTest (existing IT, MainSuite1a) — new end-to-end case Test_Non_Incremental_Job_Stored_With_Raw_Index_Name_Expect_Custom_Alias_Preserved: after a full crawl the custom alias must follow onto the new index, and the replaced index's name must never become an alias.

Validation run

  • dotcms-core install: BUILD SUCCESS · dotcms-integration test-compile: BUILD SUCCESS
  • Unit: 73/73 across every touched area — the new tests plus SiteSearchRouterReconciliationTest, ContentIndexMirrorReconcilerTest, MigrationReadinessServiceTest, MigrationReadinessResourceTest, OSSiteSearchAliasMapTest and ESIndexHelperTest (no regression).
  • The new IT was not executed locally (needs the integration stack); it runs in CI via MainSuite1a.

Scope
Bug 1 in full. Bugs 2 and 3 turned out to be the same root cause seen from other screens, and the aggregated alias view fixes their symptoms as well (the Search tab dropdown, and the blank Alias column in both cross-phase directions). They are left open in the issue for QA to re-verify against a build rather than closed from here.

🤖 Generated with Claude Code

This PR fixes: #36983

The Site Search job scheduler resolved index aliases through the
content-index router (ESIndexAPI), which is not site-search .os-aware. In
Phases 2/3 the physical index lives in OpenSearch tagged with .os, so the
lookup missed and the index selector fell back to the raw internal index
name — which was then saved as the job's `indexAlias`.

From there a full crawl destroyed the alias: it deletes the old index
(taking the real alias with it) and re-applies the job's stored string to
the new index, so a dead index's NAME became the new index's alias.

- site_search_job_schedule.jsp: resolve aliases via the phase-aware
  SiteSearchAPI#getAliasToIndexMap(), like the Indices tab already does.
- SiteSearchJobImpl: when the stored value is a raw index name, recover
  that index's real alias (or none) instead of carrying the raw name
  forward. This also repairs jobs already saved with a raw name.
- site_search.jsp: the scheduler's alias field caps at 255 chars (the
  engine limit) instead of 60 — a crawl-built name is 62 chars, so the
  old cap made those indices impossible to schedule at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Claude finished @fabrizzio-dotCMS's task in 3m 7s —— View job


PR Review

I reviewed the full diff against origin/main (the 4 PR commits): SiteSearchJobImpl, SiteSearchAPIImpl#getAliasToIndexMapAllEngines, the SiteSearchAPI interface default, SiteSearchMirrorReconciler/MirrorStatus, the four JSPs, and the new/updated tests.

The fix is coherent and the resolution rules are pinned by tests. A few observations below; none block merge.

New Issues

  • 🟡 Medium: dotCMS/src/main/java/com/dotcms/content/index/migration/SiteSearchMirrorReconciler.java:80INDEX_NAME_SHAPED (^sitesearch_\d{8,}.*) will false-positive on a legitimate operator alias that happens to be shaped like an index name (e.g. sitesearch_20260101), appending the "the alias … is an index name, not a real alias" NOTE to a healthy index. Impact is contained: it's advisory text only and correctly does not touch verdict/needsAttention() (confirmed by aliasShapedLikeAnIndexName_isFlaggedWithoutChangingTheVerdict). Acceptable as a fingerprint heuristic, but worth a one-line caveat in the NOTE ("verify before re-crawling") so an operator isn't sent to re-crawl a healthy index. Assumption: no product rule forbids operators from choosing such an alias.

Notes (non-blocking, no action required)

  • SiteSearchAPIImpl.java:224-229 — the "read provider wins" contract holds regardless of the provider != readProvider identity filter, since merged.putAll(readProvider.getAliasToIndexMap()) runs unconditionally last. If readProvider() ever returned an instance not == to the one in writeProviders(), the only cost is one redundant getAliasToIndexMap() call — correctness is unaffected. Good defensive shape.
  • getIndexMetaData (SiteSearchJobImpl.java:432-443) — the recovered raw-name branch no longer runs the isDefaultIndex check, so defaultIndex stays false. This matches the prior behavior (that branch previously set indexAlias = null and also left defaultIndex false), so it's not a regression.
  • The two JSPs that switched to new HashMap<String,String>() (site_search_job_schedule.jsp, test_site_search.jsp) both <%@ include file="/html/common/init.jsp"%> and already use bare Map/HashMap, so no missing import.
  • MirrorStatus.EngineCopy — the new 3-arg convenience constructor keeps content-index callers (alias null) compiling; @JsonInclude(NON_NULL) on the record component omits alias for those rows as intended.

Test coverage

Solid. SiteSearchJobAliasResolutionTest pins all four resolution outcomes (real alias carried through, raw name → real alias, raw name → no alias, unknown → new-index alias); SiteSearchMirrorReconcilerTest covers per-engine alias, one-sided asymmetry, the index-name-shaped flag not changing the verdict, the legit-prefix non-match, and the one-lookup-per-engine cost. The cross-engine aggregation is covered by the added SiteSearchRouterReconciliationTest cases.

Overall: no blocking issues. The one Medium is a diagnostic-message refinement, not a correctness defect.

· issue-36983-sitesearch-alias-phase-aware

…ness endpoint (#36983)

An operator knows a site-search index by its alias, never by its
sitesearch_<timestamp>_<uuid> name, so the readiness report was hard to
act on. Each Site Search row now carries the alias each engine has
attached to the index.

Per engine on purpose: an index can hold its alias on one side and not
the other (created before dual-write started, counterpart built later),
and that asymmetry is exactly what has to be visible before promoting a
phase. One alias lookup per engine covers the whole set.

An alias that is itself shaped like an index name gets a NOTE appended to
`recommendation`: that is the fingerprint of the crawl overwrite fixed in
this same PR, which cannot be repaired retroactively — this is the only
way to find the indices that still need their alias restored. It does not
change `verdict`: the verdict measures data integrity, and a damaged
alias costs no data, so it must not block a phase change.

Content rows are unaffected — `alias` is null there and omitted from the
JSON.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the Area : Documentation PR changes documentation files label Aug 11, 2026
…iews (#36983)

listIndices() is a UNION of both engines in the dual-write phases, while
getAliasToIndexMap() resolves against a single engine (the read
provider). Any index living only on the other engine therefore appears in
the list with a blank alias — two mirror images of one defect:

- Phase 2 + an index created in Phase 0 (Elasticsearch only).
- Phase 1 + an index created in Phase 3 (OpenSearch only), which is what
  a tester hits after downgrading 3 -> 2 -> 1.

Adds SiteSearchAPI#getAliasToIndexMapAllEngines(), resolved over the same
provider set listIndices() uses, with the read provider applied last so
it wins a mirror desync and the view never contradicts what a search
would hit. The single-engine method stays as is: searching must resolve
against the engine that serves the query.

Switched to it: the Indices tab, the crawl index selector, the Search tab
selector (which also stops showing raw index IDs there) and
SiteSearchJobImpl — where an invisible alias made the crawl treat an
existing index as new and drop its alias, the same loss this PR fixes for
the raw-name case.

Docs: the two alias views and when to use each; how to read the readiness
report (including the admin + migration-role gate, 403 otherwise) with
worked examples for the downgrade case and for activating a
pre-migration backup content index, whose OpenSearch counterpart is never
built and which only a full reindex repairs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…36983)

The field table gave the sign convention but not the formula, and omitted
+100.0 (original empty, mirror holds data) — which is the value the
downgrade example prints, so a reader could not reconcile the two. Also
names which verdict each sign blocks: negative blocks advance, positive
blocks rollback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code Area : Documentation PR changes documentation files

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[QA-G17] Site Search portlet: alias overwritten after crawl, index selector shows IDs, cross-phase alias visibility gaps (Phase 2–3)

1 participant