Skip to content

fix(tests): fail fast when a reindex run ends in a terminal non-success status - #31127

Merged
mohityadav766 merged 1 commit into
mainfrom
fix/java-it-reindex-status-fastfail-main
Aug 6, 2026
Merged

fix(tests): fail fast when a reindex run ends in a terminal non-success status#31127
mohityadav766 merged 1 commit into
mainfrom
fix/java-it-reindex-status-fastfail-main

Conversation

@mohityadav766

Copy link
Copy Markdown
Member

Companion to #31125 (1.13). Test-harness only — no product code.

What breaks

SearchIndexAppPage.waitForLatestRunStatus asserted containsText("Success") directly. Playwright keeps re-evaluating that until it matches or the timeout expires — so a run that had already settled on the terminal Failed kept getting polled for the whole budget: 60 minutes in external mode.

On 2.0 that turned a single failed reindex into a whole-run outage. From run 31060569969:

SelectiveFieldReindexUIIT.selectiveFieldReindexPreservesUiCriticalFields -- Time elapsed: 3888 s <<< FAILURE!
  Locator expected to contain text: Success
  Received: Failed
    5 x locator resolved to <div class="status-badge running"  ...>  - unexpected value "Running"
 3594 x locator resolved to <div class="status-badge failure" ...>  - unexpected value "Failed"

3594 polls against a badge that was never going to change. The ui-it job then hit its own 2h cap, and search-it and scale-it were cancelled behind it and inherited a server mid-reindex — the "fresh runs are accepted but fail within seconds with an empty failureContext" window that ReindexHelpers already documents. Roughly six hours to report something the badge showed in seconds.

The change

Wait for any terminal status first, then assert it is the expected one. Same assertion, same failure message, seconds instead of an hour. The terminal set mirrors ReindexHelpers.TERMINAL_STATUSES; badge text is upperFirst(AppRunRecord.status).

Scope

This is the amplifier, not the cause. The underlying reindex failure on main + 2.0 is #30364 un-gating the staged chunk recreate, so every full reindex now runs preflightEmbedding() -> Bedrock InvokeModel -> 403 on a cluster whose node role has no bedrock:InvokeModel. That is fixed separately by pointing the Java IT deployment at DJL (in-process) embeddings. This PR just makes the failure legible when it does happen.

Test plan

  • workflow_dispatch of k8s-java-it.yml after the DJL change lands; SelectiveFieldReindexUIIT should either pass or fail in seconds, never in ~65 min.

🤖 Generated with Claude Code

…ss status

waitForLatestRunStatus asserted containsText("Success") directly, so a run that
had already settled on "Failed" kept getting re-polled for the whole timeout —
60 minutes in external mode. In the nightly Java IT run on 2.0 that turned one
failed reindex into a job-level hang: SelectiveFieldReindexUIIT burned 3888s
before reporting, the ui-it job then hit its own 2h cap, and search-it and
scale-it were cancelled behind it and inherited a server mid-reindex. The build
spent about six hours to report a failure the badge had shown in seconds.

Wait for any terminal status first, then assert it is the expected one. Same
assertion, same message, seconds instead of an hour. Terminal set mirrors
ReindexHelpers.TERMINAL_STATUSES; badge text is upperFirst(AppRunRecord.status).
Copilot AI review requested due to automatic review settings August 6, 2026 16:03
@gitar-bot

gitar-bot Bot commented Aug 6, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Updates SearchIndexAppPage to wait for any terminal reindex status before asserting success, eliminating unnecessary polling when runs fail. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source

@mohityadav766
mohityadav766 merged commit ed5ce95 into main Aug 6, 2026
44 of 68 checks passed
@mohityadav766
mohityadav766 deleted the fix/java-it-reindex-status-fastfail-main branch August 6, 2026 16:20
@mohityadav766
mohityadav766 removed the request for review from Copilot August 6, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant