Skip to content

e2e: fail fast when every interpreter quick pick row is deprioritized - #15240

Open
midleman wants to merge 1 commit into
mainfrom
mi/strict-deprioritized-interpreter-pick
Open

e2e: fail fast when every interpreter quick pick row is deprioritized#15240
midleman wants to merge 1 commit into
mainfrom
mi/strict-deprioritized-interpreter-pick

Conversation

@midleman

@midleman midleman commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

selectQuickInputElementContaining silently clicked the first match when every row matching the version was deprioritized. On the Linux CI images that means clicking the uv-managed base install instead of /root/.venv before it has registered -- that interpreter cannot start, so the failure surfaced 90s later as a /started/ timeout instead of at the picker. Source of a 4.3% ubuntu/electron flake in the Quarto inline-output persistence test.

  • Throw when deprioritize is set and no non-deprioritized row exists, listing the rows that were seen
  • The existing retry in startAndSkipMetadata now reopens the picker, giving the venv time to register
  • Windows (System) and macOS (Pyenv) CI have only deprioritized rows, so they keep the first-match fallback via ALLOW_DEPRIORITIZED_PYTHON_FALLBACK
  • Applied to the session-start and notebook kernel-selection call sites

QA Notes

Test-infra only; no product code. Windows and macOS lanes matter here -- they take the fallback path, not the new throw.

Validation Steps

@:quarto @:sessions @:notebooks @:positron-notebooks @:interpreter @:win @:web

E2E Triage Diagnosis

🟢 High confidence -- Test-side: when every quick-pick match is deprioritized, selectQuickInputElementContaining silently clicks the first match (the broken uv managed base interpreter), so the session never starts and the 90s /started/ wait times out.
  • Test: Quarto - Inline Output: Persistence > Python - Verify kernel status persists after window reload
  • Targeted failure: Pattern A -- expect(locator).toBeVisible() failed on locator('.console-instance[style*="z-index: auto"]').getByText(/started/); 6 occurrences, 4.2% on main, ubuntu/electron
  • Signal: Trace timeline t=3093637 shows exactly 2 rows matching aria-label*="Python 3.10.12" (nth=0 and nth=1 read, no nth=2). Both matched DEPRIORITIZED_PYTHON_SOURCES (sessions.ts:27), so the loop at quickInput.ts:203-210 never broke out and fell through to matches.first(). Error-context snapshot, console tabpanel: "Python 3.10.12 (Global) failed to start. Could not start runtime: failed to resolve interpreter /root/.local/share/uv/python/cpython-3.10.12-linux-x86_64-gnu/bin/python". Failure line is sessions.ts:574 inside startAndSkipMetadata setup -- not in any Quarto or reload assertion.
  • Frequency: 6/139 runs (4.3%) on main, ubuntu/electron
  • Hypothesis: /root/.venv -- the only non-deprioritized 3.10.12 in the ubuntu24 CI image (Dockerfile.ubuntu24_04:159-176; asserted as the expected winner in remote-ssh.test.ts:85-96) -- had not yet registered as a runtime at click time. waitForInterpreterDiscoveryToComplete (quickInput.ts:94-108) only clears the "Discovering interpreters" placeholder and does not gate on the venv locator registering. The deprioritize fallback then converts "the interpreter I want is not listed yet" into "start a known-bad interpreter and wait 90s for a status that can never arrive". Ruled out: ordering flip (the image has exactly one non-deprioritized 3.10.12, so 2-rows-both-deprioritized means the venv row was missing, not reordered); never-rendered console (console-instance was present and showed explicit failure text); locator drift (/started/ is the correct locator, the session genuinely never started); Quarto/reload mechanics (failure precedes them). Surviving alternative (low): $HOME/.venv is structurally unregisterable on this image, making the third row impossible rather than late -- contradicted by the 95.8% pass rate.

selectQuickInputElementContaining silently clicked the first match when
every row matching the version was deprioritized, which on the Linux CI
images means clicking the uv-managed base install instead of /root/.venv
before it has registered. That interpreter cannot start, so the failure
surfaced 90s later as a /started/ timeout.

Throw instead, so the caller's existing retry reopens the picker and gives
the venv time to register. Windows and macOS CI install only a bare
System/Pyenv Python, where every row is legitimately deprioritized, so
those platforms keep the first-match fallback via
ALLOW_DEPRIORITIZED_PYTHON_FALLBACK.
@github-actions

Copy link
Copy Markdown

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:quarto @:sessions @:notebooks @:positron-notebooks @:interpreter @:win @:web

Why these tags?
Tag Source
@:critical Always runs (required)
@:quarto PR description
@:sessions PR description
@:notebooks PR description
@:positron-notebooks PR description
@:interpreter PR description
@:win PR description
@:web PR description

More on automatic tags from changed files.

readme  valid tags

@midleman
midleman requested a review from jonvanausdeln July 30, 2026 21:10

@jonvanausdeln jonvanausdeln left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants