Skip to content

Fix: install the integration fixtures from a project-scoped vitest globalSetup - #863

Open
AmaadMartin wants to merge 7 commits into
mainfrom
fix/serialize-integration-fixture-installs
Open

Fix: install the integration fixtures from a project-scoped vitest globalSetup#863
AmaadMartin wants to merge 7 commits into
mainfrom
fix/serialize-integration-fixture-installs

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

  2. Or, if no issue exists, describe the change:

Problem: The integration project installs 14 fixture projects from beforeAll hooks. Those hooks run in parallel fork workers, so several npm processes race for the registry and for the shared ~/.npm cache lock. Three of the four suites pass a 40 s or 60 s hook timeout, which a contended install exceeds, and the run fails with Hook timed out.

Solution: A project-scoped globalSetup installs each fixture once and in sequence, before the worker pool starts. The four suites drop their install hooks and the matching node_modules teardown. The entry sits on the integration project only, so npm run test:unit still installs nothing. build_setup_test.ts keeps its npm run build hook, because that hook asserts on the build output.

global_setup.ts finds the fixtures by walking tests/integration and stopping at the first directory that holds a package.json. That gives the 14 fixtures and excludes the two nested file: sub-packages their parents install, so the list cannot drift from the tree.

Cost, measured on a 64-core Linux host. The installs are now a serial pre-pool phase of about 190 s (14 installs, 12-17 s each, warm ~/.npm). vitest run --project integration goes from 189 s to 299 s with the default pool, and from 283 s to 383 s with --maxWorkers=2. Skipping an install whose tree is already current is queued as a follow-up.

Two cleanups the diff exposed. app_loader_test.ts was left with a beforeAll around one synchronous statement, now a const. build_setup_test.ts had a local HOOK_TIMEOUT of 120000 that duplicated the project-wide hookTimeout; it is deleted, and the surviving comment in vitest.config.ts is re-justified against a measured 3.6 s fixture build instead of an install that no longer runs in a hook. No timeout value changed.

Collision check. I listed the 749 open PRs on the fork and read the diffs of the eight adjacent ones: #218, #237, #257, #276, #299, #405, #407 and #723. None moves the fixture installs into a globalSetup; #218 and #405 keep the installs in beforeAll, #299 removes them by hoisting resolution to the workspace root, and #237 proposes a separate project. None has landed. I branched from main rather than stacking, because eight branches overlap and no one of them is the natural base.

Testing Plan

Please describe the tests that you ran to verify your changes.

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

The 72 existing integration files are the test for this change: every one of them runs against the fixtures globalSetup installed. I added no test file. An earlier revision carried a unit test over an exported installFixture/cleanFixture pair; deriving the fixture list removed the drift the test guarded, and the two helpers are now inline execSync and fs.rm calls with one caller each.

Commands run on the pushed commit:

npm run build
npx vitest run --project integration      # 72 files, 151 passed | 8 skipped, 289 s
npx vitest run --project unit:core        # 202 files, 2745 passed, zero npm output
git status --porcelain tests/integration  # empty
npm run lint && npm run format:check && npm run docs:check   # all pass
grep -rn "npm install" tests/integration --include=*_test.ts # no hits

npm run ts:check reports pre-existing errors under core/test and two integration files, all present on main. It reports none in the files this PR touches.

Proof the change is load-bearing. I made setup() install nothing and re-ran the four affected suites. build_setup_test.ts failed on ts_commonjs_native_addon and ts_esm_native_addon, which depend on onnxruntime-node: file:./fake-onnxruntime-node and can only resolve it after an install. The other 12 fixtures passed, because npm resolves @google/adk and @google/adk-devtools from the workspace root node_modules. Making the installs cheaper therefore has more headroom than the queued follow-up assumes.

One CI attempt failed on an unrelated flake. The macOS job timed out on the two app_loader discovery tests at their own 40 s per-test budget, which this PR does not touch: the diff only removes TEST_EXECUTION_TIMEOUT from hooks. That job took 603 s against 479 s on the passing rerun, so the runner was slow. The rerun is green on all three operating systems. #506 and #858 already target this test.

I could not reproduce the reported flake on this host. Each install finished in about 13 s here, inside the 40 s budget, so main passed at --maxWorkers=2 as well. The fix removes the contention rather than the symptom, so the reproduction is not a precondition for it.

Manual End-to-End (E2E) Tests:

npm install && npm run build
rm -rf tests/integration/**/node_modules
npx vitest run --project integration
git status --porcelain tests/integration

The run installs the 14 fixtures one at a time, prints npm's output for each, and the teardown leaves the working tree unchanged.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

Amaad Martin added 7 commits August 9, 2026 04:39
…balSetup

The integration project installs 14 fixture projects. Every fixture
resolves the same transitive tree, so concurrent npm processes contend
for the registry and the shared npm cache lock.

Add tests/integration/global_setup.ts, which installs each fixture once
and in sequence before the worker pool starts, and register it on the
integration project only.
…n beforeAll

The four install-shelling suites now rely on the globalSetup install, so
the hooks and their teardowns drop the npm install and the node_modules
and package-lock.json removal. build_setup keeps its npm run build and
its dist teardown.
Naming a case after each absolute fixture path made 14 unreadable test
names. The loop reports the offending directory in the assertion
message instead.
…nifest

npm 11 accepts a dependency on a missing `file:` path and exits zero, so
the old trigger only failed on npm 9 and the test failed on CI. An
unparseable package.json exits non-zero on both.
…pers

A directory holding a package.json is a fixture, and the walk stops
there, so the two nested file: sub-packages are excluded by
construction. That removes the hand-maintained 14-entry list, the
exception list and the drift test that existed to catch the two
disagreeing. installFixture and cleanFixture were pass-throughs over
execSync and fs.rm with one caller each, so they move into setup() and
teardown() and fixture_installs.ts goes away.

Also drop the leftover single-statement beforeAll in app_loader_test.ts
and the local HOOK_TIMEOUT in build_setup_test.ts, which duplicated the
project-wide hookTimeout, and re-justify that budget against a measured
fixture build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant