Fix: stop Vitest SSR-transforming the compiled agent bundle in app_loader integration tests - #499
Open
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 1, 2026 22:30
…ation project AgentLoader esbuilds each agent into a throwaway multi-MB bundle under <tmpdir>/adk_agent_loader/<uuid>/ and dynamically imports it. That path sits outside the project root and has no node_modules segment, so Vitest's default heuristics inline it and push the whole bundle through Vite's SSR transform inside the worker. Measured on app_loader_test.ts, the Vitest-reported transform total for the file drops from 38.1s to 1.1s once the bundle is handed to Node's loader instead.
…ery suite preloadAgents() is lazy, so the one-off discovery cost (an esbuild bundle and a dynamic import per fixture) was charged to whichever test touched the loader first rather than to shared setup. Warming it in beforeAll puts that cost under the hook budget, where setup work belongs. This is cost attribution, not the timeout fix: the externalization in the parent commit is what takes the first discovery test from 19,273ms to 3,234ms, and this takes it from 3,234ms to 3ms. No timeout value is raised and no assertion changes.
AmaadMartin
force-pushed
the
fix/app-loader-macos-test-timeout
branch
from
August 2, 2026 05:30
510a9b1 to
a670079
Compare
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
Problem:
tests/integration/app_loader/app_loader_test.tsintermittently fails withTest timed out in 40000mson therun-tests (macos-latest)leg only, always on the first test of theAgentLoader discovery and loading integrationblock. The test is not hanging — it is doing bounded work that costs far more than it should, and all of it is billed to one assertion. Two defects compound:AgentLoader.preloadAgents()esbuilds each of the four discovery fixtures into a self-contained multi-MB bundle under<os.tmpdir()>/adk_agent_loader/<uuid>/and dynamicallyimport()s it. That path is outside the project root and has nonode_modulessegment, so Vitest's default externalization heuristics do not match it: the module runner inlines the bundle and pushes all of it through Vite's SSR transform inside the test worker, once per fixture.it().preloadAgents()is lazy, and the first thing to trigger it wasloader.listApps()inside the failing test. Every later test hits the cache, which is why the two tests after it report ~1 ms.Measured locally: the first discovery test takes 19,273 ms and Vitest reports a 38.35 s transform total for a file that transforms almost no source. macOS runners are slower than this box for CPU/IO-bound work, which pushes that single test across the 40 s cap — matching the observed signature (only macOS, only this file, only this test, only intermittently).
Solution: two test-infrastructure changes; no
core/src,dev/src, orintegrations/srcfile is touched, so runtime behaviour of the shipped packages is identical.vitest.config.ts— addAGENT_LOADER_BUNDLE_PATTERN(/[\\/]adk_agent_loader[\\/]/) next to the two existing integration budget constants and wire it into theintegrationproject only asserver: {deps: {external: [...]}}. A match makes Vitest bypass Vite and hand the module to Node's native loader — which is what happens in production anyway (adk run/adk webimport the bundle from a plain Node process), so the test now exercises a more faithful path. The pattern brackets the directory component with[\\/]so it holds on Windows as well as POSIX;os.tmpdir()differs per platform (/tmp,/var/folders/.../T,%LOCALAPPDATA%\Temp) but theadk_agent_loaderprefix does not.app_loader_test.ts— call the already-publicloader.preloadAgents()in the sharedbeforeAllso the one-off discovery cost is attributed to setup. (The "hooks must not pass their own timeout" invariant is documented once, on the constants invitest.config.ts, rather than restated here; the identical hook-argument removal landed inagent_dirname_test.ts,build_setup_test.ts, andskills/script_js/agent_test.tswith no local comment, and this file should match.)No timeout value is raised anywhere, and no assertion is weakened, deleted, or rewritten — all six existing cases keep their current semantics and their 40 s per-test budget, which after this change they clear by four orders of magnitude. Raising the timeout was considered and rejected: it would leave 19 s of setup inside an assertion and hide the transform overhead that is the actual defect.
Why the visibility rule is respected:
preloadAgents()is already a public method onAgentLoader; nothing was widened and noloader['…']access was added.Collision check (run before implementing,
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000): this bug has attracted several overlapping PRs, so the scope here is deliberately narrow.fix/integration-hook-timeout-single-sourcealready removes the four file-localTEST_EXECUTION_TIMEOUThook arguments that were narrowing the project's 120 shookTimeoutback to 40 s. This PR is stacked on it rather than duplicating it. That branch was chosen as the base because it is current withmainand carries Chore: enforce the node: protocol for Node built-in imports in src (no new deps) #548'sINTEGRATION_HOOK_TIMEOUT_MS.fix/app-loader-vitest-transform-timeoutproposes the sameserver.deps.externalidea. It was not usable as a base: it predates Chore: enforce the node: protocol for Node built-in imports in src (no new deps) #548, so the integration project has nohookTimeoutat all there, and dropping the hook arguments on top of it would silently fall back to Vitest's 10 s default — a regression. The Windows-safe[\\/]bracketing used here is credited to that PR.preloadAgents()hoist. Fix: stop per-file vitest budgets undercutting the integration project defaults #478 was not used as a base because it also deletes the per-it()budgets entirely, which this change intentionally preserves.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
No new production code is introduced and no new line is added to any coverage
includeglob (the generated temp bundle was never in one), so there is no new line or branch to cover and thetest:coveragethresholds are unaffected. No new test case was added deliberately: the correct regression suite here is the six existing cases, and the plan explicitly warns against adding a wall-clock assertion such asexpect(elapsed).toBeLessThan(...), which would reintroduce exactly the machine-speed-dependent flake this change removes.[x] All unit tests pass locally.
Proving the change is load-bearing (mutation testing). Each change was mutated away and re-measured, scoped to the discovery block. This shows neither half is redundant and neither alone is sufficient:
transformtotalWhich change is load-bearing for what — stated plainly, because the two are not equal:
it().It is one line plus one comment line and a deliberate part of the approved plan, so I kept it — but it should be judged as structural hygiene, not as part of the timeout fix. It is genuinely cheap to drop if a reviewer disagrees:
preloadAgents()is idempotent and already called internally bylistApps()/listAgents()/getAppFile(), so removing it changes scheduling only, not behaviour or any assertion.Note on
tsc. Rootnpx tsc --noEmit -p tsconfig.jsonreports 280 errors, but the output is byte-identical with and without this change (verified by diffing the stripped output against a stashed tree). They are pre-existingcore/distvscore/srcduplicate-declaration conflicts that appear afternpm run build, in files this PR does not touch.validation.yamlhas no standalonetscstep; it gates on build /test:coverage/ lint / format / docs, and those pass.CI status: absent, validated locally instead. This is a stacked PR whose base is
fix/integration-hook-timeout-single-source, andvalidation.yamltriggers onpull_request: branches: [main], so the matrix will not run here. Every command above was run against the exact pushed commit. The macOS leg — the actual end-to-end signal — can only be observed once this lands on amain-based PR.Manual End-to-End (E2E) Tests:
Not applicable: this is CI/test-harness only, with no user-facing surface. To reproduce the measurements above:
npm install && npm run build npx vitest run --project integration \ tests/integration/app_loader/app_loader_test.ts --reporter=verboseThen delete the
server: {deps: {external: [AGENT_LOADER_BUNDLE_PATTERN]}}line fromvitest.config.tsand re-run: thetransformtotal jumps back to ~38 s. Additionally remove theawait loader.preloadAgents()call and the first discovery test returns to ~19 s.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.