Fix: correct the integration timeout comments to the budgets in effect - #792
Open
AmaadMartin wants to merge 2 commits into
Open
Fix: correct the integration timeout comments to the budgets in effect#792AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 7, 2026 15:01
…n effect Two comments under tests/integration justified their constants with Vitest's stock defaults. The `integration` project sets hookTimeout to 120000 and testTimeout to 60000, so those defaults never apply to these files. run_skill_script_tool_test.ts keeps its 40000 per-test budget; the comment now names the 60000 it replaces and the 30s executor timeout it must clear. build_setup_test.ts's HOOK_TIMEOUT was byte-identical to the project hook budget, so passing it selected the value the runner already applies. Deleting it removes one number kept in two places. Its measurement moves to vitest.config.ts, where the budget it explains lives.
Three claims in the new comments were not supported by the code. build_setup_test.ts said each test spawns the sample app, but only one of the four does; the others run test:db, test:devtools and the devtools CLI. run_skill_script_tool_test.ts speculated about where the margin above 30s goes. That sentence is replaced by the fact it omitted: only the four it.skipIf(!IS_WINDOWS) tests pass the constant. vitest.config.ts attributed the shared hook budget to build_setup's ~16s. That figure justifies nothing, since the ~70s cold install sets the budget, and four other suites also install in beforeAll.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A
Problem: Two comments under
tests/integrationjustify their timeout constants with Vitest's stock defaults. Theintegrationproject invitest.config.tssetshookTimeoutto 120000 andtestTimeoutto 60000, so those defaults never apply to these files.run_skill_script_tool_test.tsclaims 40000 clears a 5000ms default, when a per-test argument replaces the project budget and so lowers it from 60000.build_setup_test.tsclaims its hooks would otherwise hit a 10s default, and itsHOOK_TIMEOUT = 120000restates the project budget the runner already applies.Solution: I corrected both comments and deleted the duplicate constant.
run_skill_script_tool_test.tskeeps 40000 and all fourit()overrides; only the stated reason changes.build_setup_test.tslosesHOOK_TIMEOUTand inherits the identical project budget, and the cold-install measurement moves tovitest.config.tsnext to the number it explains. No timeout value in effect changes.A complexity review found three claims in my first draft that the code did not support, and the second commit fixes them. The
build_setupcomment said each test spawns the sample app, but only one of the four does. Therun_skill_scriptcomment speculated about where the margin above 30s goes; it now states the fact it had omitted, that only the fourit.skipIf(!IS_WINDOWS)tests pass the constant. The config comment attributed the shared budget to one suite's~16s.Notes for the reviewer:
HOOK_TIMEOUTdeletion is behaviour-neutral, and I measured it rather than assuming it. Every entry point that collects these files passes--project integration:test,test:integration,test:coverageandrecord:samplesinpackage.json, and.github/workflows/validation.yamlrunsnpm run test:coverage. See the proof below.app_loader_test.ts,agent_dirname_test.tsandskills/script_js/agent_test.tspass 40000/40000/60000, which really does lower the inherited 120000. Removing them would change a value in effect, so I left them alone.build_setup_test.tsis the only file whose hook override is a no-op.~70sfigure is the repo's own recorded measurement, moved from the comment being deleted. I did not re-measure it. The old comment also carried a~16sfigure forbuild_setup's twelve hook runs. I dropped that one: the~70scold install is what sizes 120000, andapp_loader,agent_loaderandskills/script_jsalso install inbeforeAll, so pinning a shared budget to one suite misleads.Overlap with the open timeout pull requests. I checked the open pull requests on this fork before starting. Five touch these lines and none lands this change:
build_setup'sHOOK_TIMEOUTand rewrites theINTEGRATION_HOOK_TIMEOUT_MScommentapp_loader,agent_dirnameandscript_js, which are in effectrun_skill_script_tool_test.ts's 40000 constant and all four overridesa2a,adk_web,agent_loader,app_loader,script_jsandbuild_setupHOOK_TIMEOUTinside anapp_loaderdiagnostics changeTEST_EXECUTION_TIMEOUTvalues by a platform multiplierThis PR is the minimal
main-based subset: it corrects the two false comments and removes the single override whose removal changes no effective value. It is deliberately narrower than those sweeps and does not stack on them.Testing Plan
No test is added. The change adds no executable line, so there is nothing new to cover. The coverage thresholds in
vitest.config.tsmeasurecore/src,dev/srcandintegrations/src, none of which is touched.The change carries exactly one risk: that deleting
HOOK_TIMEOUTleaves the hooks with a different budget. I proved it does not.Proof that the hooks inherit 120000. I wrote a throwaway file at
tests/integration/build_setup/inherited_budget_test.tswhosebeforeAllsleeps 11000ms with no local timeout argument, then deleted it. It is not in this diff.An 11s hook passes only because the project
hookTimeoutof 120000 applies.Proof that the check can fail. I mutated the exact line it pins,
hookTimeout: INTEGRATION_HOOK_TIMEOUT_MStohookTimeout: 10000, and re-ran the same file:I then restored the line.
Unit Tests:
Manual End-to-End (E2E) Tests:
I ran the touched suite in full, including six real
npm installruns, and the suite whose comment changed:No "Hook timed out" failure.
The four skipped cases are the
skipIf(!IS_WINDOWS)ones that carry the 40000 budget. They cannot run on a Linux host.CI covers what my host cannot. The
windows-latestleg ran both suites and both passed:The four Windows-gated tests on the 40000 budget ran there, and
build_setupcompleted its twelve hooks on the inherited budget with no "Hook timed out".ubuntu-latestandmacos-latestalso pass.The first
windows-latestattempt failed on two suites this PR does not touch:core/test/code_executors/unsafe_local_code_executor_test.ts(Test timed out in 5000ms, theunit:coreproject) andtests/integration/adk_web/webui_test.ts(CLI exited prematurely with code 1). Both are known Windows flakes on this fork: run 31206516855 hit the identical executor failure on an unrelatedintegrations/build.jsbranch, andmainitself fails validation the same way (run 31162222533). Open PRs #498 and #620 target the executor flake. A re-run of the same commit is green.Repo checks on the pushed commit:
npm run build— passes.npm run lint— clean.npm run format:check— clean.npm run ts:check— 287 errors, identical onmainand on this branch. None is in a file this PR touches. Open PRs such as Fix: type-check core/test/tools (29 tsc errors, test sources only) #742 and Fix: type-check the code-executor, telemetry and A2A test files (23 tsc errors) #743 are clearing them.To repeat the behaviour-neutrality check yourself:
tests/integration/whosebeforeAllsleeps 11000ms and passes no timeout argument.npx vitest run --project integration <that file>. It passes.hookTimeoutto 10000 and re-run. It fails withHook timed out in 10000ms.Checklist