Chore(ci): cap the validation job at 30 minutes - #713
Open
AmaadMartin wants to merge 1 commit into
Open
Conversation
Every job inherits GitHub's default 360-minute timeout, so a hung leg of the validation matrix holds a hosted runner for six hours and gives no signal on the pull request. The slowest successful leg over the last 25 runs on main was 11.2 minutes (windows-latest), so a 30-minute cap leaves 2.7x headroom and cuts the cost of a hang by 12x.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A
Problem: Every job in this repository inherits GitHub's default 360-minute job timeout. A hung leg of the validation matrix therefore holds a hosted runner for six hours, gives no signal on the pull request, and blocks the queue for other contributors. No workflow sets a timeout today:
grep -rn "timeout" .github/returns nothing onmain.Solution: Set
timeout-minutes: 30on therun-testsjob in.github/workflows/validation.yaml. The cap applies per matrix leg; it is not one budget shared across the three legs. I sized it from the last 25 successfulvalidationruns ongoogle/adk-jsmain(75 legs,completed_at - started_atfrom the jobs API): the slowest leg waswindows-latestat 11.2 minutes, with a windows median of 8.9 minutes and a p95 of 9.6 minutes across all legs. 30 minutes is 2.7x the worst observed leg, so a healthy slow run stays green, and the cost of a hang drops 12x.Collision check: I scanned all 605 open pull requests on the staging fork. 44 touch
validation.yaml, and only #218 adds a timeout to it (timeout-minutes: 60, bundled inside a 528-line test-stabilisation change last updated 29 July). I did not stack on #218, because both changes set the same key on the same job and a stack would produce a duplicate YAML mapping key. #403 adds timeouts to the other five workflows and does not touch this file, so the two changes compose.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[ ] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
The diff adds no executable code, so there is nothing to cover and no mutation to record. I did not add a test that reads the workflow and asserts
timeout-minutes === 30: it would restate the diff, and it would hard-code a value meant to be tuned. This omission is deliberate.Manual End-to-End (E2E) Tests:
The YAML parses, and the job keys are in the intended order.
python3 -c "import yaml; print(list(yaml.safe_load(open('.github/workflows/validation.yaml'))['jobs']['run-tests']))"->
['runs-on', 'timeout-minutes', 'strategy', 'steps'], exit 0.Exactly one key was added.
grep -c "timeout-minutes" .github/workflows/validation.yaml->1The change is one line in one file.
git diff --stat main->.github/workflows/validation.yaml | 1 +,1 file changed, 1 insertion(+)The workflow still schedules and passes. GitHub refuses to schedule a malformed workflow, so three green
run-testslegs on this pull request are the end-to-end proof. All three passed, each well inside the new cap:ubuntu-latest5m43s,macos-latest5m50s,windows-latest8m44s.The first
macos-latestattempt failed on a known flake unrelated to this change:tests/integration/app_loader/app_loader_test.ts > "should discover apps vs agents across directories and standalone files"took 43210ms against its 40000ms budget, and that cancelledwindows-latestthroughfail-fast. A one-line workflow key cannot change a test's duration. Both legs passed on re-run.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.