Fix: select vitest unit projects by wildcard so new unit suites cannot be silently excluded - #311
Open
AmaadMartin wants to merge 1 commit into
Conversation
The three root test scripts hand-enumerated every vitest unit project, so a project added to vitest.config.ts was silently excluded until someone remembered to update package.json. That drift is what orphaned unit:integrations for a whole release cycle. Replace the enumeration with --project "unit:*" in test, test:unit and test:coverage. Vitest anchors each filter as /^<pattern>$/i, so the wildcard selects exactly unit:core, unit:dev and unit:integrations and does not pull in integration, e2e or cross-language. Non-unit suites stay enumerated by hand so nothing slow or credential-dependent can join the CI gate by accident.
This was referenced Jul 30, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
Problem: The three root test scripts hand-enumerate every Vitest unit project:
vitest.config.tsis the place a project is defined, andpackage.jsonis the place it is selected. Nothing keeps the two in sync, so adding a project tovitest.config.tssilently excludes it fromnpm test,npm run test:unitandnpm run test:coverageuntil someone remembers to also editpackage.json. Since.github/workflows/validation.yaml:41gates every PR onnpm run test:coverage, an unselected project never runs in CI at all.That is not hypothetical — it already happened.
unit:integrationswas added tovitest.config.ts:51together with theintegrationsworkspace but never added to the script lists, sointegrations/test/never executed. It stayed unrun long enough for its only test to rot against a release-please version bump. #236 (this PR's base) repairs the orphan and de-rots the assertion; this PR removes the mechanism that produced it.Solution: Select unit projects by wildcard instead of by name.
Three lines, each shorter than before, no new file and no new abstraction.
Why this is safe and why it is scoped to
unit:*:^3.2.4in the manifest,3.2.6resolved inpackage-lock.json) compiles each--projectfilter to an anchored, case-insensitive regex, sounit:*becomes/^unit:.*$/i. It matchesunit:core,unit:dev,unit:integrationsand cannot matchintegration,e2eorcross-language. Conversely--project integrationbecomes/^integration$/i, so it does not additionally pull inunit:integrations; combining the two runs each suite exactly once. Both properties are measured below, not assumed.unit:*is globbed.unit:<workspace>is this repo's naming convention for the hermetic per-workspace unit suite, and the three that exist map 1:1 onto theworkspacesarray. Suites of that shape are fast and dependency-free and should always gate. Anything slow, networked or credential-dependent (integration,e2e,cross-language) stays hand-enumerated, so joining the blocking CI gate still requires a conscious, reviewed edit. A broader--project "!cross-language"("everything except") was rejected for exactly that reason: it would enrol any future project, including a slow or network-bound one.test:integration,test:e2eandtest:cross-languageare untouched — each targets one named suite and has no drift hazard.cross-languageremains reachable only throughnpm run test:cross-language, as before.No projects matched the filter "<pattern>"and exits 1 (measured below). A quoting regression on any OS therefore turns CI red; it cannot skip the unit suites and report a false green.package.json:18-21, which already passes\"**/*.ts\"to eslint and prettier. Those scripts run vianpm run lint/npm run format:checkon all three legs of thevalidation.yamlmatrix (ubuntu-latest,windows-latest,macos-latest) today, so the idiom is already CI-proven on Windows in this repo. Double quotes also stop POSIXshfrom glob-expandingunit:*against the repo root. Single quotes were deliberately not used: they are not a quote character tocmd.exe.vitest.config.tsis not modified. Project names, include globs,coverage.includeand the coverage thresholds are all unchanged, and no threshold was raised.Collision check (per contribution process).
gh pr list --repo AmaadMartin/adk-js --state open --limit 100plusgh pr diff --name-onlyon every plausibly adjacent PR found three that touch this area:fix/run-unit-integrations-vitest-project--project unit:integrationsto the three scripts by hand and de-rotsintegrations/test/version_test.ts.main, and its two commits appear ingit log main..HEAD. Reviewing #236 first.fix/run-unit-integrations-project-in-cipackage.jsonchange to #236 (same blobd5b10ea5), plus an extra semver-shape assertion.feat/split-slow-integration-testsintegration:slowproject and a CI job; touchespackage.jsonandvitest.config.ts.integration:slowdoes not match/^unit:.*$/i, so this PR neither selects nor deselects it; #237 continues to control it explicitly. Textual conflict in the scripts block is possible depending on merge order and is trivial to resolve.No competing implementation was written.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Environment:
vitest/3.2.6 linux-x64 node-v22.22.2, afternpm install && npm run build.CI note — this PR's checks are
absent, not green.validation.yamltriggers onpull_request: branches: [main](.github/workflows/validation.yaml:7) and this is a stacked PR whose base isfix/run-unit-integrations-vitest-project, so therun-testsmatrix never fires. Everything below was therefore run locally on the exact pushed commit and is reported honestly rather than deferred to CI.Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
No new automated test file is added, deliberately. The change is three npm script strings; the thing under test is which projects the CLI selects, which is a property of the invocation, not of any module. A meta-test asserting
package.jsonagainstvitest.config.tswould re-introduce the very duplication this PR deletes. Verification is therefore behavioural and each command and its output is recorded below.1. Baseline — the drift produced a real, hidden failure. On unmodified
main, before this stack:The suite was not merely unrun — it was broken, and no script selected it. (Repaired by base #236.)
2. Mutation proof — this change is not vacuous. The behaviour this PR adds is "a newly defined
unit:*project is picked up with nopackage.jsonedit". Mutation: add a throwaway seventh projectunit:scratchto a local, uncommittedvitest.config.ts, then run the script both ways.With the wildcard (this PR):
Reverting only the
package.jsonhunk back to the base's enumeration (the mutation):unit:scratchis silently absent — 175 files instead of 176, no error, exit 0. That silent omission is exactly the bug.vitest.config.tswas then restored;git diff --quiet vitest.config.tsconfirms it is unmodified and it is not part of this diff.3. The wildcard selects exactly the right set.
No
[integration],[e2e]or[cross-language]entries.4. No double-execution.
integrations/test/version_test.tsappears exactly once, labelled[unit:integrations];--project integrationdid not additionally pull it in. Zerocross-languagefiles.5. Behaviour is byte-identical to the base today. The decisive check that this PR cannot change any test outcome — the wildcard and the base's enumeration select the same files:
6. Scripts exercised through
npm, notnpx, so the string passes through the shell exactly as CI invokes it (CI=1becausetest/test:unitare watch-mode by design):175 files = 160 + 14 + 1, confirming the wildcard survives npm's shell. The one failure is
dev/test/cli/cli_create_test.ts > should handle Vertex AI selection with gcloud defaults, which reads the machine's ambientgcloudconfig (expected 'gcloud-project', received 'cloud-ai-agentic-coding'). It is pre-existing and unrelated — reproduced on pristinemainat1210acc7with this PR's changes absent, and already targeted by #203 / #259.All 20 failing files are environmental in this sandbox and none are attributable to this diff — guaranteed by step 5, since the selected file set is identical to the base's: 16
e2efiles fail withAPI key must be provided via constructor or GOOGLE_GENAI_API_KEY or GEMINI_API_KEY; 3integrationfiles (agent_loader/agent_dirname_test.ts,build_setup/build_setup_test.ts,skills/script_js/agent_test.ts) time out on their fixturenpm installwith no network; plus the ambient-gcloudcli_create_test.tsabove.7. Coverage. Not re-measured as a before/after pair, and deliberately so: step 5 proves the coverage run executes a byte-identical 235-file set before and after this diff, so the numbers cannot move. Coverage thresholds in
vitest.config.ts:117-122(statements 86 / branches 87 / functions 88 / lines 86) are not touched. The genuine coverage increase fromintegrations/src/**finally executing belongs to base #236, not here.8. Fail-loud property.
A quoting or shell-expansion regression on any OS turns CI red rather than silently skipping the unit suites.
9. Static checks, on the exact pushed commit:
npm run ts:checkhas pre-existing errors onmain(see #207), unchanged here: this diff contains no.tsfile. The diff introduces noany, no@ts-expect-error, no@ts-ignore, noeslint-disableand no coverage-tool suppression — verified by grepping the diff.Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
To reproduce the self-maintenance property (step 2 above) yourself:
npm install && npm run build.vitest.config.ts, copying thealiasblock from a sibling project (without the alias,@google/adk-integrationswill not resolve and the project errors instead of passing):CI=1 npm run test:unit—|unit:scratch|is picked up with nopackage.jsonedit.git stashthepackage.jsonhunk to restore the enumerated form and re-run:unit:scratchsilently disappears.git checkout vitest.config.tsto discard the scratch project.Cross-OS matrix.
windows-latestandmacos-latestcannot be exercised locally, andvalidation.yamldoes not run for a stacked base (see the CI note above). Once base #236 merges and this PR is retargeted tomain, the matrix will run; check the Windows leg's "Run tests and check code coverage" step for|unit:integrations|and for the absence ofNo projects matched the filter. The residual risk is low and bounded: the identical double-quoted-glob idiom is already green on that leg vianpm run lint, and any quoting failure is loud (step 8), not silent.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.