Skip to content

Test: fail the build when a test file is collected by no vitest project - #769

Open
AmaadMartin wants to merge 3 commits into
mainfrom
feat/test-file-collection-guard
Open

Test: fail the build when a test file is collected by no vitest project#769
AmaadMartin wants to merge 3 commits into
mainfrom
feat/test-file-collection-guard

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

  2. Or, if no issue exists, describe the change:
    Problem: Vitest runs a test file only when some project's include glob matches it. A file that matches no glob is skipped in silence: no warning, no error, no "0 tests found". The contributor then reads a green build as a passing test.

Solution: Add one repo-config test to the integration project. It reads the include globs out of vitest.config.ts, walks the working tree for test-looking filenames, and fails with every path that no project collects. It reads the globs instead of hardcoding them, so a new project widens the guard on its own.

Dependency: The guard imports globSync from tinyglobby, the library Vitest and Vite themselves use to expand include globs, so "is this file collected?" is the runner's answer rather than an approximation. tinyglobby@0.2.17 is already installed and root-hoisted through vitest@3.2.6 and vite@7.3.5; this PR only declares it in the root devDependencies so the import is not a phantom. The lockfile diff is one line and no package is added to the tree.

Scope: vitest.config.ts and the package.json scripts are unchanged.

Collision check: gh pr list --repo AmaadMartin/adk-js --state all --limit 400 returns no PR that guards test-file collection. #343 and #418 add sibling guards in the same new tests/integration/repo_config/ directory, but both check project-and-script wiring, under different filenames. This PR branches from main and does not depend on either.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

npx vitest run --project integration tests/integration/repo_config/test_file_coverage_test.ts — 6 passed.

Every test was proven able to fail. Each mutation was applied alone, then reverted:

Mutation Test that failed Message
Drop ignore: IGNORED_DIRS reports every test file the include globs miss expected [ 'dist/built_test.ts', …(3) ] to deeply equal [ 'pkg/stray_test.ts', …(1) ]
Replace .filter((file) => !collected.has(file)) with .filter(() => true) same, plus the repo-level assertion reports the collected fixtures as uncollected
Accept an empty include array rejects a project whose include it cannot read expected [Function] to throw error matching /project #0/ but it didn't
Drop the !projects guard rejects a config that declares no projects throws TypeError: Cannot read properties of undefined instead of the named error
projects.map instead of projects.flatMap flattens the includes of every project the nested arrays are returned unflattened

The extension list also has signal. tests/scratch/probe_test.mts and tests/scratch/probe.spec.cts both turn the guard red and are named in the diff.

The whole repository is clean today, so the guard is green on main and only ever goes red on a regression.

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

npm install && npm run build

# 1. Green on a clean tree.
npx vitest run --project integration tests/integration/repo_config/test_file_coverage_test.ts

# 2. Every stray form must turn it red and name the path.
mkdir -p tests/config core/tests
touch tests/config/scratch_test.ts core/tests/typo_dir_test.ts \
      core/test/dot_form.test.ts dev/src/parked_test.ts
npx vitest run --project integration tests/integration/repo_config/test_file_coverage_test.ts

# 3. Clean up and confirm it is green again.
rm -rf tests/config core/tests core/test/dot_form.test.ts dev/src/parked_test.ts
npx vitest run --project integration tests/integration/repo_config/test_file_coverage_test.ts

Step 2 fails and lists all four:

AssertionError: These files look like tests but no vitest project collects them,
so they never run. Move each one under a path an existing project collects, or
add its directory to that project's `include` in vitest.config.ts. Declared
includes: core/test/**/*_test.ts, dev/test/**/*_test.ts, ...
- []
+ [
+   "core/test/dot_form.test.ts",
+   "core/tests/typo_dir_test.ts",
+   "dev/src/parked_test.ts",
+   "tests/config/scratch_test.ts",
+ ]

Toolchain, on the pushed commit: npm run lint clean, npm run format:check clean. npm run ts:check reports 287 errors both with and without this branch, so the new file adds none; those errors are pre-existing on main.

CI is green on all three legs of the run-tests matrix. The guard passed on windows-latest too, which is the leg that proves the path handling: both the collected set and the test-looking set come from the same globSync call shape, so both are POSIX-separated paths relative to the repo root on every OS.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

Amaad Martin added 3 commits August 7, 2026 05:00
tinyglobby is already installed and root-hoisted as a transitive dependency
of vitest and vite. Declaring it lets a test import it without relying on a
phantom hoisted package. No package is added to the tree.
Vitest skips a test file that matches no project include glob without any
warning, so a contributor reads a green build as a passing test. This guard
reads the include globs from vitest.config.ts, walks the working tree for
test-looking filenames, and names every file that no project collects.
Vitest's default include is `**/*.{test,spec}.?(c|m)[jt]s?(x)`, so a
`foo.test.mts` a contributor adds outside every declared glob was still
silently uncollected and unguarded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant