Skip to content

Fix: spread the real module in three incomplete vi.mock factories - #875

Open
AmaadMartin wants to merge 1 commit into
mainfrom
fix/vi-mock-factory-missing-exports
Open

Fix: spread the real module in three incomplete vi.mock factories#875
AmaadMartin wants to merge 1 commit into
mainfrom
fix/vi-mock-factory-missing-exports

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    N/A

  2. Or, if no issue exists, describe the change:

Problem: An inline vi.mock() factory that omits an export the code under test uses does not fail loudly. Vitest throws only when the missing key is accessed, so a try/catch around the call site swallows the error and the test silently takes the fallback branch. The suite stays green and coverage still counts the lines as executed. An audit of every vi.mock site in the repo found four incomplete factories.

Solution: Three of the four factories now spread the real module and override only the export the test controls, which is the form already used in six other suites. A spread factory cannot omit an export, because its key set is the module's own export list. The fourth site is fixed by sibling PR #778, which touches a different file.

Audit result

I wrote a throwaway checker and ran it from the repo root. It scans every vi.mock('<relative path>', ...) site, resolves the specifier to its .ts source, walks the test's import graph (relative imports, barrel re-exports, and the @google/adk / @google/adk-integrations Vitest aliases), prunes the graph at any module the same test file also mocks, and compares the factory key set against the names that survive. The checker is not part of this change. It lives in /tmp, outside the working tree, and nothing in this diff references it.

Totals on aee56e07: 35 sites with a relative specifier — 27 inline-factory, 6 spread-actual, 2 automock. 0 stale keys, 0 unresolved specifiers, 0 unparsed factories. 4 flagged:

# Site Mocked module Missing Verdict Fix
1 dev/test/utils/agent_loader_test.ts:31 dev/src/utils/file_utils.ts loadFileData active sibling PR #778
2 core/test/tools/skills/run_skill_script_tool_test.ts:23 core/src/utils/file_utils.ts guessMimeType latent this PR
3 core/test/tools/skills/run_skill_inline_script_tool_test.ts:25 core/src/utils/file_utils.ts guessMimeType latent this PR
4 core/test/a2a/event_processor_utils_test.ts:23 core/src/utils/env_aware_utils.ts getBooleanEnvVar, isBrowser, base64Decode, base64Encode, isBase64Encoded latent this PR

The other 31 sites are complete, spread-actual, or automocks. I report them so the audit is visible even though only one site was an active defect.

Active vs latent, and the probe behind each verdict

I classified each hit by running the code, not by reading it. I added the missing key to the factory as a counter that increments on call, plus an afterAll that asserts the counter is zero. A counter detects invocation even when a try/catch swallows a throw, which a throwing probe does not.

Sites #2, #3 and #4: the counter stayed at zero, so the export is reachable through the barrel but nothing the suite exercises calls it. To prove the probe can detect a call at all, I ran a control that counted a known-invoked export instead. It failed as expected:

FAIL |unit:core| core/test/a2a/event_processor_utils_test.ts
AssertionError: expected 6 to be +0 // Object.is equality        <- randomUUID
FAIL |unit:core| core/test/tools/skills/run_skill_script_tool_test.ts
AssertionError: expected 3 to be +0 // Object.is equality        <- materializeFiles

Site #1 is the active defect. Its probe is swallowed by the bare catch in getTypeFromPackageJson, which is the reported failure mode. PR #778 fixes it.

For a latent hit, adding the missing key as a bare vi.fn() is the wrong fix: it replaces a real implementation with a silent undefined-returning stub, so a future call gets a wrong answer instead of an error. Spreading the real namespace preserves the real implementation and closes the hole permanently.

Regression guard

Site #1 is pinned by behavioural tests in PR #778. For sites #2-#4 the structural change is the guard: with the real namespace spread in, there is no factory key left to delete, so the factory cannot go incomplete again. Reverting the spread is not "dropping an export", it is reverting the fix, and the checker detects that. With ...actual removed from site #2 the checker flags it again:

core/test/tools/skills/run_skill_script_tool_test.ts:23  ->  core/src/utils/file_utils.ts
    factoryKeys: materializeFiles
    MISSING:     guessMimeType

With the fix in place the tally moves to 24 inline-factory / 9 spread-actual / 2 automock. The checker reports one remaining flagged site on this branch, site #1, which PR #778 closes. With both branches applied it reports 0 flagged site(s).

One checker note, since it changes a number. My first version classified a factory as spread-actual whenever it mentioned importOriginal. That is a false negative: a factory can await the original and still return a literal without spreading it, which is exactly the mutation above. The checker now decides on the returned literal, not on the mention.

Collision check

gh pr list --repo AmaadMartin/adk-js --state open --limit 100 plus gh pr diff --name-only on every plausibly adjacent PR. Two overlap:

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

This PR changes test setup only. It adds no new test, because the fix removes a failure mode by construction rather than adding behaviour. No src/ file is touched.

Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

CI is green on this PR: run-tests passes on ubuntu-latest, macos-latest and windows-latest, plus check-license. I also validated locally on the pushed commit fc717f5f:

bash scripts/check_license.sh
  -> All files have the correct license header.
npm run build -w core
  -> succeeds
npx vitest run --project unit:core core/test/tools/skills core/test/a2a
  -> 22 files, 283 tests passed
npx vitest run --project unit:dev dev/test/utils/agent_loader_test.ts
  -> 1 file, 36 tests passed
npx vitest run --project unit:dev dev/test/cli
  -> 5 files passed, 1 pre-existing failure (see below)
npx eslint <the three changed files>            -> clean
npx prettier --check <the three changed files>  -> clean

Two pre-existing failures, both present on aee56e07 before this change:

  • dev/test/cli/cli_create_test.ts — "should handle Vertex AI selection with gcloud defaults". Fails identically with my change stashed.
  • npm run ts:check reports 292 errors repo-wide, the same 292 on aee56e07 and on this branch. One of them, run_skill_script_tool_test.ts TS2352, sits in a file I touch, at line 209 before my edit and 214 after. I add zero errors and did not fix these, as they are outside this task.

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

  1. npm run build -w core
  2. npx vitest run --project unit:core core/test/tools/skills core/test/a2a
  3. To see the trap this removes, delete ...actual from any of the three factories and add guessMimeType('x.png') to a test body. Before this change that call throws No "guessMimeType" export is defined on the mock; after it, it returns image/png.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works. No new test: the fix removes the failure mode by construction, as described under "Regression guard".
[x] New and existing unit tests pass locally with my changes.

An inline vi.mock factory that omits an export the module graph under test
uses does not fail loudly. Vitest throws only at property-access time, so a
try/catch around the call site swallows the error and the test silently takes
the fallback branch.

An audit of all 35 vi.mock sites with a relative specifier found three
factories that omit an export reachable through the code under test:
guessMimeType from core/src/utils/file_utils.ts in the two skill-script
suites, and five env_aware_utils exports in event_processor_utils_test.ts.
A runtime probe showed none of the three is invoked today, so each one is a
latent trap rather than a live defect.

Spreading the real namespace and overriding only the stubbed export makes the
factory incapable of going stale, and keeps the real implementation for every
export the test does not control.
@AmaadMartin
AmaadMartin force-pushed the fix/vi-mock-factory-missing-exports branch from 7ef6400 to fc717f5 Compare August 9, 2026 16:48
@AmaadMartin
AmaadMartin changed the base branch from fix/agent-loader-test-package-json-module-type to main August 9, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant