Fix: pin GOOGLE_CLOUD_* env vars in cli_create_test so the gcloud defaults are actually testable - #569
Open
AmaadMartin wants to merge 2 commits into
Open
Fix: pin GOOGLE_CLOUD_* env vars in cli_create_test so the gcloud defaults are actually testable#569AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 2, 2026 23:45
…table getGcpProject()/getGcpRegion() read GOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATION before shelling out to gcloud, so on a developer machine that exports them the execSync mock in this suite was never consulted and the gcloud-defaults test asserted against the developer's real project id. Pin both variables to absent for the whole suite with vi.stubEnv(name, undefined) and restore them with vi.unstubAllEnvs() in afterEach (vi.restoreAllMocks() restores spies only). Add coverage for the two branches the empty CI environment never reached: env vars winning over gcloud, and gcloud being unavailable. No production code changes.
The env-precedence test also asserts execSync is never called, so the execSync implementation it installed could never run. Remove it. Remove the comment claiming the throwing execSync implementation survives into later tests. Measured on the pinned vitest 3.2.6: clearAllMocks() alone does keep implementations, but the restoreAllMocks() already in this suite's afterEach clears them, so the ordering of that test is not load-bearing.
This was referenced Aug 3, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Link to an existing issue (if applicable):
N/A — no public issue is open for this.
Or, if no issue exists, describe the change:
Problem:
dev/test/cli/cli_create_test.tsis not hermetic.getGcpProject()andgetGcpRegion()(dev/src/cli/cli_create.ts:103-131) readGOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATIONfirst and only shell out togcloud config get-valueas a fallback. The suite mocksnode:child_processexecSyncto return'gcloud-project'/'gcloud-region', but never touchesprocess.env. On a developer machine that exports those variables the env branch short-circuits, theexecSyncmock is dead code, and the prompt is seeded with the developer's real project id:CI runners have neither variable set, so the suite is permanently green there and permanently red for a large fraction of contributors — the worst combination, because the signal that should protect
adk createis inverted depending on who runs it.Solution: A test-only change, entirely additive (68 insertions, 0 deletions, one file).
vi.stubEnv('GOOGLE_CLOUD_PROJECT', undefined)/vi.stubEnv('GOOGLE_CLOUD_LOCATION', undefined)in the existingbeforeEach, paired withvi.unstubAllEnvs()in the existingafterEach. In Vitest 3.2.xstubEnv(name, undefined)deletes the key, andunstubAllEnvs()restores the true original.vi.restoreAllMocks()does not unstub environment variables, so theunstubAllEnvs()call is required, not decorative. This idiom is already used indev/test/server/adk_api_server_test.ts:964,973andcore/test/telemetry/setup_test.ts:30,96.should handle Vertex AI selection with gcloud defaultswas strengthened in place, not replaced, and the edit is additive only.initialValue: 'gcloud-project'is preserved verbatim; the addedmessage: 'Enter the Google Cloud Project ID'key only narrows the match; the region block is a net-new assertion. No assertion was removed, relaxed, or split out, so no regression signal is lost — with the env pin in place this test is now the explicit "env var absent -> gcloud value wins" case. Pinningmessagematters: without ittoHaveBeenCalledWithmatches anytextcall, which is why the original failure reported the region prompt rather than the project prompt and looked mysterious.gcloud(withexpect(execSync).not.toHaveBeenCalled()), andgcloudbeing unavailable (thecatchreturning'').On mock-implementation leakage between tests. An earlier revision carried a two-line comment claiming the throwing
execSyncimplementation survives into later tests, and I briefly added(execSync as Mock).mockReset()tobeforeEachto defend against it. Both are gone, because the premise is false for the Vitest version this repo pins (3.2.6, perpackage-lock.json). Measured directly with a scratch suite (fn.mockImplementation(() => 'LEAKED')in one test, read back in the next):beforeEachafterEachclearAllMocks()restoreAllMocks()undefined— resetclearAllMocks()'LEAKED'— leaksrestoreAllMocks()undefined— resetvi.clearAllMocks()alone does keep implementations, but thevi.restoreAllMocks()already present in this suite'safterEachclears them. Confirmed against the real module mock too: a probe assertingexpect(() => execSync('gcloud config get-value project')).not.toThrow(), placed immediately after the throwing test with nomockReset, passes. So the ordering of thegcloud-unavailable test does not matter, there is no landmine to document, and amockReset()call would have been dead code.No production code changed.
dev/src/cli/cli_create.tsis byte-identical; env-over-gcloudprecedence is deliberate and matches how the rest of ADK resolves these variables.vitest.config.tsis untouched — no repo-wideunstubEnvsflag was added, since that is a separate concern from this file's hermeticity.Collision check (required disclosure). Ran
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000before writing any code. Two open PRs already implement this same fix in this same single file and should be de-duplicated against this one:fix/cli-create-test-env-hermeticity— samebeforeEach/afterEachpin, same additive region assertion, same two new tests. Effectively equivalent to this PR.fix/hermetic-gcloud-defaults-cli-create-test— same pin, but adds a near-duplicate of the existing gcloud-defaults test rather than strengthening it in place.Only one of {this PR, #259, #203} should be merged. I flagged this and initially stopped rather than build a third implementation; this branch exists because the automated review gate required the fix to be present here. Adjacent but non-colliding PRs reviewed and ruled out: #281 (repo-wide vitest
unstubEnvs), #302 (sharedtests/unit_setup.tsenv scrub), #308 (core tracing env gate) — none touchcli_create_test.ts.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
Coverage of
dev/src/cli/cli_create.tsfrom this file (v8,--coverage.include='dev/src/cli/cli_create.ts'):All four previously-unexercised branches of
getGcpProject()/getGcpRegion()(two env short-circuits, twocatchfallbacks) are now covered. New test code is 100% executed by construction; the residual uncovered lines are unrelated paths in the same file (e.g.npm installerror handling) that this task does not touch.Proof each test can fail. Every assertion was run against mutated code and confirmed red. Source restored to pristine after each (
git diff --stat dev/src/cli/cli_create.tsempty).vi.stubEnv(..., undefined)lines; run withGOOGLE_CLOUD_PROJECT=some-project GOOGLE_CLOUD_LOCATION=globalshould handle Vertex AI selection with gcloud defaultsandshould fall back to an empty initial value when gcloud is unavailable- "initialValue": "gcloud-project"/+ "initialValue": "some-project";- "initialValue": ""/+ "initialValue": "some-project"— 2 failed | 11 passedGOOGLE_CLOUD_PROJECTshort-circuit (cli_create.ts:104-106)should prefer the GCP environment variables over the gcloud defaultsexpected "spy" to be called with arguments: [ ObjectContaining{…} ]— 1 failed | 12 passedGOOGLE_CLOUD_LOCATIONshort-circuit (cli_create.ts:119-121)should prefer the GCP environment variables over the gcloud defaultscli_create.ts:114return '';->return 'x';should fall back to an empty initial value when gcloud is unavailablegetGcpRegion()return stdout.trim();->return '';(cli_create.ts:127)should handle Vertex AI selection with gcloud defaults(the newly added region assertion)Mutation 1 is the reported bug reproducing verbatim; mutation 5 proves the assertion added to the existing test is load-bearing rather than decorative.
Manual End-to-End (E2E) Tests:
No E2E surface: this is a unit-test hermeticity fix, and
adk createperforms realnpm installcalls that must not run in CI. To reproduce the original failure and verify the fix by hand, from the repository root:Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.