Skip to content

Refactor: hoist repeated Vertex AI placeholders to module-level constants in core tests - #565

Open
AmaadMartin wants to merge 3 commits into
mainfrom
feat/hoist-vertex-placeholder-constants-core-tests
Open

Refactor: hoist repeated Vertex AI placeholders to module-level constants in core tests#565
AmaadMartin wants to merge 3 commits into
mainfrom
feat/hoist-vertex-placeholder-constants-core-tests

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    Closes: #issue_number
    Related: #issue_number
  2. Or, if no issue exists, describe the change:

Problem: Two core/test suites repeat the same Vertex AI Agent Engine
placeholder literals dozens of times.

  • core/test/sessions/vertex_ai_session_service_test.ts repeats the bare app
    name '12345' on 66 lines — 47 as appName: '12345', 3 in expect
    assertions, and 16 embedded in 'reasoningEngines/12345/...' expected
    resource names. The relationship the service actually implements
    (appName -> reasoningEngines/<appName>) is invisible in the source: it
    reads as two magic numbers that happen to match.
  • core/test/code_executors/agent_engine_sandbox_code_executor_test.ts repeats
    an 88-character sandbox resource name 9 times and its 64-character agent
    engine prefix 3 times, plus 'test-project' x4 and 'us-central1' x2.
    The same coupling is invisible here: the executor parses project and location
    out of the resource name and falls back to the stubbed env vars, so those
    tests only mean anything if the two agree — and nothing in the source says so.

Solution: A test-only refactor. Each file gets its own module-level
constants and every repeated literal is replaced by a reference to them. No
production file is touched, no test is added, removed, renamed, reordered or
skipped, and no assertion changes meaning — every substituted site evaluates to
a byte-identical string.

core/test/sessions/vertex_ai_session_service_test.ts:

/** Vertex AI requires an app name that is digits or a full resource name. */
const APP_NAME = '12345';

APP_NAME covers the 47 inputs, the 3 pass-through echo assertions
(expect(session.appName).toBe(APP_NAME)) and the 6 mock fixtures. It is
deliberately not used in the expected value of the 10
toHaveBeenCalledWith assertions: those pin the service's
appName -> reasoningEngines/<id> mapping, and deriving the expectation from
the same symbol fed in as the input would stop them pinning it independently.
Those 10 keep their hardcoded strings, unchanged from before this branch. The
constant's doc comment states that split so the remaining literals read as
deliberate rather than as missed occurrences. The inline
// Must be digits or resource name comment moves onto the constant, where the
reason lives with the value.

An earlier revision of this PR did interpolate all 16 sites; a reviewer caught
that it silently removed the regression signal, and the mutation results below
confirm it.

core/test/code_executors/agent_engine_sandbox_code_executor_test.ts:

const TEST_PROJECT = 'test-project';
const TEST_LOCATION = 'us-central1';
const AGENT_ENGINE_NAME = `projects/${TEST_PROJECT}/locations/${TEST_LOCATION}/reasoningEngines/123`;
const SANDBOX_NAME = `${AGENT_ENGINE_NAME}/sandboxEnvironments/456`;

AGENT_ENGINE_NAME is composed rather than written flat so the required
agreement between the stubbed env vars and the resource name is explicit, and
SANDBOX_NAME is composed from it rather than repeating the prefix.

Why the constants are file-local and not a shared fixture module.
vitest.config.ts defines unit:core, unit:dev, unit:integrations,
integration, e2e and cross-language as separate projects, and this repo
has no precedent for a test importing a fixture across that boundary. The one
shared-fixture module that does exist,
core/test/artifacts/artifact_service_test_utils.ts, is imported only by its
three directory-local siblings. So each file declares its own constants, neither
is exported, and nothing is imported from tests/ into core/test/. This also
matches adk-python, which declares these placeholders file-locally
(MOCK_APP_NAME / MOCK_USER_ID in
tests/unittests/memory/test_vertex_ai_memory_bank_service.py), and the
existing in-repo precedent in core/test/runner/runner_test.ts.

Literals deliberately left alone. Each of these is distinct by design
substituting a constant into any of them would turn a real assertion into a
tautology and silently destroy the regression signal:

Literal Why it stays
expect(executor['location']).toBe('us-central1') in defaults location to us-central1 if missing in env Pins the library's hard-coded default, which merely happens to equal the test placeholder. Using TEST_LOCATION here makes the test assert itself.
'custom-engine-id' / 'reasoningEngines/custom-engine-id' Proves an explicit agentEngineId overrides the appName passed to createSession. The two values must visibly disagree.
'invalid-app-name' Negative case; also appears inside the asserted error message.
'projects/my-project/locations/us-central1/reasoningEngines/999', 'reasoningEngines/999' Proves engine-id extraction from an arbitrary full resource name.
'projects/custom-p/locations/custom-l/...' (x2) Proves project/location are parsed out of the resource name in preference to the env defaults. Must differ from TEST_PROJECT/TEST_LOCATION.
'custom-location' Proves the location option beats the env var.
vi.stubEnv('GOOGLE_CLOUD_PROJECT', '') Empty-string clearing, not a placeholder.
appName: '123' on the InvocationContext session fixture (x2) Looks like the engine id but is not — agent_engine_sandbox_code_executor.ts never reads session.appName. The resemblance is coincidental.

Single-occurrence literals (projectId: 'test-project' / location: 'us-central1'
in the session service's "can initialize without passing a client explicitly"
case) are also left alone — a constant for a single use is noise.

Collision check. gh pr list --repo AmaadMartin/adk-js --state open --limit 1000
was checked before any code was written. No open PR performs this refactor: a
gh pr diff grep across every open PR for const APP_NAME / const TEST_PROJECT
/ const AGENT_ENGINE_NAME / const SANDBOX_NAME returned nothing. #439
("extract shared Vertex AI placeholder constants for the tests tree") is the
closest neighbour and is disjoint — it touches
tests/integration/** only, which is a different vitest project with no import
relationship to core/test/**, so the two can land in either order. Six PRs
(#512, #503, #287, #270, #201, #331) touch
core/test/sessions/vertex_ai_session_service_test.ts incidentally by adding
tests to it, but none does this refactor; stacking on six independent branches
is not possible, so this branches from main and will need a trivial textual
rebase against whichever of them lands first.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

No test was added: this is a pure refactor, and adding a test to it would make
the value-preservation claim harder to verify, not easier. The existing suites
are the test.

Baseline recorded on the unmodified files, then re-run after the change —
identical, with zero failures and zero skips:

$ npx vitest run --project unit:core vertex_ai_session_service agent_engine_sandbox_code_executor
 Test Files  2 passed (2)
      Tests  83 passed (83)          # before: 83 passed (83)

Refactor-correctness proof. A literal-to-constant refactor has one failure
mode — a missed occurrence, leaving a stale hard-coded literal that no longer
tracks the constant. Proved in both directions:

  1. The mapping is still independently pinned (File 1). Temporarily set
    const APP_NAME = '99999' (still digits, so it still satisfies the
    service's app-name validation) and re-ran the suite: 9 tests failed
    (Tests 9 failed | 48 passed (57)), each reporting
    expected "reasoningEngines/12345", received "reasoningEngines/99999".
    That is the intended signal — the hardcoded expectations catch the input
    changing out from under them. In the earlier revision that interpolated the
    expectations, this same mutation left the suite fully green (57/57), which
    is what the reviewer flagged. Reverted.

  2. Substitution is total (File 2). Temporarily set
    const TEST_PROJECT = 'mutant-project' and re-ran the suite. It passed:
    Tests 26 passed (26). Reverted.

  3. The remaining fixture substitutions are value-preserving (File 1). The
    6 mock-fixture sites that do use `reasoningEngines/${APP_NAME}/...`
    feed values into the mocks rather than asserting on them. Mutation 1 above
    covers them: with APP_NAME = '99999' none of the 9 failures came from a
    fixture mismatch, and with APP_NAME = '12345' all 57 pass, so each fixture
    still produces the exact string it did before.

  4. The constants are load-bearing (File 2). Changed one expectation from
    sandbox_name_language_python: SANDBOX_NAME to
    sandbox_name_language_python: `${SANDBOX_NAME}x`. Exactly one test
    failed:

    × AgentEngineSandboxCodeExecutor > executeCode > initializes session state if missing
    -   "sandbox_name_language_python": ".../sandboxEnvironments/456x",
    +   "sandbox_name_language_python": "projects/test-project/locations/us-central1/reasoningEngines/123/sandboxEnvironments/456",
     Tests  1 failed | 25 passed (26)
    

    The + line proves the composed SANDBOX_NAME evaluates to exactly the
    88-character literal it replaced. Reverted.

Quality gate:

$ npm run format:check    # All matched files use Prettier code style!
$ npm run lint            # exit 0, clean repo-wide
$ npx eslint <the two files>   # exit 0
$ npm run ts:check        # pre-existing failure, unchanged by this PR (see below)

npm run ts:check is already red on the base commit with 281 errors across the
repo. I ran it on the base and on this branch and diffed the normalized
per-file error counts: identical, 281 before and 281 after. This PR adds
zero type errors and fixes none; the four base errors whose source snippet now
reads appName: APP_NAME are the pre-existing ListSessionsRequest errors that
previously read appName: '12345'.

Post-conditions verified:

$ grep -c "12345" core/test/sessions/vertex_ai_session_service_test.ts
11                                    # 1 APP_NAME declaration + the 10
                                      # deliberately hardcoded expectations
$ grep -c "'projects/test-project" core/test/code_executors/agent_engine_sandbox_code_executor_test.ts
0                                     # AGENT_ENGINE_NAME composes it instead
$ git diff --stat                     # exactly 2 files, both under core/test/
$ git diff -- core/src dev/src integrations/src tests/
                                      # empty

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

Not applicable — there is no user-visible behaviour to exercise, and no
production code is touched. The manual verification is the diff review: read
both files end to end and confirm that (a) every literal in the
"deliberately left alone" table above is intact and still visibly different from
the new constants, and (b) no comment, test name, or mock ordering moved. To
reproduce the automated verification:

npm install
npm run build      # the unit:core setup file imports @google/adk via core/dist
npx vitest run --project unit:core vertex_ai_session_service agent_engine_sandbox_code_executor

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

CI status

All checks pass on the current head (6612141e), including windows:

Job Result
run-tests pass
run-tests (ubuntu-latest) pass
run-tests (macos-latest) pass
run-tests (windows-latest) pass
check-license pass

For the record, earlier commits on this branch saw red windows-latest and one
red macos-latest run. Those were pre-existing platform flakes, not this
change: the same windows-latest job fails on the untouched base commit
b390217e (run 30669370416, branch main, ubuntu and macos green) with
tests/integration/adk_web/webui_test.ts > 'Run from ADK CLI': CLI exited prematurely with code 1 and Tests 2679 passed | 43 skipped — zero assertion
failures, a spawned-server teardown error. Across three runs the windows job
failed three different ways (adk_web/webui_test.ts, then
unsafe_local_code_executor_test.ts timing out in 5000ms), and the macos
failure (tests/integration/app_loader/app_loader_test.ts, 40000ms timeout)
went green on re-run — it had also cancelled windows that round, since the
matrix does not set fail-fast: false. None involve the two files changed here,
which pass on every platform (57 and 26 tests). The repo-wide total is 2722
tests on both the base commit and this branch, confirming no test was added,
removed or skipped.

Review round 1 — changes made

A complexity reviewer raised three points. Two are addressed in commit
6612141e; the third is answered below.

  1. Blocking — derived expectations (fixed). Interpolating APP_NAME into
    the expected value of the 10 toHaveBeenCalledWith assertions made them
    recompute the appName -> reasoningEngines/<id> mapping from the same
    symbol supplied as the input, so they stopped pinning it independently. This
    was real and measurable: mutating APP_NAME left the suite 57/57 green.
    All 10 are restored to their original hardcoded strings — byte-for-byte as
    on the base commit, so those lines drop out of the diff entirely — and the
    same mutation now fails 9 tests. The 6 mock-fixture sites keep the constant,
    as the reviewer noted they are fixtures, not expectations.
  2. Location-default ambiguity (addressed, not by inlining). The reviewer
    suggested inlining TEST_LOCATION because its value collides with the
    library default asserted as a bare literal one screen below. I kept the
    constant — the executor parses project and location out of the resource name
    and falls back to the env vars, so the agreement between the stubbed env and
    the composed AGENT_ENGINE_NAME is a real coupling worth making explicit —
    and instead removed the ambiguity at its source with a note on the
    assertion explaining why it must not reuse the constant. That guards
    against a future contributor "completing" the substitution and silently
    making the test vacuous, which inlining would not.
  3. Dropping the app-name commit entirely (declined). The reviewer judged
    the APP_NAME hoist as not paying for itself, largely because the
    derived-expectation problem made it "a small net loss". With that fixed, the
    remaining change is 47 input sites where a bare '12345' is replaced by a
    name that says what it is — the value is not in saving characters but in the
    fact that a reader currently has to reach one inline comment at one of 66
    sites to learn the literal must be digits or a resource name. Dropping it
    would also leave the stated task half-done. Flagging it here for the
    reviewer rather than silently keeping it.

Amaad Martin added 3 commits August 2, 2026 22:32
core/test/sessions/vertex_ai_session_service_test.ts repeated the bare app
name '12345' on 66 lines. Replace every occurrence with one module-level
APP_NAME constant, and derive the expected resource names from it
(`reasoningEngines/${APP_NAME}`) so the appName -> reasoningEngines/<id>
transformation the service performs is visible in the test source instead of
implied by two matching magic numbers.

The inline '// Must be digits or resource name' comment moves onto the
constant, where the reason lives with the value.

No assertion changes meaning: every substituted site evaluates to a
byte-identical string. The custom-engine-id, invalid-app-name and
my-project/999 literals are deliberately left alone -- each proves a specific
override or parsing behaviour and must stay visibly distinct from APP_NAME.
core/test/code_executors/agent_engine_sandbox_code_executor_test.ts repeated
an 88-character sandbox resource name 9 times and its 64-character agent
engine prefix 3 times. Replace them with TEST_PROJECT, TEST_LOCATION,
AGENT_ENGINE_NAME and SANDBOX_NAME at module scope.

AGENT_ENGINE_NAME is composed from TEST_PROJECT/TEST_LOCATION rather than
written flat because several of these tests only mean anything if the
project and location stubbed into the environment agree with the ones
embedded in the resource name -- the executor parses project and location out
of the resource name and falls back to the env vars. Composing makes that
required agreement explicit.

The 'us-central1' on the 'defaults location to us-central1 if missing in env'
assertion stays a literal: it pins the library's hard-coded default, which
merely happens to equal the test placeholder. The custom-p/custom-l,
custom-location and appName: '123' literals stay literal for the same reason
-- each proves a specific override or parsing behaviour.
…ently pinned

Review feedback: interpolating APP_NAME into the *expected* value of the
toHaveBeenCalledWith assertions made those expectations derive from the same
symbol fed in as input, so they no longer pinned the
appName -> reasoningEngines/<id> mapping on their own. Mutating APP_NAME left
the whole suite green, which is exactly the lost signal.

Restore the hardcoded expected strings at all 10 assertion sites (byte-for-byte
as before this branch, so those lines leave the diff entirely). Mutating
APP_NAME now fails 9 tests instead of 0.

APP_NAME still covers the 47 inputs, the 3 pass-through echo assertions and the
6 mock fixtures, where naming the placeholder is the whole point and nothing is
being pinned. Its doc comment now states that split so the remaining literals
read as deliberate rather than as missed occurrences.

Also note at the location-default assertion why it must not reuse
TEST_LOCATION: it pins the library default, which only happens to share the
value, and reusing the constant would make the test vacuous.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant