Skip to content

Test: drop the unresolvable @google/adk deep import from the sandbox executor test (stacked on #486) - #632

Open
AmaadMartin wants to merge 2 commits into
fix/code-execution-response-processor-defaultfrom
fix/auto-wire-code-execution-response-processor
Open

Test: drop the unresolvable @google/adk deep import from the sandbox executor test (stacked on #486)#632
AmaadMartin wants to merge 2 commits into
fix/code-execution-response-processor-defaultfrom
fix/auto-wire-code-execution-response-processor

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    No public GitHub issue was supplied with this task.
  2. Or, if no issue exists, describe the change:

Stacked on #486 (--base fix/code-execution-response-processor-default). This PR is the residual cleanup that #486 leaves behind; it is deliberately not a competing implementation of the auto-wiring.

Collision check (performed before any code was written)

gh pr list --repo AmaadMartin/adk-js --state open --limit 1000 --json number,title,headRefName

531 open PRs scanned. Four are adjacent to the code-execution response processor:

PR What it does Relation
#486 Wires CODE_EXECUTION_RESPONSE_PROCESSOR into the LlmAgent defaults Already lands the auto-wiring. I did not reimplement it — I stacked on it.
#42 Earlier form of the same wiring (aliased import) Superseded by #486
#561 Stacked on #486, extends the same default path Sibling
#460 (+#512) Exports the singleton from core/src/common.ts Conflicting design — see note below

The assigned task was "auto-wire the code-execution response processor into LlmAgent". #486 implements that in full, so building a fourth implementation would have been duplicate work colliding on the same lines. What #486 does not do is satisfy the task's final postcondition — no @google/adk/... deep subpath specifier anywhere under tests/. This PR closes exactly that gap and nothing else.

Problem: tests/integration/agents/agent_with_sandbox_executor_test.ts imports the response processor across the package boundary:

import {CODE_EXECUTION_RESPONSE_PROCESSOR} from '@google/adk/agents/processors/code_execution_request_processor.js';

core/package.json declares only "." in its exports map, so this is not a resolvable subpath of the published package. It only resolves in-repo because vitest.config.ts aliases @google/adk to ./core/src as a prefix alias. This is not merely stylistic — it is a real type error that tsc already reports today:

tests/integration/agents/agent_with_sandbox_executor_test.ts(10,49): error TS2307:
Cannot find module '@google/adk/agents/processors/code_execution_request_processor.js'
or its corresponding type declarations.

(npm run ts:check is not one of the CI steps in .github/workflows/validation.yaml, which is why this has survived.)

Solution: Now that #486 installs the processor by default, no test needs to reach across the package boundary to obtain it. Drop the import, and retarget the one test that consumed it.

That test (executes code generated by the agent) hand-installed the processor via responseProcessors: [CODE_EXECUTION_RESPONSE_PROCESSOR]. Simply deleting that property would have made it a byte-for-byte duplicate of the executes code with no explicit responseProcessors case #486 added. Instead it now pins the complementary branch of config.responseProcessors ?? [...]: a caller-supplied array replaces the default, so no code is executed. The two tests together now cover both sides of that ??, where before neither test distinguished them.

On the coverage the retarget displaced. The old assertions were events.length >= 3 and hasExecutionResult (any part whose text contains 'hello'). The "code executes" half of that is subsumed, strictly more strongly, by the surviving executes code with no explicit responseProcessors case, which asserts executeCodeInternal was called exactly once, resultParts has length 1, outcome Outcome.OUTCOME_OK, text containing 'hello', and final event text 'Execution was successful.'. The old hasExecutionResult predicate was in fact too weak to be a regression signal at all: the model's first mocked response text already contains print("hello"), so it passed even with the response processor absent.

Subsumption did not hold for the other thing that test pinned, however: an explicit, non-empty responseProcessors list containing the code-execution processor still executes code. That is a distinct branch of config.responseProcessors ?? [...] — and it is the branch whose semantics this stack changes. It is restored as its own case in core/test/agents/llm_agent_test.ts:

executes a model-emitted code block when an explicit responseProcessors list
includes the code-execution processor

All four branches are now covered: default with executor, default without executor, explicit empty list, explicit non-empty list.

It is restored in the unit test rather than the integration test deliberately: reconstructing it under tests/integration/ would require re-adding the very @google/adk deep import this PR exists to remove. In core/test/, importing the non-exported singleton via a relative ../../src/... path is the established in-package pattern and resolves correctly.

One subtlety worth flagging, because it constrains how the test is written. The explicit list is read back from a default-constructed agent (new LlmAgent({name: 'probe_agent'}).responseProcessors) rather than written literally as [CODE_EXECUTION_RESPONSE_PROCESSOR]. Passing the relatively-imported singleton straight into an LlmAgentConfig typed through @google/adk mixes the core/src and core/dist/types copies of BaseLlmResponseProcessor and is a hard TS2322. Reading the array back keeps both sides on one side of that boundary — and it mirrors the only route actually available to an external caller, since the singleton is not public API. The test asserts expect(explicitProcessors).toContain(CODE_EXECUTION_RESPONSE_PROCESSOR) first, so it cannot silently degenerate into the default case.

Shared setup and assertions for the two end-to-end unit cases are hoisted into runCodeExecutionAgent / expectCodeWasExecuted; the assertions are unchanged in substance. In the sandbox test, createSandboxFixture() now builds the mock client and executor together, reducing the as unknown as Client boundary casts from two to one.

Design decision (inherited from #486, restated for the reviewer)

Auto-wire the processor; keep the singleton internal. Justified by adk-python, where src/google/adk/flows/llm_flows/single_flow.py installs _code_execution.request_processor (line 68) and _code_execution.response_processor (line 80) by default, and the module is _code_execution — underscore-prefixed, i.e. deliberately private. Users get it by construction, never by import. This PR is consistent with that: it removes the only import site rather than adding a public export.

This is the point on which #460 and #486 disagree. #460 removes the same deep specifier by exporting the singleton from core/src/common.ts, which makes the adk-js public surface strictly larger than Python's and is asymmetric with CODE_EXECUTION_REQUEST_PROCESSOR, which is likewise unexported. #460 and #486 cannot both merge as written — that is a maintainer call, flagged here rather than pre-empted.

Behavior change note (from the stack as a whole)

Setting codeExecutor on an LlmAgent is now sufficient to execute model-written fenced code blocks; previously the feature was inert for every non-built-in executor. Two consequences worth calling out:

  • An agent that sets codeExecutor purely to back the run_skill_script_tool fallback (core/src/tools/skill/run_skill_script_tool.ts) will now also execute fenced code blocks the model writes in free text. That is the intended meaning of codeExecutor and matches adk-python.
  • One pre-existing throw becomes reachable that previously was not: postProcessCodeExecutionResult in core/src/agents/processors/code_execution_request_processor.ts throws 'Artifact service is not initialized.' when invocationContext.artifactService is undefined. This only fires on a configuration that was already non-functional, and InMemoryRunner/Runner supply an artifact service. Behavior deliberately left unchanged.

BuiltInCodeExecutor and runConfig.supportCfc runs are unaffected — the response processor returns early for built-in executors.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

npx vitest run tests/integration/agents/agent_with_sandbox_executor_test.ts \
                core/test/agents/llm_agent_test.ts \
                core/test/agents/processors/code_execution_request_processor_test.ts
-> Test Files  3 passed (3)
        Tests  42 passed (42)

Proof the tests can fail (mutation testing). Two mutations were applied to core/src/agents/llm_agent.ts, each caught by the complementary test — neither test passes vacuously.

Mutation A — swap nullish coalescing for a truthy-length check (the realistic bug this PR's retargeted test guards against, where an explicitly-supplied empty array is wrongly treated as "not provided"):

-    this.responseProcessors = config.responseProcessors ?? [
-      CODE_EXECUTION_RESPONSE_PROCESSOR,
-    ];
+    this.responseProcessors = config.responseProcessors?.length
+      ? config.responseProcessors
+      : [CODE_EXECUTION_RESPONSE_PROCESSOR];
× Agent with AgentEngineSandboxCodeExecutor > does not execute code when responseProcessors is overridden
  → AssertionError: expected "spy" to not be called at all, but actually been called 1 times
    Number of calls: 1
✓ Agent with AgentEngineSandboxCodeExecutor > executes code with no explicit responseProcessors
  Tests  1 failed | 1 passed (2)

Mutation B — revert the default to [] (i.e. undo #486's fix entirely):

-    this.responseProcessors = config.responseProcessors ?? [
-      CODE_EXECUTION_RESPONSE_PROCESSOR,
-    ];
+    this.responseProcessors = config.responseProcessors ?? [];
AssertionError: expected "spy" to be called 1 times, but got 0 times
AssertionError: expected [] to include CodeExecutionResponseProcessor{}
AssertionError: expected [] to include CodeExecutionResponseProcessor{}
AssertionError: expected [] to have a length of 1 but got +0
  Tests  4 failed | 29 passed (33)

Under Mutation B the retargeted override test correctly still passes — it pins the other branch.

Mutation C — discard any caller-supplied list (the mutation that isolates the restored explicit-non-empty branch):

-    this.responseProcessors = config.responseProcessors ?? [
-      CODE_EXECUTION_RESPONSE_PROCESSOR,
-    ];
+    this.responseProcessors = config.responseProcessors
+      ? []
+      : [CODE_EXECUTION_RESPONSE_PROCESSOR];
× LlmAgent Default Response Processors > executes a model-emitted code block when an
  explicit responseProcessors list includes the code-execution processor
  → AssertionError: expected [] to have a length of 1 but got +0
× LlmAgent Abort Handling > should stop execution when abortSignal is aborted during
  response processors
  → AssertionError: expected 'test_agent' to be 'processor'
  Tests  2 failed | 32 passed (34)

The other three branch tests pass under Mutation C, confirming the restored case carries signal none of them do. (The pre-existing abort test also supplies a custom list, so it is legitimately caught too.)

Source was restored from a pristine copy after each mutation and re-verified green (42 passed).

Type-error evidence. Diffing tsc --noEmit --pretty false output before and after this change, the single error removed is exactly the deep import:

281 errors on the base (fix/code-execution-response-processor-default)
280 errors with this change
< tests/integration/agents/agent_with_sandbox_executor_test.ts(10,49): error TS2307:
    Cannot find module '@google/adk/agents/processors/code_execution_request_processor.js'

The remaining 280 are pre-existing and identical on the base — they are core/dist/types vs core/src duplicate-declaration conflicts produced by running tsc --noEmit after npm run build in one checkout, in files this PR does not touch. ts:check is not a CI step.

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

CI note: ci_status = absent. .github/workflows/validation.yaml triggers on pull_request: branches: [main], and this PR targets fix/code-execution-response-processor-default, so the run-tests job will not fire. Every CI gate was therefore run locally on the exact pushed commit:

npm run build          -> ok
npm run lint           -> clean (no output)
npx prettier tests/integration/agents/agent_with_sandbox_executor_test.ts --check
                       -> All matched files use Prettier code style!
npm run docs:check     -> EXIT=0
npx secretlint tests/integration/agents/agent_with_sandbox_executor_test.ts
                       -> clean (no output)

To walk the loop by hand, run the sandbox integration test above: the mocked model emits a fenced ```python block, the executor's executeCodeInternal is invoked once, a codeExecutionResult part with Outcome.OUTCOME_OK is emitted, and the model then summarises. No credentials or live model are required.

Verification that the postcondition now holds:

grep -rn "@google/adk/" tests/ --include=*.ts   -> no matches

(Two deep specifiers remain under core/test/sessions/vertex_ai_session_service_test.ts; those are outside this postcondition's scope and are already addressed by #512.)

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

Amaad Martin added 2 commits August 4, 2026 05:08
…executor test

The sandbox integration test imported the code-execution response processor
via '@google/adk/agents/processors/code_execution_request_processor.js'.
core/package.json declares only "." in its exports map, so that specifier is
not a resolvable subpath of the published package -- it only resolved locally
because vitest.config.ts aliases @google/adk to ./core/src as a prefix. tsc
reports it as TS2307 "Cannot find module".

Now that the processor is installed by default, no test needs to reach across
the package boundary to get it. Drop the import and retarget the test that was
its only consumer: instead of hand-installing the processor (which the new
default-wiring case already covers with strictly stronger assertions), it now
pins the complementary override branch -- a caller-supplied responseProcessors
array replaces the default, so no code is executed.
The previous commit retargeted the sandbox test that used to pin "an explicit
responseProcessors list containing the code-execution processor still executes
code". The default-wiring case does not subsume that branch, so it was left
untested. Restore it as its own case in the unit test, where importing the
non-exported singleton relatively is the established in-package pattern and
does not need an unresolvable @google/adk subpath.

The explicit list is read back from a default-constructed agent rather than
built from the relatively-imported singleton directly. That keeps the value and
the LlmAgentConfig type on the same side of the core/src vs core/dist boundary
(mixing them is a TS2322), and it mirrors the only route actually available to
an external caller, since the singleton is not part of the public API.

Shared setup and assertions for the two end-to-end cases are hoisted into
runCodeExecutionAgent / expectCodeWasExecuted; assertions are unchanged in
substance. In the sandbox test, the mock client and executor are now built by
one createSandboxFixture helper, which drops one of the two boundary casts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant