Fix: require TEST_API_SERVER_PORT in the A2A multi-hop test agent - #314
Open
AmaadMartin wants to merge 3 commits into
Open
Fix: require TEST_API_SERVER_PORT in the A2A multi-hop test agent#314AmaadMartin wants to merge 3 commits into
AmaadMartin wants to merge 3 commits into
Conversation
added 3 commits
July 30, 2026 08:07
The multi_hop_remote_agent test agent is loaded inside the spawned test ADK API server and points back at that same server, so the port it needs is the random one the harness picked. The '|| 40000' fallback could therefore never be correct: it made a dropped TEST_API_SERVER_PORT surface much later as an opaque connection or agent-card failure that never named the variable. Validate the variable at module evaluation time and throw a self-contained error naming both the variable and AdkTsApiServer, the harness component responsible for propagating it. Parse with Number() rather than parseInt() so '41234abc' is rejected instead of silently coerced to 41234.
…gent Re-evaluate the agent module per case with vi.resetModules() and a stubbed environment, asserting the message for the missing and the malformed branches and pinning the agent card URL to the supplied port.
Drop the hardcoded harness file path from the runtime message (it rots on the next file move; AdkTsApiServer is greppable) and stop restating the doc comment in the error string.
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
No existing issue.
Problem:
tests/integration/a2a/input_required/test_agents/multi_hop_remote_agent.tssilently fell back to a hardcoded port:That fallback can never be correct. This agent is a self-loopback multi-hop: it is loaded inside the spawned test ADK API server and its agent card points back at that same server, whose port the harness picks at random —
BaseTestServer.getRandomPort()(tests/integration/test_case_utils.ts:275) returns40000 + Math.floor(Math.random() * 10000), so40000is right 1 time in 10000, by coincidence.The consequence is a misattributed failure.
AdkTsApiServer.start()(tests/integration/test_api_server.ts:39-53) is the only writer ofTEST_API_SERVER_PORT; if it ever stops propagating it, the agent still constructs fine and the breakage surfaces layers away, either as a connection error againstlocalhost:40000or — worse — as a successful connection to an unrelated process listening there. I simulated exactly that regression onmain(see the E2E section below): the suite fails withAssertionError: expected [] to include 'call-hop', which names neither the variable nor the harness. Non-numeric garbage was accepted too, since the value was only checked for truthiness.Solution: validate
TEST_API_SERVER_PORTat module evaluation time and throw a descriptiveErrorwhen it is missing, empty, or not a positive integer. Because the module is evaluated duringinitA2A()->AgentLoader.preloadAgents(), the throw fails server startup, the CLI logsError starting API server: <message>, and the harness reportsCLI exited prematurely with code 1with the message in the captured output.Design notes:
input_required_test.tsfail rather than one. That is intended: if the harness is not propagating the port, the harness is broken and the suite's result is meaningless.Number(), notparseInt().parseInt('41234abc', 10)returns41234— exactly the silent coercion this change removes.Number('41234abc')isNaN. There is a test case pinning this.<= 65535) check. Deliberately omitted, not an oversight: the only producer isgetRandomPort()(range[40000, 49999]), and an extra branch here would be scope creep with no real-world input to exercise it.error.messageis logged by the CLI (never the stack), so the message is self-contained. It namesAdkTsApiServeras the remediation point but deliberately does not hardcode that file's path, which would rot on the next file move.grep -rn TEST_API_SERVER_PORTover the repo returns exactly two hits (the writer and this reader), andgrep -rniE 'process\.env\.[A-Z_]*PORT'returns only this reader pluscore/src/telemetry/setup.ts, which reads OTLP endpoint URLs and is out of scope. No other test agent reads a port from the environment, so the change is one file plus one new test file. Nocore/src,dev/srcorintegrations/srcfile is touched; there is no public API, export, or TypeDoc impact, and no dependency change.gh pr list --repo AmaadMartin/adk-js --state open --limit 300(222 open PRs) plus a keyword search forTEST_API_SERVER_PORT/multi_hop_remote_agent/input_required. Nothing lands or overlaps this change. The closest neighbours were checked by diff: Fix: use REQUEST_CONFIRMATION_FUNCTION_CALL_NAME instead of the hardcoded wire literal in tests #269 touches the siblingtest_agents/tool_confirmation.tsandinput_required_test.ts(a wire-literal constant swap — different files, no conflict), Fix: stabilise install-bound and server-spawning integration suites #218 mentionsTEST_API_SERVER_PORTin its body but its diff does not touch it, and Fix: publish the bound port in A2A agent card URLs #300 concerns ephemeral-port agent card URLs indev/src/server/adk_api_server.ts, not the test agent. Branched frommain; no stacking needed.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
New file
tests/integration/a2a/input_required/multi_hop_remote_agent_test.ts(9 cases). It sits one directory abovetest_agents/on purpose:input_required_test.ts:20passestest_agents/as the server'sagentsDirandAgentLoader.preloadAgents()esbuild-bundles every file it finds there, so a test file dropped inside would be bundled withvitestinto the spawned server. It is still matched by theintegrationproject'stests/integration/**/*_test.tsglob.Each case stubs the environment with
vi.stubEnvand re-imports the module aftervi.resetModules()(the port is read once at module evaluation, so without the reset every case after the first silently reuses the first evaluation):TEST_API_SERVER_PORT'41234'rootAgent.name === 'multi_hop'; agent card ishttp://localhost:41234/a2a/multi_hop/''TEST_API_SERVER_PORT is not set.'abc','41234abc','0','-1','1.5',' 'TEST_API_SERVER_PORT must be a positive integer, got "<raw>".The agent card URL is not publicly readable off a constructed
RemoteA2AAgent(a2aConfigisprivate), so the happy-path case captures the constructor argument via avi.mockof@google/adkthat subclasses the realRemoteA2AAgent. Reaching in withrootAgent['a2aConfig']would have been a banned private-field escape hatch. No suppressions of any kind were added — noany,@ts-expect-error,eslint-disable, or coverage pragma — and the mock factory is typed viaimportOriginal<typeof import('@google/adk')>().Commands run (targeted only):
Note on
npx tsc --noEmit: the repo has ~350 pre-existing type errors incore/test/**and elsewhere onmain(the subject of other open PRs). Neither of my two files appears in that output.Proving the tests can fail (mutation runs). Coverage is not proof, and
vitest.config.ts:109-113restricts instrumentation tocore/src/dev/src/integrations/src, so files undertests/produce no coverage delta at all. Both mutations were run explicitly:Mutation 1 — restore the original
const port = process.env.TEST_API_SERVER_PORT || '40000';. All 8 negative cases FAIL; the happy path still passes, correctly, since it never reaches the fallback:Mutation 2 — keep the validation but hardcode the URL (
http://localhost:40000/a2a/multi_hop/), which pins the remaining case, i.e. that the validated port is actually the one used:Both mutations were reverted and the suites re-run green.
Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
This reproduces the harness regression the change exists to catch. It runs a real spawned ADK API server — no mocks.
npm install && npm run build(the harness spawnsdev/dist/esm/cli_entrypoint.js).tests/integration/test_api_server.ts:45, delete theTEST_API_SERVER_PORT: this.port.toString(),line so the child is spawned with plainprocess.env.npx vitest run --project integration tests/integration/a2a/input_required/input_required_test.ts.Before this change (step 3 against
main) — two tests pass and the multi-hop one fails with an assertion that names neither the variable nor the harness:After this change (step 3 on this branch) — startup fails immediately and the output says exactly what is wrong and who is responsible:
Step 4 (harness restored) returns the suite to
Tests 3 passed (3).Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.