Skip to content

Fix: force established sockets shut in AdkApiServer.stop() so teardown always settles - #452

Open
AmaadMartin wants to merge 1 commit into
mainfrom
fix/adk-api-server-stop-close-all-connections
Open

Fix: force established sockets shut in AdkApiServer.stop() so teardown always settles#452
AmaadMartin wants to merge 1 commit into
mainfrom
fix/adk-api-server-stop-close-all-connections

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    No existing issue.
  2. Or, if no issue exists, describe the change:
    Problem: AdkApiServer.stop() (dev/src/server/adk_api_server.ts) is a bare wrapper around http.Server#close. Per the Node.js docs, close() "stops the server from accepting new connections and closes all connections connected to this server which are not sending a request or waiting for a response", and its callback only fires once every remaining connection has drained. It never forces an established socket shut, so a single client parking a connection wedges teardown and await server.stop() never settles. This is not theoretical for this server: the SSE route sets Connection: keep-alive, calls res.flushHeaders(), and streams for as long as the agent runs, so one attached /run_sse client — or one browser tab on the dev UI mid-connect — is enough to hang shutdown and hold the port.

Measured on Node v22.22.2 (node -v in this workspace), with a plain http.Server and no mocks:

Socket state when close() is called Plain close() close() + closeAllConnections()
No connections at all settles ~0 ms settles ~0 ms
Idle keep-alive socket (previous request completed) settles ~2 ms settles ~0 ms
TCP socket connected, no request bytes sent never settles settles ~1 ms
Partial request headers written, never terminated never settles settles ~0 ms
In-flight SSE response (headers flushed, stream open) never settles (measured: still hung at 2000 ms) settles ~1 ms

Solution: call this.server!.closeAllConnections() inside the same promise executor, immediately after this.server!.close(...). Two statements plus a comment; no new file, type, option, export, or dependency.

Design notes:

  • Why closeAllConnections() and not manual socket tracking. The alternative — a Set<net.Socket>, a connection listener, a per-socket close listener to evict entries, and teardown on the error path — is a leak-prone accumulator reimplementing what the runtime already maintains correctly.
  • Why after close(). The Node docs say explicitly: "Whenever using this in conjunction with server.close, calling this after server.close is recommended as to avoid race conditions where new connections are created between a call to this and a call to server.close." Both calls stay synchronous within the executor, so nothing can be accepted in between.
  • Why not closeIdleConnections(). It only reaps sockets that are neither sending a request nor awaiting a response — precisely the set close() already reaps on Node >= 19. Verified: an in-flight SSE stream still hangs with close() + closeIdleConnections().
  • Runtime floor. closeAllConnections() is @since v18.2.0 per the declaration in the repo's own pinned @types/node (dev/package.json pins ^20.12.7; node_modules/@types/node/http.d.ts:444). README.md:55 states the only declared requirement — "ADK for TypeScript requires a current Node.js LTS release" — and every supported LTS line clears 18.2.0. core/test/a2a/agent_to_a2a_body_parsing_test.ts already calls closeAllConnections() for this exact reason, so the API is established precedent here. This PR deliberately does not add an engines field; that gap is separate and unrelated to the hang.
  • No error-path churn. The close(err => ...) callback still rejects verbatim, so ERR_SERVER_NOT_RUNNING on a double stop() or a failed listen() is unchanged — dev/test/server/adk_api_server_test.ts:1213 relies on that via .catch(() => {}) and still passes. closeAllConnections() is a synchronous void method that does not throw on a never-listened, listen-failed, already-closed, or listening server, so it is not wrapped in try/catch, and no ?.() guard is added (this.server is already narrowed and the method is non-optional in @types/node).

Behavioural change, stated plainly: stop() no longer waits for in-flight requests to complete — established connections are destroyed, and clients see an abrupt close (ECONNRESET / TypeError: terminated from fetch). A caller relying on stop() as a graceful drain would notice. This is intended and assessed as acceptable: this is the local development / test API server, stop() has no production call site in dev/src (grepped — the only callers are tests; the CLI has no SIGINT handler that invokes it), and the previous "graceful" behaviour was in practice an unbounded hang rather than a bounded drain. A desirable side effect: destroying an SSE socket fires the existing req.on('close') handler, which calls abortController.abort(), so shutdown cancels in-flight agent runs through the cancellation path that already exists instead of orphaning them.

Collision check (required before implementation): gh pr list --repo AmaadMartin/adk-js --state open --limit 400 returned 353 open PRs; I filtered titles/branches for stop|close|connection|socket|server|shutdown|listen|hang|teardown|drain|destroy|settle and diffed every plausibly adjacent PR. Four open PRs touch dev/src/server/adk_api_server.ts (#252, #199, #300) or mention stop() (#359), and none of them adds closeAllConnections() or otherwise changes stop(): #252 consolidates the run handlers, #199 adds a cross-origin guard, #300 fixes A2A agent-card URLs, #359 only names hook timeouts in webui_test.ts. No collision, so this is branched from main rather than stacked. (#359's PR body asserts stop() is "http.Server.close()" when sizing its afterAll budget; that description becomes stale with this change, but the two diffs do not overlap.)

Out of scope, untouched: tests/integration/adk_web/webui_test.ts, core/test/a2a/agent_to_a2a_body_parsing_test.ts, any engines declaration, stop() idempotence (a second call still rejects with ERR_SERVER_NOT_RUNNING), and any new configuration knob — a grace-period/timeout option would be write-only config with no reader.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

Two tests added to dev/test/server/adk_api_server_test.ts as a new describe('Shutdown', ...) block. No existing test was edited, skipped, weakened, or deleted.

  1. should resolve stop() while a client holds a connection open — the regression test. Starts a dedicated AdkApiServer, opens a raw net.connect socket and awaits its connect event, then asserts stop() resolves. The socket is deliberately connected but has sent no request bytes: an idle keep-alive socket would be reaped by close() on Node >= 19 and the test would pass with and without the fix. An error listener is attached before stopping because the server-side destroy surfaces as ECONNRESET and an unhandled net.Socket error crashes the worker; the socket is destroyed in a finally. The host is pinned to 127.0.0.1 because the default localhost resolves to ::1 on some machines (verified locally), which would make the client connect fail rather than exercise the bug. Bound is Vitest's default 5 s timeout — the gap being measured is ~1 ms vs. infinite, so no Promise.race against a hand-rolled timer was added.
  2. should resolve stop() with no connections attached — pins that the forceful destroy did not break the ordinary path, and guards the close()-then-closeAllConnections() ordering.
npx vitest run --project unit:dev dev/test/server/adk_api_server_test.ts
  Test Files  1 passed (1)
       Tests  53 passed (53)

Proof the tests can fail (mutation). Deleted exactly the added line this.server!.closeAllConnections(); from stop(), leaving everything else in place, and re-ran the file:

 × AdkWebServer > Shutdown > should resolve stop() while a client holds a connection open 5025ms
   → Test timed out in 5000ms.
 ✓ AdkWebServer > Shutdown > should resolve stop() with no connections attached 6ms

 Test Files  1 failed (1)
      Tests  1 failed | 1 passed | 51 skipped (53)

Test 1 hangs and fails on the 5 s timeout without the fix; Test 2 still passes, confirming it pins the ordinary path rather than the bug. The line was then restored and the full file re-run green.

Coverage. The new production statement is on the already-exercised stop() happy path. npx vitest run --project unit:dev dev/test/server/adk_api_server_test.ts --coverage.include='dev/src/server/adk_api_server.ts' reports the file at 84.06% lines / 68.65% branches overall (pre-existing, unrelated routes), and the added line is not in the uncovered list — 100% of the new code is covered. The one uncovered line inside stop() is the pre-existing return Promise.resolve() early return for a never-started server (991-992), whose behaviour this change does not touch; per the plan no test was added for it.

Regression suites that already exercise real start()/stop() over real sockets (unedited, still green):

npx vitest run --project integration tests/integration/adk_web/webui_test.ts
  Test Files  1 passed (1)
       Tests  2 passed (2)

CI gates, run locally on the pushed commit:

npm run build          # ok
npm run lint           # clean
npm run format:check   # "All matched files use Prettier code style!"
npm run docs:check     # clean (no exported surface changed)
npm run ts:check       # no diagnostics in either changed file

npm run ts:check reports pre-existing errors in 41 unrelated files on main; neither dev/src/server/adk_api_server.ts nor dev/test/server/adk_api_server_test.ts appears in that output.

CI: run-tests green on ubuntu-latest, macos-latest, and windows-latest. The first windows-latest attempt failed on core/test/code_executors/unsafe_local_code_executor_test.ts > should execute shell code and return stdout ("Test timed out in 5000ms") — an unrelated pre-existing Windows shell flake in a file this PR does not touch (already being addressed by #254 and #246); it passed on re-run with no code change.

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

Note that the CLI installs no SIGINT handler that calls stop() (grepped dev/src), so a Ctrl-C on adk web does not route through this code path. The equivalent real e2e drives the built package over real sockets with no mocks:

  1. npm install && npm run build.
  2. Run the following against the built @google/adk-devtools (adjust the import path to your checkout):
import net from 'node:net';
import {AdkApiServer} from './dev/dist/esm/index.js';

const server = new AdkApiServer({host: '127.0.0.1', serveDebugUI: true});
await server.start();
const port = Number(new URL(server.url).port);

// A real client holding keep-alive after a completed request.
await (await fetch(`http://127.0.0.1:${port}/list-apps`)).text();
// A parked socket that never sends a request (a connecting browser tab).
const parked = net.connect(port, '127.0.0.1');
parked.on('error', () => {});
await new Promise((r) => parked.once('connect', r));
// A half-written request that never terminates its headers.
const partial = net.connect(port, '127.0.0.1');
partial.on('error', () => {});
await new Promise((r) => partial.once('connect', r));
partial.write('GET /list-apps HTTP/1.1\r\nHost: localhost\r\n');

const t0 = Date.now();
await server.stop();
console.log('stop() settled after', Date.now() - t0, 'ms');

// The port must be free immediately.
const rebind = new AdkApiServer({host: '127.0.0.1', port});
await rebind.start();
await rebind.stop();

Before the fix: stop() never settles. After the fix (measured): stop(): settled after 1ms; port 43873 rebindable: true. 3. SSE path, verified separately against a plain http.Server reproducing the /run_sse shape (Connection: keep-alive + res.flushHeaders() + a written chunk + a response that never ends), with the client holding the stream open: close() alone was still hung at the 2000 ms cutoff, while close() + closeAllConnections() settled in 1 ms and fired the server-side req.on('close') handler — the same handler the dev server uses to call abortController.abort() and log "HTTP connection closed. Aborting agent SSE execution for session …".

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

http.Server#close() stops the listener but then waits for every established
connection to drain, so a client parking a socket -- a browser tab on the dev
UI, or an in-flight /run_sse stream -- left stop() unsettled forever. Call
closeAllConnections() after close(), per the Node guidance, so teardown is
bounded regardless of attached clients.

Adds a regression test that parks a raw connected socket across stop(), plus a
no-connections test pinning the ordinary path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant