Test: Pin the ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS tracing gate with hermetic unit tests - #308
Open
AmaadMartin wants to merge 2 commits into
Open
Test: Pin the ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS tracing gate with hermetic unit tests#308AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
July 30, 2026 05:52
…ESSAGE_CONTENT_IN_SPANS
The suite never stubbed the capture kill switch, so three of its assertions
silently depended on the ambient environment of the machine running them.
On a developer box or CI image that exports
ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=false, the gated span attributes collapse
to '{}' and the suite goes red through no fault of the code under test.
Pin the variable to its default in beforeEach and unstub it in afterEach
(vi.restoreAllMocks() does not unstub envs, and unstubEnvs is not enabled in
vitest.config.ts, so this has to be explicit).
…ing gate shouldAddRequestResponseToSpans() decides whether ADK writes potentially PII-bearing request/response payloads into OpenTelemetry spans, and had zero test coverage: all six gated call sites were only ever exercised on the default capture-ON path. Add a characterization matrix over the eight env values the implementation distinguishes, driven entirely through the public tracing entrypoints (traceCallLlm, traceToolCall, traceMergedToolCalls, traceSendData) so the module-private helper stays private. The empty-string and 'TRUE' rows are the load-bearing ones: '' is falsy so it takes the || 'true' default, and the comparison is case-sensitive so 'TRUE' disables capture. Both pin behaviour that contradicts the helper's JSDoc; the tests deliberately pin what the code does today. One further test asserts the gate only redacts payloads and never suppresses ungated telemetry.
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
Problem:
core/src/telemetry/tracing.tshas a module-private privacy kill switch,shouldAddRequestResponseToSpans(), that decides whether ADK writes potentially PII-bearing request/response payloads into OpenTelemetry spans. It is read fromADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS. Two defects:ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANSreturned exactly one hit —core/src/telemetry/tracing.ts:408. All six gated call sites (tracing.ts:122,:148,:189,:228,:247,:309) were only ever executed on the default capture-ON path.core/test/telemetry/tracing_test.tsnever stubbed the variable, so three of its assertions silently depended on the ambient environment of the machine running them. Notevi.restoreAllMocks()does not unstub env vars andunstubEnvsis not enabled invitest.config.ts, so this has to be explicit.Reproduction, before this change:
After this change all three env states pass identically (
2241 passed); see Testing Plan.Solution: A test-only characterization change touching exactly one file,
core/test/telemetry/tracing_test.ts.core/src/telemetry/tracing.tsis untouched —git diff main --stat -- core/srcprints nothing, andgit diff main --statis a single test file (+227/-1).beforeEach(vi.stubEnv(CAPTURE_ENV_VAR, undefined)deletes it in Vitest 3) andvi.unstubAllEnvs()inafterEach, making the suite immune to ambient environment. Existing test bodies are unchanged.describe('ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS gate', ...)matrix over the eight values the implementation distinguishes, each exercised against all four public entrypoints (traceCallLlm,traceToolCall,traceMergedToolCalls,traceSendData).The matrix, and why each row exists:
envValueundefined(deleted)undefined || 'true'→'true'''''is falsy, so|| 'true'applies'true''1''false''0''TRUE''not-a-boolean'These tests pin what the code does today, not what its JSDoc claims. The docstring on
shouldAddRequestResponseToSpanssays capture is disabled "only when ... explicitly set to'false'or'0'", but the implementation isenvValue === 'true' || envValue === '1', so'TRUE'and'not-a-boolean'also disable it. The'TRUE'and'not-a-boolean'rows deliberately encode the implementation, and Mutation B below shows they are the only tests that would notice if someone "fixed" the code to match the docstring. Reconciling the docstring with the implementation is intentionally out of scope here and is tracked separately.Design notes:
shouldAddRequestResponseToSpans"for the test", and noobj['privateField']access is used.collectSpanAttributesflattens the span recording so an assertion need not know whether an attribute arrived viasetAttributes(bulk) orsetAttribute(single). No gated key arrives via both forms. It is typed with Vitest'sMock<...>— noany.traceToolCall/traceMergedToolCallshardcodegcp.vertex.agent.llm_request/llm_responseto'{}'and do not gate them (tracing.ts:120-121, 183-184). Asserting'{}'on those in the capture-disabled case would pass either way, so only genuinely gated keys are asserted.traceSendDatacurrently has no caller inadk-js(its live flow is still a stub) but is an exported entrypoint that reads the gate, so it is covered. It is not wired into any flow here.git diff main -U0 | grep -E '@ts-expect-error|@ts-ignore|eslint-disable|as any|: any\b|as unknown as|v8 ignore'returns nothing. The single pre-existing// eslint-disable-next-line @typescript-eslint/no-explicit-anyonlet mockSpan: anyis untouched and was not widened.One disclosure:
npm run ts:checkis not a CI gate today and reports 314 pre-existing errors repo-wide (8 of them already in this file). My new call sites add 6 more of the identical pre-existing class — fixtures typed via@google/adk(which plaintscresolves tocore/dist/types) passed into functions imported fromcore/src. This is thets:checkdist/src resolution problem, not a defect in these tests; the only local "fix" would be a cast, which would be a type-checker suppression. I deliberately did not add one, and followed the pattern already established at the 8 existing sites in the same file.Collision check (required before starting):
gh pr list --repo AmaadMartin/adk-js --state open --limit 100plusgh pr diff --name-onlyacross all 100 open PRs found no PR touchingcore/src/telemetry/tracing.tsorcore/test/telemetry/tracing_test.ts. Two PRs are thematically adjacent but disjoint in files: #302 (tests/hermetic_env*.ts,vitest.config.ts) and #281 (core/test/telemetry/setup_test.ts,vitest.config.ts). This change does not touchvitest.config.tsand uses explicitvi.unstubAllEnvs(), so it is compatible with either landing first. Branched frommain, not stacked.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
Proof the tests can fail (mutation testing). Coverage is a floor, not proof, so every new test was run against a deliberately broken gate. Five mutations of
core/src/telemetry/tracing.ts, each reverted afterwards:|| 'true'→|| 'false'unset,''rowsreturn envValue === 'true' || envValue === '1'→return envValue !== 'false' && envValue !== '0'(i.e. implement the docstring)'TRUE','not-a-boolean'rowsreturn true(gate always on)'1'disjunct'1'row'true'disjunct'true',unset,''rowsEvery one of the 33 new tests is killed by at least one mutation. Sample failure messages:
Mutation B is the important one: it is exactly the behaviour the JSDoc describes, and only the two new rows that pin the case-sensitive implementation catch it. Every mutation was reverted;
git diff main --stat -- core/srcprints nothing.Branch coverage of the code under test.
shouldAddRequestResponseToSpansand all six gated ternaries are fully covered, verified from the v8 JSON report (no zero-count branch attracing.ts:122, 148, 189, 228, 247, 309, 407-409). File-leveltracing.tscoverage from this suite is 79.32% lines / 75.55% branches; the shortfall is entirely inbindOtelContextToAsyncGenerator/runAsyncGeneratorWithOtelContext/ parts ofbuildLlmRequestForTrace, which are unrelated to this gate and out of scope.vitest.config.tscoverage thresholds were not touched.Manual End-to-End (E2E) Tests:
No integration or e2e test is added: the gate is a pure
process.envread inside the telemetry module with no I/O, no network, and no cross-component contract, so an integration test would only re-exercise the same four functions behind more machinery.To verify manually, from the repository root:
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.