Feat: add the detectErrorInResponse tool telemetry hook - #606
Open
AmaadMartin wants to merge 3 commits into
Open
Feat: add the detectErrorInResponse tool telemetry hook#606AmaadMartin wants to merge 3 commits into
AmaadMartin wants to merge 3 commits into
Conversation
added 3 commits
August 3, 2026 18:49
A tool that reports a failure in-band, by resolving with a response such as
{status: 'ERROR'} rather than throwing, currently renders as a successful
execute_tool span, so the failure is invisible to operators.
Declare an optional detectErrorInResponse hook on BaseTool, consult it in
callToolAsync once the tool resolves, and set error.type plus an ERROR span
status from the detected type. Detection is skipped while the tool is
requesting auth or a confirmation, and any exception it raises is logged and
swallowed so telemetry can never break tool execution.
Adds unit tests for detectErrorTypeForTelemetry through the public handleFunctionCallList entry point (detected type, clean response, no hook, auth and confirmation skips, throwing detector, non-string detector result) and a real-span suite that records execute_tool spans with an in-memory OpenTelemetry exporter, so error.type and the ERROR span status are asserted on an exported span rather than on a stub.
…acer Relocates the seven tool error-detection cases out of functions_test.ts and into the real-span suite. The file-scope vi.mock they needed stubbed traceToolCall for the whole file, so all 30 pre-existing cases in functions_test.ts ran against a no-op tracer; that file is now untouched again, and every relocated case asserts the error.type attribute on a span recorded by an in-memory OpenTelemetry exporter. Also drops the redundant empty-string check from detectErrorTypeForTelemetry: traceToolCall already treats an empty error type as no error, and that behaviour is pinned by its own test.
AmaadMartin
pushed a commit
that referenced
this pull request
Aug 4, 2026
createAdkEventFromMetadata() restored `branch` straight from a remote
A2A peer's own response metadata (adk_branch), unlike `author` which is
always force-set by the caller. getContents() (content_processor_utils.ts)
uses an event's branch to keep sibling sub-agent conversation contexts
isolated from each other (a branch is visible in a given context only if
it is an ancestor of, or equal to, that context's current branch).
A malicious or compromised remote peer delegated a sub-task therefore
had two ways to break that isolation and inject its response into an
unrelated sibling sub-agent's LLM context:
- setting adk_branch to a shared ancestor branch (e.g. the parent
coordinator's branch instead of its own), or
- omitting adk_branch entirely, which the filter treats as "always
visible, in every branch".
This is the same class of bug fixed in #596 for actions.transferToAgent
(peer-controlled metadata able to corrupt local orchestrator state), on
a field that fix's allowlist didn't cover.
Fix: stop restoring `branch` in createAdkEventFromMetadata at all, and
thread it as an explicit parameter through toAdkEvent and its internal
per-event-type helpers instead, mirroring how `author` is already
force-set by the caller rather than trusted from peer metadata. The one
caller, A2ARemoteAgent.runAsyncImpl, now passes its own
InvocationContext.branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Link to an existing issue (if applicable):
N/A
Or, if no issue exists, describe the change:
Problem: Many tools never throw. They report failure in-band, resolving with
{status: 'ERROR', ...}or{error: '...'}, and the agent loop treats that as anormal return. The
execute_toolspan therefore renders as a green, successfulspan and the failure is invisible to operators. adk-python solves this with an
optional per-tool hook (
_detect_error_in_response) whose result is put on thetool span as
error.typeplus anERRORspan status; adk-js had no equivalent.Solution: Port the framework side of that mechanism — the extension point and
its single reader — in three coordinated pieces:
core/src/tools/base_tool.ts: declare an optional methoddetectErrorInResponse?(response: unknown): string | undefined. An optionalmethod declaration with no body is erased at compile time, so
BaseToolgainszero runtime surface and every existing subclass, in-repo and downstream,
still typechecks.
core/src/agents/functions.ts: a module-leveldetectErrorTypeForTelemetry(tool, toolContext, response)consulted fromcallToolAsyncwhere the function-response event handed totraceToolCallisbuilt. It returns
undefinedwhen the tool is requesting auth or confirmation(that response is a control signal, not a failure), when the tool declares no
hook, and when the hook returns a non-string. It never throws: an exception
from a detector is logged via
logger.errorand swallowed.core/src/telemetry/tracing.ts:TraceToolCallParamsgains an optionalerrorType; when present,traceToolCallsets theerror.typeattribute andspan.setStatus({code: SpanStatusCode.ERROR, message: errorType}). The statusis what actually stops the span rendering as successful. The message repeats
the type, never the response body, so no tool content escapes into an
attribute the
ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANStoggle cannot gate.Behaviour is unchanged for every tool that does not declare the hook: no new
attribute, no status change, no extra log line, and the emitted
function-response event is byte-for-byte what it is today.
Two deliberate divergences from adk-python, called out explicitly
1.
setStatusis parity — but only against current adk-pythonmain, socheck the revision.
trace_tool_callinsrc/google/adk/telemetry/tracing.pycomputes a
failure_typeand does:An adk-python checkout from before that landed sets only the attribute and
contains no
StatusCodeat all, so a stale reference reads as though adk-jsinvented the status. It did not — the comment in
tracing.tsis a paraphrase ofthe one above. The status is also the point of the change: the attribute alone
still leaves the span rendering green in a trace UI, which is the bug being
fixed.
Known asymmetry this PR does not fix:
callToolAsyncis a baretry/finallywith nocatch, so a tool that genuinely throws skipstraceToolCallentirely and its span endsUNSETwith noerror.type— anin-band failure is therefore reported more severely than a real exception.
adk-python has no such gap (it passes the exception to
trace_tool_call, whichrecords it and sets the same status). Closing it means tracing the throw path,
which changes the telemetry of every currently-crashing tool, so it is left as
its own change rather than half-fixed here.
2. The hook is public SDK surface, where adk-python's is not. adk-python
reaches the detector with
getattr(tool, '_detect_error_in_response', None)specifically "to avoid adding a public API on
BaseTool". TypeScript has noequivalent: calling an undeclared member requires a cast, and
as any/as unknown asare exactly what the contribution guidelines forbid. Declaringthe member optional is the only cast-free option, so
BaseToolgains six linesof public, documented extension point. The name drops Python's leading
underscore per the adk-js style guide; the observable strings (
error.type, anderror-type values such as
TOOL_ERROR) match adk-python byte for byte.Other notes
reader —
functions.ts— is added in this same change, and it is anextension point any user-defined tool can implement on day one. Landing a
producer first (e.g. on
McpTool) with nothing calling it would be thegenuinely dead ordering. Implementing the hook on concrete tools changes the
observable telemetry of shipping tools and is left to follow-ups.
instrumentation context, so detection happens on the post-after-callback
response. In adk-js the
execute_toolspan is opened and closed insidecallToolAsync, which wraps onlytool.runAsync. Detection therefore runs onthe tool's own return value — which is also the correct input, since the hook
is defined as a tool classifying its own response, not a value an after-tool
callback substituted.
SpanStatusCodecomes from@opentelemetry/api,already a
coredependency;error.typeis a localconstalongside theexisting attribute-key constants rather than pulling in
@opentelemetry/semantic-conventions. Nopackage.json/ lockfile change.any,as any,as never,as unknown as,@ts-expect-error,eslint-disableor coverage-ignore directive is addedanywhere in this diff (verified by grep over
git diff).Collision check (run before starting): I listed all 380 open PRs on the fork
(
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000) and read thefile lists of every plausibly adjacent one. No open PR implements
detectErrorInResponseor putserror.typeon the tool span(
gh search code --owner AmaadMartin detectErrorInResponsereturns nothing).The nearest neighbours are #399/#400 (port
resolveErrorTypeand the OTelmetrics contract —
error.typeon metric attributes intelemetry/metrics.ts,not on the
execute_toolspan), #27 (implementstraceToolCallat the mergedtool-call site), and #352/#441/#394 (other optional
BaseToolmembers). Theseoverlap in file only, not in behaviour, so this branches from
mainrather thanstacking.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
All tests live in one new file,
core/test/telemetry/tool_error_span_test.ts,and use no mocks of the code under test: a real
NodeTracerProvider+InMemorySpanExporter+SimpleSpanProcessorrecords the actual spans and everyassertion reads the exported span. No existing test file is modified.
describe('traceToolCall error reporting')— 3 cases callingtraceToolCallinside a real active span:
errorTypepresent →error.typeattribute +{code: ERROR, message: 'TOOL_ERROR'}status;errorTypeabsent → no attribute,UNSET;errorType: ''→ treated as absent.describe('execute_tool span for a tool that classifies its own response')— 7cases driven end to end through the public
functionsExportedForTestingOnly.handleFunctionCallList, each also asserting thefunction-response event handed back to the agent is exactly what the tool
returned:
{status: 'ERROR'}+ hook →error.type = 'TOOL_ERROR'andERRORstatus{status: 'OK'}+ same hook → no attribute,UNSETstatusUNSETstatustoolContext.requestConfirmation(...)duringrunAsync→ skippedtoolContext.requestCredential(...)duringrunAsync→ skippedlogger.erroris called exactly once with the tool nameAn earlier revision put cases 1-7 in
core/test/agents/functions_test.tsbehind afile-scope
vi.mockof the tracing module. That stubbedtraceToolCallfor thewhole file, so all 30 pre-existing cases there would have run against a no-op
tracer — the harness equivalent of a file-scope
eslint-disable. The cases weremoved here instead, where a real exporter proves strictly more, and
functions_test.tsis back to untouched.Commands run (targeted only, never the whole suite), all green on the pushed
commit:
Coverage of the new code: 100% of new lines and branches. Measured with
--coverage.includelimited to the touched sources; every uncovered line andbranch reported for
functions.tsandtracing.tsis pre-existing codeelsewhere in those files (e.g. the
tool.description || ''fallback and thecontent-toggle ternary at
tracing.ts:121,129-131), none is in the new helper,the new call-site argument, or the new
errorTypeblock (tracing.ts:134-141).Proof that each new test can fail. Every mutation below was applied to the
source, the targeted tests re-run, and the source restored:
isEmpty(...requestedAuthConfigs/requestedToolConfirmations)skip guardexpected { …(11) } to not have property "error.type"(confirmation and auth cases)try/catcharound the detectorexpected { error: 'detection exploded' } to deeply equal { result: 'tool executed' }(the detector's exception escapes into the tool's own response)typeof … === 'string'expected { …(11) } to not have property "error.type"errorTypeat thetraceToolCallcall siteexpected undefined to be 'TOOL_ERROR'andexpected "error" to be called 1 times, but got 0 timesspan.setStatus(...)intraceToolCallexpected { code: +0 } to deeply equal { code: 2, message: 'TOOL_ERROR' }if (errorType !== undefined)expected { …(11) } to not have property "error.type"error.typeattributeexpected undefined to be 'TOOL_ERROR'One honest caveat:
npm run ts:check(tsc --noEmit, not run by CI) reports 281pre-existing errors on
main, caused by@google/adkresolving tocore/distdeclarations inside
core/test. The new test file adds exactly one more error ofthat same pre-existing class (a tool typed from
@google/adkpassed to thesrc-importedtraceToolCall, which is not part of the public surface). Severalopen PRs are fixing that resolution duality repo-wide; nothing here works around
it with a cast.
Manual End-to-End (E2E) Tests:
No credentials, network or model access are needed — the real-span suite is the
manual E2E procedure, and it can be run directly:
To observe it by hand in an application, implement the hook on a tool and export
spans anywhere:
The
execute_tool InventoryToolspan now carrieserror.type = "TOOL_ERROR"andan
ERRORstatus; a tool that requests OAuth or a confirmation gate stillreports clean.
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.