Feat: Port TokenUsage and resolveErrorType from adk-python (Part 1/2) - #399
Open
AmaadMartin wants to merge 1 commit into
Open
Feat: Port TokenUsage and resolveErrorType from adk-python (Part 1/2)#399AmaadMartin wants to merge 1 commit into
AmaadMartin wants to merge 1 commit into
Conversation
adk-js reads token counts straight off usageMetadata in traceCallLlm, with its own idea of what "input tokens" means: prompt tokens only, dropping tool-use tokens, and candidate tokens only, dropping reasoning tokens. Both are under-reported on every span, and there is no way at all to turn a caught error into an error.type label. TokenUsage centralises the aggregation the GenAI semantic conventions ask for -- prompt + tool-use as input, candidates + reasoning as output, keeping the distinction between "not reported" and "reported as zero" that a truthiness check would lose -- and traceCallLlm now sets its attributes, matching what adk-python's _set_usage_metadata_attributes does. Spans consequently gain the cache-read and reasoning token attributes, and their input/output counts are no longer short. resolveErrorType is the JS twin of tracing.resolve_error_type: a self-classified errorType wins, then the HTTP status of a @google/genai ApiError (matched on shape, not instanceof, because two copies of the SDK can coexist in one dependency tree), then the error name. It takes unknown, because anything can be thrown in JavaScript, and lives in utils/error_utils.ts rather than in tracing.ts -- it is not tracing specific and tracing has no caller for it. Neither module is added to the public entry points.
AmaadMartin
force-pushed
the
feat/telemetry-metrics-token-usage-part1
branch
from
July 31, 2026 17:19
2027124 to
22824da
Compare
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: N/A
Related: N/A
Problem:
adk-jshas no equivalent of adk-python'stelemetry/_token_usage.pyortracing.resolve_error_type. Token counts are read ad hoc wherever they are needed —tracing.tssetsgen_ai.usage.input_tokensfrompromptTokenCountalone, ignoring tool-use tokens, andgen_ai.usage.output_tokensfromcandidatesTokenCountalone, ignoring reasoning tokens — and there is no way to turn a caught error into anerror.typelabel at all. The metrics port that follows (Part 2/2) needs both.Solution: Port the two pieces, faithfully:
core/src/telemetry/token_usage.ts— aTokenUsageclass overGenerateContentResponseUsageMetadatathat aggregates prompt + tool-use tokens as input and candidate + reasoning tokens as output, plustoAttributes()and the four GenAI usage attribute-name constants. Ported field for field fromadk-python/src/google/adk/telemetry/_token_usage.py.core/src/telemetry/tracing.ts—resolveErrorType(), the JS twin oftracing.resolve_error_type(same file placement as Python).Stacked PR (Part 1/2). Part 2 rebuilds
core/src/telemetry/metrics.tson top of these.Collision check. Before writing anything I checked the open PRs on this fork:
299 open PRs; the adjacent ones are #22 (
feat/opt-in-usage-metrics), #177, #348, #357, #387, #394. #22 ownscore/src/telemetry/metrics.tsandcore/test/telemetry/metrics_test.tswith a different, non-semantic-convention metric contract, and wires three call sites. It does not land this change (noTokenUsage, noresolveErrorType), but it overlaps the same module, so this work is stacked onfeat/opt-in-usage-metricsrather than branched frommain. This PR itself is purely additive and touches nothing #22 touches; Part 2 is where the two contracts are reconciled. Nothing else in the list overlaps: #357 and #308 touchtracing.ts/tracing_test.tsbut only theADK_CAPTURE_MESSAGE_CONTENT_IN_SPANSgate, which is a different region of both files.Notes for review
gen_ai.usage.experimental.system_instruction_tokens. Renaming any of them would split the dashboard series between the two runtimes.systemInstructionTokensis read through a local extension interface, not a cast. Python reaches it with a defensivegetattr;@google/genai2.9.0 (the versioncore/package.jsondeclares, and the copy installed undercore/node_modules) does not declare the field either, soUsageMetadataWithSystemInstructionTokens extends GenerateContentResponseUsageMetadataadds the optional field with noanyinvolved.undefinedvs0is load-bearing.addTokenCountsreturnsundefinedonly when both inputs areundefined; a reported0stays0. A truthiness check (if (!a && !b)) would silently turn "the model used zero prompt tokens" into "no data", which is a different point on a dashboard. This is pinned by two tests and by mutation 1 below.resolveErrorTypematches a genaiApiErrorstructurally, not withinstanceof. This is not hypothetical here: this workspace resolves two copies of@google/genai—node_modules/@google/genaiis 1.52.0 andcore/node_modules/@google/genaiis 2.9.0 — so an error thrown by one copy is not aninstanceofthe class exported by the other. The test constructs a realApiErrorfrom the SDK to prove the structural read works against the actual class.error.nameis notconstructor.name-style type detection.error.nameis a standard, serialisableErrorproperty, which is the JS analogue of Python'stype(error).__name__; the repo's "noconstructor.namefor type detection" rule is about cross-package class identity, which this is not.error.constructor.nameis used only as a fallback whennamehas been blanked out, preserving behaviour that already has three tests on the base branch.error.codefor itsAPIError; this readsstatus. The JS SDK'sApiErrordeclaresstatus: numberand nocode(core/node_modules/@google/genai/dist/genai.d.ts:444), sostatusis the same value under the SDK's own name. No speculativecodefallback was added — there is nothing in the dependency tree that would set it.core/src/index.tsorcore/src/common.ts; tests reach them by relative path, exactly like the existingtracing.tstests.resolveErrorTypelives inutils/error_utils.ts, not intracing.ts. Python keeps it intracing.py, but nothing in adk-js'stracing.tscalls it (traceToolCalltakes no error parameter), and it is not tracing-specific — it is a generic error-classification helper, so it belongs withcase_utils.ts/file_utils.tsand is named for what it does rather than for the feature that needed it first.unknown, notError. Anything can be thrown in JavaScript, and every caller gets its value from acatch. Typing the parameter honestly is what removes thee as Errorcast from the three catch clauses in Part 2.errorTypeis only honoured when it is astring; the shape is matched withinnarrowing, so there is no interface declaring anunknownfield.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
core/test/telemetry/token_usage_test.tsports all 18 cases fromadk-python/tests/unittests/telemetry/test_token_usage.py, plus two JS-only ones for thesystemInstructionTokensbranch present and absent. Key absence is asserted withnot.toHaveProperty, because the parity point is that the key is not emitted at all — not that it is emitted asundefined. The attribute-name constants are no longer exported (toAttributes()is their only consumer), so the tests spell the wire keys out as literals instead of importing them — which is a stronger guard, since an alias cannot drift with the module.core/test/utils/error_utils_test.tscoversresolveErrorType, including the non-object andnullthrown values theunknownparameter now admits.tracing_test.tsgains two new cases for the span attributes; no existing case was modified.Coverage.
core/src/telemetry/token_usage.tsandcore/src/utils/error_utils.tsare both at 100% statements / branches / functions / lines.Proof that the tests can fail. Each new test was run against mutated source and observed to fail:
if (a === undefined && b === undefined)->if (!a && !b)inaddTokenCountsinputTokenCount > returns 0, not undefined, when both counts are zeroexpected undefined to be +0 // Object.is equalitysystemInstructionTokensbranch oftoAttributestoAttributes > emits the system instruction tokens the SDK does not declareexpected { 'gen_ai.usage.input_tokens': 10 } to deeply equal { …(2) }statusbranch ofresolveErrorTyperesolveErrorType > reports the HTTP status of a genai API errorexpected 'ApiError' to be '429' // Object.is equalityerror.name || error.constructor.name->error.nameresolveErrorType > falls back to the class name when the name has been blanked outexpected '' to be 'TypeError' // Object.is equalityType suppressions: none. This PR adds zero
@ts-expect-error,@ts-ignore,eslint-disable,any,as any,as neveror coverage pragmas, insrc/or in tests.npm run ts:checkgoes from 308 to 312 errors, and here is exactly why. No error is insrc/. The four are incore/test/telemetry/tracing_test.ts, at the two new test cases, and they are four more instances of a defect that already produces eight errors in that same file: the shared fixtures are typed through@google/adk(the builtcore/distdeclarations) while the module under test is imported fromcore/src, so TypeScript sees two nominally differentInvocationContext/LlmRequest. Every existingtraceCallLlmcase in the file reports the same pair. Fixing it means re-pointing the test tree's program, which is what fork PRs #326 and #370 are for; dodging it here would mean either dropping the tests or duplicating the fixtures against deepsrcpaths, and neither is worth it. Flagging it rather than claiming the count is unchanged.Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
Both modules are internal, so the manual check is through the API they expose:
npm run build,npm run lint,npm run format:checkandnpm run docs:checkall pass locally.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.
CI status: absent (validated locally)
This is a stacked PR whose base is
feat/opt-in-usage-metrics, notmain. Therepo's test workflow triggers on
pull_request: branches: [main](
.github/workflows/validation.yaml:5-6), so norun-testsjob exists for thisPR — the only check that runs is
auto-assign, which validates nothing. It wastherefore validated locally on the exact pushed commit
20271240:Revision — complexity review round 1
resolveErrorTypemoved fromcore/src/telemetry/tracing.tstocore/src/utils/error_utils.ts, and its tests fromtracing_test.tstocore/test/utils/error_utils_test.ts. It had no caller insidetracing.tsand is not tracing-specific.unknown, notError, and theClassifiedErrorinterface with theerrorType?: unknownfield is gone — the shape is matched withinnarrowing anderrorTypeis honoured only when it is astring. The case that pinnedString()-ing a numericerrorTypewas dropped with the behaviour it described; a new case asserts a non-stringerrorTypeis ignored, and two more cover the non-object thrown valuesunknownnow admits.toAttributes()and the attribute constants are no longer dead. The reviewer was right that nothing in production called them; the fix is the one adk-python already makes —traceCallLlmsets them — rather than deleting a faithful port. The four constants are now module-private, sincetoAttributes()is their only consumer.attribute namestest (asserting an imported constant equals its own literal) was dropped when the constants were un-exported; the same wire keys are now asserted as literals in everytoAttributescase, which is what that test was reaching for.