Feat: Port the adk-python OpenTelemetry metrics contract (Part 2/2)

[AmaadMartin]

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):
   Closes: N/A
   Related: N/A
2. Or, if no issue exists, describe the change:
   Problem: core/src/telemetry/metrics.ts (added by Feat: Implement Opt-in Usage metrics in adk-js #22) emits metric names that adk-python does not emit, in units adk-python does not use: gen_ai.agent.invocation.duration in ms where adk-python has gen_ai.invoke_agent.duration in s, gen_ai.tool.execution.duration in ms where adk-python has gen_ai.execute_tool.duration in s, and gen_ai.client.token.usage in 1 where adk-python uses {token}. It also omits all of the bucket-boundary advisories. A dashboard or OTel backend already consuming adk-python reads nothing from adk-js, which is the whole point of emitting semantic-convention metrics.

Two behavioural defects came with it:

• The meter is memoised on first use. OpenTelemetry JS has no proxy meter for metrics — unlike tracing, and unlike Python, whose metrics.get_meter() proxy re-binds when a provider is installed. MetricsAPI.getMeter() is this.getMeterProvider().getMeter(...), resolved at call time against getGlobal('metrics') || NOOP_METER_PROVIDER. Caching the meter forever means a MeterProvider registered after the first recording — e.g. an app that runs an agent before calling maybeSetOtelProviders() — silently never receives a measurement.
• error.type is error.name, so every failed model call reports ApiError instead of the HTTP status that distinguishes a 429 from a 400.

Solution: Bring the module onto the adk-python contract, porting src/google/adk/telemetry/_metrics.py field for field.

• Rename and re-unit the four overlapping instruments, and copy their bucket-boundary advisories element for element from _metrics.py:48-137.
• Key the instrument cache on the provider identity rather than memoising once, so a late-registered provider is picked up. With no provider registered, getMeterProvider() returns the stable no-op singleton and every record* call is a silent no-op — that is how the "opt-in" property is kept, with no if (!provider) return branch.
• Resolve error.type through resolveErrorType (Part 1), and token counts through TokenUsage (Part 1).
• Add getElapsedS(span, fallbackStartMs), the port of _metrics.py:287-314, and use it at the three existing call sites so every duration comes from one time source.

The three adk-js-only instruments (gen_ai.agent.request.size, gen_ai.agent.response.size, gen_ai.agent.workflow.steps) and their getContentSize machinery are kept exactly as they are — they have no adk-python counterpart, but they are this branch's feature and nothing about the parity port requires removing them.

Stacked PR (Part 2/2). Targets feat/telemetry-metrics-token-usage-part1, which targets feat/opt-in-usage-metrics (#22).

Collision check. gh pr list --repo AmaadMartin/adk-js --state open --limit 300 returns 299 open PRs; gh pr diff <n> --name-only on every plausibly adjacent one (#22, #177, #348, #357, #387, #394) shows only #22 (feat/opt-in-usage-metrics) touching this module. It does not land this change — no semconv names, no bucket advisories, no TokenUsage, no resolveErrorType, no workflow/inference/tool-call instruments — but it owns the same file, so this work is stacked on its branch instead of branched from main, and this PR is the reconciliation of the two contracts rather than a competing second metrics.ts.

Scope note. The approved plan for this task said "modules and unit tests only, no call sites". Because the branch this is stacked on has already wired three call sites, keeping the build green required updating them for the changed signatures (ms -> s, the new toolType argument, the options objects). That is the only scope difference, and it is a consequence of stacking rather than a decision to widen the change.

Notes for review

1. Every emitted string is copied verbatim from _metrics.py — names, units, descriptions, and the attribute values generate_content, input, output, gemini, vertex_ai. gen_ai.client.operation.duration and gen_ai.client.token.usage come from opentelemetry-semantic-conventions via gen_ai_metrics.create_gen_ai_client_*; their names, units (s, {token}) and descriptions (GenAI operation duration., Number of input and output tokens used.) are the semconv values, and neither declares bucket boundaries.
2. GEN_AI_AGENT_VERSION and GEN_AI_TOOL_VERSION were deliberately not ported. _metrics.py:39-40 declares them and nothing reads them; a constant with no reader is dead config.
3. Provider-name detection uses adk-js's own getGoogleLlmVariant(). Python's _guess_gemini_system_name() additionally honours an enterprise-mode env var that adk-js does not model (variant_utils.ts reads only GOOGLE_GENAI_USE_VERTEXAI). The emitted strings are byte-identical; only the detection differs, and local convention wins for process-internal behaviour rather than inventing an enterprise-mode env var in adk-js.
4. The try/catch around getGoogleLlmVariant() was removed (and with it the test that mocked the util into throwing). It was unreachable defence: every recorder already wraps its whole body in try/catch + logger.debug, so a throw from any dependency is caught one level up. That path is still covered — see "never throws a telemetry failure at the caller", which drives all seven recorders through a provider whose getMeter() throws and asserts both that nothing propagates and that seven debug logs were emitted. The two provider values it also covered are now pinned by records the request and response models (gemini) and reports vertex_ai when GOOGLE_GENAI_USE_VERTEXAI is set, which exercise the real env var through vi.stubEnv instead of a module mock.
5. The attribute sets are asymmetric on purpose, matching Python. A root workflow omits gen_ai.workflow.nested entirely rather than reporting false; an empty workflowName is not emitted (if workflow_name:); recordClientOperationDuration omits gen_ai.response.model while responses is empty, whereas recordClientTokenUsage always sets it — it has already returned in the empty case.
6. Only the last response is read for model version and token counts: streaming usage metadata is cumulative, so summing chunks over-counts. A test drives two responses and asserts the second one's count is what lands.
7. getElapsedS narrows HrTime structurally, with a small isHrTime guard, rather than importing ReadableSpan from @opentelemetry/sdk-trace-base into a runtime position — the API Span type exposes no timings, but the SDK implementation carries them. No instanceof, no cast: TimedSpan extends Span adds the two fields as optional unknown. Python's GenerateContentSpan unwrap has no adk-js counterpart and was dropped.
8. gen_ai.tool.type at the call site is tool.constructor.name, matching both _metrics.py's tool_type (Python passes type(tool).__name__) and the existing adk-js precedent at tracing.ts:117, which already sets the same span attribute the same way. This is a display label, not type detection, so the "no constructor.name for type detection" rule does not apply.
9. recordWorkflowInvocationDuration, recordInvokeAgentInferenceCalls and recordInvokeAgentToolCalls have no caller yet. adk-python records them from its workflow-agent and invocation paths, which adk-js does not have an equivalent hook for; wiring them is queued as separate work. They are fully tested here.
10. core/src/common.ts no longer re-exports this module. export * from './telemetry/metrics.js' was publishing every recorder — and getElapsedS — as @google/adk API. These are internal telemetry plumbing, they are called from inside core/src, and nothing outside core imports them. adk-python keeps the equivalent module private (_metrics.py), and the approved plan for this port says the same.
11. Three of adk-python's instruments are deliberately not ported yet. gen_ai.invoke_workflow.duration, gen_ai.invoke_agent.inference_calls and gen_ai.invoke_agent.tool_calls need a hook adk-js does not have: the first a duration and nesting signal in the workflow agents, the other two a per-invocation counter of model and tool calls (adk-python gets them from its TelemetryContext). Building that plumbing means changing InvocationContext and the workflow agents, which is a different change from a telemetry port, and an exported recorder with no caller is dead code. They land with the PR that wires them, restored from _metrics.py:48-137 (definitions and bucket advisories) and _metrics.py:151-182 (recorders); the wiring is queued as its own task. Every instrument in this PR has a live call site.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

npx vitest run --project unit:core core/test/telemetry core/test/agents/base_agent_test.ts \
  core/test/agents/functions_test.ts core/test/agents/llm_agent_test.ts core/test/models/llm_response_test.ts
 ✓ core/test/telemetry/metrics_test.ts     (50 tests)
 ✓ core/test/telemetry/token_usage_test.ts (21 tests)
 ✓ core/test/telemetry/tracing_test.ts     (13 tests)
 ✓ core/test/agents/base_agent_test.ts     (21 tests)
 ✓ core/test/agents/functions_test.ts      (27 tests)
 ✓ core/test/agents/llm_agent_test.ts      (29 tests)
 ✓ core/test/models/llm_response_test.ts   (21 tests)
 Test Files  7 passed (7)
      Tests  182 passed (182)

npx vitest run --project e2e tests/e2e/telemetry/metrics_e2e_test.ts
 Test Files  1 passed (1)

The unit tests no longer mock @opentelemetry/api. They register a real MeterProvider from @opentelemetry/sdk-metrics with an in-memory reader and assert the exported data points, so the metric names, units, descriptions, bucket boundaries, attribute sets and values are checked against what a collector would actually receive. This was also forced by the change: the old file mocked the whole @opentelemetry/api module down to metrics.getMeter, which cannot work now that the module resolves the provider (and imports tracing.ts, whose module-level trace.getTracer would be undefined under that mock).

Cases ported from adk-python/tests/unittests/telemetry/test_metrics.py (all 8), plus the gaps that file leaves:

• a table-driven check that all seven instruments declare the expected name, unit, description and bucket boundaries (the two with an advisory assert the copied array; the five without assert the SDK's default boundaries, i.e. that no advisory was given);
• client duration with no response, with a response lacking modelVersion, and with no request model;
• token usage with no response, with a missing usageMetadata (asserting the logger.warn message), with only input, with only output, with all-zero counts, and with two responses to prove only the last is read;
• getElapsedS for an ended SDK span with exact HrTime timings (2.25 s to the nanosecond), for a non-recording span with no timings, and for no span at all;
• the no-op path: with metrics.disable() and no provider, all seven recorders run without throwing;
• late provider registration: record with no provider, register one, record again, and assert exactly one measurement of the second value — the regression test for the memoisation defect;
• the failure path: a provider whose getMeter() throws, asserting none of the seven recorders propagates it and each logs once.

Three existing assertions were updated, none deleted: the two recordToolExecutionDuration spies in functions_test.ts gained the toolType argument, and the recordClientOperationDuration/recordClientTokenUsage spies in llm_agent_test.ts moved to the options-object form. metrics_e2e_test.ts follows the two renamed instruments and asserts the new gen_ai.tool.type attribute. The one removed test is the mocked getGoogleLlmVariant throw, covered as described in note 4 above.

Coverage. core/src/telemetry/metrics.ts and core/src/telemetry/token_usage.ts are both at 100% statements, branches, functions and lines:

npx vitest run --project unit:core core/test/telemetry --coverage \
  --coverage.include='core/src/telemetry/metrics.ts' --coverage.include='core/src/telemetry/token_usage.ts'
 metrics.ts     |     100 |      100 |     100 |     100 |
 token_usage.ts |     100 |      100 |     100 |     100 |

Proof that the tests can fail. Each was run against mutated source and observed to fail:

Mutation	Failing test	Result
0.04 -> 0.05 in the gen_ai.execute_tool.duration bucket advisory	instrument definitions > declares 'gen_ai.execute_tool.duration' …	1 failed | 34 passed
gen_ai.client.token.usage unit {token} -> 1	instrument definitions > declares 'gen_ai.client.token.usage' …	expected '1' to be '{token}'
provider-identity cache -> one-shot memo (if (!cache))	meter provider resolution > records against a provider registered after the first call, plus every test that installs its own provider	21 failed | 14 passed
delete resolveErrorType's instanceof Error branch	resolveErrorType > falls back to the error name, and 2 more	3 failed | 5 passed

Type suppressions: none. Zero @ts-expect-error, @ts-ignore, eslint-disable, any, as any, as never, as unknown as or coverage pragmas are added by this PR, in src/ or in tests — including in the test harness, which builds a real MeterProvider and a one-method ApiMeterProvider object literal rather than casting a partial mock. This part also removes three e as Error casts, by typing the recorders' error parameter as unknown.

npm run ts:check goes from 308 to 312 errors across the stack, all four in core/test/telemetry/tracing_test.ts and all introduced by Part 1's two new test cases. They are more instances of a defect that already produces eight errors in that file — shared fixtures typed through the built @google/adk declarations, passed to a module imported from core/src — not a new category, and none of them are in src/. See Part 1 for the detail.

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

npm run build

# 1. Metrics reach a real collector, under the adk-python names.
npx vitest run --project e2e tests/e2e/telemetry/metrics_e2e_test.ts
# Runs an agent with a fake model and a real FunctionTool through InMemoryRunner
# with a real MeterProvider, and asserts gen_ai.invoke_agent.duration,
# gen_ai.execute_tool.duration (with gen_ai.tool.type=FunctionTool),
# gen_ai.client.operation.duration and gen_ai.client.token.usage (30 input /
# 50 output tokens) on the collected ResourceMetrics.

# 2. Nothing is emitted, and nothing throws, with no provider configured:
node --input-type=module -e "
import {recordAgentInvocationDuration} from './core/dist/esm/telemetry/metrics.js';
recordAgentInvocationDuration('smoke', 0.42);
console.log('no provider: silent no-op');
"

npm run build, npm run lint, npm run format:check and npm run docs:check all pass locally.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

CI status: absent (validated locally)

This is a stacked PR whose base is feat/telemetry-metrics-token-usage-part1,
not main. The repo's test workflow triggers on
pull_request: branches: [main] (.github/workflows/validation.yaml:5-6), so
no run-tests job exists for this PR — the only check that runs is
auto-assign, which validates nothing. It was therefore validated locally on
the exact pushed commit 36f6f09f:

npm run build                                                   # pass
npx vitest run --project unit:core core/test/telemetry \
  core/test/agents/base_agent_test.ts core/test/agents/functions_test.ts \
  core/test/agents/llm_agent_test.ts core/test/models/llm_response_test.ts
                                                                # 195 passed (9 files)
npx vitest run --project e2e tests/e2e/telemetry/metrics_e2e_test.ts
                                                                # 1 passed (1 file)
npm run lint                                                    # clean
npm run format:check                                            # All matched files use Prettier code style!
npm run docs:check                                              # clean (typedoc --treatWarningsAsErrors)
npm run ts:check                                                # 308 pre-existing errors, unchanged from the base branch

Revision — complexity review round 1

Everything blocking was addressed; two findings were addressed differently from the suggestion, with reasons.

Done as asked

• core/src/common.ts no longer re-exports the module (export * was publishing the recorders and getElapsedS as @google/adk API).
• The byte-sizing cluster moved to core/src/utils/content_size_utils.ts, generically named, with its tests moved to core/test/utils/content_size_utils_test.ts. Testing contentSize directly immediately paid for itself: the recorder tests never exercised an unpadded base64 payload, so one branch of the padding calculation had never run. metrics_test.ts keeps only the two cases that are about the recorder (the attributes, and that a size is recorded).
• The ten copies of try/catch/logger.debug collapsed into safeRecord(what, record), and catch (e: unknown) became catch (e).
• e as Error is gone from all three catch clauses, fixed at the root: resolveErrorType takes unknown (Part 1) and the recorders' error parameters follow.
• errorType?: unknown is gone. The interface it lived on is gone too; the shape is matched with in narrowing and the field is honoured only when it is a string.
• resolveErrorType moved to core/src/utils/error_utils.ts (Part 1).
• The two client recorders take response?: LlmResponse instead of an array they immediately .at(-1). The "usage is cumulative, only the last chunk counts" rationale moved to the parameter's doc comment, where the caller can see it.

Addressed differently

• toAttributes() and the usage constants: given a caller, not deleted. The finding was right that nothing in production reached them. But adk-python's tracing.py does call the equivalent, and adk-js's hand-rolled substitute was wrong — it reported input tokens from promptTokenCount alone and output from candidatesTokenCount alone. Part 1 now wires traceCallLlm to TokenUsage.toAttributes(), which deletes the buggy code instead of the correct code and makes spans emit the cache-read and reasoning attributes. The four constants became module-private, so nothing dead is exported.
• The three uncalled recorders were kept. See note 11 above: they are no longer published API, they are the expensive-to-re-derive part of the port (names, units, bucket advisories), and the wiring they need is queued as its own task. Their tests stay because the code stays.

Not changed, with reasons

• getBase64ByteLength -> Buffer.byteLength(s, 'base64'). This helper is reachable from the browser bundle — core/src/index_web.ts re-exports common.ts, which reaches base_agent.ts and from there the content sizing — and Buffer is not available there. The repo already treats it that way: utils/env_aware_utils.ts guards its Buffer.from behind isBrowser(). The arithmetic is kept, now as a two-line function with the constraint written down.
• getElapsedS(span, fallbackStartMs) keeps the span path. It is _metrics.py:287-314's contract, the two call sites that have an ended span (base_agent, functions) get the span's own start/end rather than a second clock, and the fallback is what the third caller uses. The doc no longer claims a single time source for all callers.
• AgentEventTally / tallyAgentEvent in base_agent.ts are the base branch's code, unchanged by this PR.
• logger.warn for absent usageMetadata matches _metrics.py:244-249. It fires only when a provider returns no usage at all, which is a real observability gap worth surfacing once rather than hiding at debug level.

Re-verified after the revision

npx vitest run --project unit:core core/test/telemetry core/test/utils \
  core/test/agents/base_agent_test.ts core/test/agents/functions_test.ts \
  core/test/agents/llm_agent_test.ts        # 445 passed (30 files)
npx vitest run --project e2e tests/e2e/telemetry/metrics_e2e_test.ts   # 1 passed
npm run build / lint / format:check / docs:check                       # all clean

Coverage on all four modules — metrics.ts, token_usage.ts, error_utils.ts, content_size_utils.ts — is 100% statements, branches, functions and lines. The three mutations reported above were re-run against the refactored code and still fail (1, 2 and 26 failures respectively), and a fourth was added for the moved resolveErrorType: deleting its instanceof Error branch fails 3 of its 8 cases.

Revision — complexity review round 2

The one remaining blocking item, B1, is fixed by deletion: recordWorkflowInvocationDuration, recordInvokeAgentInferenceCalls and recordInvokeAgentToolCalls are gone, along with the three HISTOGRAMS entries that existed only for them and the GEN_AI_WORKFLOW_NAME / GEN_AI_WORKFLOW_NESTED constants. Their test blocks went with them, and no other test was touched. Every instrument left in this PR has a live call site.

Wiring them here was the alternative, and it is a bigger change than the port itself: gen_ai.invoke_workflow.duration needs a duration and a nesting signal threaded through the workflow agents, and the two call-count instruments need a per-invocation counter of model and tool calls on InvocationContext. That work is queued separately; the definitions come straight back from _metrics.py:48-137 and the recorders from _metrics.py:151-182, so nothing is lost by not carrying them ahead of their callers.

Re-verified after the deletion

npx vitest run --project unit:core core/test/telemetry core/test/utils \
  core/test/agents/base_agent_test.ts core/test/agents/functions_test.ts \
  core/test/agents/llm_agent_test.ts core/test/models/llm_response_test.ts
                                                            # 458 passed (31 files)
npx vitest run --project e2e tests/e2e/telemetry/metrics_e2e_test.ts   # 1 passed
npm run build / lint / format:check / docs:check                       # all clean
npm run ts:check                                            # 312, unchanged by this round

metrics.ts, token_usage.ts, error_utils.ts and content_size_utils.ts are all still at 100% statements, branches, functions and lines, and every row of the mutation table above was re-run against the trimmed module.