Skip to content
This repository was archived by the owner on May 20, 2026. It is now read-only.

Add APIUsage tracking to extChatEndpoint#5102

Open
Shinrai wants to merge 2 commits into
microsoft:mainfrom
Shinrai:patch-1
Open

Add APIUsage tracking to extChatEndpoint#5102
Shinrai wants to merge 2 commits into
microsoft:mainfrom
Shinrai:patch-1

Conversation

@Shinrai

@Shinrai Shinrai commented May 3, 2026

Copy link
Copy Markdown

Token usage reported by third-party language models is currently thrown out entirely — unceremoniously discarded before it can reach the context-window meter. This patch corrects that oversight by plumbing the usage data already present in the LLM response stream through to the appropriate locations within vscode-copilot-chat, so the meter finally gets the numbers it deserves.


fix(ext-endpoint): propagate usage from DataPart stream in ExtensionContributedChatEndpoint

The extension-contributed chat endpoint previously returned a hardcoded zeroed-out usage object on every successful response, causing the Copilot Chat context-window meter to never update when routing through third-party language model providers.

Add a usageFromDataPart() helper that checks incoming LanguageModelDataPart chunks for a native usage shape (prompt_tokens, completion_tokens, total_tokens — all numbers) without keying on any MIME type. This mirrors the shape-based check already present in the non-provider (internal) streaming path.

In makeChatRequest2():

  • Declare usage: APIUsage | undefined before the stream loop.
  • After handling known MIME types (StatefulMarker, ContextManagement), call usageFromDataPart() on every DataPart and capture the result.
  • Return usage ?? <zeroed fallback> so existing behaviour is preserved when no usage chunk arrives.

No new MIME branch is introduced; any provider that emits a DataPart whose JSON payload matches the OpenAI usage shape will automatically populate the meter.


image
image

fix(ext-endpoint): propagate usage from DataPart stream in ExtensionContributedChatEndpoint

The extension-contributed chat endpoint previously returned a hardcoded
zeroed-out usage object on every successful response, causing the
Copilot Chat context-window meter to never update when routing through
third-party language model providers.

Add a `usageFromDataPart()` helper that checks incoming
`LanguageModelDataPart` chunks for a native usage shape
(`prompt_tokens`, `completion_tokens`, `total_tokens` — all numbers)
without keying on any MIME type. This mirrors the shape-based check
already present in the non-provider (internal) streaming path.

In `makeChatRequest2()`:
- Declare `usage: APIUsage | undefined` before the stream loop.
- After handling known MIME types (StatefulMarker, ContextManagement),
  call `usageFromDataPart()` on every DataPart and capture the result.
- Return `usage ?? <zeroed fallback>` so existing behaviour is
  preserved when no usage chunk arrives.

No new MIME branch is introduced; any provider that emits a DataPart
whose JSON payload matches the OpenAI usage shape will automatically
populate the meter.
Copilot AI review requested due to automatic review settings May 3, 2026 18:57
@Shinrai

Shinrai commented May 3, 2026

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="CLDMV"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the extension-contributed chat endpoint so usage data emitted by third-party language model providers can flow back into Copilot Chat’s normal response pipeline instead of always being replaced with zeros. It fits into the endpoint layer that adapts VS Code LanguageModelChat providers to the extension’s internal ChatResponse/usage model.

Changes:

  • Adds a helper in extChatEndpoint.ts to detect OpenAI-shaped usage payloads inside LanguageModelDataPart chunks.
  • Captures usage while consuming the provider response stream and returns it on successful responses.
  • Keeps the previous zeroed fallback when no usage chunk is present.

Comment thread src/platform/endpoint/vscode-node/extChatEndpoint.ts
Comment thread src/platform/endpoint/vscode-node/extChatEndpoint.ts Outdated
…arsing

- Extend usageFromDataPart to include completion_tokens_details when
  present in the payload, preventing reasoning/prediction token breakdown
  from being silently dropped for extension-contributed models.
- Add extChatEndpoint.spec.ts with provider-specific regression tests
  covering shape-based usage chunk detection and completion_tokens_details
  passthrough, mirroring the coverage pattern of the regular SSE stream.
@alexdima

Copy link
Copy Markdown
Member

Thanks for the contribution! This repository has been archived because the project has moved into the main VS Code repository.

Could you please reopen/recreate this PR against:
https://github.com/microsoft/vscode/tree/main/extensions/copilot

We’ll continue reviewing contributions there. Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants