fix(weave_ts): link an evaluation to the agent spans it produced - #7708
Merged
Conversation
…ts through The eval span linker was registered on the OpenTelemetry global provider, but Weave emits every GenAI span through a provider it owns, so the linker never saw a span. Register it where the provider is built instead — lazily, and again after a project switch rebuilds it. Also write `weave.eval.run_id`: eval results only accept a span that carries it alongside the predict-and-score call ID, and the TypeScript SDK never wrote it.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Its onEnd hook appended a span ref to the prediction call's childSummary, which propagates to the parent, and merging two rows' arrays turns the list into an object with numeric keys — garbage on the eval root of every multi-row run. The hook has never executed, so nothing depended on it, and Python's linker is onStart-only for the same reason: the promoted eval columns carry the link and the server derives one representative ref per trace from them. Also puts the linker after the exporting processor, matching how Python registers the two, and notes both provider invariants in AGENTS.md.
ro31337
changed the title
Match the op names through the constants the sibling eval tests use, let the eval helper hand back the two calls both tests look up, and drop the comment in getOrBuildProvider — the rebuild test states the same thing executably.
ro31337
force-pushed
the
roman/WB-38444-ts-eval-span-linking
branch
from
August 9, 2026 07:21
c477ca9 to
46835c8
Compare
ro31337
changed the title
The genai suite already snapshots the whole attribute bag on spans emitted with no eval on the stack, across five files, so a stray weave.eval.* breaks those more strictly than the assertion added here did — it goes. Names: the direct-call test is only there for the write order now that the public-path test covers injection, and the provider test builds one provider, not every one. Also drop the prescription from the attribute-limit note, since Python's own eval linker does not follow the rule it implied.
ro31337
force-pushed
the
roman/WB-38444-ts-eval-span-linking
branch
from
August 9, 2026 07:34
46835c8 to
1ee7c28
Compare
This was referenced Aug 10, 2026
ro31337
marked this pull request as ready for review
August 10, 2026 18:38
rgao-coreweave
approved these changes
Aug 11, 2026
rgao-coreweave
left a comment
rgao-coreweave
left a comment
Contributor
There was a problem hiding this comment.
Amazing work, learned a lot!
LGTM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three PRs, in review order:
JIRA Issue(s)
https://coreweave.atlassian.net/browse/WB-38444
Description
In TypeScript an evaluation never links to the agent spans its predictions produce. No "View spans" button, and the eval columns on
spansstay empty. Python has had this link since May.The code for it was here all along, just never connected. The processor registered on the OTel global provider, but weave emits through its own provider, which is kept out of that registry. The call never even ran: with no user-installed provider,
trace.getTracerProvider()has noaddSpanProcessor, so the helper returnedfalseand nobody checked. Nothing regressed; the wire was never there.The fix registers the processor where the provider is built, because that happens lazily on the first span and again when
init()switches projects.One attribute was missing too. The server counts a span as linked only with both
weave.eval.run_idandweave.eval.predict_and_score_call_id, and TypeScript wrote only the second. It now writes both, run id first, since a full span drops whatever arrives next.That is enough for the "View spans" drill-in. Full parity is a separate ticket, and so is
EvaluationLogger, which cannot be linked this way at all.Why this approach
Turning the processor on also turns on its
onEndhook, which appended a span ref to the prediction call'schildSummary. That hook has never run, and once it does the result is wrong:childSummarypropagates to the parent, and merging two rows' arrays gives an object with numeric keys, so every multi-row eval would ship a malformedweave.genai_span_refon its root call. So it is gone. An existing test already says an SDK must not write that field; the promoted columns carry the link.The linker goes after the exporting processor, the same order Python uses. That is neutral for the default pipeline. A custom
settings.genai.spanProcessornow runs before it and will not seeweave.eval.*, but no processor has ever seen those attributes, so nothing is lost today.Testing
The new tests go through the public path: run an
Evaluation, emit a real GenAI span, read the exported span. That is the only kind of test that catches this, since the old ones drove the processor directly and passed for months against a processor nobody had attached; the fullsdks/nodesuite (503 tests), eslint, prettier and both typechecks are green.