diff --git a/.changeset/strict-audience-controls.md b/.changeset/strict-audience-controls.md new file mode 100644 index 0000000000..b9d5c08f14 --- /dev/null +++ b/.changeset/strict-audience-controls.md @@ -0,0 +1,5 @@ +--- +"eve": minor +--- + +Channel audience is now resolved into a delivery-scoped decision used to construct the harness instrumentation. `tracePolicy` returns an explicit drop or record decision with input/output controls, and audience is no longer exposed through harness lifecycle events or span export policy context. diff --git a/docs/guides/instrumentation.md b/docs/guides/instrumentation.md index 71c26eea64..3526324a05 100644 --- a/docs/guides/instrumentation.md +++ b/docs/guides/instrumentation.md @@ -60,6 +60,10 @@ The third configurable surface, [runtime context events](#runtime-context), atta Built-in messaging channels classify their instrumentation metadata with an `audience`: `public`, `private`, or `unknown`. Slack public channels and Chat SDK workspace-visible threads are public; direct and private conversations are private; platform surfaces without enough visibility evidence remain unknown. +With instrumentation providers enabled, `otel({ tracePolicy })` maps that classification to a delivery-scoped decision before the harness runs. eve constructs the harness instrumentation from that decision, including its provider set, content-filtered hooks, telemetry, and tracing context. Return `{ action: "drop" }` to omit the trace, or return `{ action: "record", recordInputs, recordOutputs }` to create it with an explicit content ceiling. By default, eve records public deliveries with inputs and outputs and drops private and unknown deliveries. The harness and span export policies receive neither the audience value nor the decision. + +This decision governs durable agent and AI telemetry. The optional inbound server span created by `traceChannelRequests: true` is request-scoped, contains no body or session content, and begins before a channel can classify the audience. + ## Channel delivery traces Instrumentation providers receive `channel.delivery.started` followed by diff --git a/packages/eve/extension-contracts/reports/channel/v8.json b/packages/eve/extension-contracts/reports/channel/v8.json new file mode 100644 index 0000000000..aaef072755 --- /dev/null +++ b/packages/eve/extension-contracts/reports/channel/v8.json @@ -0,0 +1,17 @@ +{ + "kind": "eve-extension-capability-contract", + "capability": "channel", + "epoch": 8, + "sha256": "2a84b0ba011d17f37b9fd48f3dde84ee2a04aa523af53713b41f782847f8e8f9", + "exports": [ + "DELETE", + "GET", + "PATCH", + "POST", + "PUT", + "WS", + "createWebSocketUpgradeServer", + "defineChannel", + "isChannel" + ] +} diff --git a/packages/eve/src/channel/types.ts b/packages/eve/src/channel/types.ts index 623f2e907d..ec13e92b6c 100644 --- a/packages/eve/src/channel/types.ts +++ b/packages/eve/src/channel/types.ts @@ -9,6 +9,7 @@ import type { ChannelAdapter } from "#channel/adapter.js"; import type { AgentLimitsDefinition } from "#shared/agent-definition.js"; import type { JsonObject } from "#shared/json.js"; import type { TaskView } from "#tasks/types.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; export type { ContextAccessor } from "#context/key.js"; export type { ChannelInstrumentationProjection } from "#channel/instrumentation.js"; @@ -162,6 +163,8 @@ export interface DeliverPayload { readonly message?: string | UserContent; readonly context?: readonly string[]; readonly outputSchema?: JsonObject; + /** Framework-only instrumentation ceiling ferried to local child sessions. */ + readonly instrumentationControls?: InstrumentationControls; /** Framework-only task envelopes consumed before adapter/model delivery. */ readonly task?: { /** Task HITL input-request batches for the parent's pre-model router. */ @@ -470,6 +473,8 @@ export interface RunInput { * (root session behavior). */ readonly initiatorAuth?: SessionAuthContext | null; + /** Framework-owned instrumentation ceiling inherited by local subagents. */ + readonly instrumentationControls?: InstrumentationControls; readonly input: { readonly message: string | UserContent; readonly context?: readonly string[]; diff --git a/packages/eve/src/compiler/extension-compatibility.ts b/packages/eve/src/compiler/extension-compatibility.ts index 29c5e0554a..d36fed61c8 100644 --- a/packages/eve/src/compiler/extension-compatibility.ts +++ b/packages/eve/src/compiler/extension-compatibility.ts @@ -31,7 +31,13 @@ const EXTENSION_CAPABILITY_CONTRACTS = { supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18], dropped: {}, }, - channel: { current: 7, supported: [1, 2, 3, 4, 5, 6, 7], dropped: {} }, + channel: { + current: 8, + supported: [1, 2, 3, 4, 5, 6, 8], + dropped: { + 7: "Instrumentation callbacks no longer expose channel audience; eve maps it to internal controls at delivery.", + }, + }, schedule: { current: 3, supported: [1, 2, 3], dropped: {} }, subagent: { current: 2, supported: [1, 2], dropped: {} }, connection: { current: 5, supported: [1, 2, 3, 4, 5], dropped: {} }, diff --git a/packages/eve/src/context/dynamic-resolve-context.test.ts b/packages/eve/src/context/dynamic-resolve-context.test.ts index 2b846335a3..08f0760c2e 100644 --- a/packages/eve/src/context/dynamic-resolve-context.test.ts +++ b/packages/eve/src/context/dynamic-resolve-context.test.ts @@ -26,7 +26,7 @@ describe("buildResolveContext", () => { ctx.set(ChannelKey, { kind: "http" }); ctx.set(ChannelInstrumentationKey, { kind: "channel:slack", - metadata: { threadTs: "1234.5678", userId: "U123" }, + metadata: { audience: "private", threadTs: "1234.5678", userId: "U123" }, }); const resolveCtx = buildResolveContext(ctx, []); diff --git a/packages/eve/src/context/dynamic-resolve-context.ts b/packages/eve/src/context/dynamic-resolve-context.ts index f447876a12..7c1df52644 100644 --- a/packages/eve/src/context/dynamic-resolve-context.ts +++ b/packages/eve/src/context/dynamic-resolve-context.ts @@ -11,6 +11,7 @@ import { } from "#context/keys.js"; import { ChannelKey } from "#runtime/sessions/runtime-context-keys.js"; import { getAdapterKind } from "#channel/adapter.js"; +import { withoutChannelAudience } from "#shared/channel-audience.js"; type ReadableContext = Pick; @@ -42,7 +43,10 @@ export function buildResolveContext( channel: { kind: channelAdapter !== undefined ? getAdapterKind(channelAdapter) : undefined, continuationToken, - metadata: channelInstrumentation?.metadata, + metadata: + channelInstrumentation === undefined + ? undefined + : withoutChannelAudience(channelInstrumentation.metadata), }, messages, }; diff --git a/packages/eve/src/context/keys.ts b/packages/eve/src/context/keys.ts index e44d05c394..6cfcbe18b3 100644 --- a/packages/eve/src/context/keys.ts +++ b/packages/eve/src/context/keys.ts @@ -27,6 +27,7 @@ import type { SandboxAccess } from "#sandbox/state.js"; import type { RunMode } from "#shared/run-mode.js"; import type { RuntimeModelReference } from "#runtime/agent/bootstrap.js"; import type { PreparedRuntimeDelegationTool } from "#runtime/sessions/turn.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; // Re-export so consumers don't need a direct channel/ import. export type { SessionAuthContext, SessionParent, SessionTurn } from "#channel/types.js"; @@ -89,6 +90,9 @@ export const ActiveChannelDeliveriesKey = new ContextKey( "eve.channelInstrumentation", ); +export const InstrumentationControlsKey = new ContextKey( + "eve.instrumentationControls", +); export const ModeKey = new ContextKey("eve.mode"); export const ParentSessionKey = new ContextKey("eve.parentSession"); /** Separate from {@link ParentSessionKey} so it stays out of what extensions read. */ diff --git a/packages/eve/src/execution/agent-handle-dispatch.ts b/packages/eve/src/execution/agent-handle-dispatch.ts index be7beb776a..51338016cb 100644 --- a/packages/eve/src/execution/agent-handle-dispatch.ts +++ b/packages/eve/src/execution/agent-handle-dispatch.ts @@ -8,7 +8,7 @@ * dead (handle deleted) or retryable (handle restored to `parked`). */ -import type { SessionAuthContext } from "#channel/types.js"; +import type { DeliverPayload, SessionAuthContext } from "#channel/types.js"; import { AGENT_BUSY, AGENT_MISMATCH, AGENT_UNREACHABLE } from "#harness/agent-handle-errors.js"; import { deriveAgentOperationId } from "#harness/handles/operation-id.js"; import { @@ -106,6 +106,7 @@ export async function dispatchToAgentHandle(input: { readonly auth: SessionAuthContext | null; readonly bundle: CompiledBundle; readonly currentSession: RuntimeSession; + readonly instrumentationControls?: DeliverPayload["instrumentationControls"]; readonly parentToken: string; readonly parentTurnId: string; }): Promise { @@ -185,6 +186,7 @@ export async function dispatchToAgentHandle(input: { auth: input.auth, bundle, identity: handle.identity, + instrumentationControls: input.instrumentationControls, parentToken: input.parentToken, }); if (!delivery.ok) { @@ -233,6 +235,7 @@ export async function dispatchToTaskAgentAddress(input: { readonly auth: SessionAuthContext | null; readonly bundle: CompiledBundle; readonly currentSession: RuntimeSession; + readonly instrumentationControls?: DeliverPayload["instrumentationControls"]; readonly parentToken: string; }): Promise { const { action, agentId } = input; @@ -271,6 +274,7 @@ export async function dispatchToTaskAgentAddress(input: { auth: input.auth, bundle: input.bundle, identity: record.identity, + instrumentationControls: input.instrumentationControls, parentToken: input.parentToken, }); if (!delivery.ok) { @@ -324,6 +328,7 @@ async function deliverToAgentAddress(input: { readonly auth: SessionAuthContext | null; readonly bundle: CompiledBundle; readonly identity: AgentIdentity; + readonly instrumentationControls?: DeliverPayload["instrumentationControls"]; readonly parentToken: string; }): Promise< Result< @@ -390,6 +395,7 @@ async function deliverToAgentAddress(input: { }, kind: "send", payload: { + instrumentationControls: input.instrumentationControls, message: readSubagentMessage(action), outputSchema: normalizeRequestedOutputSchema(action.input.outputSchema), }, diff --git a/packages/eve/src/execution/cancel-descendant-turns-step.ts b/packages/eve/src/execution/cancel-descendant-turns-step.ts index 1aaedde7dd..655fc8f5e0 100644 --- a/packages/eve/src/execution/cancel-descendant-turns-step.ts +++ b/packages/eve/src/execution/cancel-descendant-turns-step.ts @@ -13,6 +13,7 @@ import { createLogger, logError } from "#internal/logging.js"; import type { RuntimeSubagentRegistry } from "#runtime/subagents/registry.js"; import { getDynamicSubagentSelection } from "#context/dynamic-subagent-lifecycle.js"; import type { ContextContainer } from "#context/container.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; // Retry through transient world contention (queue wakes, hook-claim // conflicts), then log loudly: a silently dropped cancel leaves the child @@ -31,6 +32,15 @@ export async function cancelDescendantTurnsStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + cancelDescendantTurns(input), + ); +} + +async function cancelDescendantTurns(input: { + readonly serializedContext: Record; + readonly sessionState: DurableSessionState; +}): Promise { let running: readonly RunningAgentHandle[]; try { const session = await readDurableSession(input.sessionState); diff --git a/packages/eve/src/execution/channel-context.ts b/packages/eve/src/execution/channel-context.ts index 883166ecda..abdf5b9aab 100644 --- a/packages/eve/src/execution/channel-context.ts +++ b/packages/eve/src/execution/channel-context.ts @@ -11,13 +11,18 @@ export function setChannelContext( readonly channelName?: string; } = {}, ): void { + const existing = ctx.get(ChannelInstrumentationKey); + const projection = buildChannelInstrumentationProjection({ + adapter, + channelName: options.channelName, + existingKind: existing?.kind, + }); ctx.set(ChannelKey, adapter); - ctx.set( - ChannelInstrumentationKey, - buildChannelInstrumentationProjection({ - adapter, - channelName: options.channelName, - existingKind: ctx.get(ChannelInstrumentationKey)?.kind, - }), - ); + ctx.set(ChannelInstrumentationKey, { + ...projection, + metadata: + projection.kind === "subagent" && existing !== undefined + ? existing.metadata + : projection.metadata, + }); } diff --git a/packages/eve/src/execution/delegated-parent-notification.ts b/packages/eve/src/execution/delegated-parent-notification.ts index 7c89a0c871..8d8e1cf6b0 100644 --- a/packages/eve/src/execution/delegated-parent-notification.ts +++ b/packages/eve/src/execution/delegated-parent-notification.ts @@ -21,6 +21,7 @@ import type { AgentTurnOutcome } from "#shared/agent-turn-outcome.js"; import { toErrorMessage } from "#shared/errors.js"; import { parseJsonValue } from "#shared/json.js"; import type { TokenUsage } from "#shared/token-usage.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { resumeHook } from "#internal/workflow/runtime.js"; import { postSessionCallbackRequest } from "#execution/session-callback-request.js"; import type { TaskInboundTurnStarted } from "#tasks/types.js"; @@ -43,6 +44,16 @@ export async function notifyDelegatedParentStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + notifyDelegatedParent(input), + ); +} + +async function notifyDelegatedParent(input: { + readonly result: RuntimeSubagentChildResult | undefined; + readonly serializedContext: Record; + readonly usage?: TokenUsage; +}): Promise { if (input.result === undefined) { return; } @@ -104,11 +115,24 @@ const ZERO_TOKEN_USAGE: TokenUsage = { export async function notifyTurnCallerStep(input: { readonly caller: TurnCaller | undefined; readonly lifecycle: AgentTurnOutcome["kind"]; + readonly serializedContext?: Record; readonly sessionId: string; readonly settled: SettledTurnNotification; }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext ?? {}).run(() => + notifyTurnCaller(input), + ); +} + +async function notifyTurnCaller(input: { + readonly caller: TurnCaller | undefined; + readonly lifecycle: AgentTurnOutcome["kind"]; + readonly serializedContext?: Record; + readonly sessionId: string; + readonly settled: SettledTurnNotification; +}): Promise { if (input.caller === undefined) { return; } diff --git a/packages/eve/src/execution/dispatch-runtime-actions-shared.ts b/packages/eve/src/execution/dispatch-runtime-actions-shared.ts index 0cdb3095dd..a17941d53e 100644 --- a/packages/eve/src/execution/dispatch-runtime-actions-shared.ts +++ b/packages/eve/src/execution/dispatch-runtime-actions-shared.ts @@ -18,6 +18,7 @@ import { CapabilitiesKey, ChannelInstrumentationKey, InitiatorAuthKey, + InstrumentationControlsKey, SandboxKey, } from "#context/keys.js"; import { type AlsContext, ContextContainer } from "#context/container.js"; @@ -154,6 +155,9 @@ export interface PreparedRuntimeActionDispatch { */ readonly fanoutSize: number; readonly initiatorAuth: Parameters[0]["initiatorAuth"]; + readonly instrumentationControls: Parameters< + typeof buildSubagentRunInput + >[0]["instrumentationControls"]; readonly parentTraceContext: Parameters[0]["parentTraceContext"]; readonly sandboxSessionId: string; readonly serializedContext: Record; @@ -280,6 +284,7 @@ async function prepareActionDispatch(input: { input.fanoutSize ?? plan.filter((entry) => entry.kind === "start" && entry.target.kind === "local").length, initiatorAuth: ctx.get(InitiatorAuthKey) ?? null, + instrumentationControls: ctx.get(InstrumentationControlsKey), parentTraceContext: readSessionTraceContext(input.serializedContext, session.sessionId), plan, sandboxSessionId, @@ -542,6 +547,9 @@ export async function startSubagent(input: { readonly currentSession: RuntimeSession; readonly fanoutSize: number; readonly initiatorAuth: Parameters[0]["initiatorAuth"]; + readonly instrumentationControls: Parameters< + typeof buildSubagentRunInput + >[0]["instrumentationControls"]; readonly parentContinuationToken: string | undefined; readonly parentTraceContext: Parameters[0]["parentTraceContext"]; readonly persistentSessions: boolean; @@ -572,6 +580,7 @@ export async function startSubagent(input: { dynamicSubagentAgentConfig: input.target.dynamicSubagentAgentConfig, fanoutSize: input.fanoutSize, initiatorAuth: input.initiatorAuth, + instrumentationControls: input.instrumentationControls, parentContinuationToken: input.parentContinuationToken, parentTraceContext, persistentSessions: input.persistentSessions, diff --git a/packages/eve/src/execution/dispatch-runtime-actions-step.integration.test.ts b/packages/eve/src/execution/dispatch-runtime-actions-step.integration.test.ts index aba14dc282..b0c47c0e20 100644 --- a/packages/eve/src/execution/dispatch-runtime-actions-step.integration.test.ts +++ b/packages/eve/src/execution/dispatch-runtime-actions-step.integration.test.ts @@ -29,6 +29,7 @@ import { CapabilitiesKey, ChannelInstrumentationKey, InitiatorAuthKey, + InstrumentationControlsKey, SessionIdKey, SessionKey, } from "#context/keys.js"; @@ -41,6 +42,7 @@ import type { import type { RuntimeSandboxRegistry } from "#runtime/sandbox/registry.js"; import type { ResolvedSandboxDefinition } from "#runtime/types.js"; import { mockSandbox } from "#internal/testing/mocks/mock-sandbox.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; const mocks = vi.hoisted(() => ({ continueRemoteAgentSession: vi.fn(), @@ -850,7 +852,11 @@ describe("dispatchRuntimeActionsStep agent delivery", () => { handle: LOCAL_PARKED_HANDLE, agentId: LOCAL_PARKED_HANDLE.identity.id, }); - installContext(session); + installContext(session, undefined, false, null, { + action: "drop", + recordInputs: false, + recordOutputs: false, + }); const writes: Uint8Array[] = []; const result = await dispatchRuntimeActionsStep({ @@ -871,6 +877,11 @@ describe("dispatchRuntimeActionsStep agent delivery", () => { kind: "send", auth: null, payload: { + instrumentationControls: { + action: "drop", + recordInputs: false, + recordOutputs: false, + }, message: "continue with raw input", outputSchema: undefined, }, @@ -1271,6 +1282,7 @@ function installContext( remote?: { readonly definition: unknown; readonly nodeId: string }, tasks = false, auth: SessionAuthContext | null = null, + instrumentationControls?: InstrumentationControls, ): void { const subagentsByNodeId = new Map(); if (remote !== undefined) { @@ -1295,6 +1307,7 @@ function installContext( [CapabilitiesKey, undefined], [ChannelInstrumentationKey, undefined], [InitiatorAuthKey, null], + [InstrumentationControlsKey, instrumentationControls], [ChannelKey, ADAPTER], ]); mocks.deserializeContext.mockResolvedValue({ diff --git a/packages/eve/src/execution/dispatch-runtime-actions-step.ts b/packages/eve/src/execution/dispatch-runtime-actions-step.ts index 8a3ee94749..4057ef6e32 100644 --- a/packages/eve/src/execution/dispatch-runtime-actions-step.ts +++ b/packages/eve/src/execution/dispatch-runtime-actions-step.ts @@ -25,6 +25,7 @@ import { startSubagent, } from "#execution/dispatch-runtime-actions-shared.js"; import { createDurableSessionState } from "#execution/durable-session-store.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import type { RuntimeActionResult } from "#runtime/actions/types.js"; export async function dispatchRuntimeActionsStep( @@ -32,6 +33,14 @@ export async function dispatchRuntimeActionsStep( ): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + dispatchRuntimeActions(input), + ); +} + +async function dispatchRuntimeActions( + input: RuntimeActionDispatchInput, +): Promise { const prepared = await prepareRuntimeActionDispatch({ serializedContext: input.serializedContext, sessionState: input.sessionState, @@ -75,6 +84,7 @@ export async function dispatchRuntimeActionsStep( dynamicRemoteAgent: entry.dynamicRemoteAgent, }), currentSession: nextSession, + instrumentationControls: prepared.instrumentationControls, parentToken: input.parentContinuationToken ?? session.continuationToken, parentTurnId: batch.event.turnId, }); @@ -90,6 +100,7 @@ export async function dispatchRuntimeActionsStep( currentSession: nextSession, fanoutSize: prepared.fanoutSize, initiatorAuth: prepared.initiatorAuth, + instrumentationControls: prepared.instrumentationControls, parentContinuationToken: input.parentContinuationToken, parentTraceContext: prepared.parentTraceContext, persistentSessions, diff --git a/packages/eve/src/execution/dispatch-turn-step.ts b/packages/eve/src/execution/dispatch-turn-step.ts index 1cd87aeed7..198bf7ad45 100644 --- a/packages/eve/src/execution/dispatch-turn-step.ts +++ b/packages/eve/src/execution/dispatch-turn-step.ts @@ -5,6 +5,7 @@ import { import { buildTurnAttributes, readRootSessionId } from "#execution/eve-workflow-attributes.js"; import { startWorkflowPreferLatest, turnWorkflowReference } from "#execution/workflow-runtime.js"; import { normalizeEveAttributes } from "#runtime/attributes/normalize.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; /** Starts a per-turn child workflow for the current driver session. */ export async function dispatchTurnStep( @@ -12,6 +13,12 @@ export async function dispatchTurnStep( ): Promise<{ readonly runId: string }> { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + dispatchTurn(input), + ); +} + +async function dispatchTurn(input: TurnWorkflowDispatchInput): Promise<{ readonly runId: string }> { const run = await startWorkflowPreferLatest( turnWorkflowReference, [createTurnWorkflowInput(input)], diff --git a/packages/eve/src/execution/dispatch-workflow-runtime-actions-step.ts b/packages/eve/src/execution/dispatch-workflow-runtime-actions-step.ts index 6606c015e7..07b6f10253 100644 --- a/packages/eve/src/execution/dispatch-workflow-runtime-actions-step.ts +++ b/packages/eve/src/execution/dispatch-workflow-runtime-actions-step.ts @@ -23,6 +23,7 @@ import type { RuntimeActionResult, } from "#runtime/actions/types.js"; import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; const log = createLogger("execution.dispatch-workflow-runtime-actions"); @@ -44,6 +45,26 @@ export async function dispatchWorkflowRuntimeActionsStep(input: { }> { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + dispatchWorkflowRuntimeActions(input), + ); +} + +async function dispatchWorkflowRuntimeActions(input: { + readonly callbackBaseUrl?: string; + readonly parentContinuationToken?: string; + readonly parentWritable: WritableStream; + readonly serializedContext: Record; + readonly sessionState: DurableSessionState; +}): Promise<{ + readonly results: readonly RuntimeActionResult[]; + readonly sessionState: DurableSessionState; + readonly pendingTasks: readonly { + readonly taskInboxToken: string; + readonly taskId: string; + readonly taskRunId: string; + }[]; +}> { const durableSession = await readDurableSession(input.sessionState); const pending = getPendingWorkflowInterrupt(durableSession.state); if (pending === undefined) { diff --git a/packages/eve/src/execution/execute-prepared-turn-step.ts b/packages/eve/src/execution/execute-prepared-turn-step.ts new file mode 100644 index 0000000000..6488c3d095 --- /dev/null +++ b/packages/eve/src/execution/execute-prepared-turn-step.ts @@ -0,0 +1,604 @@ +import { buildAdapterContext } from "#channel/adapter-context.js"; +import { + callAdapterEventHandler, + defaultDeliverResult, + type ChannelAdapter, +} from "#channel/adapter.js"; +import type { DeliverHookPayload } from "#channel/types.js"; +import { contextStorage, type ContextContainer } from "#context/container.js"; +import { dispatchStreamEventHooks } from "#context/hook-lifecycle.js"; +import { + dispatchDynamicInstructionEvent, + drainDynamicInstructionUserMessages, + prepareDynamicInstructionPreamble, +} from "#context/dynamic-instruction-lifecycle.js"; +import { dispatchDynamicModelEvent } from "#context/dynamic-model-lifecycle.js"; +import { preserveSerializedSessionDynamicModelSelection } from "#context/serialized-dynamic-model-selection.js"; +import { dispatchDynamicSkillEvent } from "#context/dynamic-skill-lifecycle.js"; +import { + dispatchDynamicSubagentEvent, + refreshDynamicSessionSubagentsForRuntimeRevision, +} from "#context/dynamic-subagent-lifecycle.js"; +import { + dispatchDynamicToolEvent, + refreshDynamicSessionToolsForRuntimeRevision, +} from "#context/dynamic-tool-lifecycle.js"; +import { + CapabilitiesKey, + HandleEventKey, + ModeKey, + SessionDynamicSubagentRuntimeRevisionKey, + SessionDynamicToolRuntimeRevisionKey, + TurnTaskDeliveryKey, +} from "#context/keys.js"; +import type { CompiledBundle } from "#runtime/sessions/runtime-context-keys.js"; +import { serializeContext } from "#context/serialize.js"; +import { + emitTurnPreamble, + getHarnessEmissionState, + isHarnessBetweenTurns, + setHarnessEmissionState, +} from "#harness/emission.js"; +import { + channelDeliveryErrorCode, + instrumentChannelDelivery, +} from "#harness/channel-delivery-instrumentation.js"; +import { preserveSerializedInstrumentationState } from "#harness/instrumentation/state.js"; +import { RuntimeActionSettlementTimesKey } from "#harness/runtime-action-settlement-state.js"; +import { preserveSerializedAgentTraceState } from "#tracing/agent-trace-context-store.js"; +import { matchAuthorizationCallbacks } from "#execution/authorization-callback-match.js"; +import { readTurnSleepDurationMs } from "#harness/turn-sleep.js"; +import { isTurnCancellation, throwIfTurnAborted } from "#harness/turn-cancellation.js"; +import { setChannelContext } from "#execution/channel-context.js"; +import { hasPendingInputBatch } from "#harness/input-requests.js"; +import { activeTurnId } from "#harness/active-turn-id.js"; +import { coalesceTurnInputs } from "#harness/messages.js"; +import { + getRuntimeActionKeysFromWorkflowInterrupt, + isWorkflowRuntimeActionInterrupt, +} from "#harness/workflow-runtime-action-state.js"; +import { getPendingWorkflowInterrupt } from "#harness/workflow-interrupt-state.js"; +import type { HarnessSession, StepInput, StepResult } from "#harness/types.js"; +import { getTurnUsageState, takeSessionUsageDelta, toUsage } from "#harness/turn-tag-state.js"; +import type { DurableStepResult } from "#execution/next-driver-action.js"; +import { + createAuthorizationCompletedEvent, + createSessionStartedEvent, + encodeMessageStreamEvent, + type UnstampedMessageStreamEvent, + stampMessageStreamEvent, + type MessageStreamEvent, +} from "#protocol/message.js"; +import { getPendingAuthorization } from "#harness/authorization.js"; +import { forwardTaskEventToSessionCallback } from "#execution/task-event-callback.js"; +import { resolveEffectiveOutputSchema } from "#execution/effective-output-schema.js"; +import { createDurableSessionState, readDurableSession } from "#execution/durable-session-store.js"; +import type { TurnStepInput } from "#execution/durable-session-migrations/turn-workflow.js"; +import { buildRuntimeIdentity, createExecutionNodeStep } from "#execution/node-step.js"; +import { appendTaskAgentAnnouncement } from "#execution/tasks/parent/agent-views.js"; +import { resolveTaskDeliveryContext } from "#tasks/delivery-context.js"; +import { + readRetainedBackgroundToolResult, + runBackgroundStep, +} from "#execution/tasks/parent/tool-execution.js"; +import { TASK_UPDATE_SESSION_INSTRUCTION } from "#execution/tasks/child/instructions.js"; +import { prepareWorkflowPreambleTrace } from "#execution/workflow-trace-context.js"; +import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.js"; +import { recordSubagentUsageSpans } from "#execution/subagent-usage-span.js"; +import { reconcileSessionContinuationToken } from "#execution/reconcile-session-continuation-token.js"; +import { refreshSessionFromTurnAgent } from "#execution/session.js"; +import { createExecutionHistoryView } from "#execution/history-view.js"; +import { prepareDeliveryInstrumentation } from "#execution/instrumentation-controls.js"; +import { preserveSerializedInstrumentationControls } from "#execution/serialized-instrumentation-controls.js"; +import { resolveRuntimeCompiledArtifactsVersionedCacheKey } from "#runtime/cache-key.js"; +import { createWorkflowRuntime } from "#execution/workflow-runtime.js"; +import { derivePendingState } from "#execution/pending-turn-state.js"; + +const TASK_DONE_WITH_PENDING_INPUT_ERROR_MESSAGE = + "Task mode cannot complete while input requests remain pending."; + +interface PreparedTurnStepInput { + readonly adapter: ChannelAdapter; + readonly bundle: CompiledBundle; + readonly completedAuths: ReturnType["matches"] | undefined; + readonly ctx: ContextContainer; + readonly durableSession: Awaited>; + readonly effectiveAgent: ReturnType; + readonly history: ReturnType; + readonly initialEmissionState: ReturnType; + readonly initialSession: HarnessSession; + readonly input: TurnStepInput; + readonly pendingAuth: ReturnType; + readonly preparedInstrumentation: ReturnType; + readonly rawInput: TurnStepInput; + readonly tasksEnabled: boolean; + readonly taskUpdatesEnabled: boolean; +} + +export async function executePreparedTurnStep( + prepared: PreparedTurnStepInput, +): Promise { + const { + adapter, + bundle, + completedAuths, + ctx, + durableSession, + effectiveAgent, + history, + initialEmissionState, + initialSession, + input, + pendingAuth, + preparedInstrumentation, + rawInput, + tasksEnabled, + taskUpdatesEnabled, + } = prepared; + if (rawInput.input?.kind === "deliver") { + await contextStorage.run(ctx, () => + instrumentChannelDelivery({ + agentName: bundle.turnAgent.id, + ctx, + delivery: rawInput.input as DeliverHookPayload, + hooks: preparedInstrumentation.scope.harness?.hooks, + rootSessionId: initialSession.rootSessionId ?? initialSession.sessionId, + sequence: initialEmissionState.sequence, + sessionId: initialSession.sessionId, + turnId: activeTurnId(initialEmissionState), + }), + ); + } + + const failChannelDeliveries = async (error: unknown): Promise => { + await contextStorage.run(ctx, () => + instrumentChannelDelivery({ + ctx, + error, + errorCode: channelDeliveryErrorCode(error), + hooks: preparedInstrumentation.scope.harness?.hooks, + includeTurn: false, + outcome: "failed", + }), + ); + await preparedInstrumentation.scope.harness?.forceFlush?.(); + }; + const adapterCtx = buildAdapterContext(adapter, ctx); + + // Run the adapter's deliver hook for each queued payload and + // coalesce the resulting StepInput values. + let resolved: StepInput | undefined; + if (input.input?.kind === "deliver") { + const results: StepInput[] = []; + try { + for (const payload of input.input.payloads) { + const deliver = async () => + await (adapter.deliver + ? adapter.deliver(payload, adapterCtx) + : defaultDeliverResult(payload)); + const result = await deliver(); + + if (result !== undefined && result !== null) { + results.push(result); + } + } + } catch (error) { + await failChannelDeliveries(error); + throw error; + } + resolved = results.length === 0 ? undefined : results.reduce(coalesceTurnInputs); + } else if (input.input?.kind === "runtime-action-result") { + const results = input.input.results; + recordSubagentUsageSpans(results); + if (input.input.acceptedAtMsByCallId !== undefined) { + ctx.set(RuntimeActionSettlementTimesKey, input.input.acceptedAtMsByCallId); + } + resolved = { runtimeActionResults: input.input.results }; + } + + if ( + resolved !== undefined && + rawInput.input?.kind === "deliver" && + rawInput.input.taskDeliveryId !== undefined + ) { + const taskContext = resolveTaskDeliveryContext({ + state: durableSession.state, + taskDeliveryId: rawInput.input.taskDeliveryId, + }); + if (taskContext !== undefined) { + ctx.set(TurnTaskDeliveryKey, taskContext.phase); + resolved = { + ...resolved, + context: [...(resolved.context ?? []), taskContext.context], + }; + } + } + + // Persist adapter-state mutations across the step boundary. + if (input.input?.kind === "deliver") { + const updatedAdapter = { ...adapter, state: { ...adapterCtx.state } }; + setChannelContext(ctx, updatedAdapter); + } + + // Adapter handled the delivery inline; re-park and skip unchanged snapshot writes. + if (input.input?.kind === "deliver" && resolved === undefined) { + await contextStorage.run(ctx, () => + instrumentChannelDelivery({ + ctx, + hooks: preparedInstrumentation.scope.harness?.hooks, + includeTurn: false, + outcome: "completed", + }), + ); + await preparedInstrumentation.scope.harness?.forceFlush?.(); + const rekeyed = reconcileSessionContinuationToken(ctx, initialSession); + const nextSerializedContext = serializeContext(ctx); + const nextState = + rekeyed === initialSession + ? input.sessionState + : createDurableSessionState({ session: rekeyed }); + + return { + action: "park", + ...derivePendingState(rekeyed), + serializedContext: nextSerializedContext, + sessionState: nextState, + tasksEnabled, + }; + } + + const hookRegistry = bundle.hookRegistry; + const dynamicInstructionsResolvers = bundle.resolvedAgent.dynamicInstructionsResolvers ?? []; + const dynamicSkillResolvers = bundle.resolvedAgent.dynamicSkillResolvers ?? []; + const dynamicSubagentResolvers = bundle.subagentRegistry.dynamicResolvers ?? []; + const persistentSubagentSessions = + tasksEnabled || bundle.resolvedAgent.config?.experimental?.subagentPersistentSessions === true; + const dynamicToolResolvers = bundle.resolvedAgent.dynamicToolResolvers ?? []; + const effectiveNode = { + ...bundle.graph.root, + turnAgent: effectiveAgent.turnAgent, + }; + const runtimeIdentity = buildRuntimeIdentity(effectiveNode); + try { + const deploymentId = process.env.VERCEL_DEPLOYMENT_ID?.trim(); + const dynamicRuntimeRevision = deploymentId + ? `deployment:${deploymentId}` + : await resolveRuntimeCompiledArtifactsVersionedCacheKey(bundle.compiledArtifactsSource); + const sessionStarted = initialEmissionState.sessionStarted; + + if (!sessionStarted) { + ctx.set(SessionDynamicSubagentRuntimeRevisionKey, dynamicRuntimeRevision); + ctx.set(SessionDynamicToolRuntimeRevisionKey, dynamicRuntimeRevision); + } else { + const refreshEvent = createSessionStartedEvent({ runtime: runtimeIdentity }); + await Promise.all([ + refreshDynamicSessionSubagentsForRuntimeRevision({ + ctx, + resolvers: dynamicSubagentResolvers, + event: refreshEvent, + messages: history.initial.messages, + persistentSessions: persistentSubagentSessions, + runtimeRevision: dynamicRuntimeRevision, + }), + refreshDynamicSessionToolsForRuntimeRevision({ + ctx, + resolvers: dynamicToolResolvers, + event: refreshEvent, + messages: history.initial.messages, + runtimeRevision: dynamicRuntimeRevision, + }), + ]); + } + } catch (error) { + await failChannelDeliveries(error); + throw error; + } + + const writer = input.parentWritable.getWriter(); + + // Stamp once: the persisted chunk and the hooks below must agree on the id. + const emit = async (event: UnstampedMessageStreamEvent): Promise => { + const toEmit = await callAdapterEventHandler(adapter, event, adapterCtx); + setChannelContext(ctx, { ...adapter, state: { ...adapterCtx.state } }); + const stamped = stampMessageStreamEvent(toEmit); + await writer.write(encodeMessageStreamEvent(stamped)); + return stamped; + }; + + const handleEvent = async ( + event: UnstampedMessageStreamEvent, + messages?: readonly import("ai").ModelMessage[], + ): Promise => { + // A remote task's parent owns its HITL. Forward blocking events over + // the task callback and keep them out of the child's local channel; + // otherwise two TUIs can present and answer the same request. + const forwardedToTaskParent = await forwardTaskEventToSessionCallback(ctx, event); + const emitted = forwardedToTaskParent ? stampMessageStreamEvent(event) : await emit(event); + await dispatchStreamEventHooks({ ctx, registry: hookRegistry, event: emitted }); + if (emitted.type !== "step.started") { + await dispatchDynamicModelEvent({ + ctx, + dynamicModel: effectiveAgent.turnAgent.dynamicModel, + event: emitted, + messages: messages ?? [], + scope: { + moduleMap: bundle.moduleMap, + nodeId: bundle.nodeId, + }, + }); + } + await dispatchDynamicSubagentEvent({ + ctx, + resolvers: dynamicSubagentResolvers, + event: emitted, + messages: messages ?? [], + persistentSessions: persistentSubagentSessions, + }); + await dispatchDynamicToolEvent({ + ctx, + resolvers: dynamicToolResolvers, + event: emitted, + messages: messages ?? [], + }); + await dispatchDynamicSkillEvent({ + ctx, + resolvers: dynamicSkillResolvers, + event: emitted, + messages: messages ?? [], + }); + await dispatchDynamicInstructionEvent({ + ctx, + resolvers: dynamicInstructionsResolvers, + event: emitted, + messages: messages ?? [], + }); + }; + + const mode = ctx.require(ModeKey); + + let stepResult: StepResult; + try { + // A signal already aborted at entry (cancellation during an in-line + // runtime-action wait) must settle before the park-resume stages run, + // or the pending batch would re-park and later re-dispatch. + throwIfTurnAborted(input.abortSignal); + stepResult = await runBackgroundStep(ctx, initialSession, async (enrichedSession) => { + ctx.setVirtualContext(HandleEventKey, handleEvent); + let schemaSession = resolveEffectiveOutputSchema({ + agentOutputSchema: effectiveAgent.turnAgent.outputSchema, + input: resolved, + mode, + session: enrichedSession, + }); + if (completedAuths) { + let emissionState = getHarnessEmissionState(schemaSession.state); + if (isHarnessBetweenTurns(schemaSession)) { + prepareDynamicInstructionPreamble(ctx, history.messages(schemaSession)); + let instructionMessages: readonly import("ai").ModelMessage[] = []; + const traceContext = await prepareWorkflowPreambleTrace({ + ctx, + emissionState, + instrumentation: preparedInstrumentation.scope.harness, + runtimeIdentity, + session: schemaSession, + }); + try { + emissionState = await emitTurnPreamble( + handleEvent, + {}, + emissionState, + runtimeIdentity, + traceContext, + ); + } finally { + instructionMessages = drainDynamicInstructionUserMessages(ctx); + schemaSession = { + ...schemaSession, + history: [...schemaSession.history, ...instructionMessages], + }; + } + schemaSession = setHarnessEmissionState(schemaSession, emissionState); + } + for (const { authorization, result } of completedAuths) { + const candidateId = pendingAuth?.challenges.find( + (challenge) => challenge.attemptId === result.attemptId, + )?.candidateId; + await handleEvent( + createAuthorizationCompletedEvent({ + attemptId: result.attemptId, + authorization, + candidateId, + name: result.name, + outcome: "authorized", + sequence: emissionState.sequence, + stepIndex: emissionState.stepIndex, + turnId: emissionState.turnId, + }), + ); + } + } + + const capabilities = ctx.get(CapabilitiesKey); + + const runHarnessStep = async ( + lifecycleSession: HarnessSession, + stepInput: StepInput | undefined, + ): Promise => { + const refreshedSession = refreshSessionFromTurnAgent({ + compactionOverrides: { + thresholdPercent: effectiveAgent.thresholdPercent, + }, + session: lifecycleSession, + systemPromptAdditions: taskUpdatesEnabled ? [TASK_UPDATE_SESSION_INSTRUCTION] : undefined, + turnAgent: effectiveAgent.turnAgent, + }); + const modelSession = tasksEnabled + ? await appendTaskAgentAnnouncement(refreshedSession, history.messages(refreshedSession)) + : refreshedSession; + + const step = createExecutionNodeStep({ + abortSignal: input.abortSignal, + capabilities, + clearOnly: input.input?.kind === "clear", + compactOnly: input.input?.kind === "compact", + createRuntime: createWorkflowRuntime, + handleEvent, + historyProjector: history.projector, + historyView: history.prepare(modelSession), + instrumentation: preparedInstrumentation.scope.harness, + instrumentationChannel: preparedInstrumentation.channel, + mode, + modelResolutionScope: { + moduleMap: bundle.moduleMap, + nodeId: bundle.nodeId, + }, + node: effectiveNode, + workflowMaxSubagents: refreshedSession.workflowMaxSubagents, + }); + return step(modelSession, stepInput); + }; + + return runHarnessStep(schemaSession, resolved); + }); + } catch (error) { + if (!isTurnCancellation(error)) { + await failChannelDeliveries(error); + throw error; + } + writer.releaseLock(); + // Trace and instrumentation state are needed by the cancellation + // epilogue to close the operation the discarded step opened. + // The session model is also kept because `session.started` is not emitted + // again after this cancellation settles. + const interrupted = serializeContext(ctx); + const retained = readRetainedBackgroundToolResult(ctx); + return { + action: "cancelled", + ...(retained === undefined + ? {} + : { + backgroundTaskState: createDurableSessionState({ + session: retained.backgroundTaskSession, + }), + backgroundTasks: retained.backgroundTasks, + }), + serializedContext: preserveSerializedInstrumentationControls( + preserveSerializedInstrumentationState( + preserveSerializedAgentTraceState( + preserveSerializedSessionDynamicModelSelection(input.serializedContext, interrupted), + interrupted, + ), + interrupted, + ), + interrupted, + ), + sessionState: input.sessionState, + }; + } + + // Re-stamp if a handler called `session.continuation.rekey(...)` (eg. Slack auto-anchor). + const rekeyed = reconcileSessionContinuationToken(ctx, stepResult.session); + const nextSerializedContext = serializeContext(ctx); + stepResult = { ...stepResult, session: rekeyed }; + + const nextState = createDurableSessionState({ session: stepResult.session }); + const sleepDurationMs = readTurnSleepDurationMs(ctx); + const sleepTransition = sleepDurationMs === undefined ? {} : { sleepDurationMs }; + const backgroundTransition = + stepResult.backgroundTasks === undefined || stepResult.backgroundTaskSession === undefined + ? {} + : { + backgroundTaskState: createDurableSessionState({ + session: stepResult.backgroundTaskSession, + }), + backgroundTasks: stepResult.backgroundTasks, + }; + + if ( + stepResult.next !== null && + typeof stepResult.next === "object" && + "done" in stepResult.next + ) { + if (mode === "task" && hasPendingInputBatch(stepResult.session.state)) { + writer.releaseLock(); + throw new Error(TASK_DONE_WITH_PENDING_INPUT_ERROR_MESSAGE); + } + await writer.close(); + const sessionTotals = getTurnUsageState(stepResult.session.state)?.session; + return { + action: "done", + ...backgroundTransition, + output: stepResult.next.output, + isError: stepResult.next.isError, + ...sleepTransition, + serializedContext: nextSerializedContext, + sessionState: nextState, + usage: sessionTotals === undefined ? undefined : toUsage(sessionTotals), + usageDelta: takeSessionUsageDelta(stepResult.session).delta, + }; + } + + if (stepResult.next === null) { + writer.releaseLock(); + + const workflowInterrupt = getPendingWorkflowInterrupt(stepResult.session.state); + if ( + workflowInterrupt !== undefined && + isWorkflowRuntimeActionInterrupt(workflowInterrupt.interrupt) + ) { + return { + action: "dispatch-workflow-runtime-actions", + ...backgroundTransition, + pendingRuntimeActionKeys: getRuntimeActionKeysFromWorkflowInterrupt( + workflowInterrupt.interrupt, + ), + ...sleepTransition, + serializedContext: nextSerializedContext, + sessionState: nextState, + }; + } + + const pending = derivePendingState(stepResult.session); + + // `settledTurn` is the harness's explicit settlement verdict. Pending + // state may predate this turn, while newly created parks omit the verdict. + // `usage` carries only this turn's delta: the take marks the totals + // reported, so a persistent child never re-reports earlier spend. + if (stepResult.settledTurn !== undefined) { + const { delta, session: reportedSession } = takeSessionUsageDelta(stepResult.session); + return { + action: "park", + ...backgroundTransition, + ...pending, + ...sleepTransition, + serializedContext: nextSerializedContext, + sessionState: createDurableSessionState({ session: reportedSession }), + settled: { + output: stepResult.settledTurn.output, + isError: stepResult.settledTurn.isError, + usage: delta, + }, + tasksEnabled, + }; + } + + return { + action: "park", + ...backgroundTransition, + ...pending, + ...sleepTransition, + serializedContext: nextSerializedContext, + sessionState: nextState, + tasksEnabled, + }; + } + + writer.releaseLock(); + return { + action: "continue", + ...backgroundTransition, + ...sleepTransition, + serializedContext: nextSerializedContext, + sessionState: nextState, + }; +} diff --git a/packages/eve/src/execution/instrumentation-controls.test.ts b/packages/eve/src/execution/instrumentation-controls.test.ts new file mode 100644 index 0000000000..d0e27321e8 --- /dev/null +++ b/packages/eve/src/execution/instrumentation-controls.test.ts @@ -0,0 +1,116 @@ +import { describe, expect, it } from "vitest"; + +import { ContextContainer } from "#context/container.js"; +import { ChannelInstrumentationKey, InstrumentationControlsKey } from "#context/keys.js"; +import { prepareDeliveryInstrumentation } from "#execution/instrumentation-controls.js"; +import { consumeDeliveryInstrumentationControls } from "#execution/instrumentation-controls.js"; +import { setChannelContext } from "#execution/channel-context.js"; +import { + createInstrumentationRuntime, + type InstrumentationRuntime, +} from "#harness/instrumentation/runtime.js"; + +function runtime(): InstrumentationRuntime { + const hooks = { capturesContent: true, publish: async () => undefined }; + return createInstrumentationRuntime({ + createHooks: () => hooks, + forceFlush: async () => undefined, + otelSettings: { + recordInputs: true, + recordOutputs: true, + traceChannelRequests: false, + }, + resolveDecision: () => ({ action: "record", recordInputs: true, recordOutputs: true }), + runInContext: (_operation, execute) => execute(), + runWithTracingSuppressed: (execute) => execute(), + shutdown: async () => undefined, + }); +} + +describe("prepareDeliveryInstrumentation", () => { + it("removes audience from the channel projection before the harness", () => { + const ctx = new ContextContainer(); + ctx.set(ChannelInstrumentationKey, { + kind: "channel:slack", + metadata: { audience: "private", threadTs: "1234.5678" }, + }); + + const prepared = prepareDeliveryInstrumentation({ + adapter: { kind: "slack" }, + ctx, + delivery: { kind: "deliver" }, + instrumentation: runtime(), + rootSessionId: "session-1", + sessionId: "session-1", + }); + + expect(prepared.channel).toEqual({ + kind: "channel:slack", + metadata: { threadTs: "1234.5678" }, + }); + }); + + it("allows a local subagent decision to tighten but not loosen inherited controls", () => { + const ctx = new ContextContainer(); + ctx.set(ChannelInstrumentationKey, { kind: "subagent", metadata: { audience: "public" } }); + ctx.set(InstrumentationControlsKey, { + action: "record", + recordInputs: false, + recordOutputs: true, + }); + + prepareDeliveryInstrumentation({ + adapter: { kind: "subagent" }, + ctx, + delivery: { kind: "deliver" }, + instrumentation: runtime(), + rootSessionId: "session-1", + sessionId: "session-1", + }); + + expect(ctx.get(InstrumentationControlsKey)).toEqual({ + action: "record", + recordInputs: false, + recordOutputs: true, + }); + }); + + it("consumes controls ferried to a persistent local child", () => { + const ctx = new ContextContainer(); + const delivery = consumeDeliveryInstrumentationControls(ctx, { + kind: "deliver", + payloads: [ + { + instrumentationControls: { + action: "drop", + recordInputs: false, + recordOutputs: false, + }, + message: "private", + }, + ], + }); + + expect(ctx.get(InstrumentationControlsKey)).toEqual({ + action: "drop", + recordInputs: false, + recordOutputs: false, + }); + expect(delivery.payloads).toEqual([{ message: "private" }]); + }); + + it("preserves inherited audience metadata when subagent state is refreshed", () => { + const ctx = new ContextContainer(); + ctx.set(ChannelInstrumentationKey, { + kind: "subagent", + metadata: { audience: "public", parent: "session-1" }, + }); + + setChannelContext(ctx, { kind: "subagent", state: { updated: true } }); + + expect(ctx.get(ChannelInstrumentationKey)?.metadata).toEqual({ + audience: "public", + parent: "session-1", + }); + }); +}); diff --git a/packages/eve/src/execution/instrumentation-controls.ts b/packages/eve/src/execution/instrumentation-controls.ts new file mode 100644 index 0000000000..381fd09403 --- /dev/null +++ b/packages/eve/src/execution/instrumentation-controls.ts @@ -0,0 +1,111 @@ +import { getAdapterKind, type ChannelAdapter } from "#channel/adapter.js"; +import type { DeliverHookPayload, DeliverPayload } from "#channel/types.js"; +import type { ContextContainer } from "#context/container.js"; +import { ChannelInstrumentationKey, InstrumentationControlsKey } from "#context/keys.js"; +import { + constructInstrumentation, + getInstrumentationRuntime, + type ConstructedInstrumentation, + type InstrumentationRuntime, +} from "#harness/instrumentation/runtime.js"; +import { normalizeChannelAudience, withoutChannelAudience } from "#shared/channel-audience.js"; +import { intersectInstrumentationControls } from "#shared/instrumentation-controls.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; + +interface PreparedDeliveryInstrumentation { + readonly channel?: { + readonly kind?: string; + readonly metadata: Readonly>; + }; + readonly scope: ConstructedInstrumentation; +} + +const UNINSTRUMENTED: ConstructedInstrumentation = { + run: (execute) => execute(), +}; + +/** Resolves one delivery's audience at the execution boundary and returns bound controls. */ +export function prepareDeliveryInstrumentation(input: { + readonly adapter: ChannelAdapter; + readonly agentName?: string; + readonly ctx: ContextContainer; + readonly delivery?: { readonly deliveryMetadata?: unknown; readonly kind: string }; + readonly instrumentation?: InstrumentationRuntime; + readonly rootSessionId: string; + readonly sessionId: string; +}): PreparedDeliveryInstrumentation { + const channel = input.ctx.get(ChannelInstrumentationKey); + const projectedChannel = + channel === undefined + ? undefined + : { + kind: channel.kind, + metadata: withoutChannelAudience(channel.metadata), + }; + const { instrumentation } = input; + if (instrumentation === undefined) { + return { channel: projectedChannel, scope: UNINSTRUMENTED }; + } + + const existing = input.ctx.get(InstrumentationControlsKey); + const shouldResolve = existing === undefined || input.delivery?.kind === "deliver"; + const resolved = shouldResolve + ? instrumentation.resolveDecision({ + agentName: input.agentName, + audience: normalizeChannelAudience(channel?.metadata.audience), + rootSessionId: input.rootSessionId, + sessionId: input.sessionId, + }) + : existing; + const controls = + existing !== undefined && getAdapterKind(input.adapter) === "subagent" + ? intersectInstrumentationControls(existing, resolved) + : resolved; + input.ctx.set(InstrumentationControlsKey, controls); + return { + channel: projectedChannel, + scope: constructInstrumentation(instrumentation, controls), + }; +} + +/** Consumes controls ferried to a local child before its adapter sees the payload. */ +export function consumeDeliveryInstrumentationControls( + ctx: ContextContainer, + delivery: DeliverHookPayload, +): DeliverHookPayload { + let inherited: InstrumentationControls | undefined; + let changed = false; + const payloads = delivery.payloads.map((payload): DeliverPayload => { + const controls = payload.instrumentationControls; + if (controls === undefined) return payload; + inherited = + inherited === undefined ? controls : intersectInstrumentationControls(inherited, controls); + changed = true; + const { instrumentationControls: _controls, ...visible } = payload; + return visible; + }); + if (inherited !== undefined) ctx.set(InstrumentationControlsKey, inherited); + return changed ? { ...delivery, payloads } : delivery; +} + +/** Constructs the active delivery capability from a persisted decision. */ +export function constructExecutionInstrumentation( + controls: InstrumentationControls | undefined, + instrumentation: InstrumentationRuntime | undefined, +): ConstructedInstrumentation { + return controls === undefined || instrumentation === undefined + ? UNINSTRUMENTED + : constructInstrumentation(instrumentation, controls); +} + +/** Constructs an out-of-band execution capability from serialized state. */ +export function constructSerializedInstrumentation( + serializedContext: Record, +): ConstructedInstrumentation { + const value = serializedContext[InstrumentationControlsKey.name]; + const controls = + typeof value === "object" && value !== null && "action" in value + ? (value as InstrumentationControls) + : undefined; + return constructExecutionInstrumentation(controls, getInstrumentationRuntime()); +} diff --git a/packages/eve/src/execution/node-step.ts b/packages/eve/src/execution/node-step.ts index eb40d0ceeb..38dcdaad30 100644 --- a/packages/eve/src/execution/node-step.ts +++ b/packages/eve/src/execution/node-step.ts @@ -9,9 +9,14 @@ import { import type { HarnessToolDefinition } from "#harness/execute-tool.js"; import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js"; import { createToolLoopHarness } from "#harness/tool-loop.js"; -import type { HandleEventFn, HarnessToolMap, StepFn } from "#harness/types.js"; +import type { + HandleEventFn, + HarnessToolMap, + StepFn, + ToolLoopHarnessConfig, +} from "#harness/types.js"; +import type { HarnessInstrumentation } from "#harness/instrumentation/runtime.js"; import { resolveInstalledPackageInfo } from "#internal/application/package.js"; -import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; import { createLogger } from "#internal/logging.js"; import type { RuntimeIdentity } from "#protocol/message.js"; import { UNSPECIFIED_INPUT_SCHEMA } from "#shared/tool-schema.js"; @@ -81,6 +86,8 @@ export interface CreateExecutionNodeStepInput { readonly handleEvent?: HandleEventFn; readonly historyProjector?: HistoryViewProjector; readonly historyView?: PreparedHistoryView; + readonly instrumentation?: HarnessInstrumentation; + readonly instrumentationChannel?: ToolLoopHarnessConfig["instrumentationChannel"]; readonly mode: RunMode; readonly modelResolutionScope: RuntimeModelResolutionScope; readonly node: ResolvedRuntimeAgentNode; @@ -105,7 +112,7 @@ export function createExecutionNodeStep(input: CreateExecutionNodeStepInput): St input.node.turnAgent.dynamicModel, ); const tools = createNodeHarnessTools({ node: input.node }); - const instrumentation = getInstrumentationRuntime(); + const instrumentation = input.instrumentation; const step = createToolLoopHarness({ abortSignal: input.abortSignal, capabilities: input.capabilities, @@ -118,6 +125,7 @@ export function createExecutionNodeStep(input: CreateExecutionNodeStepInput): St historyProjector: input.historyProjector, historyView: input.historyView, instrumentation, + instrumentationChannel: input.instrumentationChannel, mode: input.mode, onCompaction: preserveFrameworkStateOnCompaction, persistentSubagentSessions: @@ -133,7 +141,7 @@ export function createExecutionNodeStep(input: CreateExecutionNodeStepInput): St try { return await step(session, stepInput); } finally { - await instrumentation.forceFlush(); + await instrumentation.forceFlush?.(); } }; } diff --git a/packages/eve/src/execution/pending-turn-state.ts b/packages/eve/src/execution/pending-turn-state.ts new file mode 100644 index 0000000000..bddb41f49d --- /dev/null +++ b/packages/eve/src/execution/pending-turn-state.ts @@ -0,0 +1,31 @@ +import { getPendingAuthorization } from "#harness/authorization.js"; +import { hasPendingInputBatch } from "#harness/input-requests.js"; +import { getPendingRuntimeActionBatch } from "#harness/runtime-actions.js"; +import type { HarnessSession } from "#harness/types.js"; +import { getRuntimeActionRequestKey } from "#runtime/actions/keys.js"; + +/** Projects the pending fields consumed by the turn workflow at a park boundary. */ +export function derivePendingState(session: HarnessSession): { + readonly authorizationAttemptIds?: readonly string[]; + readonly authorizationNames?: readonly string[]; + readonly hasPendingAuthorization: boolean; + readonly hasPendingInputBatch: boolean; + readonly pendingRuntimeActionKeys?: readonly string[]; +} { + const batch = getPendingRuntimeActionBatch(session.state); + const pendingAuth = getPendingAuthorization(session.state); + const base = { + authorizationAttemptIds: pendingAuth?.challenges.flatMap((challenge) => + challenge.attemptId === undefined ? [] : [challenge.attemptId], + ), + authorizationNames: pendingAuth?.challenges.map((challenge) => challenge.name), + hasPendingAuthorization: pendingAuth !== undefined, + hasPendingInputBatch: hasPendingInputBatch(session.state), + }; + return batch === undefined + ? base + : { + ...base, + pendingRuntimeActionKeys: batch.actions.map(getRuntimeActionRequestKey), + }; +} diff --git a/packages/eve/src/execution/proxied-deliver-step.ts b/packages/eve/src/execution/proxied-deliver-step.ts index 8cf692f7c3..d0fa2838ec 100644 --- a/packages/eve/src/execution/proxied-deliver-step.ts +++ b/packages/eve/src/execution/proxied-deliver-step.ts @@ -1,5 +1,6 @@ import type { DeliverHookPayload, DeliverPayload, SessionAuthContext } from "#channel/types.js"; import { coalesceDeliverPayloads } from "#execution/deliver-payloads.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { type DurableSessionState, readDurableSession, @@ -83,6 +84,27 @@ export async function routeProxiedDeliverStep( ): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext ?? {}).run(() => + routeProxiedDeliver(input), + ); +} + +async function routeProxiedDeliver( + input: + | { + readonly delivery: DeliverHookPayload; + readonly parentWritable: WritableStream; + readonly serializedContext?: Record; + readonly sessionState: DurableSessionState; + } + | { + readonly auth?: SessionAuthContext | null; + readonly parentWritable: WritableStream; + readonly payload: DeliverPayload; + readonly serializedContext?: Record; + readonly sessionState: DurableSessionState; + }, +): Promise { let durableSession = await readDurableSession(input.sessionState); const legacyInput = !("delivery" in input); const sourceDelivery: DeliverHookPayload = diff --git a/packages/eve/src/execution/runtime-context.ts b/packages/eve/src/execution/runtime-context.ts index ebae5ec9bd..b639c64bda 100644 --- a/packages/eve/src/execution/runtime-context.ts +++ b/packages/eve/src/execution/runtime-context.ts @@ -10,6 +10,7 @@ import { ContinuationTokenKey, DynamicSubagentAgentConfigKey, InitiatorAuthKey, + InstrumentationControlsKey, ModeKey, ParentSessionKey, ParentTraceContextKey, @@ -48,6 +49,9 @@ export function buildRunContext(input: { ctx.set(ModeKey, run.mode); ctx.set(AuthKey, auth); ctx.set(InitiatorAuthKey, run.initiatorAuth ?? auth); + if (run.instrumentationControls !== undefined) { + ctx.set(InstrumentationControlsKey, run.instrumentationControls); + } if (input.dynamicSubagentAgentConfig !== undefined) { ctx.set(DynamicSubagentAgentConfigKey, input.dynamicSubagentAgentConfig); diff --git a/packages/eve/src/execution/serialized-instrumentation-controls.ts b/packages/eve/src/execution/serialized-instrumentation-controls.ts new file mode 100644 index 0000000000..841a754a91 --- /dev/null +++ b/packages/eve/src/execution/serialized-instrumentation-controls.ts @@ -0,0 +1,12 @@ +import { InstrumentationControlsKey } from "#context/keys.js"; + +/** Retains a delivery decision while rolling back the rest of a cancelled turn. */ +export function preserveSerializedInstrumentationControls( + before: Record, + after: Record, +): Record { + const controls = after[InstrumentationControlsKey.name]; + return controls === undefined + ? before + : { ...before, [InstrumentationControlsKey.name]: controls }; +} diff --git a/packages/eve/src/execution/session-callback-step.ts b/packages/eve/src/execution/session-callback-step.ts index f77ea13766..9754381bc1 100644 --- a/packages/eve/src/execution/session-callback-step.ts +++ b/packages/eve/src/execution/session-callback-step.ts @@ -6,6 +6,10 @@ import { SESSION_FAILED } from "#harness/agent-handle-errors.js"; import { createLogger } from "#internal/logging.js"; import { toErrorMessage } from "#shared/errors.js"; import type { TokenUsage } from "#shared/token-usage.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; +import { constructExecutionInstrumentation } from "#execution/instrumentation-controls.js"; +import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; import type { UnstampedMessageStreamEvent } from "#protocol/message.js"; const log = createLogger("execution.session-callback"); @@ -18,9 +22,25 @@ export async function fireTaskEventCallbackStep(input: { readonly event: | SubagentAuthorizationEvent | Extract; + readonly instrumentationControls?: InstrumentationControls; }): Promise { "use step"; + return await constructExecutionInstrumentation( + input.instrumentationControls, + getInstrumentationRuntime(), + ).run(() => fireTaskEventCallback(input)); +} + +async function fireTaskEventCallback(input: { + readonly callback: unknown; + readonly childContinuationToken: string; + readonly childSessionId: string; + readonly event: + | SubagentAuthorizationEvent + | Extract; + readonly instrumentationControls?: InstrumentationControls; +}): Promise { const callback = parseSerializedSessionCallback(input.callback); if (callback.taskId === undefined) return; const inputRequested = input.event.type === "input.requested"; @@ -49,9 +69,24 @@ export async function fireTaskUpdateCallbackStep(input: { readonly updateIndex: number; readonly updateEpoch: string; readonly message: string; + readonly instrumentationControls?: InstrumentationControls; }): Promise { "use step"; + return await constructExecutionInstrumentation( + input.instrumentationControls, + getInstrumentationRuntime(), + ).run(() => fireTaskUpdateCallback(input)); +} + +async function fireTaskUpdateCallback(input: { + readonly callback: unknown; + readonly callId: string; + readonly updateIndex: number; + readonly updateEpoch: string; + readonly message: string; + readonly instrumentationControls?: InstrumentationControls; +}): Promise { const callback = parseSerializedSessionCallback(input.callback); if (callback.taskId === undefined) return undefined; const response = await postSessionCallbackRequest({ @@ -94,6 +129,18 @@ export async function fireSessionCallbackStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + fireSessionCallback(input), + ); +} + +async function fireSessionCallback(input: { + readonly error?: unknown; + readonly output?: unknown; + readonly serializedContext: Record; + readonly status: "completed" | "failed"; + readonly usage?: TokenUsage; +}): Promise { const sessionId = (input.serializedContext["eve.sessionId"] as string | undefined) ?? ""; const value = input.serializedContext[SessionCallbackKey.name]; if (value === undefined) { diff --git a/packages/eve/src/execution/settle-cancelled-turn-step.ts b/packages/eve/src/execution/settle-cancelled-turn-step.ts index 5a0193659e..bc9d90e461 100644 --- a/packages/eve/src/execution/settle-cancelled-turn-step.ts +++ b/packages/eve/src/execution/settle-cancelled-turn-step.ts @@ -5,6 +5,7 @@ import { withContextScope } from "#context/run-step.js"; import { deserializeContext, serializeContext } from "#context/serialize.js"; import { ChannelInstrumentationKey } from "#context/keys.js"; import { setChannelContext } from "#execution/channel-context.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { createDurableSessionState, type DurableSessionState, @@ -28,7 +29,7 @@ import { import { abandonRunningAgentTurns } from "#harness/handles/transitions.js"; import { clearPendingRuntimeActionBatch } from "#harness/runtime-actions.js"; import { createInstrumentationHandleEvent } from "#harness/instrumentation/native-events.js"; -import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; +import type { ConstructedInstrumentation } from "#harness/instrumentation/runtime.js"; import { getTurnUsageState, toUsage } from "#harness/turn-tag-state.js"; import { clearPendingWorkflowInterrupt } from "#harness/workflow-interrupt-state.js"; import { @@ -59,13 +60,24 @@ export async function settleCancelledTurnStep(input: { }): Promise { "use step"; + const instrumentationScope = constructSerializedInstrumentation(input.serializedContext); + return await instrumentationScope.run(() => settleCancelledTurn(input, instrumentationScope)); +} + +async function settleCancelledTurn( + input: { + readonly parentWritable: WritableStream; + readonly serializedContext: Record; + readonly sessionState: DurableSessionState; + }, + instrumentationScope: ConstructedInstrumentation, +): Promise { const durableSession = await readDurableSession(input.sessionState); const ctx = await deserializeContext(input.serializedContext); const adapter = ctx.require(ChannelKey); const adapterCtx = buildAdapterContext(adapter, ctx); const bundle = ctx.require(BundleKey); const effectiveAgent = resolveEffectiveAgentRuntime(bundle, ctx); - const instrumentation = getInstrumentationRuntime(); let session = hydrateDurableSession({ compactionOverrides: { @@ -91,37 +103,39 @@ export async function settleCancelledTurnStep(input: { if (!alreadyEpilogued) { const writer = input.parentWritable.getWriter(); try { - const scoped = await withContextScope(ctx, session, async (enrichedSession) => { - const baseEmit = async (event: UnstampedMessageStreamEvent): Promise => { - const transformed = await callAdapterEventHandler(adapter, event, adapterCtx); - setChannelContext(ctx, { ...adapter, state: { ...adapterCtx.state } }); - // Stamp once: the persisted chunk and the hooks must agree on the id. - const stamped = stampMessageStreamEvent(transformed); - await writer.write(encodeMessageStreamEvent(stamped)); - await dispatchStreamEventHooks({ - ctx, - event: stamped, - registry: bundle.hookRegistry, - }); - }; - const emit = - createInstrumentationHandleEvent({ - agentName: bundle.turnAgent.id, - channelKind: ctx.get(ChannelInstrumentationKey)?.kind, - handleEvent: baseEmit, - hooks: instrumentation?.hooks, - sessionId: session.sessionId, - turnId: activeTurnId(emissionState), - }) ?? baseEmit; - return { - result: await emitCancelledTurn(emit, emissionState), - session: enrichedSession, - }; - }); + const settle = () => + withContextScope(ctx, session, async (enrichedSession) => { + const baseEmit = async (event: UnstampedMessageStreamEvent): Promise => { + const transformed = await callAdapterEventHandler(adapter, event, adapterCtx); + setChannelContext(ctx, { ...adapter, state: { ...adapterCtx.state } }); + // Stamp once: the persisted chunk and the hooks must agree on the id. + const stamped = stampMessageStreamEvent(transformed); + await writer.write(encodeMessageStreamEvent(stamped)); + await dispatchStreamEventHooks({ + ctx, + event: stamped, + registry: bundle.hookRegistry, + }); + }; + const emit = + createInstrumentationHandleEvent({ + agentName: bundle.turnAgent.id, + channelKind: ctx.get(ChannelInstrumentationKey)?.kind, + handleEvent: baseEmit, + hooks: instrumentationScope.harness?.hooks, + sessionId: session.sessionId, + turnId: activeTurnId(emissionState), + }) ?? baseEmit; + return { + result: await emitCancelledTurn(emit, emissionState), + session: enrichedSession, + }; + }); + const scoped = await settle(); emissionState = scoped.result; session = scoped.session; } finally { - await instrumentation?.forceFlush(); + await instrumentationScope.harness?.forceFlush?.(); writer.releaseLock(); } } diff --git a/packages/eve/src/execution/subagent-event-proxy-step.ts b/packages/eve/src/execution/subagent-event-proxy-step.ts index acb2e6654a..6a0aef752c 100644 --- a/packages/eve/src/execution/subagent-event-proxy-step.ts +++ b/packages/eve/src/execution/subagent-event-proxy-step.ts @@ -25,6 +25,7 @@ import type { UnstampedMessageStreamEvent } from "#protocol/message.js"; import { encodeMessageStreamEvent, stampMessageStreamEvent } from "#protocol/message.js"; import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.js"; import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; type SubagentEventHookPayload = | SubagentAuthorizationEventHookPayload @@ -46,15 +47,7 @@ export async function runProxySubagentEventStep(input: { }): Promise { "use step"; - const durableSession = await readDurableSession(input.sessionState); - const ctx = await deserializeContext(input.serializedContext); - - return emitProxiedSubagentEvent({ - ctx, - durableSession, - hookPayload: input.hookPayload, - parentWritable: input.parentWritable, - }); + return runSerializedProxyEvent(input, true); } /** Emits a task request whose proxy routes were committed by a prior step. */ @@ -66,15 +59,7 @@ export async function emitRecordedTaskInputRequestStep(input: { }): Promise { "use step"; - const durableSession = await readDurableSession(input.sessionState); - const ctx = await deserializeContext(input.serializedContext); - return emitProxiedSubagentEvent({ - ctx, - durableSession, - hookPayload: input.hookPayload, - parentWritable: input.parentWritable, - recordProxyInputRequests: false, - }); + return runSerializedProxyEvent(input, false); } /** Emits a task authorization event after its ownership was validated. */ @@ -86,14 +71,28 @@ export async function emitRecordedTaskAuthorizationEventStep(input: { }): Promise { "use step"; - const durableSession = await readDurableSession(input.sessionState); - const ctx = await deserializeContext(input.serializedContext); - return emitProxiedSubagentEvent({ - ctx, - durableSession, - hookPayload: input.hookPayload, - parentWritable: input.parentWritable, - recordProxyInputRequests: false, + return runSerializedProxyEvent(input, false); +} + +async function runSerializedProxyEvent( + input: { + readonly hookPayload: SubagentEventHookPayload; + readonly parentWritable: WritableStream; + readonly serializedContext: Record; + readonly sessionState: DurableSessionState; + }, + recordProxyInputRequests: boolean, +): Promise { + return await constructSerializedInstrumentation(input.serializedContext).run(async () => { + const durableSession = await readDurableSession(input.sessionState); + const ctx = await deserializeContext(input.serializedContext); + return emitProxiedSubagentEvent({ + ctx, + durableSession, + hookPayload: input.hookPayload, + parentWritable: input.parentWritable, + recordProxyInputRequests, + }); }); } diff --git a/packages/eve/src/execution/subagent-start-local.ts b/packages/eve/src/execution/subagent-start-local.ts index 63cbdd15e4..1aa3170679 100644 --- a/packages/eve/src/execution/subagent-start-local.ts +++ b/packages/eve/src/execution/subagent-start-local.ts @@ -33,6 +33,9 @@ export async function startLocalSubagent(input: { readonly dynamicSubagentAgentConfig?: DynamicSubagentAgentConfig; readonly fanoutSize: number; readonly initiatorAuth: Parameters[0]["initiatorAuth"]; + readonly instrumentationControls: Parameters< + typeof buildSubagentRunInput + >[0]["instrumentationControls"]; readonly parentContinuationToken: string | undefined; readonly parentTraceContext: Parameters[0]["parentTraceContext"]; readonly persistentSessions: boolean; @@ -55,6 +58,7 @@ export async function startLocalSubagent(input: { channelMetadata: input.channelMetadata, fanoutSize: input.fanoutSize, initiatorAuth: input.initiatorAuth, + instrumentationControls: input.instrumentationControls, graph: input.bundle.graph, parentContinuationToken: input.parentContinuationToken, parentTraceContext: input.parentTraceContext, diff --git a/packages/eve/src/execution/subagent-tool.ts b/packages/eve/src/execution/subagent-tool.ts index a4887a3042..fec4493131 100644 --- a/packages/eve/src/execution/subagent-tool.ts +++ b/packages/eve/src/execution/subagent-tool.ts @@ -16,6 +16,7 @@ import type { RuntimeSubagentCallActionRequest } from "#runtime/actions/types.js import { mintSubagentContinuationToken } from "#execution/session.js"; import { resolveSubagentDepth } from "#harness/subagent-depth.js"; import { resolveRemainingSessionTokenLimits } from "#harness/subagent-token-budget.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; /** * Pending runtime-action batch event metadata needed for child run lineage. @@ -85,6 +86,7 @@ export function buildSubagentRunInput(input: { */ readonly fanoutSize?: number; readonly initiatorAuth: SessionAuthContext | null; + readonly instrumentationControls?: InstrumentationControls; /** * Runtime graph used to detect whether this declared child selected the * dispatching parent's sandbox. Absence means no inheritance. @@ -112,6 +114,7 @@ export function buildSubagentRunInput(input: { capabilities, channelMetadata, initiatorAuth, + instrumentationControls, session, source, } = input; @@ -161,6 +164,7 @@ export function buildSubagentRunInput(input: { channelMetadata, continuationToken: childContinuationToken, initiatorAuth, + instrumentationControls, input: { message: formatSubagentCallInputMessage({ action, diff --git a/packages/eve/src/execution/task-event-callback.ts b/packages/eve/src/execution/task-event-callback.ts index efbbadce87..f76f6d8b8b 100644 --- a/packages/eve/src/execution/task-event-callback.ts +++ b/packages/eve/src/execution/task-event-callback.ts @@ -1,5 +1,10 @@ import type { ContextContainer } from "#context/container.js"; -import { ContinuationTokenKey, SessionCallbackKey, SessionIdKey } from "#context/keys.js"; +import { + ContinuationTokenKey, + InstrumentationControlsKey, + SessionCallbackKey, + SessionIdKey, +} from "#context/keys.js"; import { fireTaskEventCallbackStep } from "#execution/session-callback-step.js"; import type { UnstampedMessageStreamEvent } from "#protocol/message.js"; @@ -30,6 +35,7 @@ export async function forwardTaskEventToSessionCallback( childContinuationToken: ctx.require(ContinuationTokenKey), childSessionId: ctx.require(SessionIdKey), event, + instrumentationControls: ctx.get(InstrumentationControlsKey), }); return true; } diff --git a/packages/eve/src/execution/tasks/child/update.ts b/packages/eve/src/execution/tasks/child/update.ts index 7777acef35..e3282f2715 100644 --- a/packages/eve/src/execution/tasks/child/update.ts +++ b/packages/eve/src/execution/tasks/child/update.ts @@ -1,5 +1,5 @@ import type { ChannelAdapter } from "#channel/adapter.js"; -import { SessionCallbackKey } from "#context/keys.js"; +import { InstrumentationControlsKey, SessionCallbackKey } from "#context/keys.js"; import { fireTaskUpdateCallbackStep } from "#execution/session-callback-step.js"; import { isSubagentAdapterState, @@ -10,6 +10,7 @@ import { resumeHook } from "#internal/workflow/runtime.js"; import type { RuntimeActionResult, RuntimeToolCallActionRequest } from "#runtime/actions/types.js"; import { readTaskIdFromInboxToken } from "#tasks/task-id.js"; import type { TaskInboundUpdate } from "#tasks/types.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; /** Sends one child-authored progress update over its existing parent transport. */ export async function executeTaskUpdate(input: { @@ -32,6 +33,9 @@ export async function executeTaskUpdate(input: { updateIndex: input.updateIndex, updateEpoch: input.updateEpoch, message, + instrumentationControls: input.serializedContext?.[InstrumentationControlsKey.name] as + | InstrumentationControls + | undefined, }); if (taskId === undefined) { return createTaskControlError(input.action, "This session is not owned by a parent task."); diff --git a/packages/eve/src/execution/tasks/parent/dispatch-task-step.ts b/packages/eve/src/execution/tasks/parent/dispatch-task-step.ts index 9fda6c9da9..337ee5e5b9 100644 --- a/packages/eve/src/execution/tasks/parent/dispatch-task-step.ts +++ b/packages/eve/src/execution/tasks/parent/dispatch-task-step.ts @@ -28,6 +28,7 @@ import { type RuntimeActionDispatchResult, startSubagent, } from "#execution/dispatch-runtime-actions-shared.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { createDurableSessionState } from "#execution/durable-session-store.js"; import { beginDelegatedTask, @@ -49,6 +50,14 @@ export async function dispatchTaskStep( ): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + dispatchTasks(input), + ); +} + +async function dispatchTasks( + input: RuntimeActionDispatchInput, +): Promise { const prepared = await prepareRuntimeActionDispatch({ serializedContext: input.serializedContext, sessionState: input.sessionState, @@ -57,7 +66,6 @@ export async function dispatchTaskStep( if (prepared === undefined) { return { results: [], sessionState: input.sessionState, pendingTasks: [] }; } - const { batch, bundle, session } = prepared; // Acquired only once preflight can no longer throw, so a planning failure // never leaks the writer lock. @@ -141,6 +149,7 @@ export async function dispatchTaskStep( dynamicRemoteAgent: entry.dynamicRemoteAgent, }), currentSession: nextSession, + instrumentationControls: prepared.instrumentationControls, parentToken: delegated.taskInboxToken, }); break; @@ -155,6 +164,7 @@ export async function dispatchTaskStep( currentSession: nextSession, fanoutSize: prepared.fanoutSize, initiatorAuth: prepared.initiatorAuth, + instrumentationControls: prepared.instrumentationControls, parentContinuationToken: delegated.taskInboxToken, parentTraceContext: prepared.parentTraceContext, // Background tasks require resumable children, so task mode diff --git a/packages/eve/src/execution/terminal-session-completion-step.ts b/packages/eve/src/execution/terminal-session-completion-step.ts index 509765c4c7..d04fd642c6 100644 --- a/packages/eve/src/execution/terminal-session-completion-step.ts +++ b/packages/eve/src/execution/terminal-session-completion-step.ts @@ -2,6 +2,7 @@ import { buildAdapterContext } from "#channel/adapter-context.js"; import { callAdapterEventHandler } from "#channel/adapter.js"; import { deserializeContext } from "#context/serialize.js"; import { createLogger } from "#internal/logging.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { createSessionCompletedEvent, encodeMessageStreamEvent, @@ -18,6 +19,15 @@ export async function emitTerminalSessionCompletionStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + emitTerminalSessionCompletion(input), + ); +} + +async function emitTerminalSessionCompletion(input: { + readonly parentWritable: WritableStream; + readonly serializedContext: Record; +}): Promise { const event = createSessionCompletedEvent(); const sessionId = (input.serializedContext["eve.sessionId"] as string | undefined) ?? ""; diff --git a/packages/eve/src/execution/terminal-session-failure-step.ts b/packages/eve/src/execution/terminal-session-failure-step.ts index 69f61baa6b..07b82c8ed8 100644 --- a/packages/eve/src/execution/terminal-session-failure-step.ts +++ b/packages/eve/src/execution/terminal-session-failure-step.ts @@ -2,6 +2,7 @@ import { buildAdapterContext } from "#channel/adapter-context.js"; import { callAdapterEventHandler } from "#channel/adapter.js"; import { deserializeContext } from "#context/serialize.js"; import { summarizeKnownError } from "#harness/semantic-errors/index.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; import { createLogger, formatError } from "#internal/logging.js"; import { createSessionFailedEvent, @@ -20,6 +21,16 @@ export async function emitTerminalSessionFailureStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext).run(() => + emitTerminalSessionFailure(input), + ); +} + +async function emitTerminalSessionFailure(input: { + readonly error: unknown; + readonly parentWritable: WritableStream; + readonly serializedContext: Record; +}): Promise { // Cataloged failures replace the raw identity with the curated one; the // `detail` dump stays attached to the private event so the session trace // keeps the raw evidence while the transcript shows the actionable summary. diff --git a/packages/eve/src/execution/terminate-child-sessions-step.ts b/packages/eve/src/execution/terminate-child-sessions-step.ts index 34452427fe..440f92971c 100644 --- a/packages/eve/src/execution/terminate-child-sessions-step.ts +++ b/packages/eve/src/execution/terminate-child-sessions-step.ts @@ -8,6 +8,7 @@ import { createLogger, logError } from "#internal/logging.js"; import { cancelRun, getWorld } from "#internal/workflow/runtime.js"; import { getSessionTaskIndex } from "#tasks/session-index.js"; import { BundleKey } from "#runtime/sessions/runtime-context-keys.js"; +import { constructSerializedInstrumentation } from "#execution/instrumentation-controls.js"; const log = createLogger("execution.terminate-child-sessions"); @@ -28,6 +29,15 @@ export async function terminateChildSessionsStep(input: { }): Promise { "use step"; + return await constructSerializedInstrumentation(input.serializedContext ?? {}).run(() => + terminateChildSessions(input), + ); +} + +async function terminateChildSessions(input: { + readonly serializedContext?: Record; + readonly sessionState: DurableSessionState; +}): Promise { let session; try { session = await readDurableSession(input.sessionState); diff --git a/packages/eve/src/execution/turn-workflow.ts b/packages/eve/src/execution/turn-workflow.ts index 05cf2986a9..a1d4c44434 100644 --- a/packages/eve/src/execution/turn-workflow.ts +++ b/packages/eve/src/execution/turn-workflow.ts @@ -30,6 +30,7 @@ import { TurnExecutionCursor } from "#execution/turn-execution-cursor.js"; import { resolveWorkflowCallbackBaseUrl } from "#execution/workflow-callback-url.js"; import { normalizeSerializableError } from "#execution/workflow-errors.js"; import { turnStep } from "#execution/workflow-steps.js"; +import { preserveSerializedInstrumentationControls } from "#execution/serialized-instrumentation-controls.js"; import { activeTurnId } from "#harness/active-turn-id.js"; import { getRuntimeActionResultKey } from "#runtime/actions/keys.js"; import { resolveRuntimeActionResultsForKeys } from "#runtime/actions/results.js"; @@ -145,8 +146,11 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise { // normally after the workflow observes cancellation. Roll that result // back except for a session model selected by its one-time preamble. await cursor.adopt({ - serializedContext: preserveSerializedSessionDynamicModelSelection( - beforeStep.serializedContext, + serializedContext: preserveSerializedInstrumentationControls( + preserveSerializedSessionDynamicModelSelection( + beforeStep.serializedContext, + result.serializedContext, + ), result.serializedContext, ), sessionState: cursor.sessionState, diff --git a/packages/eve/src/execution/workflow-entry.test.ts b/packages/eve/src/execution/workflow-entry.test.ts index 3a38454021..d3081a4167 100644 --- a/packages/eve/src/execution/workflow-entry.test.ts +++ b/packages/eve/src/execution/workflow-entry.test.ts @@ -365,6 +365,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledWith({ caller: undefined, lifecycle: "terminal", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "" }, }); @@ -557,6 +558,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledWith({ caller, lifecycle: "terminal", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { isError: true, @@ -606,6 +608,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledWith({ caller, lifecycle: "terminal", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { isError: true, @@ -704,6 +707,7 @@ describe("workflowEntry", () => { subagentName: "researcher", }, lifecycle: "terminal", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { isError: true, @@ -763,6 +767,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledWith({ caller, lifecycle: "parked", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "settled answer" }, }); @@ -809,6 +814,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledExactlyOnceWith({ caller, lifecycle: "parked", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "approved answer" }, }); @@ -1044,6 +1050,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledExactlyOnceWith({ caller: undefined, lifecycle: "terminal", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "ok" }, }); @@ -1091,6 +1098,7 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenCalledWith({ caller: undefined, lifecycle: "parked", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "hello" }, }); @@ -1231,12 +1239,14 @@ describe("workflowEntry", () => { expect(notifyTurnCallerStep).toHaveBeenNthCalledWith(1, { caller: firstCaller, lifecycle: "parked", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "first answer" }, }); expect(notifyTurnCallerStep).toHaveBeenNthCalledWith(2, { caller: secondCaller, lifecycle: "parked", + serializedContext: expect.any(Object), sessionId: "wrun_test_123", settled: { output: "second answer" }, }); diff --git a/packages/eve/src/execution/workflow-entry.ts b/packages/eve/src/execution/workflow-entry.ts index 3d78e11cbd..e832e141a8 100644 --- a/packages/eve/src/execution/workflow-entry.ts +++ b/packages/eve/src/execution/workflow-entry.ts @@ -262,6 +262,7 @@ export async function workflowEntry(input: WorkflowEntryInput): Promise { + delete (globalThis as Record)[Symbol.for("eve.instrumentation-runtime")]; getRunMock.mockReset(); resumeHookMock.mockReset(); startMock.mockReset(); @@ -249,6 +256,101 @@ afterEach(() => { mockIdentityHistoryViewProjector.mockImplementation(({ messages }) => messages); }); +describe("delivery instrumentation controls", () => { + it("maps audience before delivery publication and injects only bound controls", async () => { + vi.mocked(getCompiledRuntimeAgentBundle).mockResolvedValue({ + adapterRegistry: { + adaptersByKind: new Map([[threadContextAdapter.kind, threadContextAdapter]]), + }, + compiledArtifactsSource: {} as never, + graph: { + nodesByNodeId: new Map(), + root: { sandboxRegistry: { sandbox: null }, turnAgent: TestTurnAgent }, + }, + hookRegistry: createEmptyHookRegistry(), + moduleMap: { nodes: {} }, + resolvedAgent: { config: {} }, + subagentRegistry: {}, + toolRegistry: {}, + turnAgent: TestTurnAgent, + } as never); + const resolveDecision = vi.fn(() => ({ + action: "record" as const, + recordInputs: false, + recordOutputs: true, + })); + let controlsAtDelivery: unknown; + const hooks = { + capturesContent: true, + publish: async ( + event: import("#harness/instrumentation/lifecycle.js").InstrumentationEvent, + ) => { + if (event.type === "channel.delivery.started") { + controlsAtDelivery = loadContext().get(InstrumentationControlsKey); + } + }, + }; + registerInstrumentationRuntime( + createInstrumentationRuntime({ + createHooks: () => hooks, + forceFlush: async () => undefined, + otelSettings: undefined, + resolveDecision, + runInContext: (_operation, execute) => execute(), + runWithTracingSuppressed: (execute) => execute(), + shutdown: async () => undefined, + }), + undefined, + ); + const session = createStubSession(); + installSessionStoreMocks([session]); + vi.mocked(createExecutionNodeStep).mockReturnValue(async (stepSession) => ({ + next: { done: true, output: "ok" }, + session: stepSession, + })); + const serializedContext = createSerializedContext(); + serializedContext[ChannelInstrumentationKey.name] = { + kind: "channel:test", + metadata: { audience: "private" }, + }; + + const result = await turnStep({ + input: { + deliveryMetadata: [ + { + channelKind: "channel:test", + channelName: "test", + deliveryId: "delivery-1", + payloadIndex: 0, + }, + ], + kind: "deliver", + payloads: [{ message: "secret" }], + }, + parentWritable: createTestWritable(), + serializedContext, + sessionState: createStubSessionState(), + }); + + expect(resolveDecision).toHaveBeenCalledWith( + expect.objectContaining({ audience: "private", sessionId: session.sessionId }), + ); + expect(controlsAtDelivery).toEqual({ + action: "record", + recordInputs: false, + recordOutputs: true, + }); + expect(result.serializedContext[InstrumentationControlsKey.name]).toEqual(controlsAtDelivery); + expect( + vi.mocked(createExecutionNodeStep).mock.calls[0]?.[0].instrumentation, + ).not.toHaveProperty("resolveDecision"); + expect(vi.mocked(createExecutionNodeStep).mock.calls[0]?.[0].instrumentationChannel).toEqual({ + kind: "channel:test", + metadata: {}, + }); + }); +}); + describe("routeProxiedDeliverStep", () => { it("forwards descendant input responses as session send commands", async () => { const auth = { diff --git a/packages/eve/src/execution/workflow-steps.ts b/packages/eve/src/execution/workflow-steps.ts index ef05c9d37b..16f575432b 100644 --- a/packages/eve/src/execution/workflow-steps.ts +++ b/packages/eve/src/execution/workflow-steps.ts @@ -1,75 +1,10 @@ -import { buildAdapterContext } from "#channel/adapter-context.js"; -import { callAdapterEventHandler, defaultDeliverResult } from "#channel/adapter.js"; -import type { DeliverHookPayload } from "#channel/types.js"; -import { contextStorage } from "#context/container.js"; -import { dispatchStreamEventHooks } from "#context/hook-lifecycle.js"; -import { - dispatchDynamicInstructionEvent, - drainDynamicInstructionUserMessages, - prepareDynamicInstructionPreamble, -} from "#context/dynamic-instruction-lifecycle.js"; -import { dispatchDynamicModelEvent } from "#context/dynamic-model-lifecycle.js"; -import { preserveSerializedSessionDynamicModelSelection } from "#context/serialized-dynamic-model-selection.js"; -import { dispatchDynamicSkillEvent } from "#context/dynamic-skill-lifecycle.js"; -import { - dispatchDynamicSubagentEvent, - refreshDynamicSessionSubagentsForRuntimeRevision, -} from "#context/dynamic-subagent-lifecycle.js"; -import { - dispatchDynamicToolEvent, - refreshDynamicSessionToolsForRuntimeRevision, -} from "#context/dynamic-tool-lifecycle.js"; -import { - AuthKey, - CapabilitiesKey, - HandleEventKey, - ModeKey, - SessionDynamicSubagentRuntimeRevisionKey, - SessionDynamicToolRuntimeRevisionKey, - TasksEnabledKey, - TurnTaskDeliveryKey, -} from "#context/keys.js"; +import { AuthKey, TasksEnabledKey, TurnTaskDeliveryKey } from "#context/keys.js"; import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.js"; -import { deserializeContext, serializeContext } from "#context/serialize.js"; -import { - emitTurnPreamble, - getHarnessEmissionState, - isHarnessBetweenTurns, - setHarnessEmissionState, -} from "#harness/emission.js"; -import { - channelDeliveryErrorCode, - instrumentChannelDelivery, -} from "#harness/channel-delivery-instrumentation.js"; +import { deserializeContext } from "#context/serialize.js"; +import { getHarnessEmissionState } from "#harness/emission.js"; import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; -import { preserveSerializedInstrumentationState } from "#harness/instrumentation/state.js"; -import { RuntimeActionSettlementTimesKey } from "#harness/runtime-action-settlement-state.js"; -import { preserveSerializedAgentTraceState } from "#tracing/agent-trace-context-store.js"; import { matchAuthorizationCallbacks } from "#execution/authorization-callback-match.js"; -import { readTurnSleepDurationMs } from "#harness/turn-sleep.js"; -import { isTurnCancellation, throwIfTurnAborted } from "#harness/turn-cancellation.js"; -import { setChannelContext } from "#execution/channel-context.js"; -import { hasPendingInputBatch } from "#harness/input-requests.js"; -import { activeTurnId } from "#harness/active-turn-id.js"; -import { coalesceTurnInputs } from "#harness/messages.js"; -import { - getRuntimeActionKeysFromWorkflowInterrupt, - isWorkflowRuntimeActionInterrupt, -} from "#harness/workflow-runtime-action-state.js"; -import { getPendingWorkflowInterrupt } from "#harness/workflow-interrupt-state.js"; -import { getPendingRuntimeActionBatch } from "#harness/runtime-actions.js"; -import type { HarnessSession, StepInput, StepResult } from "#harness/types.js"; -import { getTurnUsageState, takeSessionUsageDelta, toUsage } from "#harness/turn-tag-state.js"; import type { DurableStepResult } from "#execution/next-driver-action.js"; -import { getRuntimeActionRequestKey } from "#runtime/actions/keys.js"; -import { - createAuthorizationCompletedEvent, - createSessionStartedEvent, - encodeMessageStreamEvent, - type UnstampedMessageStreamEvent, - stampMessageStreamEvent, - type MessageStreamEvent, -} from "#protocol/message.js"; import { CallbackBaseUrlKey, clearPendingAuthorization, @@ -77,34 +12,19 @@ import { PendingAuthorizationResultKey, } from "#harness/authorization.js"; import { resolveWorkflowCallbackBaseUrl } from "#execution/workflow-callback-url.js"; -import { forwardTaskEventToSessionCallback } from "#execution/task-event-callback.js"; -import { resolveEffectiveOutputSchema } from "#execution/effective-output-schema.js"; -import { createDurableSessionState, readDurableSession } from "#execution/durable-session-store.js"; +import { readDurableSession } from "#execution/durable-session-store.js"; import type { TurnStepInput } from "#execution/durable-session-migrations/turn-workflow.js"; -import { buildRuntimeIdentity, createExecutionNodeStep } from "#execution/node-step.js"; -import { appendTaskAgentAnnouncement } from "#execution/tasks/parent/agent-views.js"; -import { resolveTaskDeliveryContext } from "#tasks/delivery-context.js"; -import { - readRetainedBackgroundToolResult, - runBackgroundStep, -} from "#execution/tasks/parent/tool-execution.js"; -import { - isTaskOwnedSerializedContext, - TASK_UPDATE_SESSION_INSTRUCTION, -} from "#execution/tasks/child/instructions.js"; -import { prepareWorkflowPreambleTrace } from "#execution/workflow-trace-context.js"; +import { isTaskOwnedSerializedContext } from "#execution/tasks/child/instructions.js"; import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.js"; -import { recordSubagentUsageSpans } from "#execution/subagent-usage-span.js"; -import { reconcileSessionContinuationToken } from "#execution/reconcile-session-continuation-token.js"; -import { hydrateDurableSession, refreshSessionFromTurnAgent } from "#execution/session.js"; +import { hydrateDurableSession } from "#execution/session.js"; import { createExecutionHistoryView } from "#execution/history-view.js"; -import { resolveRuntimeCompiledArtifactsVersionedCacheKey } from "#runtime/cache-key.js"; -import { createWorkflowRuntime } from "#execution/workflow-runtime.js"; +import { + consumeDeliveryInstrumentationControls, + prepareDeliveryInstrumentation, +} from "#execution/instrumentation-controls.js"; +import { executePreparedTurnStep } from "#execution/execute-prepared-turn-step.js"; import { isTaskToolAvailable, TASK_UPDATE_TOOL_NAME } from "#runtime/framework-tools/tasks.js"; -const TASK_DONE_WITH_PENDING_INPUT_ERROR_MESSAGE = - "Task mode cannot complete while input requests remain pending."; - export type { TurnStepInput }; /** @@ -117,6 +37,12 @@ export async function turnStep(rawInput: TurnStepInput): Promise - instrumentChannelDelivery({ - agentName: bundle.turnAgent.id, - ctx, - delivery: rawInput.input as DeliverHookPayload, - hooks: instrumentation?.hooks, - rootSessionId: initialSession.rootSessionId ?? initialSession.sessionId, - sequence: initialEmissionState.sequence, - sessionId: initialSession.sessionId, - turnId: activeTurnId(initialEmissionState), - }), - ); - } - - const failChannelDeliveries = async (error: unknown): Promise => { - await contextStorage.run(ctx, () => - instrumentChannelDelivery({ - ctx, - error, - errorCode: channelDeliveryErrorCode(error), - hooks: instrumentation?.hooks, - includeTurn: false, - outcome: "failed", - }), - ); - await instrumentation?.forceFlush(); - }; - const adapterCtx = buildAdapterContext(adapter, ctx); - - // Run the adapter's deliver hook for each queued payload and - // coalesce the resulting StepInput values. - let resolved: StepInput | undefined; - if (input.input?.kind === "deliver") { - const results: StepInput[] = []; - try { - for (const payload of input.input.payloads) { - const result = adapter.deliver - ? await adapter.deliver(payload, adapterCtx) - : defaultDeliverResult(payload); - - if (result !== undefined && result !== null) { - results.push(result); - } - } - } catch (error) { - await failChannelDeliveries(error); - throw error; - } - resolved = results.length === 0 ? undefined : results.reduce(coalesceTurnInputs); - } else if (input.input?.kind === "runtime-action-result") { - recordSubagentUsageSpans(input.input.results); - if (input.input.acceptedAtMsByCallId !== undefined) { - ctx.set(RuntimeActionSettlementTimesKey, input.input.acceptedAtMsByCallId); - } - resolved = { runtimeActionResults: input.input.results }; - } - - if ( - resolved !== undefined && - rawInput.input?.kind === "deliver" && - rawInput.input.taskDeliveryId !== undefined - ) { - const taskContext = resolveTaskDeliveryContext({ - state: durableSession.state, - taskDeliveryId: rawInput.input.taskDeliveryId, - }); - if (taskContext !== undefined) { - ctx.set(TurnTaskDeliveryKey, taskContext.phase); - resolved = { - ...resolved, - context: [...(resolved.context ?? []), taskContext.context], - }; - } - } - - // Persist adapter-state mutations across the step boundary. - if (input.input?.kind === "deliver") { - const updatedAdapter = { ...adapter, state: { ...adapterCtx.state } }; - setChannelContext(ctx, updatedAdapter); - } - - // Adapter handled the delivery inline; re-park and skip unchanged snapshot writes. - if (input.input?.kind === "deliver" && resolved === undefined) { - await contextStorage.run(ctx, () => - instrumentChannelDelivery({ - ctx, - hooks: instrumentation?.hooks, - includeTurn: false, - outcome: "completed", - }), - ); - await instrumentation?.forceFlush(); - const rekeyed = reconcileSessionContinuationToken(ctx, initialSession); - const nextSerializedContext = serializeContext(ctx); - const nextState = - rekeyed === initialSession - ? input.sessionState - : createDurableSessionState({ session: rekeyed }); - - return { - action: "park", - ...derivePendingState(rekeyed), - serializedContext: nextSerializedContext, - sessionState: nextState, - tasksEnabled, - }; - } - - const hookRegistry = bundle.hookRegistry; - const dynamicInstructionsResolvers = bundle.resolvedAgent.dynamicInstructionsResolvers ?? []; - const dynamicSkillResolvers = bundle.resolvedAgent.dynamicSkillResolvers ?? []; - const dynamicSubagentResolvers = bundle.subagentRegistry.dynamicResolvers ?? []; - const persistentSubagentSessions = - tasksEnabled || bundle.resolvedAgent.config?.experimental?.subagentPersistentSessions === true; - const dynamicToolResolvers = bundle.resolvedAgent.dynamicToolResolvers ?? []; - const effectiveNode = { - ...bundle.graph.root, - turnAgent: effectiveAgent.turnAgent, - }; - const runtimeIdentity = buildRuntimeIdentity(effectiveNode); - try { - const deploymentId = process.env.VERCEL_DEPLOYMENT_ID?.trim(); - const dynamicRuntimeRevision = deploymentId - ? `deployment:${deploymentId}` - : await resolveRuntimeCompiledArtifactsVersionedCacheKey(bundle.compiledArtifactsSource); - const sessionStarted = initialEmissionState.sessionStarted; - - if (!sessionStarted) { - ctx.set(SessionDynamicSubagentRuntimeRevisionKey, dynamicRuntimeRevision); - ctx.set(SessionDynamicToolRuntimeRevisionKey, dynamicRuntimeRevision); - } else { - const refreshEvent = createSessionStartedEvent({ runtime: runtimeIdentity }); - await Promise.all([ - refreshDynamicSessionSubagentsForRuntimeRevision({ - ctx, - resolvers: dynamicSubagentResolvers, - event: refreshEvent, - messages: history.initial.messages, - persistentSessions: persistentSubagentSessions, - runtimeRevision: dynamicRuntimeRevision, - }), - refreshDynamicSessionToolsForRuntimeRevision({ - ctx, - resolvers: dynamicToolResolvers, - event: refreshEvent, - messages: history.initial.messages, - runtimeRevision: dynamicRuntimeRevision, - }), - ]); - } - } catch (error) { - await failChannelDeliveries(error); - throw error; - } - - const writer = input.parentWritable.getWriter(); - - // Stamp once: the persisted chunk and the hooks below must agree on the id. - const emit = async (event: UnstampedMessageStreamEvent): Promise => { - const toEmit = await callAdapterEventHandler(adapter, event, adapterCtx); - setChannelContext(ctx, { ...adapter, state: { ...adapterCtx.state } }); - const stamped = stampMessageStreamEvent(toEmit); - await writer.write(encodeMessageStreamEvent(stamped)); - return stamped; - }; - - const handleEvent = async ( - event: UnstampedMessageStreamEvent, - messages?: readonly import("ai").ModelMessage[], - ): Promise => { - // A remote task's parent owns its HITL. Forward blocking events over - // the task callback and keep them out of the child's local channel; - // otherwise two TUIs can present and answer the same request. - const forwardedToTaskParent = await forwardTaskEventToSessionCallback(ctx, event); - const emitted = forwardedToTaskParent ? stampMessageStreamEvent(event) : await emit(event); - await dispatchStreamEventHooks({ ctx, registry: hookRegistry, event: emitted }); - if (emitted.type !== "step.started") { - await dispatchDynamicModelEvent({ - ctx, - dynamicModel: effectiveAgent.turnAgent.dynamicModel, - event: emitted, - messages: messages ?? [], - scope: { - moduleMap: bundle.moduleMap, - nodeId: bundle.nodeId, - }, - }); - } - await dispatchDynamicSubagentEvent({ - ctx, - resolvers: dynamicSubagentResolvers, - event: emitted, - messages: messages ?? [], - persistentSessions: persistentSubagentSessions, - }); - await dispatchDynamicToolEvent({ + return preparedInstrumentation.scope.run(() => + executePreparedTurnStep({ + adapter, + bundle, + completedAuths, ctx, - resolvers: dynamicToolResolvers, - event: emitted, - messages: messages ?? [], - }); - await dispatchDynamicSkillEvent({ - ctx, - resolvers: dynamicSkillResolvers, - event: emitted, - messages: messages ?? [], - }); - await dispatchDynamicInstructionEvent({ - ctx, - resolvers: dynamicInstructionsResolvers, - event: emitted, - messages: messages ?? [], - }); - }; - - const mode = ctx.require(ModeKey); - - let stepResult: StepResult; - try { - // A signal already aborted at entry (cancellation during an in-line - // runtime-action wait) must settle before the park-resume stages run, - // or the pending batch would re-park and later re-dispatch. - throwIfTurnAborted(input.abortSignal); - stepResult = await runBackgroundStep(ctx, initialSession, async (enrichedSession) => { - ctx.setVirtualContext(HandleEventKey, handleEvent); - let schemaSession = resolveEffectiveOutputSchema({ - agentOutputSchema: effectiveAgent.turnAgent.outputSchema, - input: resolved, - mode, - session: enrichedSession, - }); - if (completedAuths) { - let emissionState = getHarnessEmissionState(schemaSession.state); - if (isHarnessBetweenTurns(schemaSession)) { - prepareDynamicInstructionPreamble(ctx, history.messages(schemaSession)); - let instructionMessages: readonly import("ai").ModelMessage[] = []; - const traceContext = await prepareWorkflowPreambleTrace({ - ctx, - emissionState, - runtimeIdentity, - session: schemaSession, - }); - try { - emissionState = await emitTurnPreamble( - handleEvent, - {}, - emissionState, - runtimeIdentity, - traceContext, - ); - } finally { - instructionMessages = drainDynamicInstructionUserMessages(ctx); - schemaSession = { - ...schemaSession, - history: [...schemaSession.history, ...instructionMessages], - }; - } - schemaSession = setHarnessEmissionState(schemaSession, emissionState); - } - for (const { authorization, result } of completedAuths) { - const candidateId = pendingAuth?.challenges.find( - (challenge) => challenge.attemptId === result.attemptId, - )?.candidateId; - await handleEvent( - createAuthorizationCompletedEvent({ - attemptId: result.attemptId, - authorization, - candidateId, - name: result.name, - outcome: "authorized", - sequence: emissionState.sequence, - stepIndex: emissionState.stepIndex, - turnId: emissionState.turnId, - }), - ); - } - } - - const capabilities = ctx.get(CapabilitiesKey); - - const runHarnessStep = async ( - lifecycleSession: HarnessSession, - stepInput: StepInput | undefined, - ): Promise => { - const refreshedSession = refreshSessionFromTurnAgent({ - compactionOverrides: { - thresholdPercent: effectiveAgent.thresholdPercent, - }, - session: lifecycleSession, - systemPromptAdditions: taskUpdatesEnabled ? [TASK_UPDATE_SESSION_INSTRUCTION] : undefined, - turnAgent: effectiveAgent.turnAgent, - }); - const modelSession = tasksEnabled - ? await appendTaskAgentAnnouncement(refreshedSession, history.messages(refreshedSession)) - : refreshedSession; - - const step = createExecutionNodeStep({ - abortSignal: input.abortSignal, - capabilities, - clearOnly: input.input?.kind === "clear", - compactOnly: input.input?.kind === "compact", - createRuntime: createWorkflowRuntime, - handleEvent, - historyProjector: history.projector, - historyView: history.prepare(modelSession), - mode, - modelResolutionScope: { - moduleMap: bundle.moduleMap, - nodeId: bundle.nodeId, - }, - node: effectiveNode, - workflowMaxSubagents: refreshedSession.workflowMaxSubagents, - }); - return step(modelSession, stepInput); - }; - - return runHarnessStep(schemaSession, resolved); - }); - } catch (error) { - if (!isTurnCancellation(error)) { - await failChannelDeliveries(error); - throw error; - } - writer.releaseLock(); - // Trace and instrumentation state are needed by the cancellation - // epilogue to close the operation the discarded step opened. - // The session model is also kept because `session.started` is not emitted - // again after this cancellation settles. - const interrupted = serializeContext(ctx); - const retained = readRetainedBackgroundToolResult(ctx); - return { - action: "cancelled", - ...(retained === undefined - ? {} - : { - backgroundTaskState: createDurableSessionState({ - session: retained.backgroundTaskSession, - }), - backgroundTasks: retained.backgroundTasks, - }), - serializedContext: preserveSerializedInstrumentationState( - preserveSerializedAgentTraceState( - preserveSerializedSessionDynamicModelSelection(input.serializedContext, interrupted), - interrupted, - ), - interrupted, - ), - sessionState: input.sessionState, - }; - } - - // Re-stamp if a handler called `session.continuation.rekey(...)` (eg. Slack auto-anchor). - const rekeyed = reconcileSessionContinuationToken(ctx, stepResult.session); - const nextSerializedContext = serializeContext(ctx); - stepResult = { ...stepResult, session: rekeyed }; - - const nextState = createDurableSessionState({ session: stepResult.session }); - const sleepDurationMs = readTurnSleepDurationMs(ctx); - const sleepTransition = sleepDurationMs === undefined ? {} : { sleepDurationMs }; - const backgroundTransition = - stepResult.backgroundTasks === undefined || stepResult.backgroundTaskSession === undefined - ? {} - : { - backgroundTaskState: createDurableSessionState({ - session: stepResult.backgroundTaskSession, - }), - backgroundTasks: stepResult.backgroundTasks, - }; - - if ( - stepResult.next !== null && - typeof stepResult.next === "object" && - "done" in stepResult.next - ) { - if (mode === "task" && hasPendingInputBatch(stepResult.session.state)) { - writer.releaseLock(); - throw new Error(TASK_DONE_WITH_PENDING_INPUT_ERROR_MESSAGE); - } - await writer.close(); - const sessionTotals = getTurnUsageState(stepResult.session.state)?.session; - return { - action: "done", - ...backgroundTransition, - output: stepResult.next.output, - isError: stepResult.next.isError, - ...sleepTransition, - serializedContext: nextSerializedContext, - sessionState: nextState, - usage: sessionTotals === undefined ? undefined : toUsage(sessionTotals), - usageDelta: takeSessionUsageDelta(stepResult.session).delta, - }; - } - - if (stepResult.next === null) { - writer.releaseLock(); - - const workflowInterrupt = getPendingWorkflowInterrupt(stepResult.session.state); - if ( - workflowInterrupt !== undefined && - isWorkflowRuntimeActionInterrupt(workflowInterrupt.interrupt) - ) { - return { - action: "dispatch-workflow-runtime-actions", - ...backgroundTransition, - pendingRuntimeActionKeys: getRuntimeActionKeysFromWorkflowInterrupt( - workflowInterrupt.interrupt, - ), - ...sleepTransition, - serializedContext: nextSerializedContext, - sessionState: nextState, - }; - } - - const pending = derivePendingState(stepResult.session); - - // `settledTurn` is the harness's explicit settlement verdict. Pending - // state may predate this turn, while newly created parks omit the verdict. - // `usage` carries only this turn's delta: the take marks the totals - // reported, so a persistent child never re-reports earlier spend. - if (stepResult.settledTurn !== undefined) { - const { delta, session: reportedSession } = takeSessionUsageDelta(stepResult.session); - return { - action: "park", - ...backgroundTransition, - ...pending, - ...sleepTransition, - serializedContext: nextSerializedContext, - sessionState: createDurableSessionState({ session: reportedSession }), - settled: { - output: stepResult.settledTurn.output, - isError: stepResult.settledTurn.isError, - usage: delta, - }, - tasksEnabled, - }; - } - - return { - action: "park", - ...backgroundTransition, - ...pending, - ...sleepTransition, - serializedContext: nextSerializedContext, - sessionState: nextState, + durableSession, + effectiveAgent, + history, + initialEmissionState, + initialSession, + input, + pendingAuth, + preparedInstrumentation, + rawInput, tasksEnabled, - }; - } - - writer.releaseLock(); - return { - action: "continue", - ...backgroundTransition, - ...sleepTransition, - serializedContext: nextSerializedContext, - sessionState: nextState, - }; -} - -/** - * Derives the pending-state fields the turn workflow needs to choose - * the right `NextDriverAction` arm at the park boundary. - */ -function derivePendingState(session: HarnessSession): { - readonly authorizationAttemptIds?: readonly string[]; - readonly authorizationNames?: readonly string[]; - readonly hasPendingAuthorization: boolean; - readonly hasPendingInputBatch: boolean; - readonly pendingRuntimeActionKeys?: readonly string[]; -} { - const batch = getPendingRuntimeActionBatch(session.state); - const pendingAuth = getPendingAuthorization(session.state); - const base = { - authorizationAttemptIds: pendingAuth?.challenges.flatMap((challenge) => - challenge.attemptId === undefined ? [] : [challenge.attemptId], - ), - authorizationNames: pendingAuth?.challenges.map((c) => c.name), - hasPendingAuthorization: pendingAuth !== undefined, - hasPendingInputBatch: hasPendingInputBatch(session.state), - }; - if (batch !== undefined) { - return { - ...base, - pendingRuntimeActionKeys: batch.actions.map((action) => getRuntimeActionRequestKey(action)), - }; - } - return base; + taskUpdatesEnabled, + }), + ); } diff --git a/packages/eve/src/execution/workflow-trace-context.ts b/packages/eve/src/execution/workflow-trace-context.ts index 264d644286..7474b33a9b 100644 --- a/packages/eve/src/execution/workflow-trace-context.ts +++ b/packages/eve/src/execution/workflow-trace-context.ts @@ -1,31 +1,25 @@ import type { ContextContainer } from "#context/container.js"; -import { - ChannelInstrumentationKey, - ParentSessionKey, - ParentTraceContextKey, -} from "#context/keys.js"; +import { ParentSessionKey, ParentTraceContextKey } from "#context/keys.js"; import type { HarnessEmissionState } from "#harness/emission.js"; -import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; +import type { HarnessInstrumentation } from "#harness/instrumentation/runtime.js"; import { resolveParentLineage } from "#harness/parent-lineage.js"; import { prepareTurnTraceContext } from "#harness/prepare-trace-context.js"; import type { HarnessSession } from "#harness/types.js"; import type { RuntimeIdentity, RuntimeTraceContext } from "#protocol/message.js"; import { ChannelKey } from "#runtime/sessions/runtime-context-keys.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; /** Prepares native tracing for workflow-owned preambles emitted outside the tool loop. */ export async function prepareWorkflowPreambleTrace(input: { readonly ctx: ContextContainer; readonly emissionState: HarnessEmissionState; + readonly instrumentation?: HarnessInstrumentation; readonly runtimeIdentity: RuntimeIdentity; readonly session: HarnessSession; }): Promise { const parent = input.ctx.get(ParentSessionKey); - const channel = input.ctx.get(ChannelInstrumentationKey); return await prepareTurnTraceContext({ agentName: input.runtimeIdentity.agentName, - channelAudience: normalizeChannelAudience(channel?.metadata.audience), - instrumentation: getInstrumentationRuntime(), + instrumentation: input.instrumentation, parentLineage: resolveParentLineage(parent, input.ctx.get(ChannelKey)), parentTraceContext: input.ctx.get(ParentTraceContextKey), rootSessionId: parent?.rootSessionId ?? input.session.rootSessionId ?? input.session.sessionId, diff --git a/packages/eve/src/harness/channel-delivery-instrumentation.ts b/packages/eve/src/harness/channel-delivery-instrumentation.ts index dabfe29f60..3ede9df1e6 100644 --- a/packages/eve/src/harness/channel-delivery-instrumentation.ts +++ b/packages/eve/src/harness/channel-delivery-instrumentation.ts @@ -2,7 +2,6 @@ import type { DeliverHookPayload, DeliverPayload } from "#channel/types.js"; import type { AlsContext } from "#context/container.js"; import { ActiveChannelDeliveriesKey, - ChannelInstrumentationKey, ParentTraceContextKey, type ActiveChannelDelivery, } from "#context/keys.js"; @@ -12,7 +11,6 @@ import type { InstrumentationHooks, } from "#harness/instrumentation/lifecycle.js"; import { channelDeliveryIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; interface ChannelDeliveryStartInstrumentation { readonly agentName?: string; @@ -70,13 +68,9 @@ export async function instrumentChannelDelivery( if (input.hooks === undefined || input.delivery.deliveryMetadata === undefined) return; const active: ActiveChannelDelivery[] = []; - const channelAudience = normalizeChannelAudience( - input.ctx.get(ChannelInstrumentationKey)?.metadata.audience, - ); for (const metadata of input.delivery.deliveryMetadata) { const payload = input.delivery.payloads[metadata.payloadIndex]; const delivery = { - channelAudience, channelKind: metadata.channelKind, channelName: metadata.channelName, deliveryId: metadata.deliveryId, diff --git a/packages/eve/src/harness/instrumentation/config.test.ts b/packages/eve/src/harness/instrumentation/config.test.ts index f8afd599e1..07d1a190c9 100644 --- a/packages/eve/src/harness/instrumentation/config.test.ts +++ b/packages/eve/src/harness/instrumentation/config.test.ts @@ -71,7 +71,8 @@ describe("instrumentation-config chunk-isolation regression", () => { it("installs harness telemetry settings on the instrumentation runtime", async () => { const { registerInstrumentationConfig } = await import("#harness/instrumentation/config.js"); - const { getInstrumentationRuntime } = await import("#harness/instrumentation/runtime.js"); + const { constructInstrumentation, getInstrumentationRuntime } = + await import("#harness/instrumentation/runtime.js"); await registerInstrumentationConfig( { @@ -83,7 +84,15 @@ describe("instrumentation-config chunk-isolation regression", () => { { agentName: "test-agent" }, ); - expect(getInstrumentationRuntime()?.otelSettings).toEqual({ + const runtime = getInstrumentationRuntime()!; + expect(runtime.traceChannelRequests).toBe(true); + expect( + constructInstrumentation(runtime, { + action: "record", + recordInputs: true, + recordOutputs: false, + }).harness?.otelSettings, + ).toMatchObject({ functionId: "weather", recordInputs: true, recordOutputs: false, @@ -93,11 +102,20 @@ describe("instrumentation-config chunk-isolation regression", () => { it("disables input and output recording by default", async () => { const { registerInstrumentationConfig } = await import("#harness/instrumentation/config.js"); - const { getInstrumentationRuntime } = await import("#harness/instrumentation/runtime.js"); + const { constructInstrumentation, getInstrumentationRuntime } = + await import("#harness/instrumentation/runtime.js"); await registerInstrumentationConfig({}, { agentName: "test-agent" }); - expect(getInstrumentationRuntime()?.otelSettings).toEqual({ + const runtime = getInstrumentationRuntime()!; + expect(runtime.traceChannelRequests).toBe(false); + expect( + constructInstrumentation(runtime, { + action: "record", + recordInputs: false, + recordOutputs: false, + }).harness?.otelSettings, + ).toMatchObject({ functionId: undefined, recordInputs: false, recordOutputs: false, diff --git a/packages/eve/src/harness/instrumentation/config.ts b/packages/eve/src/harness/instrumentation/config.ts index c8bf7d8d84..200a8210ce 100644 --- a/packages/eve/src/harness/instrumentation/config.ts +++ b/packages/eve/src/harness/instrumentation/config.ts @@ -1,5 +1,8 @@ import { createInstrumentationHooks } from "#harness/instrumentation/lifecycle.js"; -import { registerInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; +import { + createInstrumentationRuntime, + registerInstrumentationRuntime, +} from "#harness/instrumentation/runtime.js"; import { createInstrumentationSetupContext } from "#harness/instrumentation/setup-context.js"; import type { InstrumentationDefinition } from "#public/instrumentation/index.js"; @@ -46,18 +49,30 @@ export async function registerInstrumentationConfig( globalContainer[INSTRUMENTATION_CONFIG_GLOBAL_KEY] = config; // This legacy layout leaves `registerOTel` to `setup`, so install only the // runtime projection consumed by the harness. - registerInstrumentationRuntime({ - forceFlush: async () => undefined, - hooks: createInstrumentationHooks([]), - otelSettings: { - functionId: config.functionId, - recordInputs: config.recordInputs === true, - recordOutputs: config.recordOutputs === true, - traceChannelRequests: config.traceChannelRequests === true, - }, - runInContext: (_operation, execute) => execute(), - shutdown: async () => undefined, - }); + const hooks = createInstrumentationHooks([]); + const otelSettings = { + functionId: config.functionId, + recordInputs: config.recordInputs === true, + recordOutputs: config.recordOutputs === true, + traceChannelRequests: config.traceChannelRequests === true, + }; + registerInstrumentationRuntime( + createInstrumentationRuntime({ + authoredConfig: config, + createHooks: () => hooks, + forceFlush: async () => undefined, + otelSettings, + resolveDecision: () => ({ + action: "record", + recordInputs: config.recordInputs === true, + recordOutputs: config.recordOutputs === true, + }), + runInContext: (_operation, execute) => execute(), + runWithTracingSuppressed: (execute) => execute(), + shutdown: async () => undefined, + }), + otelSettings, + ); await config.setup?.(createInstrumentationSetupContext(input.agentName)); } diff --git a/packages/eve/src/harness/instrumentation/content.ts b/packages/eve/src/harness/instrumentation/content.ts index 89c7138363..616b3f98d6 100644 --- a/packages/eve/src/harness/instrumentation/content.ts +++ b/packages/eve/src/harness/instrumentation/content.ts @@ -1,31 +1,56 @@ import type { InstrumentationEvent } from "#harness/instrumentation/lifecycle.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; /** Returns an immutable event projection with conversation content removed. */ export function withoutInstrumentationContent(event: InstrumentationEvent): InstrumentationEvent { + return withInstrumentationControls(event, { + action: "record", + recordInputs: false, + recordOutputs: false, + }); +} + +/** Applies one delivery's content ceiling before an event reaches any provider. */ +export function withInstrumentationControls( + event: InstrumentationEvent, + controls: InstrumentationControls, +): InstrumentationEvent { switch (event.type) { case "channel.delivery.started": - return Object.freeze({ ...event, input: undefined }); + return controls.recordInputs ? event : Object.freeze({ ...event, input: undefined }); case "action.started": - return Object.freeze({ ...event, input: undefined }); + return controls.recordInputs ? event : Object.freeze({ ...event, input: undefined }); case "action.completed": - return Object.freeze({ ...event, output: Object.freeze({ type: event.output.type }) }); + return controls.recordOutputs + ? event + : Object.freeze({ ...event, output: Object.freeze({ type: event.output.type }) }); case "input.requested": - return Object.freeze({ ...event, request: undefined }); - case "input.resolved": - return Object.freeze({ ...event, error: undefined, response: undefined }); + return controls.recordOutputs ? event : Object.freeze({ ...event, request: undefined }); + case "input.resolved": { + if (controls.recordInputs && controls.recordOutputs) return event; + return Object.freeze({ + ...event, + error: controls.recordOutputs ? event.error : undefined, + response: controls.recordInputs ? event.response : undefined, + }); + } case "tool.call.started": - return Object.freeze({ ...event, input: undefined }); + return controls.recordInputs ? event : Object.freeze({ ...event, input: undefined }); case "tool.call.completed": - return Object.freeze({ ...event, output: Object.freeze({ type: event.output.type }) }); + return controls.recordOutputs + ? event + : Object.freeze({ ...event, output: Object.freeze({ type: event.output.type }) }); case "model.call.started": - return Object.freeze({ ...event, input: undefined }); + return controls.recordInputs ? event : Object.freeze({ ...event, input: undefined }); case "model.call.completed": - return Object.freeze({ ...event, content: undefined }); + return controls.recordOutputs ? event : Object.freeze({ ...event, content: undefined }); case "step.attempt.metadata": - return Object.freeze({ - ...event, - providerMetadata: structuralProviderMetadata(event.providerMetadata), - }); + return controls.recordOutputs + ? event + : Object.freeze({ + ...event, + providerMetadata: structuralProviderMetadata(event.providerMetadata), + }); case "action.failed": case "model.call.failed": case "session.failed": @@ -33,7 +58,7 @@ export function withoutInstrumentationContent(event: InstrumentationEvent): Inst case "tool.call.failed": case "turn.failed": case "channel.delivery.failed": - return Object.freeze({ ...event, error: undefined }); + return controls.recordOutputs ? event : Object.freeze({ ...event, error: undefined }); default: return event; } diff --git a/packages/eve/src/harness/instrumentation/lifecycle.ts b/packages/eve/src/harness/instrumentation/lifecycle.ts index 9598608c0f..239e64a5a2 100644 --- a/packages/eve/src/harness/instrumentation/lifecycle.ts +++ b/packages/eve/src/harness/instrumentation/lifecycle.ts @@ -1,7 +1,6 @@ import { createInstrumentationDispatcher } from "#harness/instrumentation/dispatch.js"; import type { InstrumentationStateSlot } from "#harness/instrumentation/state.js"; import type { RuntimeTraceContext } from "#protocol/message.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; /** * Stable eve identity for one actual model attempt. @@ -13,7 +12,6 @@ import type { ChannelAudience } from "#shared/channel-audience.js"; * fire once per attempt. */ export interface InstrumentationAttemptScope { - readonly channelAudience?: ChannelAudience; readonly attemptId: string; readonly attemptIndex: number; readonly functionId?: string; @@ -162,7 +160,6 @@ export function channelDeliveryIdempotencyKey(sessionId: string, deliveryId: str } export interface InstrumentationChannelDeliveryRef { - readonly channelAudience?: ChannelAudience; readonly channelKind: string; readonly channelName: string; readonly deliveryId: string; @@ -284,7 +281,6 @@ export interface InstrumentationSessionStartedEvent { readonly type: "session.started"; readonly agentName?: string; readonly channelKind?: string; - readonly channelAudience?: ChannelAudience; readonly idempotencyKey: string; readonly parentTraceContext?: InstrumentationTraceContext; readonly rootSessionId: string; diff --git a/packages/eve/src/harness/instrumentation/native-events.ts b/packages/eve/src/harness/instrumentation/native-events.ts index 0987c371f1..5dfc62bad1 100644 --- a/packages/eve/src/harness/instrumentation/native-events.ts +++ b/packages/eve/src/harness/instrumentation/native-events.ts @@ -29,12 +29,10 @@ import type { ResolvedInputBatch } from "#harness/input-requests.js"; import { RuntimeActionSettlementTimesKey } from "#harness/runtime-action-settlement-state.js"; import type { HandleEventFn } from "#harness/types.js"; import type { RuntimeActionRequest, RuntimeActionResult } from "#runtime/actions/types.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; export interface CreateInstrumentationHandleEventInput { readonly agentName?: string; readonly channelKind?: string; - readonly channelAudience?: ChannelAudience; readonly getAttemptScope?: () => InstrumentationAttemptScope | undefined; readonly handleEvent?: HandleEventFn; readonly hooks?: InstrumentationHooks; @@ -294,7 +292,6 @@ function toLifecycleEvent( case "session.started": return { agentName: input.agentName, - channelAudience: input.channelAudience, channelKind: input.channelKind, idempotencyKey: sessionIdempotencyKey(input.sessionId), parentTraceContext: input.parentTraceContext, diff --git a/packages/eve/src/harness/instrumentation/providers.integration.test.ts b/packages/eve/src/harness/instrumentation/providers.integration.test.ts index fc2741f0d9..afdb3d7cb8 100644 --- a/packages/eve/src/harness/instrumentation/providers.integration.test.ts +++ b/packages/eve/src/harness/instrumentation/providers.integration.test.ts @@ -1,6 +1,7 @@ import { beforeEach, describe, expect, it } from "vitest"; import { turnIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; +import { constructInstrumentation } from "#harness/instrumentation/runtime.js"; import { finalizeInstrumentationProviders, registerInstrumentationProvider, @@ -68,7 +69,12 @@ describe("authored instrumentation provider dispatch", () => { expect(order).toEqual(["first:setup", "first:setup-complete", "second:setup"]); - const runtime = finalizeInstrumentationProviders({ serviceName: "weather" }); + const installedRuntime = finalizeInstrumentationProviders({ serviceName: "weather" }); + const runtime = constructInstrumentation(installedRuntime, { + action: "record", + recordInputs: true, + recordOutputs: true, + }).harness!; const publication = runtime.hooks.publish({ idempotencyKey: turnIdempotencyKey("session-1", "turn-1"), rootSessionId: "session-1", diff --git a/packages/eve/src/harness/instrumentation/providers.test.ts b/packages/eve/src/harness/instrumentation/providers.test.ts index 80a4964fb6..1c60821eae 100644 --- a/packages/eve/src/harness/instrumentation/providers.test.ts +++ b/packages/eve/src/harness/instrumentation/providers.test.ts @@ -1,6 +1,7 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; import { turnIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; +import { constructInstrumentation } from "#harness/instrumentation/runtime.js"; import { EVE_EVALUATION_ENV_FLAG, EVE_EVALUATION_RUN_ID_ENV, @@ -207,7 +208,10 @@ describe("finalizeInstrumentationProviders", () => { const started = vi.fn(); await register("rows", defineInstrumentation({ events: { "turn.started": started } })); - const runtime = finalizeInstrumentationProviders({ serviceName: "weather-agent" }); + const runtime = constructInstrumentation( + finalizeInstrumentationProviders({ serviceName: "weather-agent" }), + { action: "record", recordInputs: true, recordOutputs: true }, + ).harness!; await runtime.hooks.publish(turnStarted); expect(started).toHaveBeenCalledOnce(); @@ -220,7 +224,10 @@ describe("finalizeInstrumentationProviders", () => { // going missing. await register("rows", defineInstrumentation({})); - const runtime = finalizeInstrumentationProviders({ serviceName: "weather-agent" }); + const runtime = constructInstrumentation( + finalizeInstrumentationProviders({ serviceName: "weather-agent" }), + { action: "record", recordInputs: true, recordOutputs: true }, + ).harness!; const result = await runtime.runInContext( { idempotencyKey: "tool:session-1:turn-1:0:0:call-1:0", diff --git a/packages/eve/src/harness/instrumentation/runtime-context.test.ts b/packages/eve/src/harness/instrumentation/runtime-context.test.ts index ed48f80db6..ccb9462aff 100644 --- a/packages/eve/src/harness/instrumentation/runtime-context.test.ts +++ b/packages/eve/src/harness/instrumentation/runtime-context.test.ts @@ -2,7 +2,7 @@ import type { ModelMessage } from "ai"; import { describe, expect, it, vi } from "vitest"; import { ContextContainer, contextStorage } from "#context/container.js"; -import { AuthKey, ChannelInstrumentationKey } from "#context/keys.js"; +import { AuthKey } from "#context/keys.js"; import type { HarnessEmissionState } from "#harness/emission.js"; import { buildTelemetryRuntimeContext, @@ -160,33 +160,26 @@ describe("buildTelemetryRuntimeContext", () => { }); it("reflects the active channel kind and exposes channel metadata to the resolver", () => { - const ctx = new ContextContainer(); - ctx.set(ChannelInstrumentationKey, { - kind: "channel:support", - metadata: { triggeringUserId: "U999" }, - }); - - const runtimeContext = contextStorage.run(ctx, () => - build({ - authored: { - events: { - "step.started": ( - input: InstrumentationStepStartedEventInput, - ): InstrumentationStepStartedEventResult => - input.channel.kind === "channel:support" - ? { - runtimeContext: { - "slack.user_id": - typeof input.channel.metadata["triggeringUserId"] === "string" - ? input.channel.metadata["triggeringUserId"] - : "", - }, - } - : { runtimeContext: {} }, - }, + const runtimeContext = build({ + channel: { kind: "channel:support", metadata: { triggeringUserId: "U999" } }, + authored: { + events: { + "step.started": ( + input: InstrumentationStepStartedEventInput, + ): InstrumentationStepStartedEventResult => + input.channel.kind === "channel:support" + ? { + runtimeContext: { + "slack.user_id": + typeof input.channel.metadata["triggeringUserId"] === "string" + ? input.channel.metadata["triggeringUserId"] + : "", + }, + } + : { runtimeContext: {} }, }, - }), - ); + }, + }); expect(runtimeContext).toMatchObject({ "eve.channel.kind": "channel:support", @@ -198,10 +191,6 @@ describe("buildTelemetryRuntimeContext", () => { const roles = ["admin"]; const channelMetadata = { nested: { value: "original" }, triggeringUserId: "U999" }; const ctx = new ContextContainer(); - ctx.set(ChannelInstrumentationKey, { - kind: "channel:support", - metadata: channelMetadata, - }); ctx.set(AuthKey, { attributes: { roles }, authenticator: "jwt", @@ -212,6 +201,7 @@ describe("buildTelemetryRuntimeContext", () => { let captured: InstrumentationStepStartedEventInput | undefined; contextStorage.run(ctx, () => build({ + channel: { kind: "channel:support", metadata: channelMetadata }, authored: { events: { "step.started": (input: InstrumentationStepStartedEventInput) => { diff --git a/packages/eve/src/harness/instrumentation/runtime-context.ts b/packages/eve/src/harness/instrumentation/runtime-context.ts index 64887d6510..a2678d0641 100644 --- a/packages/eve/src/harness/instrumentation/runtime-context.ts +++ b/packages/eve/src/harness/instrumentation/runtime-context.ts @@ -3,12 +3,7 @@ import type { ModelMessage, SystemModelMessage } from "ai"; import type { SessionAuthContext } from "#channel/types.js"; import type { AlsContext } from "#context/container.js"; import { contextStorage } from "#context/container.js"; -import { - AuthKey, - ChannelInstrumentationKey, - InitiatorAuthKey, - ParentSessionKey, -} from "#context/keys.js"; +import { AuthKey, InitiatorAuthKey, ParentSessionKey } from "#context/keys.js"; import type { HarnessEmissionState } from "#harness/emission.js"; import type { HarnessSession } from "#harness/types.js"; import type { RuntimeContextResolver } from "#tracing/otel-declaration.js"; @@ -28,6 +23,10 @@ import { parseJsonObject, parseJsonValue, type JsonObject, type JsonValue } from const log = createLogger("harness.instrumentation-runtime-context"); export interface BuildTelemetryRuntimeContextInput { + readonly channel?: { + readonly kind?: string; + readonly metadata: Readonly>; + }; readonly eveVersion: string; readonly authored: InstrumentationDefinition | undefined; readonly emissionState: HarnessEmissionState; @@ -59,13 +58,10 @@ export function buildTelemetryRuntimeContext( const authoredRuntimeContext = resolveStepStartedRuntimeContext(input); const providerRuntimeContext = resolveProviderRuntimeContext(input); - const context = contextStorage.getStore(); - const projection = context?.get(ChannelInstrumentationKey); - return { ...authoredRuntimeContext, ...providerRuntimeContext, - "eve.channel.kind": normalizeInstrumentationChannelKind(projection?.kind), + "eve.channel.kind": normalizeInstrumentationChannelKind(input.channel?.kind), "eve.environment": input.environment, "eve.session.id": input.session.sessionId, "eve.step.index": String(input.emissionState.stepIndex), @@ -79,12 +75,11 @@ function buildInstrumentationStepStartedInput( input: Omit, ): InstrumentationStepStartedEventInput { const context = contextStorage.getStore(); - const projection = context?.get(ChannelInstrumentationKey); return { channel: { - kind: normalizeInstrumentationChannelKind(projection?.kind), - metadata: snapshotForInstrumentation(projection?.metadata, "channel.metadata") ?? {}, + kind: normalizeInstrumentationChannelKind(input.channel?.kind), + metadata: snapshotForInstrumentation(input.channel?.metadata, "channel.metadata") ?? {}, } as InstrumentationChannel, modelInput: snapshotForInstrumentation(input.modelInput, "modelInput") ?? { instructions: undefined, diff --git a/packages/eve/src/harness/instrumentation/runtime.test.ts b/packages/eve/src/harness/instrumentation/runtime.test.ts new file mode 100644 index 0000000000..a293e0b18e --- /dev/null +++ b/packages/eve/src/harness/instrumentation/runtime.test.ts @@ -0,0 +1,83 @@ +import { describe, expect, it, vi } from "vitest"; + +import type { InstrumentationRuntime } from "#harness/instrumentation/runtime.js"; +import { + constructInstrumentation, + createInstrumentationRuntime, +} from "#harness/instrumentation/runtime.js"; + +function runtime(publish = vi.fn()): InstrumentationRuntime { + const hooks = { capturesContent: true, publish }; + return createInstrumentationRuntime({ + createHooks: () => hooks, + forceFlush: async () => undefined, + otelSettings: { + recordInputs: true, + recordOutputs: true, + traceChannelRequests: false, + }, + resolveDecision: () => ({ action: "drop", recordInputs: false, recordOutputs: false }), + runInContext: (_operation, execute) => execute(), + runWithTracingSuppressed: (execute) => execute(), + shutdown: async () => undefined, + }); +} + +describe("constructInstrumentation", () => { + it("applies directional content controls before publishing", async () => { + const publish = vi.fn(); + const constructed = constructInstrumentation(runtime(publish), { + action: "record", + recordInputs: false, + recordOutputs: true, + }); + + await constructed.harness!.hooks.publish({ + idempotencyKey: "model-1", + input: { instructions: "secret", messages: [] }, + model: { modelId: "test", provider: "test" }, + scope: { + attemptId: "attempt-1", + attemptIndex: 0, + sessionId: "session-1", + stepIndex: 0, + turnId: "turn-1", + }, + type: "model.call.started", + }); + await constructed.harness!.hooks.publish({ + content: [{ text: "visible", type: "text" }], + finishReason: "stop", + idempotencyKey: "model-1", + scope: { + attemptId: "attempt-1", + attemptIndex: 0, + sessionId: "session-1", + stepIndex: 0, + turnId: "turn-1", + }, + type: "model.call.completed", + usage: {}, + }); + + expect(publish.mock.calls[0]?.[0]).toHaveProperty("input", undefined); + expect(publish.mock.calls[1]?.[0]).toHaveProperty("content"); + expect(constructed.harness?.otelSettings).toMatchObject({ + recordInputs: false, + recordOutputs: true, + }); + }); + + it("suppresses OTel while retaining metadata-only provider events", () => { + const constructed = constructInstrumentation(runtime(), { + action: "drop", + recordInputs: false, + recordOutputs: false, + }); + + expect(constructed.harness?.hooks.capturesContent).toBe(false); + expect(constructed.harness?.otelSettings).toBeUndefined(); + expect(constructed.harness?.prepareSessionTrace).toBeUndefined(); + expect(constructed.harness?.prepareTurnTrace).toBeUndefined(); + }); +}); diff --git a/packages/eve/src/harness/instrumentation/runtime.ts b/packages/eve/src/harness/instrumentation/runtime.ts index 188ca6aa80..382a4e0510 100644 --- a/packages/eve/src/harness/instrumentation/runtime.ts +++ b/packages/eve/src/harness/instrumentation/runtime.ts @@ -5,32 +5,158 @@ import type { InstrumentationTraceContext, InstrumentationTurnStartedEvent, } from "#harness/instrumentation/lifecycle.js"; -import type { OtelHarnessSettings, RuntimeContextResolver } from "#tracing/otel-declaration.js"; +import { withInstrumentationControls } from "#harness/instrumentation/content.js"; +import type { InstrumentationControls } from "#shared/instrumentation-controls.js"; +import type { + OtelHarnessSettings, + OtelRuntimeSettings, + RuntimeContextResolver, + TraceCaptureContext, +} from "#tracing/otel-declaration.js"; +import type { InstrumentationDefinition } from "#public/instrumentation/index.js"; +import { getRegisteredTelemetryIntegrations } from "#harness/ai-sdk-telemetry.js"; const INSTRUMENTATION_RUNTIME_KEY = Symbol.for("eve.instrumentation-runtime"); +const UPDATE_OTEL_SETTINGS = Symbol.for("eve.instrumentation-runtime.update-otel-settings"); /** Process-wide runtime consumed by every harness execution surface. */ export interface InstrumentationRuntime { + readonly [UPDATE_OTEL_SETTINGS]: (settings: OtelRuntimeSettings | undefined) => void; + readonly construct: (controls: InstrumentationControls) => ConstructedInstrumentation; readonly forceFlush: () => Promise; - readonly hooks: InstrumentationHooks; - readonly prepareSessionTrace?: ( - event: InstrumentationSessionStartedEvent, - ) => Promise; - readonly prepareTurnTrace?: ( - event: InstrumentationTurnStartedEvent, - ) => Promise; - otelSettings: OtelHarnessSettings | undefined; - /** Provider `runtimeContext` resolvers, collected at install time. */ + readonly resolveDecision: (context: TraceCaptureContext) => InstrumentationControls; + readonly shutdown: () => Promise; + readonly traceChannelRequests: boolean; +} + +type PrepareSessionTrace = ( + event: InstrumentationSessionStartedEvent, +) => Promise; +type PrepareTurnTrace = ( + event: InstrumentationTurnStartedEvent, +) => Promise; +type RunWithTracingSuppressed = (execute: () => PromiseLike) => PromiseLike; + +interface InstrumentationConstructionInput { + readonly authoredConfig?: InstrumentationDefinition; + readonly createHooks: (controls: InstrumentationControls) => InstrumentationHooks; + readonly forceFlush: InstrumentationRuntime["forceFlush"]; + readonly prepareSessionTrace?: PrepareSessionTrace; + readonly prepareTurnTrace?: PrepareTurnTrace; + readonly resolveDecision: InstrumentationRuntime["resolveDecision"]; readonly runtimeContextResolvers?: readonly RuntimeContextResolver[]; readonly runInContext: InstrumentationContextRunner; - readonly shutdown: () => Promise; + readonly runWithTracingSuppressed: RunWithTracingSuppressed; + readonly shutdown: InstrumentationRuntime["shutdown"]; + readonly otelSettings: OtelRuntimeSettings | undefined; } /** Instrumentation capabilities consumed inside one harness execution. */ -export type HarnessInstrumentation = Pick< - InstrumentationRuntime, - "hooks" | "prepareSessionTrace" | "prepareTurnTrace" | "runInContext" ->; +export interface HarnessInstrumentation { + readonly authoredConfig?: InstrumentationDefinition; + readonly forceFlush?: InstrumentationRuntime["forceFlush"]; + readonly hooks?: InstrumentationHooks; + readonly otelSettings?: OtelHarnessSettings; + readonly prepareSessionTrace?: PrepareSessionTrace; + readonly prepareTurnTrace?: PrepareTurnTrace; + readonly runtimeContextResolvers?: readonly RuntimeContextResolver[]; + readonly runInContext?: InstrumentationContextRunner; + readonly telemetryIntegrations?: readonly Telemetry[]; +} + +type BoundHarnessInstrumentation = HarnessInstrumentation & { + readonly forceFlush: NonNullable; + readonly hooks: NonNullable; + readonly runInContext: NonNullable; +}; + +export interface ConstructedInstrumentation { + readonly harness?: BoundHarnessInstrumentation; + run(execute: () => PromiseLike): PromiseLike; +} + +/** Constructs one decision-bound instrumentation capability. */ +export function constructInstrumentation( + runtime: InstrumentationRuntime, + controls: InstrumentationControls, +): ConstructedInstrumentation { + return runtime.construct(controls); +} + +export function createInstrumentationRuntime( + input: InstrumentationConstructionInput, +): InstrumentationRuntime { + let runtimeSettings = input.otelSettings; + return { + [UPDATE_OTEL_SETTINGS]: (settings) => { + runtimeSettings = settings; + }, + construct: (controls) => constructFromComponents(input, runtimeSettings, controls), + forceFlush: input.forceFlush, + resolveDecision: input.resolveDecision, + shutdown: input.shutdown, + get traceChannelRequests() { + return runtimeSettings?.traceChannelRequests === true; + }, + }; +} + +function constructFromComponents( + input: InstrumentationConstructionInput, + runtimeSettings: OtelRuntimeSettings | undefined, + controls: InstrumentationControls, +): ConstructedInstrumentation { + const runtimeHooks = input.createHooks(controls); + const hooks: InstrumentationHooks = { + capturesContent: + runtimeHooks.capturesContent && (controls.recordInputs || controls.recordOutputs), + publish: async (event) => { + const publish = () => runtimeHooks.publish(withInstrumentationControls(event, controls)); + await (controls.action === "drop" ? input.runWithTracingSuppressed(publish) : publish()); + }, + }; + const otelSettings = (() => { + if (runtimeSettings === undefined || controls.action === "drop") return undefined; + const { tracePolicy: _tracePolicy, ...settings } = runtimeSettings; + return { + ...settings, + recordInputs: + controls.action === "record" && runtimeSettings.recordInputs === true + ? controls.recordInputs + : false, + recordOutputs: + controls.action === "record" && runtimeSettings.recordOutputs === true + ? controls.recordOutputs + : false, + }; + })(); + + const run = + controls.action === "drop" + ? input.runWithTracingSuppressed + : (execute: () => PromiseLike): PromiseLike => execute(); + return { + harness: { + authoredConfig: input.authoredConfig, + forceFlush: input.forceFlush, + hooks, + otelSettings, + prepareSessionTrace: controls.action === "record" ? input.prepareSessionTrace : undefined, + prepareTurnTrace: controls.action === "record" ? input.prepareTurnTrace : undefined, + runtimeContextResolvers: + controls.action === "record" ? input.runtimeContextResolvers : undefined, + runInContext: + controls.action === "record" + ? input.runInContext + : (_operation, execute) => input.runWithTracingSuppressed(execute), + telemetryIntegrations: + controls.action === "record" && controls.recordInputs && controls.recordOutputs + ? getRegisteredTelemetryIntegrations() + : [], + }, + run, + }; +} type InstrumentationGlobal = typeof globalThis & { [INSTRUMENTATION_RUNTIME_KEY]?: InstrumentationRuntime; @@ -41,11 +167,12 @@ const globalRuntime = globalThis as InstrumentationGlobal; /** Registers the process instrumentation runtime before agent execution begins. */ export function registerInstrumentationRuntime( runtime: InstrumentationRuntime, + otelSettings: OtelRuntimeSettings | undefined, ): InstrumentationRuntime { const existing = globalRuntime[INSTRUMENTATION_RUNTIME_KEY]; if (existing !== undefined) { // A legacy config may reload without taking ownership from the installed runtime. - existing.otelSettings = runtime.otelSettings; + existing[UPDATE_OTEL_SETTINGS](otelSettings); return existing; } globalRuntime[INSTRUMENTATION_RUNTIME_KEY] = runtime; @@ -56,3 +183,4 @@ export function registerInstrumentationRuntime( export function getInstrumentationRuntime(): InstrumentationRuntime | undefined { return globalRuntime[INSTRUMENTATION_RUNTIME_KEY]; } +import type { Telemetry } from "ai"; diff --git a/packages/eve/src/harness/prepare-trace-context.ts b/packages/eve/src/harness/prepare-trace-context.ts index d8c1d67358..92873c1e07 100644 --- a/packages/eve/src/harness/prepare-trace-context.ts +++ b/packages/eve/src/harness/prepare-trace-context.ts @@ -6,14 +6,12 @@ import type { } from "#harness/instrumentation/lifecycle.js"; import { sessionIdempotencyKey, turnIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; import type { HarnessInstrumentation } from "#harness/instrumentation/runtime.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; const log = createLogger("harness.prepare-trace-context"); /** Prepares native session/turn tracing before their durable stream events. */ export async function prepareTurnTraceContext(input: { readonly agentName?: string; - readonly channelAudience?: ChannelAudience; readonly instrumentation?: HarnessInstrumentation; readonly parentLineage?: InstrumentationParentLineage; readonly parentTraceContext?: InstrumentationTraceContext; @@ -30,7 +28,6 @@ export async function prepareTurnTraceContext(input: { try { prepared = await input.instrumentation.prepareSessionTrace({ agentName: input.agentName, - channelAudience: input.channelAudience, idempotencyKey: sessionIdempotencyKey(input.sessionId), parentTraceContext: input.parentTraceContext, rootSessionId: input.rootSessionId, diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.ts index 0e171a9c09..0ef2871be4 100644 --- a/packages/eve/src/harness/tool-loop.test.ts +++ b/packages/eve/src/harness/tool-loop.test.ts @@ -5,6 +5,7 @@ import { type LanguageModelCallEndEvent, type LanguageModel, type ModelMessage, + type Telemetry, ToolLoopAgent, type UserContent, } from "ai"; @@ -16,7 +17,6 @@ import { DynamicModelSelectionError } from "#context/dynamic-model-lifecycle.js" import { dispatchDynamicInstructionEvent } from "#context/dynamic-instruction-lifecycle.js"; import { AuthKey, - ChannelInstrumentationKey, InitiatorAuthKey, LiveStepDynamicModelSelectionKey, ParentSessionKey, @@ -39,6 +39,7 @@ import type { ResolvedDynamicInstructionsResolver } from "#runtime/types.js"; import type { DynamicResolveContext } from "#shared/dynamic-tool-definition.js"; import { registerDurableDynamicCallback } from "#shared/durable-dynamic-tool-callbacks.js"; import type { RunMode } from "#shared/run-mode.js"; +import type { OtelHarnessSettings } from "#tracing/otel-declaration.js"; import { compactMessages, shouldCompact } from "#harness/compaction.js"; import { getHarnessEmissionState, isHarnessBetweenTurns } from "#harness/emission.js"; import { @@ -97,7 +98,7 @@ const { registeredOtelIntegration, } = vi.hoisted(() => ({ mockCreateAiSdkHookBridge: vi.fn((..._args: unknown[]) => ({ onStart: vi.fn() })), - mockGetRegisteredTelemetryIntegrations: vi.fn((): unknown[] => []), + mockGetRegisteredTelemetryIntegrations: vi.fn((): Telemetry[] => []), registeredAuthorIntegration: { onStart: vi.fn() }, registeredOtelIntegration: { onStart: vi.fn() }, })); @@ -125,8 +126,17 @@ vi.mock("./instrumentation/runtime.js", () => ({ * Registering an authored config writes both stores, so the tests toggle * telemetry through one call rather than keeping two mocks in step by hand. */ +let declaredOtelSettings: OtelHarnessSettings | undefined; + function declareTelemetry(config: Readonly> | undefined): void { mockGetInstrumentationConfig.mockReturnValue(config); + declaredOtelSettings = + config === undefined + ? undefined + : { + ...config, + traceChannelRequests: config["traceChannelRequests"] === true, + }; mockGetInstrumentationRuntime.mockReturnValue( config === undefined ? undefined @@ -182,6 +192,24 @@ function createTestConfig( emit?: HarnessEmitFn, overrides?: Partial, ): ToolLoopHarnessConfig { + const instrumentation = + overrides?.instrumentation === undefined + ? declaredOtelSettings === undefined + ? undefined + : { + authoredConfig: mockGetInstrumentationConfig() as never, + otelSettings: declaredOtelSettings, + telemetryIntegrations: mockGetRegisteredTelemetryIntegrations(), + } + : { + ...overrides.instrumentation, + authoredConfig: + overrides.instrumentation.authoredConfig ?? (mockGetInstrumentationConfig() as never), + otelSettings: overrides.instrumentation.otelSettings ?? declaredOtelSettings, + telemetryIntegrations: + overrides.instrumentation.telemetryIntegrations ?? + mockGetRegisteredTelemetryIntegrations(), + }; return { handleEvent: emit, mode, @@ -198,6 +226,7 @@ function createTestConfig( ], ]), ...overrides, + instrumentation, }; } @@ -4894,6 +4923,7 @@ describe("createToolLoopHarness", () => { }); const config: ToolLoopHarnessConfig = { instrumentation: { + authoredConfig: mockGetInstrumentationConfig() as never, hooks: createInstrumentationHooks([]), runInContext: (_operation, execute) => execute(), }, @@ -10588,6 +10618,23 @@ describe("createToolLoopHarness", () => { }); describe("telemetry metadata", () => { + it("disables registered integrations for dropped compaction", async () => { + vi.mocked(compactMessages).mockResolvedValue([{ content: "summary", role: "assistant" }]); + mockGetRegisteredTelemetryIntegrations.mockReturnValue([registeredAuthorIntegration]); + const runStep = createToolLoopHarness( + createTestConfig("conversation", undefined, { + compactOnly: true, + instrumentation: { + telemetryIntegrations: [], + }, + }), + ); + + await runStep(createTestSession({ history: [{ content: "private", role: "user" }] })); + + expect(vi.mocked(compactMessages).mock.calls[0]?.[4]).toMatchObject({ integrations: [] }); + }); + it("emits the authored turn trace with the session and turn preamble", async () => { const authoredTrace = { spanId: "0123456789abcdef", @@ -10652,7 +10699,7 @@ describe("createToolLoopHarness", () => { expect(runtimeContext?.["eve.version"]).not.toBe(""); expect(runtimeContext?.["eve.session.id"]).toBe("test-session"); expect(agentCall?.telemetry?.isEnabled).toBe(true); - expect(agentCall?.telemetry?.integrations).toBeUndefined(); + expect(agentCall?.telemetry?.integrations).toEqual([]); }); it("injects one provider-neutral bridge when lifecycle hooks opt in", async () => { @@ -10709,6 +10756,45 @@ describe("createToolLoopHarness", () => { ); }); + it("suppresses tracing and registered integrations for a dropped model call", async () => { + setupMockAgent({ + finishReason: "stop", + response: { messages: [{ content: "Hello!", role: "assistant" }] }, + text: "Hello!", + toolCalls: [], + toolResults: [], + }); + mockGetRegisteredTelemetryIntegrations.mockReturnValue([registeredAuthorIntegration]); + const hooks = createInstrumentationHooks([ + { capture: "content", events: {}, name: "content" }, + ]); + const runStep = createToolLoopHarness( + createTestConfig("conversation", undefined, { + instrumentation: { + hooks, + runInContext: (_operation, execute) => execute(), + telemetryIntegrations: [], + }, + }), + ); + + await runStep(createTestSession(), { message: "private" }); + + const bridge = mockCreateAiSdkHookBridge.mock.results[0]!.value; + const agentCall = vi.mocked(ToolLoopAgent).mock.calls[0]?.[0] as { + telemetry?: { + integrations?: unknown[]; + recordInputs?: boolean; + recordOutputs?: boolean; + }; + }; + expect(agentCall.telemetry).toMatchObject({ + integrations: [bridge], + recordInputs: false, + recordOutputs: false, + }); + }); + it("publishes a delegation action when the AI SDK skips execution callbacks", async () => { setupMockAgent({ finishReason: "tool-calls", @@ -10763,7 +10849,7 @@ describe("createToolLoopHarness", () => { ); }); - it("composes the bridge with every registered integration", async () => { + it("withholds partial-content calls from unfilterable registered integrations", async () => { setupMockAgent({ finishReason: "stop", response: { messages: [{ content: "Hello!", role: "assistant" }] }, @@ -10783,6 +10869,7 @@ describe("createToolLoopHarness", () => { instrumentation: { hooks, runInContext: (_operation, execute) => execute(), + telemetryIntegrations: [], }, }), ); @@ -10798,7 +10885,7 @@ describe("createToolLoopHarness", () => { }; }; expect(agentCall.telemetry).toMatchObject({ - integrations: [bridge, registeredOtelIntegration, registeredAuthorIntegration], + integrations: [bridge], recordInputs: true, recordOutputs: false, }); @@ -10842,16 +10929,13 @@ describe("createToolLoopHarness", () => { }); const ctx = new ContextContainer(); - ctx.set(ChannelInstrumentationKey, { - kind: "channel:support", - metadata: { - triggeringUserId: "U123", - }, - }); - const hidden = { content: "HIDE_FROM_INSTRUMENTATION", role: "user" as const }; const config = createTestConfig("conversation", emit, { historyProjector: ({ messages }) => messages.filter((message) => message !== hidden), + instrumentationChannel: { + kind: "channel:support", + metadata: { triggeringUserId: "U123" }, + }, }); const runStep = createToolLoopHarness(config); await contextStorage.run(ctx, () => diff --git a/packages/eve/src/harness/tool-loop.ts b/packages/eve/src/harness/tool-loop.ts index 5a1e77d5d4..7938fd0a7b 100644 --- a/packages/eve/src/harness/tool-loop.ts +++ b/packages/eve/src/harness/tool-loop.ts @@ -33,7 +33,6 @@ import { formatLanguageModelGatewayId } from "#internal/runtime-model.js"; import { contextStorage } from "#context/container.js"; import { AuthKey, - ChannelInstrumentationKey, ParentSessionKey, ParentTraceContextKey, SessionCallbackKey, @@ -55,7 +54,6 @@ import { import { buildDynamicSubagentTools } from "#context/dynamic-subagent-lifecycle.js"; import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.js"; import { toErrorMessage } from "#shared/errors.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; import { createActionResultEvent, createApprovalCandidateEvent, @@ -169,8 +167,6 @@ import { convertStaleResponsesToUserMessage, dropStaleSessionLimitContinuationResponses, } from "#harness/stale-input-responses.js"; -import { getInstrumentationConfig } from "#harness/instrumentation/config.js"; -import { getInstrumentationRuntime } from "#harness/instrumentation/runtime.js"; import type { OtelHarnessSettings } from "#tracing/otel-declaration.js"; import { normalizeModelMessages, @@ -213,10 +209,7 @@ import { TASK_DELIVERY_SETTLED_INSTRUCTION, } from "#tasks/delivery-context.js"; import { extractWorkflowStreamWriteErrorDetails } from "#harness/workflow-stream-error.js"; -import { - ensureOtelIntegration, - getRegisteredTelemetryIntegrations, -} from "#harness/ai-sdk-telemetry.js"; +import { ensureOtelIntegration } from "#harness/ai-sdk-telemetry.js"; import { getAdvertisedTools } from "#harness/advertised-tools.js"; import { createBackgroundToolCallBatch } from "#harness/background-tools.js"; import { @@ -323,8 +316,13 @@ function enrichTelemetry( agentName: string | undefined, runtimeContext?: Readonly>, bridgeIntegration?: Telemetry, + telemetryIntegrations?: readonly Telemetry[], ): TelemetryOptions | undefined { - if (settings === undefined && bridgeIntegration === undefined) { + if ( + settings === undefined && + bridgeIntegration === undefined && + telemetryIntegrations === undefined + ) { return undefined; } @@ -334,16 +332,20 @@ function enrichTelemetry( for (const key of Object.keys(runtimeContext ?? {})) { includeRuntimeContext[key] = true; } + const integrations = + bridgeIntegration === undefined && telemetryIntegrations === undefined + ? undefined + : [ + ...(bridgeIntegration === undefined ? [] : [bridgeIntegration]), + ...(telemetryIntegrations ?? []), + ]; return { functionId: settings?.functionId ?? agentName, includeRuntimeContext, // Passing integrations replaces the registered ones for this call, so the // bridge has to be composed with them rather than handed over on its own. - integrations: - bridgeIntegration === undefined - ? undefined - : [bridgeIntegration, ...getRegisteredTelemetryIntegrations()], + integrations, isEnabled: true, recordInputs: settings?.recordInputs ?? false, recordOutputs: settings?.recordOutputs ?? false, @@ -575,13 +577,12 @@ function buildHarnessToolsWithDynamicSubagents( export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { const baseEmit = config.handleEvent; - const instrumentationRuntime = getInstrumentationRuntime(); - const otelSettings = instrumentationRuntime?.otelSettings; + const otelSettings = config.instrumentation?.otelSettings; // The legacy single-file layout reads its runtime-context resolver from the // authored config object; the provider directory collects resolvers at install // time onto the runtime. Both paths feed buildTelemetryRuntimeContext. - const authoredConfig = getInstrumentationConfig(); - const providerRuntimeContextResolvers = instrumentationRuntime?.runtimeContextResolvers; + const authoredConfig = config.instrumentation?.authoredConfig; + const providerRuntimeContextResolvers = config.instrumentation?.runtimeContextResolvers; if (otelSettings !== undefined) { ensureOtelIntegration(); } @@ -615,14 +616,17 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { const parentContext = resolveStepOtelContext(tracer, turnSpan, initialSession); const executeStep = () => executeStepBody(initialSession, input, turnSpan); - try { - if (parentContext) { - return await otelContext.with(parentContext, executeStep); + const execute = async (): Promise => { + try { + if (parentContext) { + return await otelContext.with(parentContext, executeStep); + } + return await executeStep(); + } finally { + turnSpan?.end(); } - return await executeStep(); - } finally { - turnSpan?.end(); - } + }; + return await execute(); } async function executeStepBody( @@ -649,7 +653,6 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { let emissionState = getHarnessEmissionState(session.state); const store = contextStorage.getStore(); - const channelInstrumentation = store?.get(ChannelInstrumentationKey); const parent = store?.get(ParentSessionKey); const channel = store?.get(ChannelKey); const callback = store?.get(SessionCallbackKey); @@ -663,8 +666,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { let activeAttemptScope: InstrumentationAttemptScope | undefined; const emit = createInstrumentationHandleEvent({ agentName: config.runtimeIdentity?.agentName, - channelAudience: normalizeChannelAudience(channelInstrumentation?.metadata.audience), - channelKind: channelInstrumentation?.kind, + channelKind: config.instrumentationChannel?.kind, getAttemptScope: () => activeAttemptScope, handleEvent: baseEmit, hooks: config.instrumentation?.hooks, @@ -721,7 +723,6 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { : undefined; return await prepareTurnTraceContext({ agentName: config.runtimeIdentity?.agentName, - channelAudience: normalizeChannelAudience(channelInstrumentation?.metadata.audience), instrumentation: config.instrumentation, parentLineage, parentTraceContext, @@ -776,7 +777,14 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { resolveModel: config.resolveModel, runtimeIdentity: config.runtimeIdentity, session, - telemetry: enrichTelemetry(otelSettings, agentName) ?? undefined, + telemetry: + enrichTelemetry( + otelSettings, + agentName, + undefined, + undefined, + config.instrumentation?.telemetryIntegrations, + ) ?? undefined, }); session = { @@ -1270,7 +1278,14 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { resolveModel: config.resolveModel, runtimeIdentity: config.runtimeIdentity, session, - telemetry: enrichTelemetry(otelSettings, agentName) ?? undefined, + telemetry: + enrichTelemetry( + otelSettings, + agentName, + undefined, + undefined, + config.instrumentation?.telemetryIntegrations, + ) ?? undefined, }); session = compaction.session; if (compaction.compacted) { @@ -1371,13 +1386,14 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { return { instructions, telemetryRuntimeContext: buildTelemetryRuntimeContext({ + channel: config.instrumentationChannel, eveVersion, authored: authoredConfig, emissionState, environment, modelInput: { - instructions, - messages: modelMessages, + instructions: otelSettings?.recordInputs === false ? undefined : instructions, + messages: otelSettings?.recordInputs === false ? [] : modelMessages, }, providerResolvers: providerRuntimeContextResolvers, session, @@ -1501,7 +1517,6 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { : { attemptId: `${session.sessionId}:${instrumentationTurnId}:${emissionState.stepIndex}:${opts.attemptIndex}`, attemptIndex: opts.attemptIndex, - channelAudience: normalizeChannelAudience(channelInstrumentation?.metadata.audience), functionId: otelSettings?.functionId ?? agentName, rootSessionId: parent?.rootSessionId ?? session.sessionId, sessionId: session.sessionId, @@ -1570,6 +1585,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn { agentName, telemetryRuntimeContext, bridgeIntegration, + config.instrumentation?.telemetryIntegrations, ), toolApproval: buildToolApproval(modelTools), toolChoice: hasPendingApprovalBatch(session) ? ("none" as const) : undefined, diff --git a/packages/eve/src/harness/types.ts b/packages/eve/src/harness/types.ts index 437ae624a5..77ea4b6a9d 100644 --- a/packages/eve/src/harness/types.ts +++ b/packages/eve/src/harness/types.ts @@ -306,6 +306,11 @@ export interface ToolLoopHarnessConfig { readonly historyProjector?: HistoryViewProjector; /** Execution-prepared view of the history supplied to the first harness step. */ readonly historyView?: PreparedHistoryView; + /** Execution-sanitized channel projection exposed to instrumentation callbacks. */ + readonly instrumentationChannel?: { + readonly kind?: string; + readonly metadata: Readonly>; + }; /** * Internal lifecycle hooks injected into each actual model attempt. * Omitted in production until an instrumentation runtime opts in. diff --git a/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts b/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts index 3f9b0431ca..04addc5d78 100644 --- a/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts +++ b/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts @@ -69,7 +69,7 @@ export async function traceChannelRequest( input: TraceChannelRequestInput, handler: (span: Span | undefined) => Promise, ): Promise { - if (getInstrumentationRuntime()?.otelSettings?.traceChannelRequests !== true) { + if (getInstrumentationRuntime()?.traceChannelRequests !== true) { return await handler(undefined); } diff --git a/packages/eve/src/public/channels/index.ts b/packages/eve/src/public/channels/index.ts index b25a89b02b..6fe9d52294 100644 --- a/packages/eve/src/public/channels/index.ts +++ b/packages/eve/src/public/channels/index.ts @@ -50,14 +50,11 @@ export { import { getChannelInstrumentationKind } from "#channel/compiled-channel.js"; import type { Channel, InferChannelMetadata } from "#public/definitions/channel.js"; -import type { ChannelAudienceMetadata } from "#shared/channel-audience.js"; /** * Base channel metadata shape used by framework channel kinds. */ -export type InstrumentationChannelMetadata = Readonly< - Record & ChannelAudienceMetadata ->; +export type InstrumentationChannelMetadata = Readonly>; /** * Kind discriminator exposed to instrumentation and dynamic resolvers. @@ -90,7 +87,7 @@ export type InstrumentationChannelForChannel, "metadata" > & { - readonly metadata: InferChannelMetadata; + readonly metadata: Omit, "audience">; }; /** diff --git a/packages/eve/src/public/instrumentation/otel.ts b/packages/eve/src/public/instrumentation/otel.ts index 15c43e72bc..11d5cf85b2 100644 --- a/packages/eve/src/public/instrumentation/otel.ts +++ b/packages/eve/src/public/instrumentation/otel.ts @@ -37,6 +37,7 @@ export { type SpanExportPolicy, type SpanExportPredicate, type TraceCaptureContext, + type TraceCaptureDecision, type TraceCapturePolicy, } from "#tracing/otel-declaration.js"; diff --git a/packages/eve/src/runtime/framework-tools/subagent/local.ts b/packages/eve/src/runtime/framework-tools/subagent/local.ts index 607bb1613d..f768319f6b 100644 --- a/packages/eve/src/runtime/framework-tools/subagent/local.ts +++ b/packages/eve/src/runtime/framework-tools/subagent/local.ts @@ -225,6 +225,7 @@ async function dispatchSubagent(input: SubagentDispatchInput): Promise>, +): Readonly> { + if (!("audience" in metadata)) return metadata; + const { audience: _audience, ...visible } = metadata; + return visible; +} diff --git a/packages/eve/src/shared/instrumentation-controls.ts b/packages/eve/src/shared/instrumentation-controls.ts new file mode 100644 index 0000000000..dc12f0e5ff --- /dev/null +++ b/packages/eve/src/shared/instrumentation-controls.ts @@ -0,0 +1,30 @@ +export type InstrumentationControls = + | { + readonly action: "drop"; + readonly recordInputs: false; + readonly recordOutputs: false; + } + | { + readonly action: "record"; + readonly recordInputs: boolean; + readonly recordOutputs: boolean; + }; + +export const DROP_INSTRUMENTATION: InstrumentationControls = { + action: "drop", + recordInputs: false, + recordOutputs: false, +}; + +/** Intersects two decisions so inherited controls can only become more restrictive. */ +export function intersectInstrumentationControls( + inherited: InstrumentationControls, + current: InstrumentationControls, +): InstrumentationControls { + if (inherited.action === "drop" || current.action === "drop") return DROP_INSTRUMENTATION; + return { + action: "record", + recordInputs: inherited.recordInputs && current.recordInputs, + recordOutputs: inherited.recordOutputs && current.recordOutputs, + }; +} diff --git a/packages/eve/src/tracing/agent-action-instrumentation.ts b/packages/eve/src/tracing/agent-action-instrumentation.ts index 79574e2410..8963e9e40f 100644 --- a/packages/eve/src/tracing/agent-action-instrumentation.ts +++ b/packages/eve/src/tracing/agent-action-instrumentation.ts @@ -16,12 +16,11 @@ import type { } from "#harness/instrumentation/lifecycle.js"; import { actionIdempotencyKey, attemptIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; import { contentAttribute } from "#tracing/agent-otel-content.js"; +import { vercelSessionIdAttribute } from "#tracing/agent-otel-attributes.js"; import { setAgentUsage } from "#tracing/agent-otel-usage.js"; import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js"; import type { AgentActionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; import { isSampledTrace } from "#tracing/sampled-trace.js"; -import { withChannelAudience } from "#tracing/channel-audience-context.js"; export interface AgentActionInstrumentation { readonly events: Pick< @@ -67,7 +66,6 @@ export function createAgentActionInstrumentation(input: { const state: AgentActionTraceState = existing ?? { attemptIndex: event.scope.attemptIndex, callId: event.callId, - channelAudience: normalizeChannelAudience(event.scope.channelAudience), inputAttribute: input.recordInputs ? contentAttribute(event.input, false) : undefined, kind: event.kind, name: event.name, @@ -133,9 +131,7 @@ export function createAgentActionInstrumentation(input: { "agent.step.attempt": state.attemptIndex, "agent.step.index": state.stepIndex, "agent.turn.id": state.turnId, - ...(emitVercelSessionId - ? { "vercel.session_id": state.rootSessionId } - : {}), + ...vercelSessionIdAttribute(emitVercelSessionId, state.rootSessionId), }, startTime: state.startTimeMs, }, @@ -194,19 +190,13 @@ function actionContext(state: AgentActionTraceState): AgentActionContext { traceId: state.parent.traceId, }; return { - context: withChannelAudience( - trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext(spanContext)), - state.channelAudience, - ), + context: trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext(spanContext)), spanContext, }; } function contextFromActionState(state: AgentActionTraceState): Context { - return withChannelAudience( - trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext({ ...state.parent, isRemote: false })), - state.channelAudience, - ); + return trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext({ ...state.parent, isRemote: false })); } function recordError(span: Span, error: unknown): void { diff --git a/packages/eve/src/tracing/agent-approval-instrumentation.ts b/packages/eve/src/tracing/agent-approval-instrumentation.ts index 2150d09c6a..4c243f22cf 100644 --- a/packages/eve/src/tracing/agent-approval-instrumentation.ts +++ b/packages/eve/src/tracing/agent-approval-instrumentation.ts @@ -15,15 +15,14 @@ import type { } from "#harness/instrumentation/lifecycle.js"; import type { JsonValue } from "#shared/json.js"; import { contentAttribute } from "#tracing/agent-otel-content.js"; +import { vercelSessionIdAttribute } from "#tracing/agent-otel-attributes.js"; import type { AgentActionContext } from "#tracing/agent-action-instrumentation.js"; import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js"; -import { normalizeChannelAudience, type ChannelAudience } from "#shared/channel-audience.js"; interface AgentApprovalSpanState { readonly actionCallId: string; readonly actionName: string; readonly attemptIndex: number; - readonly channelAudience: ChannelAudience; readonly parent: SpanContext; readonly requestAttribute?: string; readonly requestId: string; @@ -67,7 +66,6 @@ export function createAgentApprovalInstrumentation(input: { actionCallId: event.action.callId, actionName: event.action.name, attemptIndex: event.scope.attemptIndex, - channelAudience: normalizeChannelAudience(event.scope.channelAudience), parent: { spanId: parent.spanContext.spanId, traceFlags: parent.spanContext.traceFlags, @@ -112,9 +110,7 @@ export function createAgentApprovalInstrumentation(input: { "agent.step.attempt": state.attemptIndex, "agent.step.index": state.stepIndex, "agent.turn.id": state.turnId, - ...(emitVercelSessionId - ? { "vercel.session_id": state.rootSessionId } - : {}), + ...vercelSessionIdAttribute(emitVercelSessionId, state.rootSessionId), }, startTime: state.startTimeMs, }, @@ -166,7 +162,6 @@ function readState(value: unknown): AgentApprovalSpanState | undefined { actionCallId: state["actionCallId"], actionName: state["actionName"], attemptIndex: state["attemptIndex"], - channelAudience: normalizeChannelAudience(state["channelAudience"]), parent: { isRemote: false, spanId: parentRecord["spanId"], diff --git a/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts b/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts index d388a6a896..4ab5a15249 100644 --- a/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts +++ b/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts @@ -17,14 +17,13 @@ import type { } from "#harness/instrumentation/lifecycle.js"; import { sessionIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; import type { JsonValue } from "#shared/json.js"; -import { normalizeChannelAudience, type ChannelAudience } from "#shared/channel-audience.js"; import { contentAttribute } from "#tracing/agent-otel-content.js"; +import { vercelSessionIdAttribute } from "#tracing/agent-otel-attributes.js"; import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js"; import type { AgentSessionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js"; import { isSampledTrace } from "#tracing/sampled-trace.js"; interface ChannelDeliverySpanState { - readonly channelAudience: ChannelAudience; readonly inputAttribute?: string; readonly parent: SpanContext; readonly requestTraceContext?: SpanContext; @@ -58,7 +57,6 @@ export function createAgentChannelDeliveryInstrumentation(input: { ): Promise => { const session = await input.ensureSessionContext({ agentName: event.agentName, - channelAudience: event.delivery.channelAudience, channelKind: event.delivery.channelKind, idempotencyKey: sessionIdempotencyKey(event.sessionId), parentTraceContext: event.parentTraceContext, @@ -69,7 +67,6 @@ export function createAgentChannelDeliveryInstrumentation(input: { if (!isSampledTrace(session.context)) return; const inputAttribute = input.recordInputs ? contentAttribute(event.input, false) : undefined; const state: Record = { - channelAudience: normalizeChannelAudience(event.delivery.channelAudience), parent: { isRemote: session.context.isRemote ?? false, spanId: session.context.spanId, @@ -132,9 +129,7 @@ export function createAgentChannelDeliveryInstrumentation(input: { "agent.session.window": session?.window ?? state.window, "agent.turn.id": event.turnId, "agent.turn.sequence": event.sequence, - ...(emitVercelSessionId - ? { "vercel.session_id": event.rootSessionId } - : {}), + ...vercelSessionIdAttribute(emitVercelSessionId, event.rootSessionId), }, kind: SpanKind.CONSUMER, links: @@ -189,7 +184,6 @@ function readState(value: unknown): ChannelDeliverySpanState | undefined { ? value.requestTraceContext : undefined; return { - channelAudience: normalizeChannelAudience(value.channelAudience), inputAttribute: typeof value.inputAttribute === "string" ? value.inputAttribute : undefined, parent: value.parent, requestTraceContext, diff --git a/packages/eve/src/tracing/agent-otel-attributes.ts b/packages/eve/src/tracing/agent-otel-attributes.ts new file mode 100644 index 0000000000..0f24fd7fc8 --- /dev/null +++ b/packages/eve/src/tracing/agent-otel-attributes.ts @@ -0,0 +1,6 @@ +export function vercelSessionIdAttribute( + enabled: boolean, + rootSessionId: string, +): { readonly "vercel.session_id"?: string } { + return enabled ? { "vercel.session_id": rootSessionId } : {}; +} diff --git a/packages/eve/src/tracing/agent-otel-provider.test.ts b/packages/eve/src/tracing/agent-otel-provider.test.ts index da481a2e7d..5b18040b47 100644 --- a/packages/eve/src/tracing/agent-otel-provider.test.ts +++ b/packages/eve/src/tracing/agent-otel-provider.test.ts @@ -36,8 +36,6 @@ import { type InstrumentationTraceContext, type InstrumentationUsage, } from "#harness/instrumentation/lifecycle.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; -import type { TraceCapturePolicy } from "#tracing/otel-declaration.js"; import { actionIdempotencyKey, attemptIdempotencyKey, @@ -61,7 +59,6 @@ interface TestRuntime { function createRuntime( stateStore: AgentTraceStateStore = new InMemoryAgentTraceStateStore(), - tracePolicy: TraceCapturePolicy | null = () => true, options: { readonly emitVercelSessionId?: boolean } = {}, ): TestRuntime { const exporter = new InMemorySpanExporter(); @@ -71,9 +68,7 @@ function createRuntime( spanProcessors: [new SimpleSpanProcessor(exporter)], }); const tracer = provider.getTracer("eve.agent"); - const agentOtelInput: Omit & { - tracePolicy?: TraceCapturePolicy; - } = { + const agentOtelInput: AgentOtelInstrumentationInput = { emitVercelSessionId: options.emitVercelSessionId, frameworkVersion: "test", idGenerator, @@ -82,7 +77,6 @@ function createRuntime( stateStore, tracer, }; - if (tracePolicy !== null) agentOtelInput.tracePolicy = tracePolicy; const agentOtel = createAgentOtelInstrumentation(agentOtelInput); const hooks = createInstrumentationHooks([agentOtel.hook]); return { @@ -100,7 +94,6 @@ async function emitAttempt(input: { readonly actionUsage?: InstrumentationUsage; readonly attemptIndex?: number; readonly attemptError?: Error; - readonly channelAudience?: ChannelAudience; readonly hooks: InstrumentationHooks; readonly parentTraceContext?: InstrumentationTraceContext; readonly runInContext: InstrumentationContextRunner; @@ -118,7 +111,6 @@ async function emitAttempt(input: { const scope: InstrumentationAttemptScope = { attemptId: `${input.sessionId}:${input.turnId}:0:${input.attemptIndex ?? 0}`, attemptIndex: input.attemptIndex ?? 0, - channelAudience: input.channelAudience, functionId: "weather", sessionId: input.sessionId, stepIndex: 0, @@ -288,7 +280,6 @@ async function emitAttempt(input: { } async function publishTurnStarted(input: { - readonly channelAudience?: ChannelAudience; readonly hooks: InstrumentationHooks; readonly parentLineage?: InstrumentationParentLineage; readonly parentTraceContext?: InstrumentationTraceContext; @@ -300,7 +291,6 @@ async function publishTurnStarted(input: { const rootSessionId = input.rootSessionId ?? input.sessionId; await input.hooks.publish({ agentName: "weather", - channelAudience: input.channelAudience, channelKind: "http", idempotencyKey: sessionIdempotencyKey(input.sessionId), parentTraceContext: input.parentTraceContext, @@ -648,7 +638,6 @@ describe("createAgentOtelInstrumentation", () => { const contextWith = vi.spyOn(context, "with"); await context.with(activeContext, () => emitAttempt({ - channelAudience: "public", hooks: runtime.hooks, runInContext: runtime.runInContext, sessionId: "session-1", @@ -674,7 +663,6 @@ describe("createAgentOtelInstrumentation", () => { expect(session.parentSpanContext).toBeUndefined(); expect(session.events.map((event) => event.name)).toEqual(["session.started"]); expect(session.attributes).toMatchObject({ - "agent.channel.audience": "public", "agent.session.id": "session-1", "agent.session.window": 0, "agent.trace.schema.version": 1, @@ -731,62 +719,6 @@ describe("createAgentOtelInstrumentation", () => { }); }); - it.each(["private", "unknown"] as const)( - "does not record %s conversation traces by default", - async (audience) => { - const runtime = createRuntime(new InMemoryAgentTraceStateStore(), null); - - await emitAttempt({ - channelAudience: audience, - hooks: runtime.hooks, - runInContext: runtime.runInContext, - sessionId: `session-${audience}`, - turnId: `turn-${audience}`, - turnSequence: 0, - }); - await runtime.provider.forceFlush(); - - expect(runtime.exporter.getFinishedSpans()).toEqual([]); - }, - ); - - it("applies the private audience gate to an adopted sampled trace", async () => { - const runtime = createRuntime(new InMemoryAgentTraceStateStore(), null); - - await emitAttempt({ - channelAudience: "private", - hooks: runtime.hooks, - parentTraceContext: { - spanId: "a".repeat(16), - traceFlags: 1, - traceId: "b".repeat(32), - }, - runInContext: runtime.runInContext, - sessionId: "session-private", - turnId: "turn-private", - turnSequence: 0, - }); - await runtime.provider.forceFlush(); - - expect(runtime.exporter.getFinishedSpans()).toEqual([]); - }); - - it("allows private tracing when the trace policy opts in", async () => { - const runtime = createRuntime(new InMemoryAgentTraceStateStore(), () => true); - - await emitAttempt({ - channelAudience: "private", - hooks: runtime.hooks, - runInContext: runtime.runInContext, - sessionId: "session-private", - turnId: "turn-private", - turnSequence: 0, - }); - await runtime.provider.forceFlush(); - - expect(byName(runtime.exporter.getFinishedSpans(), "agent.session")).toHaveLength(1); - }); - it("writes merged runtime context onto the step, operation, and chat spans", async () => { const runtime = createRuntime(); @@ -1852,7 +1784,7 @@ describe("createAgentOtelInstrumentation", () => { describe("emitVercelSessionId", () => { it("emits vercel.session_id on session, turn, step, and action spans when enabled", async () => { - const runtime = createRuntime(undefined, undefined, { emitVercelSessionId: true }); + const runtime = createRuntime(undefined, { emitVercelSessionId: true }); await emitAttempt({ hooks: runtime.hooks, runInContext: runtime.runInContext, diff --git a/packages/eve/src/tracing/agent-otel-provider.ts b/packages/eve/src/tracing/agent-otel-provider.ts index f706bb7d4a..e56f35461c 100644 --- a/packages/eve/src/tracing/agent-otel-provider.ts +++ b/packages/eve/src/tracing/agent-otel-provider.ts @@ -26,10 +26,9 @@ import { createAgentApprovalInstrumentation } from "#tracing/agent-approval-inst import { createAgentChannelDeliveryInstrumentation } from "#tracing/agent-channel-delivery-instrumentation.js"; import { createAgentToolInstrumentation } from "#tracing/agent-tool-instrumentation.js"; import { setAgentUsage } from "#tracing/agent-otel-usage.js"; +import { vercelSessionIdAttribute } from "#tracing/agent-otel-attributes.js"; import { createAgentOtelSessionContext } from "#tracing/agent-otel-session-context.js"; -import type { TraceCapturePolicy } from "#tracing/otel-declaration.js"; import { isSampledTrace } from "#tracing/sampled-trace.js"; -import { withChannelAudience } from "#tracing/channel-audience-context.js"; import { suppressTracing } from "#tracing/suppress-tracing.js"; import type { InstrumentationStepAttemptMetadataEvent, @@ -72,7 +71,6 @@ export interface AgentOtelInstrumentationInput { readonly idGenerator: AgentSpanIdGenerator; readonly stateStore: AgentTraceStateStore; readonly tracer: Tracer; - readonly tracePolicy?: TraceCapturePolicy; /** * When true, every agent span also carries `vercel.session_id` set to the * root session id. The VDP trace ingestion materialized view extracts this @@ -159,10 +157,7 @@ export function createAgentOtelInstrumentation( const onStepStarted = async (event: InstrumentationStepAttemptStartedEvent): Promise => { const turn = await input.stateStore.getTurn(event.scope.sessionId, event.scope.turnId); if (turn === undefined || !isSampledTrace(turn.context)) return; - const turnContext = withChannelAudience( - contextFromSpanContext(turn.context), - event.scope.channelAudience, - ); + const turnContext = contextFromSpanContext(turn.context); const activeSpanContext = trace.getSpan(context.active())?.spanContext(); const stepSpan = input.idGenerator.withSpanId( input.idGenerator.deriveSpanId(attemptIdempotencyKey(event.scope)), @@ -288,9 +283,7 @@ export function createAgentOtelInstrumentation( "agent.session.window": session?.window, "agent.turn.id": event.turnId, "agent.turn.sequence": turn.sequence, - ...(emitVercelSessionId - ? { "vercel.session_id": turn.rootSessionId } - : {}), + ...vercelSessionIdAttribute(emitVercelSessionId, turn.rootSessionId), }, startTime: turn.startTimeMs, }, @@ -499,10 +492,7 @@ export function createAgentOtelInstrumentation( operation.scope.turnId, ); if (turn !== undefined) { - parent = withChannelAudience( - contextFromSpanContext(turn.context), - operation.scope.channelAudience, - ); + parent = contextFromSpanContext(turn.context); if (!isSampledTrace(turn.context)) parent = suppressTracing(parent); } } diff --git a/packages/eve/src/tracing/agent-otel-session-context.ts b/packages/eve/src/tracing/agent-otel-session-context.ts index 3f0a1526ee..3e423f337c 100644 --- a/packages/eve/src/tracing/agent-otel-session-context.ts +++ b/packages/eve/src/tracing/agent-otel-session-context.ts @@ -7,9 +7,7 @@ import { type InstrumentationTurnStartedEvent, } from "#harness/instrumentation/lifecycle.js"; import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; -import type { TraceCapturePolicy } from "#tracing/otel-declaration.js"; +import { vercelSessionIdAttribute } from "#tracing/agent-otel-attributes.js"; import { SESSION_WINDOW_TURN_LIMIT, type AgentSessionTraceState, @@ -22,7 +20,6 @@ interface AgentOtelSessionContextInput { readonly idGenerator: AgentSpanIdGenerator; readonly stateStore: AgentTraceStateStore; readonly tracer: Tracer; - readonly tracePolicy?: TraceCapturePolicy; } interface AgentOtelSessionContext { @@ -43,33 +40,21 @@ export function createAgentOtelSessionContext( const emitVercelSessionId = input.emitVercelSessionId ?? false; const openSessionWindow = (window: { readonly agentName?: string; - readonly channelAudience: ChannelAudience; readonly index: number; readonly previousTraceId?: string; readonly rootSessionId: string; readonly sessionId: string; }): SpanContext => { - if (!shouldTrace(input.tracePolicy, window)) { - return { - isRemote: false, - spanId: input.idGenerator.deriveSpanId(`session:${window.sessionId}:${window.index}`), - traceFlags: 0, - traceId: input.idGenerator.generateTraceId(), - }; - } const span = input.tracer.startSpan("agent.session", { attributes: { "agent.framework.name": "eve", "agent.framework.version": input.frameworkVersion, - "agent.channel.audience": window.channelAudience, "agent.name": window.agentName, "agent.root.session.id": window.rootSessionId, "agent.session.id": window.sessionId, "agent.session.window": window.index, "agent.trace.schema.version": 1, - ...(emitVercelSessionId - ? { "vercel.session_id": window.rootSessionId } - : {}), + ...vercelSessionIdAttribute(emitVercelSessionId, window.rootSessionId), ...(window.previousTraceId === undefined ? {} : { "agent.session.window.previous.trace.id": window.previousTraceId }), @@ -91,26 +76,16 @@ export function createAgentOtelSessionContext( if (state === undefined) { state = { agentName: event.agentName, - channelAudience: normalizeChannelAudience(event.channelAudience), channelKind: event.channelKind, context: event.parentTraceContext === undefined ? openSessionWindow({ agentName: event.agentName, - channelAudience: normalizeChannelAudience(event.channelAudience), index: 0, rootSessionId: event.rootSessionId, sessionId: event.sessionId, }) - : adoptedSpanContext( - event.parentTraceContext, - shouldTrace(input.tracePolicy, { - agentName: event.agentName, - channelAudience: normalizeChannelAudience(event.channelAudience), - rootSessionId: event.rootSessionId, - sessionId: event.sessionId, - }), - ), + : adoptedSpanContext(event.parentTraceContext), rootSessionId: event.rootSessionId, turnsInWindow: 0, window: 0, @@ -130,7 +105,6 @@ export function createAgentOtelSessionContext( ...session, context: openSessionWindow({ agentName: session.agentName, - channelAudience: normalizeChannelAudience(session.channelAudience), index, previousTraceId: session.context.traceId, rootSessionId: session.rootSessionId, @@ -163,7 +137,6 @@ export function createAgentOtelSessionContext( event.sessionId, await ensureSessionContext({ agentName: undefined, - channelAudience: "unknown", channelKind: undefined, idempotencyKey: sessionIdempotencyKey(event.sessionId), parentTraceContext: event.parentTraceContext, @@ -200,29 +173,6 @@ export function createAgentOtelSessionContext( return { ensureSessionContext, prepareSessionTrace, prepareTurnTrace }; } -function shouldTrace( - policy: TraceCapturePolicy | undefined, - trace: { - readonly agentName?: string; - readonly channelAudience: ChannelAudience; - readonly rootSessionId: string; - readonly sessionId: string; - }, -): boolean { - try { - return ( - policy?.({ - agentName: trace.agentName, - audience: trace.channelAudience, - rootSessionId: trace.rootSessionId, - sessionId: trace.sessionId, - }) ?? trace.channelAudience === "public" - ); - } catch { - return false; - } -} - function portableSpanContext(spanContext: SpanContext): InstrumentationTraceContext { return { spanId: spanContext.spanId, @@ -231,11 +181,11 @@ function portableSpanContext(spanContext: SpanContext): InstrumentationTraceCont }; } -function adoptedSpanContext(handed: InstrumentationTraceContext, sampled = true): SpanContext { +function adoptedSpanContext(handed: InstrumentationTraceContext): SpanContext { return { isRemote: "isRemote" in handed && handed.isRemote === true, spanId: handed.spanId, - traceFlags: sampled ? handed.traceFlags : 0, + traceFlags: handed.traceFlags, traceId: handed.traceId, }; } diff --git a/packages/eve/src/tracing/agent-trace-context-store.ts b/packages/eve/src/tracing/agent-trace-context-store.ts index eba434d30e..1041013477 100644 --- a/packages/eve/src/tracing/agent-trace-context-store.ts +++ b/packages/eve/src/tracing/agent-trace-context-store.ts @@ -9,7 +9,6 @@ import type { AgentTraceStateStore, AgentTurnTraceState, } from "#tracing/agent-trace-state.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; interface AgentTraceContextState { readonly actions: Readonly>; @@ -192,7 +191,6 @@ function deserializeState(data: unknown): AgentTraceContextState { if (!isRecord(value) || !isSpanContext(value.context)) return undefined; return { agentName: typeof value.agentName === "string" ? value.agentName : undefined, - channelAudience: normalizeChannelAudience(value.channelAudience), channelKind: typeof value.channelKind === "string" ? value.channelKind : undefined, context: value.context, rootSessionId: typeof value.rootSessionId === "string" ? value.rootSessionId : "", @@ -239,7 +237,6 @@ function deserializeAction(value: unknown): AgentActionTraceState | undefined { return { attemptIndex: value.attemptIndex, callId: value.callId, - channelAudience: normalizeChannelAudience(value.channelAudience), inputAttribute: typeof value.inputAttribute === "string" ? value.inputAttribute : undefined, kind: value.kind, name: value.name, diff --git a/packages/eve/src/tracing/agent-trace-state.ts b/packages/eve/src/tracing/agent-trace-state.ts index 84f0d32cd9..21c79182da 100644 --- a/packages/eve/src/tracing/agent-trace-state.ts +++ b/packages/eve/src/tracing/agent-trace-state.ts @@ -7,13 +7,11 @@ import type { InstrumentationTurnFailedEvent, InstrumentationTurnSettledEvent, } from "#harness/instrumentation/lifecycle.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; /** Sized so an ordinary session stays one trace and only an outsized one rolls. */ export const SESSION_WINDOW_TURN_LIMIT = 200; export interface AgentSessionTraceState { - readonly channelAudience?: ChannelAudience; readonly agentName?: string; readonly channelKind?: string; readonly context: SpanContext; @@ -38,7 +36,6 @@ export interface AgentTurnTraceState { export interface AgentActionTraceState { readonly attemptIndex: number; readonly callId: string; - readonly channelAudience?: ChannelAudience; readonly inputAttribute?: string; readonly kind: InstrumentationActionKind; readonly name: string; diff --git a/packages/eve/src/tracing/channel-audience-context.ts b/packages/eve/src/tracing/channel-audience-context.ts deleted file mode 100644 index a8884c3bd8..0000000000 --- a/packages/eve/src/tracing/channel-audience-context.ts +++ /dev/null @@ -1,16 +0,0 @@ -import { createContextKey, type Context } from "#compiled/@opentelemetry/api/index.js"; -import { normalizeChannelAudience, type ChannelAudience } from "#shared/channel-audience.js"; - -const CHANNEL_AUDIENCE_KEY = createContextKey("eve.channel.audience"); - -export function channelAudienceFromContext(context: unknown): ChannelAudience { - if (typeof context !== "object" || context === null) return "unknown"; - const getValue = Reflect.get(context, "getValue"); - return typeof getValue === "function" - ? normalizeChannelAudience(Reflect.apply(getValue, context, [CHANNEL_AUDIENCE_KEY])) - : "unknown"; -} - -export function withChannelAudience(context: Context, audience: unknown): Context { - return context.setValue(CHANNEL_AUDIENCE_KEY, normalizeChannelAudience(audience)); -} diff --git a/packages/eve/src/tracing/content-span-processor.test.ts b/packages/eve/src/tracing/content-span-processor.test.ts index 26dd66535e..4f214436e0 100644 --- a/packages/eve/src/tracing/content-span-processor.test.ts +++ b/packages/eve/src/tracing/content-span-processor.test.ts @@ -242,26 +242,25 @@ describe("contentFilteringProcessor", () => { }); it.each([ - ["public", true], - ["private", false], - ["unknown", false], - ] as const)("retains content for the %s audience: %s", (audience, retained) => { + ["visible", true], + ["hidden", false], + ] as const)("retains content when the span is %s: %s", (visibility, retained) => { const downstream = recordingProcessor(); contentFilteringProcessor( downstream, composeSpanExportPolicies( - redactSpanInputs(({ audience }) => audience !== "public"), - redactSpanOutputs(({ audience }) => audience !== "public"), + redactSpanInputs(({ attributes }) => attributes["visibility"] !== "visible"), + redactSpanOutputs(({ attributes }) => attributes["visibility"] !== "visible"), ), ).onEnd( span({ - "agent.channel.audience": audience, + visibility, "ai.prompt.messages": "input", "ai.response.text": "output", }) as never, ); - const expected: Record = { "agent.channel.audience": audience }; + const expected: Record = { visibility }; if (retained) { expected["ai.prompt.messages"] = "input"; expected["ai.response.text"] = "output"; @@ -271,27 +270,6 @@ describe("contentFilteringProcessor", () => { ); }); - it("fails closed when audience attributes disagree", () => { - const downstream = recordingProcessor(); - contentFilteringProcessor( - downstream, - composeSpanExportPolicies( - redactSpanInputs(({ audience }) => audience !== "public"), - redactSpanOutputs(({ audience }) => audience !== "public"), - ), - ).onEnd( - span({ - "agent.channel.audience": "public", - "ai.prompt.messages": "private", - "ai.settings.context.eve.channel.audience": "private", - }) as never, - ); - - expect( - (downstream.ended[0] as { attributes: Record }).attributes, - ).not.toHaveProperty("ai.prompt.messages"); - }); - it("can drop an individual span", () => { const downstream = recordingProcessor(); contentFilteringProcessor(downstream, { span: ({ name }) => name !== "private-work" }).onEnd({ diff --git a/packages/eve/src/tracing/content-span-processor.ts b/packages/eve/src/tracing/content-span-processor.ts index 563162c8c7..774081e57c 100644 --- a/packages/eve/src/tracing/content-span-processor.ts +++ b/packages/eve/src/tracing/content-span-processor.ts @@ -5,15 +5,12 @@ import { type ResolvedContentOptions, } from "#tracing/content-attributes.js"; import { hasSessionRelease, type LocalTracesProcessor } from "#tracing/local-traces.js"; -import { normalizeChannelAudience } from "#shared/channel-audience.js"; -import type { ChannelAudience } from "#shared/channel-audience.js"; import { contentRedactionForSpan, spanExportPolicyStages, type SpanExportContext, type SpanExportPolicy, } from "#tracing/span-export-policy.js"; -import { channelAudienceFromContext } from "#tracing/channel-audience-context.js"; /** * Puts one destination's content policy in front of it. @@ -74,12 +71,7 @@ function policyFilteringProcessor( if (typeof span !== "object" || span === null) { return; } - const scoped = facadeFor( - span, - exportPolicy, - facades, - channelAudienceFromContext(parentContext), - ); + const scoped = facadeFor(span, exportPolicy, facades); if (!scoped.exported) { dropped.add(span); return; @@ -108,12 +100,11 @@ function facadeFor( span: object, exportPolicy: SpanExportPolicy, facades: WeakMap, - inheritedAudience?: ChannelAudience, ): SpanFacade { const existing = facades.get(span); if (existing !== undefined) return existing; - const context = spanExportContext(span, inheritedAudience); + const context = spanExportContext(span); const effectiveContent = contentForSpan(context, exportPolicy); const attributes: Record = {}; const events: unknown[] = []; @@ -212,26 +203,12 @@ function refreshAttributes( } } -function spanExportContext( - span: object, - inheritedAudience: ChannelAudience = "unknown", -): SpanExportContext { +function spanExportContext(span: object): SpanExportContext { const attributes = (span as { readonly attributes?: unknown }).attributes; const record = typeof attributes === "object" && attributes !== null ? (attributes as Readonly>) : {}; - const candidates = [ - record["agent.channel.audience"], - record["ai.settings.context.eve.channel.audience"], - ].filter((value) => value !== undefined); - const normalized = candidates.map(normalizeChannelAudience); - const audience = - normalized.length > 0 && normalized.every((value) => value === normalized[0]) - ? normalized[0]! - : normalized.length > 0 - ? "unknown" - : inheritedAudience; const spanContext = (span as { readonly spanContext?: () => unknown }).spanContext?.(); const ids = typeof spanContext === "object" && spanContext !== null @@ -240,7 +217,6 @@ function spanExportContext( const name = (span as { readonly name?: unknown }).name; return { attributes: record, - audience, name: typeof name === "string" ? name : "", spanId: typeof ids["spanId"] === "string" ? ids["spanId"] : "", traceId: typeof ids["traceId"] === "string" ? ids["traceId"] : "", diff --git a/packages/eve/src/tracing/install-instrumentation-runtime.test.ts b/packages/eve/src/tracing/install-instrumentation-runtime.test.ts index d56ad362a1..cedaf6ca1b 100644 --- a/packages/eve/src/tracing/install-instrumentation-runtime.test.ts +++ b/packages/eve/src/tracing/install-instrumentation-runtime.test.ts @@ -2,8 +2,9 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; import { ContextContainer, contextStorage } from "#context/container.js"; import { turnIdempotencyKey } from "#harness/instrumentation/lifecycle.js"; +import { constructInstrumentation } from "#harness/instrumentation/runtime.js"; import { installInstrumentationRuntime } from "#tracing/install-instrumentation-runtime.js"; -import { otelIntegration, collectOtelPipeline } from "#tracing/otel-declaration.js"; +import { otel, otelIntegration, collectOtelPipeline } from "#tracing/otel-declaration.js"; const { forceFlush, internalTerminalState, shutdown } = vi.hoisted(() => ({ forceFlush: vi.fn(async () => undefined), @@ -69,7 +70,14 @@ describe("installInstrumentationRuntime", () => { expect(forceFlush).toHaveBeenCalledOnce(); expect(providerFlush).toHaveBeenCalledOnce(); - expect(runtime.otelSettings).toEqual({ + expect(runtime.traceChannelRequests).toBe(false); + expect( + constructInstrumentation(runtime, { + action: "record", + recordInputs: true, + recordOutputs: true, + }).harness?.otelSettings, + ).toMatchObject({ functionId: undefined, recordInputs: true, recordOutputs: true, @@ -97,9 +105,14 @@ describe("installInstrumentationRuntime", () => { serviceName: "weather", }); const idempotencyKey = turnIdempotencyKey("session-1", "turn-1"); + const instrumentation = constructInstrumentation(runtime, { + action: "record", + recordInputs: true, + recordOutputs: true, + }).harness!; await contextStorage.run(new ContextContainer(), async () => { - await runtime.hooks.publish({ + await instrumentation.hooks.publish({ idempotencyKey, rootSessionId: "session-1", sequence: 0, @@ -107,7 +120,7 @@ describe("installInstrumentationRuntime", () => { turnId: "turn-1", type: "turn.started", }); - await runtime.hooks.publish({ + await instrumentation.hooks.publish({ idempotencyKey, sessionId: "session-1", turnId: "turn-1", @@ -118,4 +131,115 @@ describe("installInstrumentationRuntime", () => { expect(internalTerminalState).toHaveBeenCalledExactlyOnceWith("framework"); expect(authoredTerminalState).toHaveBeenCalledExactlyOnceWith("authored"); }); + + it("maps audience to controls before the harness", () => { + const runtime = installInstrumentationRuntime({ + collected: collectOtelPipeline([otelIntegration()]), + frameworkVersion: "test", + providers: [], + serviceName: "weather", + }); + const context = { rootSessionId: "session-1", sessionId: "session-1" }; + + expect(runtime.resolveDecision({ ...context, audience: "public" })).toEqual({ + action: "record", + recordInputs: true, + recordOutputs: true, + }); + expect(runtime.resolveDecision({ ...context, audience: "private" })).toEqual({ + action: "drop", + recordInputs: false, + recordOutputs: false, + }); + }); + + it("uses the controls returned by a custom trace policy", () => { + const runtime = installInstrumentationRuntime({ + collected: collectOtelPipeline([ + otel({ + tracePolicy: () => ({ + action: "record", + recordInputs: false, + recordOutputs: true, + }), + }), + otelIntegration(), + ]), + frameworkVersion: "test", + providers: [], + serviceName: "weather", + }); + + expect( + runtime.resolveDecision({ + audience: "private", + rootSessionId: "session-1", + sessionId: "session-1", + }), + ).toEqual({ action: "record", recordInputs: false, recordOutputs: true }); + }); + + it("fails closed when a trace policy throws", () => { + const runtime = installInstrumentationRuntime({ + collected: collectOtelPipeline([ + otel({ + tracePolicy: () => { + throw new Error("policy failed"); + }, + }), + otelIntegration(), + ]), + frameworkVersion: "test", + providers: [], + serviceName: "weather", + }); + + expect( + runtime.resolveDecision({ + audience: "public", + rootSessionId: "session-1", + sessionId: "session-1", + }), + ).toEqual({ action: "drop", recordInputs: false, recordOutputs: false }); + }); + + it("suppresses only the internal OTel provider for dropped deliveries", async () => { + const authoredTurnStarted = vi.fn(); + const runtime = installInstrumentationRuntime({ + collected: collectOtelPipeline([otelIntegration()]), + frameworkVersion: "test", + providers: [ + { + events: { "turn.started": authoredTurnStarted }, + name: "authored", + }, + ], + serviceName: "weather", + }); + const instrumentation = constructInstrumentation(runtime, { + action: "drop", + recordInputs: false, + recordOutputs: false, + }).harness!; + + await contextStorage.run(new ContextContainer(), async () => { + await instrumentation.hooks.publish({ + idempotencyKey: "turn-1", + rootSessionId: "session-1", + sequence: 0, + sessionId: "session-1", + turnId: "turn-1", + type: "turn.started", + }); + await instrumentation.hooks.publish({ + idempotencyKey: "turn-1", + sessionId: "session-1", + turnId: "turn-1", + type: "turn.completed", + }); + }); + + expect(authoredTurnStarted).toHaveBeenCalledOnce(); + expect(internalTerminalState).not.toHaveBeenCalled(); + }); }); diff --git a/packages/eve/src/tracing/install-instrumentation-runtime.ts b/packages/eve/src/tracing/install-instrumentation-runtime.ts index adbc734f78..88613aed85 100644 --- a/packages/eve/src/tracing/install-instrumentation-runtime.ts +++ b/packages/eve/src/tracing/install-instrumentation-runtime.ts @@ -1,12 +1,15 @@ -import { trace } from "#compiled/@opentelemetry/api/index.js"; +import { context, trace } from "#compiled/@opentelemetry/api/index.js"; import type { SpanProcessor } from "#compiled/@vercel/otel/index.js"; import { createInstrumentationHooks, + type InstrumentationHooks, type InstrumentationProviderDefinition, } from "#harness/instrumentation/lifecycle.js"; import { + createInstrumentationRuntime, registerInstrumentationRuntime, + type HarnessInstrumentation, type InstrumentationRuntime, } from "#harness/instrumentation/runtime.js"; import { createLogger, formatError } from "#internal/logging.js"; @@ -15,6 +18,11 @@ import { ContextAgentTraceStateStore } from "#tracing/agent-trace-context-store. import { createAgentOtelInstrumentation } from "#tracing/agent-otel-provider.js"; import { hasSessionRelease, type LocalTracesProcessor } from "#tracing/local-traces.js"; import type { CollectedOtel, RuntimeContextResolver } from "#tracing/otel-declaration.js"; +import { + DROP_INSTRUMENTATION, + type InstrumentationControls, +} from "#shared/instrumentation-controls.js"; +import { suppressTracing } from "#tracing/suppress-tracing.js"; import { registerOtelPipeline, type RegisteredOtelPipeline } from "#tracing/otel-registration.js"; const log = createLogger("tracing.install-instrumentation-runtime"); @@ -39,9 +47,10 @@ export function installInstrumentationRuntime(input: { const serialBefore: InstrumentationProviderDefinition[] = []; const serialAfter: InstrumentationProviderDefinition[] = []; let otelRuntime: RegisteredOtelPipeline | undefined; - let prepareSessionTrace: InstrumentationRuntime["prepareSessionTrace"]; - let prepareTurnTrace: InstrumentationRuntime["prepareTurnTrace"]; - let runInContext: InstrumentationRuntime["runInContext"] = (_operation, execute) => execute(); + let prepareSessionTrace: HarnessInstrumentation["prepareSessionTrace"]; + let prepareTurnTrace: HarnessInstrumentation["prepareTurnTrace"]; + let runInContext: NonNullable = (_operation, execute) => + execute(); if (input.collected.declared) { otelRuntime = registerOtelPipeline({ @@ -56,7 +65,6 @@ export function installInstrumentationRuntime(input: { recordOutputs: input.collected.settings.recordOutputs, stateStore: new ContextAgentTraceStateStore(), tracer: trace.getTracer("eve.agent", input.frameworkVersion), - tracePolicy: input.collected.settings.tracePolicy, }); // The span must exist before authored providers observe the lifecycle event. serialBefore.push({ ...agentOtel.hook, stateNamespace: "internal:otel" }); @@ -71,31 +79,66 @@ export function installInstrumentationRuntime(input: { } const allProviders = [...serialBefore, ...input.providers, ...serialAfter]; - let shutdown: Promise | undefined; - return registerInstrumentationRuntime({ - forceFlush: () => - settleAll([ - ...(otelRuntime === undefined ? [] : [otelRuntime.forceFlush]), - ...allProviders.map((provider) => () => provider.flush?.()), - ]), - hooks: createInstrumentationHooks({ + const createHooks = (controls: InstrumentationControls): InstrumentationHooks => + createInstrumentationHooks({ parallel: input.providers, serialAfter, - serialBefore, + serialBefore: controls.action === "record" ? serialBefore : [], + }); + let shutdown: Promise | undefined; + const otelSettings = input.collected.declared ? input.collected.settings : undefined; + return registerInstrumentationRuntime( + createInstrumentationRuntime({ + createHooks, + forceFlush: () => + settleAll([ + ...(otelRuntime === undefined ? [] : [otelRuntime.forceFlush]), + ...allProviders.map((provider) => () => provider.flush?.()), + ]), + otelSettings, + prepareSessionTrace, + prepareTurnTrace, + runtimeContextResolvers: input.runtimeContextResolvers, + resolveDecision: (context) => { + if (!input.collected.declared) { + return { + action: "record", + recordInputs: true, + recordOutputs: true, + }; + } + try { + const decision = + input.collected.settings.tracePolicy?.(context) ?? + (context.audience === "public" + ? { action: "record", recordInputs: true, recordOutputs: true } + : { action: "drop" }); + if (decision.action === "drop") return DROP_INSTRUMENTATION; + if ( + decision.action === "record" && + typeof decision.recordInputs === "boolean" && + typeof decision.recordOutputs === "boolean" + ) { + return decision; + } + } catch { + // Trace policy failures fail closed. + } + return DROP_INSTRUMENTATION; + }, + runInContext, + runWithTracingSuppressed: (execute) => + context.with(suppressTracing(context.active()), execute), + shutdown: () => { + shutdown ??= settleAll([ + ...(otelRuntime === undefined ? [] : [otelRuntime.shutdown]), + ...allProviders.map((provider) => () => provider.shutdown?.()), + ]); + return shutdown; + }, }), - otelSettings: input.collected.declared ? input.collected.settings : undefined, - prepareSessionTrace, - prepareTurnTrace, - runtimeContextResolvers: input.runtimeContextResolvers, - runInContext, - shutdown: () => { - shutdown ??= settleAll([ - ...(otelRuntime === undefined ? [] : [otelRuntime.shutdown]), - ...allProviders.map((provider) => () => provider.shutdown?.()), - ]); - return shutdown; - }, - }); + otelSettings, + ); } function isSpanProcessor(processor: SpanProcessor | "auto"): processor is SpanProcessor { diff --git a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts index bd6c42626e..b9f5cd5f30 100644 --- a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts +++ b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts @@ -14,6 +14,7 @@ import { } from "#compiled/@opentelemetry/api/index.js"; import { ContextContainer, contextStorage } from "#context/container.js"; import { createAiSdkHookBridge } from "#harness/ai-sdk-hook-bridge.js"; +import { constructInstrumentation } from "#harness/instrumentation/runtime.js"; import { listLocalTraces } from "#tracing/local-trace-reader.js"; import type { InstrumentationAttemptScope } from "#harness/instrumentation/lifecycle.js"; import { @@ -43,11 +44,14 @@ describe("local instrumentation runtime", () => { const authoredTracer = ( require("@opentelemetry/api") as typeof import("@opentelemetry/api") ).trace.getTracer("test-user"); - const runtime = installLocalInstrumentationRuntime({ - appRoot, - frameworkVersion: "test", - serviceName: "test-agent", - }); + const runtime = constructInstrumentation( + installLocalInstrumentationRuntime({ + appRoot, + frameworkVersion: "test", + serviceName: "test-agent", + }), + { action: "record", recordInputs: true, recordOutputs: true }, + ).harness!; const scope: InstrumentationAttemptScope = { attemptId: "session-1:turn-1:0:0", attemptIndex: 0, diff --git a/packages/eve/src/tracing/local-instrumentation-runtime.ts b/packages/eve/src/tracing/local-instrumentation-runtime.ts index 905547fb1a..9d55de4246 100644 --- a/packages/eve/src/tracing/local-instrumentation-runtime.ts +++ b/packages/eve/src/tracing/local-instrumentation-runtime.ts @@ -13,7 +13,9 @@ import { /** Zero-config local tracing keeps unclassified HTTP/TUI sessions observable. @internal */ export const localTracePolicy: TraceCapturePolicy = ({ audience }) => - audience === "public" || audience === "unknown"; + audience === "public" || audience === "unknown" + ? { action: "record", recordInputs: true, recordOutputs: true } + : { action: "drop" }; /** Installs the zero-config local OTel runtime once in an `eve dev` worker. */ export function installLocalInstrumentationRuntime(input: { diff --git a/packages/eve/src/tracing/local-traces.test.ts b/packages/eve/src/tracing/local-traces.test.ts index e0bd1e850b..d9d684f1f8 100644 --- a/packages/eve/src/tracing/local-traces.test.ts +++ b/packages/eve/src/tracing/local-traces.test.ts @@ -101,16 +101,16 @@ describe("resolveLocalTracesContent", () => { describe("localTracePolicy", () => { it.each([ - ["public", true], - ["unknown", true], - ["private", false], - ] as const)("accepts the %s audience: %s", (audience, accepted) => { + ["public", "record"], + ["unknown", "record"], + ["private", "drop"], + ] as const)("maps the %s audience to %s", (audience, action) => { expect( localTracePolicy({ audience, rootSessionId: "session-1", sessionId: "session-1", }), - ).toBe(accepted); + ).toMatchObject({ action }); }); }); diff --git a/packages/eve/src/tracing/otel-declaration.test.ts b/packages/eve/src/tracing/otel-declaration.test.ts index 57173f035d..a5121049c9 100644 --- a/packages/eve/src/tracing/otel-declaration.test.ts +++ b/packages/eve/src/tracing/otel-declaration.test.ts @@ -104,7 +104,7 @@ describe("managed export policy", () => { if (spanProcessor === undefined || spanProcessor === "auto") throw new Error("Expected policy"); spanProcessor.onEnd( testSpan({ - "agent.channel.audience": "private", + visibility: "private", "ai.prompt.messages": "private input", }), ); @@ -116,7 +116,7 @@ describe("managed export policy", () => { let visibleAttributes: Readonly> | undefined; const integration = managedOtelIntegration({ exportPolicy: composeSpanExportPolicies( - redactSpanInputs(({ audience }) => audience !== "public"), + redactSpanInputs(({ attributes }) => attributes["visibility"] !== "public"), { span: ({ attributes }) => { visibleAttributes = attributes; @@ -131,12 +131,12 @@ describe("managed export policy", () => { if (spanProcessor === undefined || spanProcessor === "auto") throw new Error("Expected policy"); spanProcessor.onEnd( testSpan({ - "agent.channel.audience": "private", + visibility: "private", "ai.prompt.messages": "private input", }), ); - expect(visibleAttributes).toEqual({ "agent.channel.audience": "private" }); + expect(visibleAttributes).toEqual({ visibility: "private" }); }); it("applies deprecated content switches before the configured export policy", () => { diff --git a/packages/eve/src/tracing/otel-declaration.ts b/packages/eve/src/tracing/otel-declaration.ts index e14ff8b229..9c3266597f 100644 --- a/packages/eve/src/tracing/otel-declaration.ts +++ b/packages/eve/src/tracing/otel-declaration.ts @@ -111,7 +111,15 @@ export interface TraceCaptureContext { readonly sessionId: string; } -export type TraceCapturePolicy = (trace: TraceCaptureContext) => boolean; +export type TraceCaptureDecision = + | { readonly action: "drop" } + | { + readonly action: "record"; + readonly recordInputs: boolean; + readonly recordOutputs: boolean; + }; + +export type TraceCapturePolicy = (trace: TraceCaptureContext) => TraceCaptureDecision; /** Where one `otelIntegration()` sends spans. */ export interface OtelIntegrationOptions extends ContentOptions { @@ -276,12 +284,16 @@ export interface OtelPipeline { export interface OtelHarnessSettings { readonly functionId?: string; readonly traceChannelRequests: boolean; - readonly tracePolicy?: TraceCapturePolicy; /** Legacy `defineInstrumentation()` capture settings. Provider destinations capture fully. */ readonly recordInputs?: boolean; readonly recordOutputs?: boolean; } +/** Process settings retained outside the harness until channel delivery. @internal */ +export interface OtelRuntimeSettings extends OtelHarnessSettings { + readonly tracePolicy?: TraceCapturePolicy; +} + /** @internal */ export type RuntimeContextResolver = ( input: InstrumentationRuntimeContextInput, @@ -296,7 +308,7 @@ export interface CollectedOtel { readonly declared: boolean; readonly pipeline: OtelPipeline; readonly runtimeContextResolvers: readonly RuntimeContextResolver[]; - readonly settings: OtelHarnessSettings; + readonly settings: OtelRuntimeSettings; } /** @@ -337,7 +349,7 @@ export function collectOtelPipeline(values: readonly unknown[]): CollectedOtel { } const options = declaration?.options ?? {}; - const settings: OtelHarnessSettings = { + const settings: OtelRuntimeSettings = { functionId: options.functionId, recordInputs: capturesContent, recordOutputs: capturesContent, diff --git a/packages/eve/src/tracing/span-export-policy.ts b/packages/eve/src/tracing/span-export-policy.ts index 116008179f..343b6556ee 100644 --- a/packages/eve/src/tracing/span-export-policy.ts +++ b/packages/eve/src/tracing/span-export-policy.ts @@ -1,8 +1,5 @@ -import type { ChannelAudience } from "#shared/channel-audience.js"; - export interface SpanExportContext { readonly attributes: Readonly>; - readonly audience: ChannelAudience; readonly name: string; readonly spanId: string; readonly traceId: string; diff --git a/research/channel-audience-content-policy.md b/research/channel-audience-content-policy.md index d88d730dcc..11d807424a 100644 --- a/research/channel-audience-content-policy.md +++ b/research/channel-audience-content-policy.md @@ -22,7 +22,7 @@ type ChannelAudience = "public" | "private" | "unknown"; The field is optional for authored channels. Eve normalizes absent, malformed, and unsupported values to `unknown`. Built-in channel metadata interfaces require the field and classify only from platform evidence already captured during dispatch; ambiguous and proactive destinations remain `unknown` rather than performing observability-only network requests. -The normalized audience is persisted with session trace state and exported as `agent.channel.audience` only on each `agent.session` window. Durable Eve state and an internal OpenTelemetry context key make the same value available to descendant export policies without duplicating a public attribute onto every span. Local subagents inherit the parent audience. Remote agents classify their receiving channel independently rather than trusting opaque metadata across deployment boundaries. +At channel delivery, eve evaluates the process-wide trace policy and persists only its generic decision for that turn. Each durable execution boundary reconstructs a decision-bound instrumentation scope whose hooks, telemetry, provider graph, and execution context already enforce the decision. The harness, lifecycle providers, trace state, and export policies never receive the audience classification or branch on the decision. Local subagents inherit the serialized decision; remote agents classify their receiving channel independently. ## Public tracing API @@ -38,7 +38,15 @@ interface TraceCaptureContext { readonly sessionId: string; } -type TraceCapturePolicy = (trace: TraceCaptureContext) => boolean; +type TraceCaptureDecision = + | { readonly action: "drop" } + | { + readonly action: "record"; + readonly recordInputs: boolean; + readonly recordOutputs: boolean; + }; + +type TraceCapturePolicy = (trace: TraceCaptureContext) => TraceCaptureDecision; interface OtelOptions { // Other process-wide OTel settings are unchanged. @@ -53,7 +61,6 @@ Agent Runs and local traces expose the managed export policy: ```ts interface SpanExportContext { readonly attributes: Readonly>; - readonly audience: ChannelAudience; readonly name: string; readonly spanId: string; readonly traceId: string; @@ -101,25 +108,26 @@ declare function localTraces(options?: ManagedTraceOptions): OtelIntegration; `composeSpanExportPolicies()` applies policies in declaration order. A later span or attribute policy sees the facade produced by earlier redactors. A span predicate returning `false` removes that span from one destination without suppressing the rest of its trace. Attribute policies run once for each attribute still visible at their stage. -For example, this admits public and private conversations at the head gate while redacting private content before applying destination-specific filtering: +For example, this records public conversations with content and private conversations without content: ```ts // agent/instrumentation/otel.ts export default otel({ - tracePolicy: ({ audience }) => audience === "public" || audience === "private", + tracePolicy: ({ audience }) => + audience === "public" + ? { action: "record", recordInputs: true, recordOutputs: true } + : audience === "private" + ? { action: "record", recordInputs: false, recordOutputs: false } + : { action: "drop" }, }); // agent/instrumentation/agent-runs.ts export default agentRuns({ - exportPolicy: composeSpanExportPolicies( - redactSpanInputs(({ audience }) => audience !== "public"), - redactSpanOutputs(({ audience }) => audience !== "public"), - { - span: ({ name }) => name !== "internal.cache.refresh", - attribute: ({ key }) => - key === "user.email" ? { action: "replace", value: "[redacted]" } : { action: "keep" }, - }, - ), + exportPolicy: { + span: ({ name }) => name !== "internal.cache.refresh", + attribute: ({ key }) => + key === "user.email" ? { action: "replace", value: "[redacted]" } : { action: "keep" }, + }, }); ``` @@ -128,37 +136,43 @@ export default agentRuns({ The default authored and production head policy is equivalent to: ```ts -({ audience }) => audience === "public"; +({ audience }) => + audience === "public" + ? { action: "record", recordInputs: true, recordOutputs: true } + : { action: "drop" }; ``` -| Audience | Trace created by default | Content when admitted by a custom trace policy | -| --------- | ------------------------ | ---------------------------------------------- | -| `public` | Yes | Unchanged unless an export policy redacts it | -| `private` | No | Unchanged unless an export policy redacts it | -| `unknown` | No | Unchanged unless an export policy redacts it | +| Audience | Trace created by default | Input content | Output content | +| --------- | ------------------------ | ------------- | -------------- | +| `public` | Yes | Yes | Yes | +| `private` | No | No | No | +| `unknown` | No | No | No | The default policy for local tracing for `eve dev` is equivalent to: ```ts -({ audience }) => audience === "public" || audience === "unknown"; +({ audience }) => + audience === "public" || audience === "unknown" + ? { action: "record", recordInputs: true, recordOutputs: true } + : { action: "drop" }; ``` This keeps unclassified local HTTP/TUI sessions observable while still rejecting channels classified as `private`. The runtime order is: -1. Derive and normalize the channel audience. -2. Evaluate the process-wide `tracePolicy` before creating `agent.session`. -3. For accepted traces, capture complete Eve and AI SDK spans. -4. Run each managed destination's composed export policies in declaration order. Custom integrations run their declared span processors. +1. Derive and normalize the channel audience at channel delivery. +2. Evaluate `tracePolicy` and persist only its drop/record and content decision for the turn. +3. Construct an instrumentation scope that includes only the permitted providers and exposes pre-filtered hooks, telemetry, and execution context to the harness. +4. Run each managed destination's composed export policies in declaration order. Destination policies may redact further but cannot restore content removed by the delivery controls. 5. Hand the resulting facade to that destination's processors or exporter. -There is no implicit content redaction after a custom trace policy admits an audience. Redaction occurs only when the export pipeline includes `redactSpanInputs()` or `redactSpanOutputs()` (or when a retained compatibility option explicitly requests the equivalent redaction). - Policies fail closed at their boundary: a throwing trace policy rejects the trace, a throwing span policy drops the span, a throwing attribute policy drops the attribute, and a throwing content-redaction predicate redacts that content direction. Missing, malformed, or conflicting audience evidence normalizes to `unknown`. +`traceChannelRequests` remains a separate opt-in request diagnostic. Its server span begins before channel delivery can classify audience and contains no body, session id, auth, cookie, token, or query content; the delivery decision governs the durable agent and AI trace beneath that request boundary. + ## Compatibility -The existing `recordInputs` and `recordOutputs` destination options remain accepted as deprecated source-compatible aliases. An explicit `false` prepends the corresponding redaction policy; these options no longer prevent accepted spans from capturing content upstream. `EVE_TRACES_CONTENT=off` similarly prepends both redactors for local traces. +The existing `recordInputs` and `recordOutputs` destination options remain accepted as deprecated source-compatible aliases. An explicit `false` prepends the corresponding redaction policy. `EVE_TRACES_CONTENT=off` similarly prepends both redactors for local traces. -Filtering remains a span-processor responsibility because local trace persistence and authored processors are processors rather than uniform exporters. Keeping the filtering boundary immediately above each destination prevents one destination's policy from mutating what another destination receives. +Boolean `tracePolicy` results are replaced by `TraceCaptureDecision`. This is a deliberate pre-1.0 API break: malformed results fail closed rather than being treated as an admission decision.