refactor(coding-agent): schedule agent-trace uploads through a disk-cursor outbox

[snimu]

Problem

Agent-trace upload progress lived only in process memory (ENG-5837's investigation). That one premise produced the whole defect class:

• disposal/exit paths had to drain uploads before the process could go away — fix(coding-agent): don't block RLM subagent deletion on trace upload or a doomed kernel snapshot #1954 fixed the worst symptom (deletes blocking up to 60s+ on a flush) with a detached flush plus a process-exit barrier, but the barrier still makes exit paths know about and wait for telemetry;
• a crash still loses the last <=60s of trace regardless of any exit choreography;
• a 429 without retry-after slept a flat 60s inside fetchWithRetry, holding whoever awaited the upload.

Shape

Upload scheduling is now a disk-cursor outbox owned entirely by agent-traces.ts:

• One entry file per session under <agentDir>/agent-traces-outbox/, named by a hash of the session-file path, each written atomically (tmp+rename). An entry holds the session-file path plus the last uploaded content signature {size, mtimeMs}, or no signature for "scheduled but never uploaded". (size, mtimeMs) is the same append-only-content signature readSessionInfo caching already trusts. Per-entry files make cross-process lost updates and wipe-on-corrupt-read structurally impossible: concurrent writers touch different files (or write equivalent content to the same one), a bad read costs only its own entry, and pruning deletes one file.
• Schedule = durable intent. The per-session controller keeps the existing 1s debounce / 60s min-interval timer shape; the first persist of a session file also writes its pending entry synchronously, so the intent is on disk the moment the transcript persist returns — no exit path has to care. All upload timers are unref()ed: telemetry never keeps the process alive.
• Cursor invariant: an automatic upload never re-sends a file whose current content signature equals its cursor; every successful upload (automatic or manual) advances the cursor. Compare-before-send re-reads the file from disk, which is also the multi-process dedupe: identical idempotent payload + cursor compare is the whole story, no lock service.
• Startup catch-up (catchUpAgentTraceUploads): iterates the outbox entries, prunes entries whose file is gone (deleted drafts/subagent catalogs), skips files owned by a live controller in this process, and sequentially uploads anything ahead of its cursor through the existing 5-per-minute request gate.
• 429 is not retried in-request anymore. The upload returns failed(429) and the controller re-arms its timer at max(60s since the last start, the server's advertised retry-after). Nothing ever sleeps inside a caller, and a fresh persist cannot re-arm inside an advertised window. In-request retry-after on 503 is still honored. A successful PUT whose cursor write fails returns a retryable failure instead of silently keeping a stale cursor (the PUT is idempotent; a resend is harmless).
• Disposal/exit code no longer knows traces exist. The flush-on-dispose calls, flushAgentTraceUpload, detachTraceFlush, and fix(coding-agent): don't block RLM subagent deletion on trace upload or a doomed kernel snapshot #1954's exit barrier (flushAllPendingAgentTraceUploads + its four try/finally call sites in daemon shutdown, update restart, worker archive-and-shutdown, and in-process connection dispose) are deleted. A pending upload of a closed session fires from its still-armed timer, or from the next process start's catch-up.

The cursor mechanism is deliberately file-kind-agnostic (path-keyed, "content ahead of cursor -> upload -> advance"); only the endpoint/payload construction in performAgentTraceUpload is transcript-specific, so a second append-only file kind (e.g. the deferred semantic-edges ledger from #1885) can reuse the scheduling/catch-up/pruning later without a registry being built today.

Ownership decision

The catch-up runs once per process, triggered by the first installAgentTraceUpload call — i.e. in whichever process actually hosts sessions (daemon workers, interactive/print/rpc processes). That is the only place trace upload is enabled and configured today (the authStorage+settingsManager pair from session services); the daemon supervisor hosts no sessions and stays trace-ignorant. Cross-worker duplication is bounded by the cursor compare + idempotent PUT.

Server-semantics evidence (wire format unchanged)

Same endpoint, same auth, same full-file body; zero server-side changes. Evidence the server overwrites by session id: the method is PUT (replace semantics) on /api/v1/agent-traces/sessions/<sessionId>; main already re-PUTs the entire raw session JSONL for the same id every <=60s while a session is active and the server answers each with bytes_stored for the full body; the existing test pins body == readFileSync(sessionFile). The cursor only decides when an upload is due.

Supersedes (#1954 overlap)

Builds on #1954 and deletes its interim exit-drain machinery — the startup catch-up makes exit draining unnecessary: flushAllPendingAgentTraceUploads (+ 4 call sites), detachTraceFlush, logDetachedAgentTraceFlushFailure, flushAgentTraceUpload, and the barrier pins. Kept from #1954: the kernelSnapshot: false disposal threading (orthogonal) and its still-valid delete pins (delete never blocks, upload fires once, snapshot flags), reshaped onto the outbox scheduler.

Numbers

Measured with the same drivers against the pre-#1954 in-memory scheduler (old) and this PR (new):

scenario	old	new
delete_subagent of a resident child with a pending upload (3s network)	6004ms	25ms
simulated active hour (append every 20s, healthy server)	59 POSTs / 3.05MB	21 POSTs / 1.06MB
simulated active hour, server always 429s (no retry-after)	67 POSTs / 3.40MB	45 POSTs / 2.34MB, and no caller ever blocked

Honest note on volume: both schedulers are bounded by the same 60s min interval and the wire format is unchanged, so steady-state volume is similar by design; the new scheduler additionally coalesces while an upload is in flight and never re-POSTs unchanged content across restarts. The wins this PR is actually for are durability (crash loses nothing past the next startup) and the removal of all blocking.

The explicit /traces upload and /traces upload-all commands intentionally bypass the cursor (an explicit command is a force re-send / escape hatch); only scheduled and catch-up uploads are cursor-gated.

Pins (each fails on the old scheduler; verified by running them against the pre-rebase origin/main sources)

• (i) schedule durably records intent on disk before any upload; a seeded outbox from a "previous process" catch-ups exactly the missed content with one POST and an fs-visible cursor advance — old: no outbox file is ever written (assert failed)
• (ii) unchanged content: repeat automatic upload returns unchanged with zero POSTs; repeat catch-up uploads nothing — old: re-POSTs the full file (got uploaded, 2 calls)
• (iii) delete_subagent with a gated in-flight upload resolves immediately (kernelSnapshot: false still asserted) and the transcript upload completes afterward — old (pre-fix(coding-agent): don't block RLM subagent deletion on trace upload or a doomed kernel snapshot #1954): hangs on the gated fetch until the 20s test timeout
• (iv) 429 without retry-after returns failed(429) after exactly one attempt and the controller re-arms and retries on the next cycle without any caller waiting — old: retried in-request with 60s sleeps (test burned its 30s timeout)
• (v) cursor entries for deleted session files are pruned by catch-up; kept files keep their cursor — old: no cursor exists
• intent marker is synchronously on disk when the transcript persist returns — fails on the async fire-and-forget marker
• upload timers never hold the process open (hasRef() === false) — fails without unref()
• an advertised Retry-After gates the next cycle (300s pin), and a fresh persist cannot re-arm earlier — fails when the header is dropped
• a successful PUT with a failed cursor write returns a retryable failure, not uploaded — fails on the swallowed cursor error
• a corrupt outbox entry costs only itself: sibling cursors survive and stay unchanged — fails on the single-map store (wipe + resend)

Also reshaped: #1954's passivation-join delete pin (kept, minus the barrier drain), and its barrier pins deleted with the barrier.

Tests

packages/coding-agent: agent-traces (33), daemon-mode (183), agent-connection-in-process, agent-session-concurrent, ipython-provisioner all pass in a Prime sandbox; full suite is at the documented environmental baseline (82 failed / 14 files: extensions-, kernel--skill bridges, tools-EACCES-as-root, config, resource-loader, sdk-session-manager, agent-session-recursion, 4428, 4603 — identical list, no new failures). Root npm run check clean.

Linear: RES-1244

---

Note

Medium Risk
Changes telemetry durability and shutdown timing (traces may complete only after exit via catch-up), but wire format is unchanged and idempotent PUTs limit duplicate-send risk.

Overview
Replaces in-process trace upload draining with a durable agent-traces-outbox/ cursor so shutdown, session disposal, and subagent deletes no longer wait on telemetry. Upload intent and last-uploaded {size, mtimeMs} live in one hashed JSON entry per session; the first installAgentTraceUpload per process runs catchUpAgentTraceUploads to finish work from a prior crash and prune entries for deleted session files.

Scheduled and catch-up uploads skip unchanged transcripts (unchanged); explicit /traces upload still force-sends. 429 responses are not retried inside the HTTP client—the controller re-arms an unref()’d timer using Retry-After (and the 60s min interval) instead of blocking callers. Successful PUTs advance the on-disk cursor; cursor write failures surface as retryable failures.

Removes flushAgentTraceUpload, flushAllPendingAgentTraceUploads, and detached flush hooks from runtime, daemon shutdown paths, in-process connection dispose, and RLM child updates. Interactive UI adds messaging for unchanged uploads.

^{Reviewed by Cursor Bugbot for commit 2eb82f2. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Replace detached agent-trace flushes with disk-cursor outbox scheduling

• Trace uploads now persist a pending outbox marker on each transcript write and record a size+mtime cursor after a successful upload, so uploads survive process restarts via a startup catch-up pass
• A new AgentTraceUploadController schedules non-blocking uploads using unreferenced timers with debounce, throttle, and Retry-After rescheduling; content arriving mid-upload triggers a follow-up cycle
• HTTP 429 is no longer retried inside the request loop; it is returned to the controller which re-arms based on Retry-After
• Removed all detached trace-upload launches and global drain barriers from session teardown, runtime disposal, in-process connection disposal, and daemon shutdown paths
• Risk: disposal and daemon shutdown no longer await trace uploads — any upload still in-flight at exit may not complete; AgentTraceUploadController timer is unreferenced so it will not keep the process alive; cursor recording failure after a successful remote upload is reported as a failed result in agent-traces.ts

^{Macroscope summarized 2eb82f2.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c90981d. Configure here.}