perf(ai): reduce streaming and native media overhead

[JustYannicc]

What changed

• Batch main-process AI stream IPC chunk sends with synchronous flushes before done, error, cancellation cleanup, and same-request replacement.
• Clean up Raycast AI.ask() caller abort listeners on every terminal stream path.
• Cache stable local ASR helper status probe results for a short TTL while force-refreshing explicit download, warmup, and transcription flows.
• Clean up Ollama pull progress/done/error listeners in Settings > AI, filter events to the active request id, and skip identical progress writes.
• Remove the redundant camera capture /bin/mkdir -p subprocess and rely on the existing main-process binary writer to create parent directories.
• Bulk-write captured native WAV samples through one contiguous little-endian PCM buffer.
• Preserve focused regression/perf harnesses for all six consolidated fixes.

Why

These hot paths were still doing avoidable work after the first performance stack: AI streams could emit one IPC message per provider chunk, local model polling could relaunch helper probes every tick, successful AI.ask() calls could retain abort listeners until the caller signal aborted, Settings > AI remounts could leave stale Ollama pull listeners behind, camera capture still spawned a directory subprocess, and native WAV serialization appended two bytes per sample. Consolidating them keeps the second-wave performance fixes together on top of the integration stack.

Compatibility impact

• Renderer-facing AI APIs, request IDs, cancellation behavior, final concatenated text, and error handling are preserved.
• Stream chunk boundaries may be larger, but content order and terminal flush behavior are unchanged.
• Local model status cache only retains stable statuses inside the TTL; downloading/error states and explicit actions still bypass cached data.
• Existing Ollama pull event subscribers can keep ignoring the unsubscribe return value; Settings > AI now uses it for cleanup while preserving single-pull UX and final status refresh behavior.
• Camera capture keeps the same save path and binary writer contract.
• WAV output remains RIFF/WAVE PCM, mono, 16-bit little-endian, with unchanged clamping/scaling and atomic write behavior.

How tested

• node --test scripts/test-ollama-pull-listener-cleanup.mjs scripts/test-ai-model-status-polling.mjs scripts/test-local-model-status-probe-cache.mjs scripts/test-ai-stream-ipc-batching.mjs scripts/test-raycast-ai-abort-listeners.mjs scripts/test-use-ai-stream-batching.mjs -> 27 pass.
• node --test scripts/test-camera-capture-single-encode.mjs -> 10 pass.
• pnpm test -> 184 pass.
• ./node_modules/.bin/tsc -p tsconfig.main.json --noEmit --pretty false -> pass.
• ./node_modules/.bin/tsc -p tsconfig.renderer.json --noEmit --pretty false -> pass.
• pnpm run build:main -> pass.
• pnpm run build:renderer -> pass.
• swiftc -O -o /tmp/supercmd-audio-capturer-check src/native/audio-capturer.swift -framework AVFoundation -framework Foundation -> pass.
• swiftc -O -o /tmp/supercmd-audio-wav-bulk-test scripts/test-audio-wav-bulk-write.swift && /tmp/supercmd-audio-wav-bulk-test -> pass.
• pnpm run build:native passed during branch validation after using a hoisted local pnpm validation install; the initial strict pnpm layout could not resolve the native helper addon's transitive node-addon-api from binding.gyp. perf(settings): clean up Ollama pull listeners #553 touches no native files.
• git diff --check codex/perf-consolidated-integration-stack..HEAD -> pass.
• Codex LSP diagnostics: no diagnostics for src/main/ai-stream-ipc.ts, src/main/local-model-status-probe.ts, src/main/main.ts, src/main/preload.ts, src/renderer/src/raycast-api/index.tsx, src/renderer/src/CameraExtension.tsx, src/renderer/src/settings/AITab.tsx, src/renderer/src/settings/ollamaPullProgress.ts, and src/renderer/types/electron.d.ts.
• SourceKit diagnostics: no diagnostics for src/native/audio-capturer.swift.

Performance evidence

• AI stream IPC focused harness: 600 provider chunks -> 600 baseline ai-stream-chunk IPC sends; batched path -> 1 IPC send, with final text preserved.
• Raycast useAI harness: 600 streamed chunks -> 600 baseline visible updates; batched path -> 1 visible update, with final flush/error/abort behavior preserved.
• Local model status probe cache focused harness: iterations=60 legacyCalls=60 legacyMs=105.067 cachedCalls=1 cachedMs=1.791 avoidedCalls=59.
• AI status polling harness: 5 unchanged ticks went from 15 baseline state updates / 15 profiler commits to 0; changing ticks commit once per tick.
• Raycast AI.ask() abort listener harness: success, stream error, aiAsk rejection, and explicit abort all end at active=0, added=1, removed=1; the pre-fix retained-listener baseline was active=1, added=1, removed=0.
• Ollama pull listener harness: 5 remounts + 1 active mount previously produced progress/done/error callbacks 6/6/6 and left listener counts 6/6/6; after cleanup callbacks are 1/1/1 and listener counts after cleanup are 0/0/0. The same harness verifies stale request-id filtering and identical-progress dedupe.
• Camera mkdir audit from the replaced PR: /bin/mkdir -p averaged 1.370 ms over 80 runs, while in-process recursive mkdir averaged 0.033 ms; this branch asserts capture no longer shells out and the main IPC writer still creates parents before writing.
• WAV bulk-write harness on this branch: 480,000 samples at 16 kHz, byte-for-byte output validated; per-sample append 14.76 ms avg, bulk Int16 append 0.81 ms avg, 18.16x speedup.

Stack validation

• Branch: codex/consolidate-ai-native-media from codex/perf-consolidated-integration-stack at 52288a0.
• Added only the six assigned consolidation commits on top of the integration stack.
• perf(settings): clean up Ollama pull listeners #553 was folded in as commit 31caa81 and verified as the only new delta after the original perf(ai): reduce streaming and native media overhead #556 head.
• Did not modify AGENTS.md, root pnpm-lock.yaml, root pnpm-workspace.yaml, or local playbook/prompt markdown.
• The worktree is clean after validation artifacts were removed.

Replaces

• Replaces perf(ai): batch stream IPC chunks #537.
• Replaces perf(ai): clean up stream abort listeners #538.
• Replaces perf(camera): remove redundant mkdir subprocess #540.
• Replaces perf(ai): cache local model status probes #541.
• Replaces perf(audio): bulk-write captured WAV samples #543.
• Replaces perf(settings): clean up Ollama pull listeners #553.

[shobhit99]

Reviewed the AI-streaming and native-media consolidation changes (per this PR's stated scope). The six targeted fixes are individually sound and well-tested; my only real reservation is the enormous, mixed review surface.

Strengths

• AI stream IPC batching is correct on every terminal path. createAIStreamIpcCoalescer (src/main/ai-stream-ipc.ts) flushes synchronously before ai-stream-done/ai-stream-error are sent (src/main/main.ts:17631, :17706), before controller.abort() in ai-cancel, and in the finally/replace paths (finishActiveAIRequest at src/main/main.ts:3170, startActiveAIRequest at :3159). Because Electron preserves webContents.send ordering across channels and the flush cancels the pending setTimeout, the final coalesced chunk always arrives before done — no dropped/duplicated/out-of-order chunks and no dangling flush timer.
• finishActiveAIRequest's identity guard (activeAIRequests.get(requestId) === request) fixes a real pre-existing bug where a stale handler's finally could delete a same-id replacement's controller from the map.
• Ollama pulls moved to a separate activeOllamaPullRequests map and ai-cancel still aborts both — cleaner isolation, no cancellation regression.
• Raycast AI.ask() abort-listener leak is properly closed. finalizeAIStream (src/renderer/src/raycast-api/index.tsx:1837) removes the caller-signal abort listener on done, error, aiAsk rejection, and abort. The _activeStreams entry is created before the signal block, so the handler always attaches/cleans up against a real entry.
• Renderer batchers are consistent. ai-chat-stream-buffer and cursorPromptResultBatcher flush on done/error, reset on new streams, and cancel on unmount. useAiChat keeps messagesRef in sync via flushStreamingBuffer() before finalizeConversation() reads it, so the persisted conversation includes the final streamed text.
• Local ASR status TTL cache is safe (src/main/local-model-status-probe.ts): only downloaded/not-downloaded are cached, downloading/error always re-probe, and every explicit flow (download/warmup/transcribe) passes forceRefresh: true.
• Camera mkdir -p removal is safe — verified fs-write-binary-file does fs.promises.mkdir(dirname, { recursive: true }) (src/main/main.ts:15723) before writing, so src/renderer/src/CameraExtension.tsx:362 still creates parents. Switching the preview to a managed object URL (revoked on dispose()) also avoids leaking large base64 data-URL strings.
• WAV bulk-write preserves bytes. Int16(littleEndian:) stored into the [Int16] then re-read via withUnsafeBytes yields identical little-endian output as the old per-sample append on both endiannesses (src/native/audio-capturer.swift).

Concerns / potential bugs

• No correctness bug found in the AI-streaming / native-media scope. The paths above hold up against dropped/duplicated/reordered chunks, abort/error propagation, request-id routing, and listener/timer leaks.
• Undisclosed native behavior change: processBuffer now also computes the level meter when not recording, straight from the input buffer via the new updateMeter(fromInputBuffer:) overloads (src/native/audio-capturer.swift:286-317). This goes beyond the described "bulk WAV write." It appears safe (skips conversion off the record path; if the tap only runs while recording it's just defensive dead code), but it isn't mentioned in the description — worth confirming it's intended.
• Review-surface / scope (process, not a bug): This is a ~17k-line, 100+-file PR that folds in six other PRs plus a large inherited stack. src/renderer/src/raycast-api/index.tsx alone mixes the (in-scope) abort-listener fix with an unrelated full Cache size-index rewrite and a Toast.Style enum→type refactor; main.ts mixes AI batching with exec-command extraction, aerospace, and async clipboard/notes quit-flush. These are outside my review remit and hard to fully vet as one unit — landing them together raises blast radius.
• Overlap with perf(ai): batch streaming and status updates #531: the ai-stream-ipc coalescer is squarely "AI streaming" work and appears to supersede/duplicate perf(ai): batch streaming and status updates #531 — flagging per instructions (not reviewed here). Please make sure only one lands.

Suggestions (optional)

• useCursorPrompt's batcher defaults to requestAnimationFrame; if the cursor-prompt overlay is ever hidden/occluded mid-stream, intermediate visible updates could be throttled. Final correctness is fine (flush() on done/error forces it) — noting only in case live streaming visibility matters there.
• Consider splitting the unrelated Cache/exec/file-search rewrites out of this "AI + native media" PR to shrink the merge blast radius.

🟡 Approve with minor comments — the in-scope AI-streaming and native-media changes look correct and safe; my reservations are the undisclosed meter change and the oversized mixed surface, not defects in the targeted fixes.

🤖 Automated review by Claude Code (Opus 4.8).

[JustYannicc]

Addressed the review follow-up from the AI-streaming/native-media pass.

What changed:

• Tightened src/native/audio-capturer.swift so idle/warmup meter updates no longer use the new raw-input updateMeter(fromInputBuffer:) path.
• Non-recording frames still skip conversion unless the 100ms meter cadence needs an update, preserving the intended overhead reduction.
• When a meter update is needed, it now uses the converted 16 kHz mono target buffer again, matching the prior semantics instead of changing meter behavior by input device format.

Review triage:

• GraphQL reports no review threads and no review submissions on perf(ai): reduce streaming and native media overhead #556; only the top-level review comment is present.
• The idle meter behavior was the only concrete code concern, and this push addresses it directly.
• The useCursorPrompt RAF note remains optional; final done/error flushing is covered by tests.
• For the noted overlap with perf(ai): batch streaming and status updates #531: perf(ai): reduce streaming and native media overhead #556 currently contains the AI stream IPC coalescer work, so that piece should be treated as superseding/overlapping perf(ai): batch streaming and status updates #531 before either PR lands.
• I also acknowledge the large mixed review surface concern. I kept this follow-up scoped to the PR feedback rather than broadening the diff.

Verification:

• node --test scripts/test-ai-stream-ipc-batching.mjs scripts/test-ai-chat-stream-batching.mjs scripts/test-use-ai-stream-batching.mjs scripts/test-raycast-ai-abort-listeners.mjs scripts/test-cursor-prompt-stream-batching.mjs scripts/test-ai-model-status-polling.mjs scripts/test-local-model-status-probe-cache.mjs scripts/test-ollama-pull-listener-cleanup.mjs scripts/test-camera-capture-single-encode.mjs ✅ 43 passing tests
• swift scripts/test-audio-wav-bulk-write.swift ✅ validated byte-for-byte WAV output; bulk path measured 1.70x faster locally
• swiftc -O -framework AVFoundation src/native/audio-capturer.swift -o /tmp/supercmd-audio-capturer-test && /tmp/supercmd-audio-capturer-test <<< '{"command":"ping"}' ✅ emitted {"pong":true}
• git diff --check ✅