perf(ai): batch streaming and status updates

[JustYannicc]

What changed

• Batches visible AI chat stream updates through a small stream buffer and forces final flushes before completion, error persistence, cancellation, and mode transitions.
• Batches Cursor Prompt streamed result rendering while keeping the authoritative result ref current for apply/finalization.
• Batches Raycast compatibility useAI streaming data updates with frame/timer scheduling and synchronous terminal flushes.
• Coalesces Whisper.cpp, Parakeet, and Qwen3 AI model status polling into one poll tick with semantic equality guards before React state updates.

Why

Streaming AI text and settings model status polling previously caused renderer-visible state updates once per token/chunk or once per individual status poll. That created avoidable React work during long responses and while the AI settings tab sat open. The new helpers preserve authoritative data immediately while reducing visible updates to flush cadence or actual semantic status changes.

Compatibility impact

No IPC contracts, public hook return shapes, user-facing strings, or locale files changed. Chat and Cursor Prompt still flush final content before persistence/apply, stream errors still surface with the accumulated text, cancellation/abort paths clear pending repaint work, useAI still calls onData with the final full text, and model download progress still updates because progress fields participate in equality checks.

How tested

• node scripts/test-ai-chat-stream-batching.mjs passed; baseline 240 chunks -> 240 visible updates / 240 layout measurements / 267,795 simulated markdown chars, batched burst -> 1 visible update / 1 layout measurement / 2,290 simulated markdown chars.
• node --test scripts/test-cursor-prompt-stream-batching.mjs passed; baseline 500 chunks -> 500 visible result updates, batched -> 1 visible update, plus final flush/apply-before-visible-flush/cancellation coverage.
• node --test scripts/test-use-ai-stream-batching.mjs passed; baseline 600 chunks -> 600 visible updates, batched -> 1 visible update, real hook harness -> 3 renders / 1 visible data update, plus final data/error/abort coverage.
• node --test scripts/test-ai-model-status-polling.mjs passed; unchanged 5-tick baseline -> 15 state updates / 15 commits, coalesced unchanged -> 0 updates / 0 commits, all-changing -> 5 updates / 5 commits.
• pnpm test passed on this standalone branch: 40 pass / 1 skipped live Electron test.
• pnpm run check:i18n completed on this standalone branch; it still reports existing Italian locale gaps unrelated to this PR.
• git diff --check origin/main..HEAD passed.
• Codex LSP diagnostics were clean on the touched streaming hooks/helpers and status polling helper; AITab.tsx still reports existing unrelated diagnostics outside this diff.
• ./node_modules/.bin/tsc --noEmit --target ES2020 --module ESNext --moduleResolution bundler --jsx react-jsx --strict --esModuleInterop --skipLibCheck src/renderer/src/utils/ai-chat-stream-buffer.ts src/renderer/src/hooks/cursorPromptResultBatcher.ts src/renderer/src/raycast-api/hooks/use-ai.ts src/renderer/src/settings/aiModelStatusPolling.ts passed.

Performance evidence

• node scripts/test-ai-chat-stream-batching.mjs baseline 240 chunks -> 240 visible updates / 240 layout measurements / 267,795 simulated markdown chars, batched burst -> 1 visible update / 1 layout measurement / 2,290 simulated markdown chars.
• node --test scripts/test-cursor-prompt-stream-batching.mjs baseline 500 chunks -> 500 visible result updates, batched -> 1 visible update.
• node --test scripts/test-use-ai-stream-batching.mjs baseline 600 chunks -> 600 visible updates, batched -> 1 visible update, real hook harness -> 3 renders / 1 visible data update.
• node --test scripts/test-ai-model-status-polling.mjs unchanged 5-tick baseline -> 15 state updates / 15 commits, coalesced unchanged -> 0 updates / 0 commits, all-changing -> 5 updates / 5 commits.

Stack validation

• Standalone pnpm exec tsc -p tsconfig.renderer.json --noEmit --pretty false was run and still fails because origin/main has existing renderer typing debt outside this patch area, including App.tsx, CameraExtension.tsx, launcher command model/keyboard controls, Raycast runtime files, AITab.tsx, and quicklink icon typing.
• Stacked locally on codex/consolidate-validation-checks (fix(validation): restore project checks) plus this branch:
  • pnpm exec tsc -p tsconfig.renderer.json --noEmit --pretty false passed.
  • pnpm test passed: 41 pass / 1 skipped live Electron test.
  • pnpm run check:i18n passed.
  • git diff --check codex/consolidate-validation-checks..HEAD passed.

Replaces

• perf(ai): batch chat stream updates #492
• perf(cursor-prompt): batch streaming result updates #504
• perf(ai): batch Raycast useAI streaming #525
• perf(settings): coalesce AI status polling #527

[shobhit99]

Reviewed the full diff. This is a clean, well-tested batching refactor across four AI subsystems (chat stream, cursor prompt, Raycast `useAI`, and settings model-status polling). I traced the behavior-critical paths and could not find a correctness regression.

Strengths

• Authoritative data is always kept current on a ref before the visible flush (`dataRef`/`cursorPromptResultRef`/buffer `getContent`), and every terminal path forces a synchronous flush: chat done/error/cancel/exit, cursor prompt done/error, and `useAI` `.then`/`.catch`. So persistence, `applyCursorPromptResultToEditor`, and `onData(fullText)` all still see complete text.
• `use-ai.ts` setting `dataRef.current = fullText` for stream mode is safe: `StreamingPromise._complete(entry.fullText)` (index.tsx:1670) resolves with `entry.fullText` accumulated from every chunk (index.tsx:1662), so it equals the streamed concatenation — no clobbering, and it guarantees the final visible text is exact.
• Status coalescing equality (`aiModelStatusPolling.ts`) compares all fields of each status type (state/modelName/path/bytes-or-progress/error) against the type defs in electron.d.ts:257-280, so download-progress and error changes still commit; unchanged polls correctly no-op. Failed fetches return `undefined` and `applyAiModelStatusPatch` preserves the prior value rather than clobbering to null.
• `useAiChat` keeps `messagesRef` in sync synchronously via `setMessagesSnapshot`/`updateMessagesSnapshot`, so `finalizeConversation` reading `messagesRef.current` (after `flushStreamingBuffer()`) is actually more correct than the old setState-updater read.
• Unmount cleanup added for all three batchers (`cancel`/`dispose`/`cancelPendingFlush`), preventing setState-after-unmount from a pending flush. Good test coverage for flush/error/abort/cancel and the coalescing matrix.

Concerns / potential bugs

• None blocking. The diff appears behavior-preserving on all the paths I checked (streamed content, final flush, error text with accumulated content, abort/cancel clearing pending repaints, and polling coalescing).

Suggestions (non-blocking)

• Flush mechanisms are inconsistent across the three helpers: `ai-chat-stream-buffer.ts` uses `setTimeout(40ms)`, while `cursorPromptResultBatcher.ts` and `use-ai.ts` use `requestAnimationFrame` (32ms timer fallback). Not a bug, but worth unifying for maintainability.
• The two rAF-based batchers pause intermediate repaints when the window/document is hidden (rAF is throttled while hidden). This is fine here because done/error force a synchronous `flush()` and cursor-prompt apply reads the ref — just flagging it as the thing to keep in mind if either surface ever needs live updates while backgrounded.

Verdict: 🟡 Approve with minor comments

🤖 Automated review by Claude Code (Opus 4.8).

[JustYannicc]

Addressed the remaining PR feedback/proof gap for #531.

What changed:

• Confirmed via GraphQL that PR perf(ai): batch streaming and status updates #531 currently has 0 review threads, 0 submitted reviews, and 1 top-level review comment.
• Addressed the non-blocking maintainability suggestion by adding a shared stream flush scheduler used by AI chat streaming, cursor prompt streaming, and Raycast useAI streaming.
• The shared scheduler keeps the existing terminal synchronous flush behavior, uses rAF for visible-window intermediate repaint batching when available, and falls back to timer flushes while the document is hidden so backgrounded windows still get bounded intermediate updates.
• Added focused hidden-document timer fallback coverage for cursor prompt and Raycast useAI batching.
• Updated the focused TS test importer to bundle local imports, so these tests keep running production source after the scheduler extraction.

Verification on commit d3b1978 and the current follow-up branch:

• node --test scripts/test-ai-chat-stream-batching.mjs scripts/test-cursor-prompt-stream-batching.mjs scripts/test-use-ai-stream-batching.mjs scripts/test-ai-model-status-polling.mjs ✅ 18 passed
• git diff --check ✅

[JustYannicc]

Follow-up after the verification pass: I pushed commit d6a9bf8 to make the scheduler intent explicit for AI chat.

Correction/clarification:

• Cursor prompt and Raycast useAI use the shared scheduler's rAF path while visible, with timer fallback when the document is hidden.
• AI chat now uses the same shared scheduler abstraction but explicitly preserves its existing 40ms timer batching cadence, including when requestAnimationFrame exists. This avoids changing the chat stream repaint window while still removing duplicated scheduler lifecycle code.
• Added a regression check: chat stream batching preserves timer scheduling when requestAnimationFrame exists.

Verification:

• node --test scripts/test-ai-chat-stream-batching.mjs scripts/test-cursor-prompt-stream-batching.mjs scripts/test-use-ai-stream-batching.mjs scripts/test-ai-model-status-polling.mjs ✅ 18 passed
• git diff --check ✅