Skip to content

perf(ai): batch Raycast useAI streaming#525

Closed
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-use-ai-stream-batching-pr
Closed

perf(ai): batch Raycast useAI streaming#525
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-use-ai-stream-batching-pr

Conversation

@JustYannicc

@JustYannicc JustYannicc commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • Added a small ref-backed stream batcher for Raycast compatibility useAI, publishing visible data with requestAnimationFrame or a short timer instead of one setData call per streamed chunk.
  • Kept terminal states synchronous: final responses and errors flush pending text immediately, and abort/unmount cancels any pending visible flush.
  • Added a focused Node test for fake __supercmdRaycastAI.ask() streams covering the legacy baseline, batched update count, final data, onData, error flush, and abort cancellation.

Why

useAI previously appended streamed chunks with setData(prev => prev + chunk). For token-sized streams this creates one visible state update and one growing string copy per chunk. The new test records a 600-chunk legacy baseline at 600 visible updates, while the batched path publishes the same text in 1 visible update and the real hook harness renders 3 times total.

Compatibility impact

Raycast useAI behavior remains compatible: data still resolves to the complete final text, onData remains a final full-text callback, onError still receives request errors, non-streaming mode still sets the full text on completion, and aborted requests do not publish stale pending chunks. No user-facing strings or locale files changed.

How tested

  • Baseline before editing: fake __supercmdRaycastAI.ask() legacy stream path in node --test scripts/test-use-ai-stream-batching.mjs reported 600 visible updates for 600 chunks.
  • After fix: the same test reported 1 batched visible update for 600 chunks, and the real useAI hook harness reported 3 renders with 1 visible data update.
  • node --test scripts/test-use-ai-stream-batching.mjs: pass, 5 tests.
  • node --test scripts/test-*.mjs: pass on the scoped origin/main PR branch, 29 pass / 1 skip.
  • ./node_modules/.bin/tsc -p tsconfig.renderer.json: pass on codex/perf-integration-stack; on raw origin/main it still reports existing unrelated renderer errors outside this PR.

Stack validation

This issue was found and validated from codex/perf-integration-stack, which contains the active performance fix stack. The implementation was first validated there with the focused stream batching tests, renderer typecheck, and the full Node integration stack suite: 134 pass / 1 skip. The upstream PR branch is intentionally scoped against origin/main by cherry-picking only this fix commit.

@JustYannicc

Copy link
Copy Markdown
Collaborator Author

Consolidated into #531, which combines the AI chat streaming, Cursor Prompt streaming, Raycast useAI streaming, and AI status polling performance work in one upstream PR. Closing this superseded scoped PR in favor of the consolidated replacement.

@JustYannicc JustYannicc closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant