Skip to content

perf(ai): cache local model status probes#541

Closed
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-ai-model-status-helper-polling-pr
Closed

perf(ai): cache local model status probes#541
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-ai-model-status-helper-polling-pr

Conversation

@JustYannicc

@JustYannicc JustYannicc commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • Cache stable Parakeet and Qwen3 local ASR helper status probe results for a short TTL so the AI Settings poll loop does not launch both helpers every second.
  • Keep downloading and error behavior compatible, and force-refresh download, warmup, and transcription action paths so explicit flows still read fresh helper state.
  • Add focused cache tests with an injected fake slow probe.

Why

  • The AI Settings poll loop was launching both local ASR helpers every second even when helper status was stable.
  • Audit baseline: 60 synchronous trivial helper launches averaged 1.763 ms each, so the two helper polls cost about 3.526 ms of main-process blocking per second when fast.

Compatibility impact

How tested

  • node scripts/test-local-model-status-probe-cache.mjs
  • ./node_modules/.bin/tsc -p tsconfig.main.json
  • On the integration worker branch: node scripts/test-ai-model-status-polling.mjs
  • On the integration worker branch: pnpm run build:main
  • LSP diagnostics for touched TypeScript files passed on the integration worker branch; src/main/local-model-status-probe.ts is clean on this branch. origin/main currently reports unrelated main-process LSP diagnostics for missing/import-mismatched files outside this change.

Performance evidence

  • New fake-probe test on the clean PR branch: iterations=60 legacyCalls=60 legacyMs=105.099 cachedCalls=1 cachedMs=1.813 avoidedCalls=59.
  • Same test on the integration worker branch before the clean cherry-pick: iterations=60 legacyCalls=60 legacyMs=105.153 cachedCalls=1 cachedMs=1.816 avoidedCalls=59.

Stack validation

@JustYannicc

Copy link
Copy Markdown
Collaborator Author

Superseded by #556, which consolidates this AI/native media performance batch.

@JustYannicc JustYannicc closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant