Cold-start follow-ups: remaining lazy-loading candidates from the ACP eager-closure audit

[doudouOUC]

Body

Context

Follow-up to #4748. An esbuild-metafile audit of the ACP child process at de962a5ecf measured an eager static import closure of 17.24 MiB / 2420 modules — all of it parsed and evaluated before the child can answer initialize, on every cold start.

The largest single cluster (telemetry OTLP exporters + NodeSDK + instrumentations, 2.16 MiB / 429 modules) is addressed by the lazy-telemetry-SDK change (facade + dynamically imported implementation). Verified on the 2C4G benchmark host with 30 paired serial cold starts against the same baseline:

Scenario	Metric (P50)	Baseline	Lazy SDK	Delta
telemetry off (default)	process → first session	1922.4 ms	1778.0 ms	−144.4 ms (−7.5%)
telemetry off (default)	cold POST /session	1237.2 ms	1154.8 ms	−82.4 ms
telemetry off (default)	daemon listener ready	687.5 ms	627.4 ms	−60.2 ms
telemetry on (outfile)	channel.initialize	1119.6 ms	1107.3 ms	−12.3 ms

The telemetry-on delta is small by design: the 2.16 MiB still loads, just concurrently instead of on the blocking import chain — and on a 2-core box the concurrent load competes for CPU with session setup (import phase −50 ms, config/bootstrap phase +50 ms). That observation motivates candidates 1 and 2 below.

This issue tracks the remaining candidates from the audit so they aren't lost. Ordered by estimated payoff:

Candidates

1. ✅ Completed — protocol-split the telemetry implementation module. Landed in perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol #7276. Telemetry now dynamically loads only the configured OTLP protocol chain; outfile mode loads neither. On the 2C4G reference host, telemetry-enabled process → first session P50 improved by 50.8 ms and config construction dropped from 45.4 ms to 23.9 ms.
2. ✅ Completed — defer ACP-child telemetry init until after the initialize response is sent. Landed in perf(cli): Defer ACP telemetry initialization #7558. In the final metrics-preserving rerun of 30 alternating paired cold starts, channel.initialize P50 improved by 43.8 ms and the paired median by 44.2 ms; cold process → first session did not show a statistically demonstrated change. Cost: a wider init window during which early spans and one-shot bootstrap events are dropped (same gate semantics as today, just longer).
3. ✅ Completed — lazy @google/genai SDK import. Landed in perf(startup): lazy-load Google GenAI SDK on first use #7512. The SDK now loads on first real use instead of during ACP bootstrap. Maintainer A/B verification measured @google/genai in the ACP static closure at 755,788 → 0 bytes and a total closure reduction of 999,320 bytes; 2C4G validation measured process → first session P50 improving by 65.9 ms and peak RSS by 8 MiB.
4. ✅ Completed — lazy undici loading and proxy-dispatcher assembly. Landed in perf(startup): Load undici lazily behind package-local dynamic imports #7455. All eight value-import sites now load undici through package-local single-flight dynamic imports while preserving proxy installation ordering. The ACP eager closure dropped by 2,057 KiB, process → first session P50 improved by 89.5 ms, and RSS after the first session dropped by 8.1 MB on the 2C4G reference host.
5. ✅ Completed — first-use loading for iconv-lite encoding tables, @xterm/headless, and simple-git (~890 KiB combined). Landed in perf(core): Lazy-load first-use dependencies #7686. On the recorded prototype artifact, the ACP static closure fell from 13,405,027 to 12,314,617 bytes and all three packages reached zero static-closure bytes; on the 2-vCPU reference host, 30 alternating cold pairs improved process → first session P50 from 1877.7 ms to 1733.3 ms, channel.initialize P50 from 896.2 ms to 831.5 ms, and peak process-tree RSS P50 from 417.0 MB to 408.1 MB. The final merged commit was rebased, rebuilt, and guarded locally but did not receive a fresh remote 30-pair rerun; synchronous package-root encoding helpers remain compatible and CI locks these dependencies out of the ACP static closure.
6. ✅ Completed — replace comment-json/esprima with a lightweight jsonc-parser editor. Landed in perf(cli): replace comment-json settings parser #7747. The ACP static closure fell from 12,449,869 to 12,145,099 bytes (−304,770 bytes, −2.45%); on the 2-vCPU reference host, 30 alternating cold pairs improved channel.initialize P50 by 35.39 ms, process → first session response P50 by 38.00 ms, and process → first session complete P50 by 48.51 ms, with 28/30 paired wins and paired-mean bootstrap intervals entirely below zero. Preheated process → session complete behavior was statistically neutral, the 10-second steady-state RSS interval spanned zero, and CI locks comment-json, esprima, and the jsonc-parser UMD build out of the ACP static closure.
7. ✅ Completed — propagate the production serve compile cache to ACP children. Landed in perf(cli): Propagate compile cache to ACP children #7594. On the 2C4G reference host, 30 warm-cache pairs improved median channel.initialize by 176.6 ms and median process → first session by 199.0 ms, with 30/30 paired wins for both. The first empty-cache startup was 117.2 ms slower at the median, so the measured workload recovers the one-time cost on the second ACP startup; median peak process-tree RSS increased by 8.6 MB and the stable cache used 9.4 MB. User-provided, disabled, failed, and unsupported cache configurations remain unchanged, and live-daemon E2E verification confirmed the configured cache reaches ACP children without leaking the internal handoff variable.

Enabler for every step: extend the fast-path bundle guard's forbidden-package lists (as done for the OTLP/gRPC packages) so each newly lazified cluster is locked out of the eager closure by CI.

Acceptance discipline

Bytes do not convert to milliseconds — per the discipline established in #4748, every step must individually pass the 2C4G gate: 30 paired serial cold starts, reporting channel.initialize P50/P95, process → first session, preheated/warm behavior, concurrent first sessions, telemetry on/off compatibility, and peak RSS. Reference points: #7182 removed 1.16 MiB for −63 ms; the lazy telemetry SDK removed 2.08 MiB from the eager closure for −144 ms P50 (telemetry off).

Candidates should land as separate PRs so each one's regression surface stays reviewable and its gain is individually attributable.

[qwen-code-ci-bot]

Triage

• Type: feature (performance enhancement tracking)
• Labels: type/enhancement, category/performance, scope/core, priority/P2
• Next: Accept for exploration — well-scoped follow-up with clear acceptance criteria

Direction assessment: Cold-start latency is a first-class UX concern for a CLI tool. This issue builds directly on accepted prior work (#4748, #7182) and the lazy-telemetry-SDK change already landed. The candidates are ordered by estimated payoff, each scoped as a separate PR with individual regression gates (30 paired serial cold starts, P50/P95 metrics). The acceptance discipline (bytes ≠ milliseconds, each step must prove its own gain) is exactly the right rigor for incremental perf work.

Progress update (2026-07-23): 4 of 7 candidates completed:

• ✅ pre-release: fix ci #1 Protocol-split telemetry (perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol #7276)
• ✅ Where is the config saved? #2 Defer ACP-child telemetry init (perf(cli): Defer ACP telemetry initialization #7558)
• ✅ 如何自定义密钥文件 .env可能与其他文件冲突 #3 Lazy @google/genai SDK (perf(startup): lazy-load Google GenAI SDK on first use #7512)
• ✅ Are you interested in AI Terminal? #4 Lazy undici loading (perf(startup): Load undici lazily behind package-local dynamic imports #7455)
• Open: TypeError in Authentication Selection Interface #5 (iconv-lite/xterm/simple-git), OpenAI API Error: 401 Incorecct API Key provided #6 (esprima), API Key是要设成阿里云的API Key吗？ #7 (compile cache — draft perf(cli): Propagate compile cache to ACP children #7594)

Verdict: ✅ Accept for exploration. Remaining candidates should land as separate PRs per the issue's own discipline.

中文说明

• 类型: feature（性能优化跟踪）
• 标签: type/enhancement, category/performance, scope/core, priority/P2
• 下一步: 接受探索——范围清晰的后续工作，有明确验收标准

方向评估： 冷启动延迟是 CLI 工具的一等 UX 关注点。此 issue 直接承接已接受的前期工作（#4748、#7182）和已合入的 lazy-telemetry-SDK 变更。候选项按预估收益排序，每个作为独立 PR 并有独立回归门控（30 次配对串行冷启动，P50/P95 指标）。验收纪律（字节数≠毫秒数，每步必须证明自身收益）对增量性能工作来说是恰当的严格度。

进度更新（2026-07-23）： 7 个候选项中已完成 4 个：

• ✅ pre-release: fix ci #1 协议拆分遥测（perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol #7276）
• ✅ Where is the config saved? #2 延迟 ACP 子进程遥测初始化（perf(cli): Defer ACP telemetry initialization #7558）
• ✅ 如何自定义密钥文件 .env可能与其他文件冲突 #3 懒加载 @google/genai SDK（perf(startup): lazy-load Google GenAI SDK on first use #7512）
• ✅ Are you interested in AI Terminal? #4 懒加载 undici（perf(startup): Load undici lazily behind package-local dynamic imports #7455）
• 待完成：TypeError in Authentication Selection Interface #5（iconv-lite/xterm/simple-git）、OpenAI API Error: 401 Incorecct API Key provided #6（esprima）、API Key是要设成阿里云的API Key吗？ #7（编译缓存——草案 perf(cli): Propagate compile cache to ACP children #7594）

结论： ✅ 接受探索。剩余候选项应按 issue 自身的纪律作为独立 PR 落地。

— Qwen Code · qwen3.7-max

[doudouOUC]

Additional orthogonal candidate: NODE_COMPILE_CACHE (V8 bytecode cache across spawns)

Re-profiled the ACP child cold start on the same 2C4G benchmark host used for this issue's baseline (2 vCPU, ~3.48 GiB RAM, no swap, bundled Node.js v22.23.1), against the phase2 protocol-split build — i.e. the already-landed lazy-telemetry SDK + OTLP protocol split from #7276. The breakdown reproduces this issue's baseline within noise and surfaces a lever that is orthogonal to every candidate listed above.

Reproduced child-startup profile (telemetry on, 30-run medians)

Stage	ms	share of process_to_response
phase.gemini_import	566	~58%
phase.bootstrap_config_initialization	124	~13%
↳ config.tool_warmup	46
↳ config.hierarchical_memory	19.5
↳ config.skills	16.5
↳ config.tool_registry / extensions_initial / other	~35
phase.process_to_profiler_ready (node boot)	60
phase.acp_import	53
child.unattributed	71
args_parse / config_construction / app_initialization	~65
child.process_to_response	976

This confirms candidate #3 (lazy @google/genai) is the single biggest remaining lever: gemini_import alone is ~58% of the child's process→response, dominated by importing and evaluating the eager module graph.

The new candidate (call it 7)

Every candidate tracked above shrinks the eager import closure (fewer bytes to parse/eval). NODE_COMPILE_CACHE (Node ≥ 22.8, module.enableCompileCache()) is a different axis: it caches the V8 bytecode of whatever remains, so it stacks on top of every lazy-loading step rather than competing with them. It is currently unused in the bundle.

Preliminary A/B on the phase2 protocol-split build, same host, warm cache (1 discarded warmup to populate, then 6 serial cold starts, medians):

Metric	Baseline	NODE_COMPILE_CACHE warm	Delta
process → first session	1240 ms	1051 ms	−189 ms (−15%)
channel.initialize	944 ms	775 ms	−169 ms (−18%)
phase.gemini_import	563 ms	418 ms	−145 ms (−26%)
phase.acp_import	52 ms	32 ms	−20 ms (−38%)
phase.bootstrap_config_initialization	118 ms	112 ms	~0 (execution, not compile — expected)
config.tool_warmup	46 ms	41 ms	~0

The savings land exactly where expected — on the compile portion of the import phases; execution-bound stages (bootstrap_config, tool_warmup) are unchanged. What remains of gemini_import after caching (~418 ms) is module evaluation, which is precisely what candidate #3 attacks. The two are complementary.

Cache footprint ~9.8 MiB, keyed by Node version.

Why this is preliminary, not a gate pass

• 6-run median signal, not the 2C4G acceptance gate. It has not been run through this issue's discipline (30 paired serial cold starts, P50/P95, preheated/warm, concurrent first sessions, telemetry on/off, peak RSS). The numbers above are directional.
• Benefit requires a warm cache. The first-ever spawn writes the cache (small penalty, no gain); the win applies to subsequent spawns — which is the daemon's normal life (ACP child re-spawn on idle timeout, daemon restart). A truly one-shot process sees no benefit.
• Cache invalidation across upgrades. The cache directory must be keyed on the bundle/version so a CLI upgrade never reads stale bytecode. Node keys by Node version + source content; a per-install cache dir under the qwen home is the likely shape.
• phase.process_to_profiler_ready moved the other way (57 → 75 ms) — plausibly early-boot cache-read overhead or noise; the fuller run should characterize it.

If worth pursuing, the natural landing is a small standalone PR (enable a versioned compile cache early in the CLI entry, before the heavy imports), gated on the same 2C4G benchmark as the other candidates here.

[doudouOUC]

Candidate 3 is implemented in #7512.

The PR removes @google/genai from the ACP bootstrap static closure while preserving eager configuration validation and initial Qwen OAuth credential acquisition. Provider construction and logging are single-flight loaded on the first asynchronous model operation; the official MCP adapter loads on discovery. A bundle guard prevents future static regressions.

On the same 2-vCPU host and the then-current main baseline, 30 alternating paired cold starts with telemetry outfile measured:

Metric	Control P50 / P95	Candidate P50 / P95	P50 delta
channel.initialize	984.9 / 1010.6 ms	954.8 / 972.5 ms	-30.1 ms
Process to first session	1924.6 / 1951.1 ms	1858.7 / 1901.0 ms	-65.9 ms
Peak RSS	414.6 / 427.1 MiB	406.5 / 420.5 MiB	-8.0 MiB

The ACP static closure decreased by 999,320 bytes, and the 755,788 bytes attributed directly to @google/genai dropped to zero. A separate 30-pair telemetry-off run with an immediate real prompt showed no demonstrated median cost shift: process-to-first-token P50 improved by 57.6 ms, while P95 remained inconclusive because both variants had unrelated multi-second model-network outliers.

The full design, failure-path audit, test matrix, and raw-result locations are included in the PR. This implements only candidate 3, so #7264 should remain open for the other candidates.