metal: reduce decode synchronization for DeepSeek V4.1 Flash SSD streaming

[Dango233]

This reduces per-layer waits during single-host DeepSeek V4.1 Flash SSD decode on Metal. The switch is off by default and excludes quality/imatrix mode. The queue drains after layer 13 before shared Engram input reuse, and at token completion; expert-ID readback and cache-replacement synchronization remain. After the upstream CUDA addition, the single-host opt-in condition is explicitly macOS-only.

Enable with DS4_METAL_ENABLE_V41_STREAM_DECODE_QUEUE=1.

Revalidation against current upstream

Measured on 2026-09-14 after merging a04f46fa423e45712c8c7e430eff422479f314a3 (DeepSeek V4.1 CUDA support). M2 Ultra, 192 GiB, macOS 15.7.4, Metal; calibrated V4.1 Flash IQ2_XXS/Q2_K. Same binary per comparison, 2,048-token prompt, 8,257 allocated context, automatic expert cache, fresh engine per run; OS/file cache was not flushed.

Both arms use DS4_METAL_DISABLE_STREAMING_EXPERT_SLABS=1 to isolate this change. No companion optimization is enabled.

Mean 512-token decode: 10.170 → 11.315 tokens/s (+11.3%).

Run	Switch	Prefill t/s	Decode t/s
1	off	67.40	10.13
2	on	66.23	11.30
3	on	66.39	11.33
4	off	68.84	10.21

All four decoded outputs match. These are two observations per variant on one host. Original 2026-09-13 measurements remain in the report.

Validation and limits

The real-weight forced-eviction parity test passes under Metal API validation: positions 121–144 compare full logits, finite values, Engram history and all saved cache spans. The 512-step balanced same-engine harness checks every logit and selected token.

Clean Metal build, CPU compilation, frontend/Engram/GGUF/quality-tool unit tests and seven targeted Metal kernel subtests pass. The full Metal kernel suite is not green: the new router check fails at tests/test_deepseek41_metal.c:105; pristine upstream reproduces the exact same actual/reference values. No kernel or tolerance was changed to bypass it.

The two existing SDK 15 unused Metal 4 symbol warnings remain. Full legacy make test and CUDA/ROCm hardware validation were not run.

Commands, model hash, raw CSVs and full results.

[adriangalilea]

Resident single-box data point for the same change, since the PR's opt-in condition is g->streaming only: on a Mac Studio M3 Ultra 512 GB with the Q4 GGUF fully resident (ds41_graph_step, no streaming, --temp 0), lifting the per-layer drain the same way (drain at layer 13 and the last layer only) gives:

prompt	drain every layer	queued	Δ
2k, greedy 128 (ABBA)	17.08 / 16.78 t/s	21.73 / 21.72 t/s	+28 %
16k, greedy 128 (ABBA)	16.67 / 16.68	21.47 / 21.42	+29 %
2k, greedy 512	16.68	21.60	+29 %

Greedy output byte-identical in every pair (md5 of the generated text equal across all runs). Per-token GPU stage timestamps on the resident path explain the size of the gain: 37.3 ms of GPU busy inside a 46.9 ms span with the per-layer drains (9.6 ms of scheduling gap per token, 40 waits), 36.9 ms busy in a 37.7 ms span queued (0.75 ms gap).

So the resident path benefits more than the streaming one (+28 % vs your +11 %), and there is nothing streaming-specific in the mechanism: the expert-ID readback and cache-replacement syncs you mention stay where they are. Suggestion: enable the queue for every single-host non-quality decode (drop the g->streaming term) and, given the TP path already queues by default behind DS4_METAL_DISABLE_V41_TP_DECODE_QUEUE, make the single-host one default-on behind a matching DS4_METAL_DISABLE_V41_DECODE_QUEUE. That is the shape running on our fork (adriangalilea/ds4 branch v41-decode-profile, commit 6768711, which also carries the stage timestamps for the V4.1 graph); happy to fold either into this PR or open it separately, whichever is easier for review.