Sync with upstream main: Gemma4 video input coexisting with our audio port - #3
Merged
Conversation
* Track the current SDK's SamplingMode.Kind case names FoundationModels renamed GenerationOptions.SamplingMode.Kind's .top/ .nucleus cases to .randomTopK/.randomProbabilityThreshold, which broke compilation against the newer SDK.
…er compiled (ml-explore#438) * Add behavior-neutral generation-event observation for tests Route every channel send through typed emit helpers that also hand a readable GenerationEvent mirror to an optional task-local observer. The observer is nil in shipping builds, so channel sends are unchanged. This gives tests a way to read streamed output that the macOS 27 SDK otherwise keeps opaque. * Make TestResponseStream yield readable GenerationEvent values Attach the executor's observation hook via the task-local and relay its readable mirror events to tests, draining and discarding the opaque SDK channel events. Tests now iterate GenerationEvent instead of pattern- matching the sealed channel actions.
…l's processor (fixes ml-explore#433) (ml-explore#435) The tool-calling path hand-built its LMInput from raw applyChatTemplate output, producing a 1-D [N] token array. LLM models tolerate that (the default LLMModel.prepare adds the batch axis), but every VLM prepare implementation requires [1, N] and indexes dim(1), so any VLM-loaded model with .toolCalling declared aborted with a fatal 'SmallVector out of range' (mlx-c array.cpp:335) the moment the constrained turn began. Route the tool-aware prompt through context.processor.prepare (as the unconstrained and guided paths already do) so the input carries the token rank its model family requires — and so image/video content in the transcript reaches the model instead of being silently dropped. The think-then-call Phase-2 continuation is built by a new rank- and media-preserving helper, with unit tests.
Merge message draft:
---
**Fix `ropeOffset` protocol witness trap in `BaseKVCache`**
`BaseKVCache` does not declare `ropeOffset`, so the `KVCache` extension default (`.scalar(offset)`) is bound as the protocol witness at its conformance. A subclass that declares a property of the same name only shadows it statically: every access through a `KVCache` reference — which is how models read it, e.g. `let offset = cache?.ropeOffset` — still resolves to `.scalar(offset)` and silently ignores the subclass.
Nothing in the tree overrides `ropeOffset` today, so no current behavior changes. But `BaseKVCache` is `open` and this is a trap for any cache that needs a non-scalar offset: a batched cache returning a per-row `.batch(...)` compiles cleanly and is then ignored, feeding every row the same scalar offset.
Declare `ropeOffset` as an `open` class property on `BaseKVCache` (same `.scalar(offset)` default) so it is the real witness and subclasses get a dynamically-dispatched override point.
**Testing**
Added a test that subclasses `BaseKVCache`, overrides `ropeOffset` with a per-row `.batch(...)`, and reads it back through a `KVCache` reference. Without this change the test does not compile ("property does not override any property from its superclass"), which is precisely the defect.
… on the FM-27 SDK (ml-explore#439) The FM-27 beta `.swiftinterface` declares `LanguageModelExecutorGenerationChannel.Response.Action.updateUsage(input:output:metadata: = [:])`, but the shipping FoundationModels dylib exports only the older two-parameter `updateUsage(input:output:)`. Because `emitUsage` relied on the `metadata:` default, the compiler resolved the call to the three-parameter symbol, which does not exist at runtime. dyld cannot bind it, so every `respond()` path crashed (SIGSEGV) the instant generation finished and usage was emitted. Drop the `channel.send(.updateUsage(...))` call, which was the only reference to the missing symbol. A runtime guard cannot help: the compiled reference alone aborts the process at load under chained-fixups linking, before any check runs. The `generationObserver` notification is preserved, so usage is still observable. Restore the send, documented inline, once a later SDK ships a dylib that matches its interface. Verified on-device: UpdateUsageEmissionTests passes on all three respond() paths (unconstrained, guided, tool-calling).
…lore#440) Set OS_ACTIVITY_MODE=disable on the IntegrationTesting scheme TestAction to suppress log spam from MLX runtime shader JIT compilation, e.g.: [Metal Compiler Warning] Warning: Compilation succeeded with: mlx/backend/metal/kernels/defines.h:10:32: warning: unused variable MAX_REDUCE_SPECIALIZED_DIMS [-Wunused-const-variable] static MTL_CONST constexpr int MAX_REDUCE_SPECIALIZED_DIMS = 4; ^ (and similarly for REDUCE_N_READS, REDUCE_N_WRITES, SOFTMAX_N_READS, RMS_N_READS, RMS_LOOPED_LIMIT) These come from mlx-swift vendored MLX C++/Metal sources, not this repo.
* Add TurboQuant KV cache compression Walsh-Hadamard rotation + Lloyd-Max codebook KV compression behind the kvScheme parameter (ml-explore#230), with JIT Metal kernels: fused encode, a SIMD-parallel two-pass online-softmax flash decode (packed-K, raw-fp16-K, and 8-bit-affine-K score variants), and an inline sparse value kernel. Schemes are asymmetric because keys are far more quality-sensitive than values (softmax amplifies key error, value error averages out): the recommended tiers keep K at fp16 (turbo0v4/v3/v2) or 8-bit affine (turbo8v4/v3/v2) while V takes the compression; symmetric turbo4/3/2 compress both sides. Symmetric key quality rides on per-dimension key calibration: at compression time the cache derives a per-dimension scale from the post-rotation key variance of the prefill cache and folds it into the query side, leaving the score kernels untouched; the calibrated encode rotates through the codec's MLX ops so kernel and reference quantize identical values. Boundary-layer protection converts the first and last attention layers to 8-bit affine for fragile schemes. Rotating (sliding-window) cache layers stay fp16 with a one-time notice; global layers on mixed models compress normally. Per-call kernel values travel in a parameter buffer rather than template arguments, so each shape compiles once. Caches stay raw fp16 through prefill, compress on the first decode step, and encode incrementally; state round-trips through prompt cache save/restore, and speculative decoding routes through the compressed path. Generation settles cache state with the token eval and drains autoreleased temporaries per step so long generations hold steady memory. * Add TurboQuant tests Codec round trips and pinned codebooks, rotation orthogonality, packing, kernel-vs-reference attention parity (GQA repeat factors 1/2/3, non-power-of-two head dims), calibrated-encode kernel exact index parity, prompt-cache round trips for all state layouts including calibration state, compressed partial trim with continued decode, speculative decoding with a turbo scheme, allocation-boundary growth, boundary-protection gating, mixed rotating/standard cache conversion, and end-to-end generation on tiny random-weight models. CI-runnable, no checkpoint downloads. * Add KV cache quantization documentation Scheme reference table, the asymmetric-first selection ladder, measured family sensitivity and calibration notes, performance guidance, and limitations.
…#387) Expose Gemma3 encoder surface behind @_spi(GemmaEncoder) for frozen-text-encoder use Multimodal generators (e.g. Lightricks LTX-2) increasingly condition on ALL hidden states of a frozen LLM text encoder rather than generated tokens. Since an encoder-style all-hidden-states tap uses a single uniform mask that bypasses Gemma3's per-layer sliding-window/global mask selection, it diverges from generic Gemma3 semantics for long texts — so it doesn't belong in MLXLLM itself. Instead, this exposes the minimal surface needed for client code to implement that tap: Gemma3TextConfiguration.hiddenSize, .hiddenLayers Gemma3TransformerBlock (class) and .callAsFunction(_:mask:cache:) Gemma3Model.embedTokens, .layers, .config All seven are gated @_spi(GemmaEncoder) public (not plain public) so they're usable via @_spi(GemmaEncoder) import MLXLLM without becoming advertised public API. GemmaEncoder is a new SPI group, distinct from the existing test-only @_spi(Testing). No behavior changes — access levels, attributes, and doc comments only. Gemma3EncoderAccessTests proves sufficiency by implementing the allHiddenStates tap in test code using only the exposed surface (no @testable), checking state count (numHiddenLayers + 1), (B, T, hiddenSize) shapes, first state == scaled embedding, and determinism on a tiny randomly-initialized model.
* Optimize Qwen3.5 interleaved M-RoPE * Optimize Qwen3-VL interleaved M-RoPE
Brings the MLXVLM Gemma4 MoE implementation to the LLM text path so the 26B-A4B (MoE) checkpoint loads through MLXLLM, not just MLXVLM. Adds Gemma4TextRouter, Gemma4TextExperts (SwitchGLU), the enable_moe_block config + dense/sparse decoder branch, and the fused-expert weight remap in sanitize. The E-series KV-sharing it pairs with is already upstream (ml-explore#342). Ported from MLXVLM/Models/Gemma4.swift (ml-explore PRs ml-explore#180, ml-explore#228).
… pair (ml-explore#445) Fix MTP integration test skip semantics; pin 31B checkpoints for fixture parity - Convert checkpoint-gated tests from `Issue.record()+return` (which Swift Testing treats as a failure, not a skip) to auto-download via the existing `Downloader` mechanism, matching the pattern used elsewhere in the suite. - Document that MTPRung4TokenParityTests' case_02_block4/case_03_block6 failures stem from stale fixtures (pre-dating a checkpoint update), not a Swift bug — confirmed by bit-for-bit match against a live mlx-vlm re-run. - Pin the 31B target/drafter downloads to the exact commits live at fixture-generation time, restoring parity for all 3 Rung4TokenParityTests without regenerating fixtures. Verified weight blobs are unchanged at the pins via HF's per-commit LFS-OID listing. - Add optional `targetRevision`/`drafterRevision` params to `loadTargetAndDrafter` so 31B call sites can pin while env-gated E4B/E2B sites stay on latest. - Trim checkpoint-pin comments to self-contained one-liners.
…t + gemma4_unified_assistant drafter) (ml-explore#415) * Add gemma4_unified_assistant MTP drafter support (12B target) The 12B Gemma 4 (model_type gemma4_unified, class Gemma4Unified) could not use MTP speculative decoding: - Its drafter (google/gemma-4-12B-it-assistant and the mlx-community conversions) ships model_type gemma4_unified_assistant, which was not registered with MTPDrafterTypeRegistry. - Gemma4Unified did not override callAsFunction(_:cache:state:), so the mtpEmitFlagKey opt-in was discarded by the protocol-extension default and the target never emitted drafter state — the MTP iterator silently fell back to single-token passthrough. - Gemma4AssistantDraftModel.draftBlock only accepted a Gemma4 target and trapped on Gemma4Unified. Fix: register gemma4_unified_assistant with the same creator (the drafter implementation is shared; the unified variant differs only in its text_config, which Gemma4TextConfiguration already decodes), add the MTP-aware state entry point to Gemma4Unified (mirroring Gemma4), and accept Gemma4Unified in draftBlock's target dispatch. Tests: config-decode pin mirroring the published 12B assistant config, and a synthetic end-to-end pin (Gemma4Unified emits state through the MTP entry point; draftBlock accepts the unified target and returns [1, blockSize-1]). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Make draftBlock target-agnostic via Gemma4BackboneProviding Replace the concrete-class switch in Gemma4AssistantDraftModel.draftBlock (case Gemma4 / case Gemma4Unified) with a cast to a new internal Gemma4BackboneProviding protocol exposing the shared text backbone. Adopted by Gemma4 and Gemma4Unified in Gemma4.swift, where the conformances can reach the private languageModel — this also reverts the visibility widening of Gemma4Unified.languageModel from the previous commit. Existing trap semantics for non-Gemma4 targets are kept.
…xplore#432) (ml-explore#434) Fix tool-schema $defs resolution for grammar compilation in tool-calling envelopes. Problem: Named sub-schemas (nested @Generable/DynamicGenerationSchema types) get serialized as root-level $defs with root-anchored #/$defs/... refs. When a tool's parameter schema is embedded under oneOf[i].properties.arguments, its $defs got buried there too, but refs still pointed to the envelope root — causing xgrammar's grammar compilation to fail (constraintCompilationFailed) for any tool with non-flat argument schemas. Fix: Hoist each tool's $defs to the envelope root, namespacing them per tool (<tool>__<def>) and rewriting that tool's refs to match. Applied once in toolCallingEnvelopeObject, covering both envelope and structural-tag encoders. Refinement (review feedback): The ref rewrite was made structure-aware — instead of a raw string find/replace, it walks the parsed schema tree and only rewrites string values under $ref keys. This prevents corrupting unrelated strings (e.g., a @Guide description that happens to mention #/$defs/Foo), and correctly handles refs nested inside other $defs bodies. Added a doubly-nested test fixture and a pinning test for the description-string edge case. Fixes ml-explore#432
…l-explore#455) The vision Attention materialized a [1, L, L] additive -1e9 mask over the concatenated patch sequence of every image and fell back to unfused SDPA (head dim 72 is outside the fused kernel's 64/80/128), so attention cost 16 * (4 * pads)^2 * 2 bytes in one Metal buffer: 19.3GB at a 6144-token image, and a two-image 8146-token request exceeded maxBufferLength. The mask was block-diagonal per image, so attend each cu_seqlens segment independently with no mask (mlx-vlm's Python implementation does the same), and zero-pad Q/K/V head dim 72 -> 80 to dispatch the fused kernel (the gemma4EnsureFusedSDPA trick; padding is exact, scale is explicit). Measured on Qwen3.5-9B, M4 Pro: single 6120-pad image peak 28.7 -> 12.5GB, two-image 8140-pad request fatal -> 14.2GB, prefill 59.8 -> 36.3s, outputs unchanged.
…ing (ml-explore#395) The mlx-community Gemma 4 QAT checkpoints (gemma-4-12B-it-qat-4bit, gemma-4-E4B-it-qat-4bit) quantize attention and embeddings at 4-bit and every mlp.{gate,up,down}_proj at 8-bit, declared as per-module overrides interleaved with the global settings in config.json's quantization dict. Loading these checkpoints works today: QuantizationContainer decodes the interleaved dict, PerLayerQuantization resolves each module path, and loadWeights quantizes each module with its per-module bits (mirroring Python mlx-lm's class_predicate). But nothing exercised that path end to end — only the config-decode unit tests in BaseConfigurationTests — and reports like ollama/ollama#16740 attribute empty/garbage generation to loaders mis-decoding the 8-bit MLP payload as 4-bit, so it is behavior worth pinning. Add MixedPrecisionQuantLoadTests, which builds a tiny gemma4_unified model, quantizes its own weights into a synthetic checkpoint with the QAT layout (8-bit MLP / 4-bit rest), and verifies through loadWeights that: - per-module overrides are honored (mlp projections come back as 8-bit QuantizedLinear, everything else 4-bit), and - a load that ignores the overrides (global 4-bit only) fails loudly on the packed-shape mismatch instead of silently mis-decoding the payload. Validated to bite: with the per-layer lookup in loadWeights replaced by the global default, the first test fails with mismatchedSize on layers.0.mlp.gate_proj ([128, 8] vs [128, 16]). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ion call (ml-explore#457) Two bugs on the cached path, so DeepSeek-V3 could not generate. 1. `kvHeads` was declared `[Int] = []` and never assigned. `newCache` derives the layer count from `kvHeads.count`, so it returned an empty array, and the first `cache?[i]` in the layer loop trapped with "Index out of range". Any generation crashed; only a `cache: nil` forward pass worked. Populate it the way the other models do: one entry per layer. 2. Attention called `cache.update(...)` explicitly and then passed the resulting keys/values to `attentionWithCacheUpdate`, which performs the update itself. The second update appended the already-returned history back into the cache, so a prompt of L tokens left the cache at 2L and every subsequent token attended over duplicated keys. Drop the explicit update and let the helper own the single one, matching Llama/Qwen3/Cohere and the Python reference. This also routes the quantized-cache path correctly, which the manual update bypassed. Adds DeepseekV3Tests with a cache-growth regression test: a 5-token forward pass must leave every layer's cache at offset 5. With (1) unfixed the test crashes; with (1) fixed and (2) unfixed it reports 10. Reported by @davidkoski in ml-explore#379, where the same double-update was found in the DeepSeek-V2 port. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add multi-round tool calling to MLXFoundationModels: replay prior tool calls/results into the prompt and keep the tool path active each round until the model answers, matching LanguageModelSession/ChatSession semantics. Along the way, fixed several correctness issues surfaced by new characterization tests: - Degenerate/non-deterministic outputs from Gemma and Qwen (thinking-mode collisions, grammar ordering, greedy free-text decoding). - Tool-call grammar/tokenizer boundary bugs (EOS residuals, LFM2 prefix duplication, unfinished protocol tails, ordered-output streaming). - Required/disallowed tool-mode handling, native allowed-tool routing, and removal of the synthetic final-answer tool in favor of native structured outputs. - Foundation Models sampling-seed preservation and propagation. Added a dormant diagnostic sink for guided-generation tracing, plus new characterization and integration tests (tool-call grammar, structured outputs, seed forwarding), with SDK/compile fixes to keep the FoundationModels integration target building.
Add DeepSeek-V2 model
Port of mlx-lm's deepseek_v2.py, reusing the in-tree V3 MLA machinery (q_a/q_b + kv_a_with_mqa/kv_b compression, qk nope/rope split, YaRN rope) and SwitchGLU MoE with shared experts. Router differs from V3: plain softmax scoring, optional group-limited-greedy masking, unconditional routed_scaling_factor. Dense layers below first_k_dense_replace use standard MLP.
Fixes along the way:
- Make `qLoraRank` optional (`Int?`); V2-Lite sets `q_lora_rank: null` and needs direct q_proj instead of the q_a/q_b LoRA path. Fixes loading mlx-community/DeepSeek-V2-Lite-Chat-4bit-mlx.
- Update the KV cache once per attention call instead of twice — the explicit `cache.update()` plus `attentionWithCacheUpdate`'s internal update was duplicating cached keys/values, doubling cache size and corrupting attention beyond the first token.
- Fix `topk_method` default typo ("gready" → "greedy"), cosmetic only.
Includes tests for logits shape, group-limited-greedy routing, expert-stacking in `sanitize()`, qLoraRank parsing, and KV cache growth.
…l-explore#347) Add Hunyuan dense V1 (hunyuan_v1_dense): Hunyuan-MT-7B and Hy-MT2-7B Adds Tencent's Hunyuan dense V1 architecture (HunYuanDenseV1ForCausalLM), used by translation models Hunyuan-MT-7B and Hy-MT2-7B. A Llama-family dense transformer close to the Qwen3 path: per-head QK RMSNorm (applied after RoPE, matching the mlx-lm reference), GQA, SwiGLU MLP, pre/post RMSNorm blocks, tied embeddings, and DynamicNTKAlphaRoPE (rescales RoPE base by alpha^(dim/(dim-2)), reusing the existing freqs-based fast-RoPE path). Changes: - RoPEUtils.swift: DynamicNTKAlphaRoPE - Models/Hunyuan.swift: model/config (flat decode, head_dim/attention_head_dim alias, conditional qk-norm) - LLMModelFactory.swift: register hunyuan_v1_dense; presets for both models (4-bit/8-bit) - HunyuanTests.swift: config decode, sanitize, forward, dynamic-RoPE, presets Validated byte-identical greedy output against mlx-lm reference on locally converted 4-bit weights for both models.
* Add GitHub workflow for IntegrationTesting tests Runs the IntegrationTestingTests target manually (workflow_dispatch) and nightly (schedule) on the self-hosted macOS runner. Kept off the PR path since the tests download HF checkpoints, require Metal, and are long-running. Uses -parallel-testing-enabled NO to avoid concurrent workers racing on the shared on-disk Hugging Face cache.
…re#391) Add video input to base Gemma4 (gemma4) VLM Gemma4 has no separate video encoder — video frames run through the same vision tower as images and scatter onto `<video>` soft-token positions. Model: - `Gemma4Configuration` decodes `video_token_id` and `vision_soft_tokens_per_video_frame` (nil/default preserves old text+vision behavior) - `getInputEmbeddings` accepts `videoPixelValues`; image/video paths share a new `scatterVisionFeatures`/`scatterVideoFeatures` helpers; per-layer mask excludes video soft tokens - `prepare` routes `input.video?.pixels` and passes real `videoTokenId` - No weight/sanitize changes — video reuses the vision tower Processor: - `Gemma4ProcessorConfiguration` decodes `video_token_id`, `video_soft_tokens_per_frame`, `video_max_frames`, and `videoFixedSize` - `Gemma4Processor.processVideos` samples ≤32 frames at ~1fps and preprocesses to a stacked `[frames, C, H, W]` tensor - `Gemma4Processor.prepare` populates `LMInput.video` and expands `<video>` placeholders per frame Tests (`Gemma4VideoInputTests`): config decode, image/video logit equivalence, multi-frame truncation, processor frame extraction. `Gemma4ChunkedPrefillTests`/`Gemma4UnifiedTests` still pass. Follow-ups: per-frame `mm:ss` timestamps; encoder-free `Gemma4Unified` processor video block.
The nightly IntegrationTesting job failed to compile on the Xcode 26.5 runner: the FoundationModels adapter (MLXFoundationModels) is gated behind canImport(FoundationModels, _version: 2) (macOS 27 SDK only), but the integration test files gated only on the always-set FoundationModelsIntegration trait, so they referenced symbols absent on the 26 SDK. - Extend the 37 FoundationModels-gated test files' top-level guard to '#if FoundationModelsIntegration && canImport(FoundationModels, _version: 2)', mirroring the library so they compile out on the 26 SDK and stay active on 27. - Workflow: prefer Xcode 27 (via DEVELOPER_DIR) when the runner has it so the full suite runs; otherwise fall back to the default toolchain and run the SDK-agnostic suites (MTP, Qwen3VL/Qwen3.5 vision, Coherence, Gemma4, tool calls).
… new models Brings the 24 upstream commits past our shared lineage, most notably: - end-to-end video input for Gemma 4 (ml-explore#391) — merged to COEXIST with our audio port: config/CodingKeys/decode unions, both scatter paths in getInputEmbeddings (audio block then video block), both placeholder expansions in prepare(input:), LMInput carries image+video+audio - upstream MTP enablement for 12B unified (ml-explore#415), Gemma4Text MoE (ml-explore#364) - effectiveEOSTokenIds (nested text configs, ml-explore#449) — adopted in both factories, keeping our sampling-params surfacing - per-layer-input masking generalized to image/audio/video soft tokens (upstream's multimodalMask supersedes our image+audio version) - unified multimodalTokenCountMismatch error replaces imageTokenCountMismatch - Hunyuan V1, DeepSeek-V2, TurboQuant KV compression, FoundationModels fixes, Qwen3.5 M-RoPE optimization Load.swift verification collision resolved strict-by-default: upstream's regression (ml-explore#395) requires verify .all so mismatched quantized shapes fail loudly; our mxfp8 relaxation now applies ONLY when an mxfp8 layer was actually built (its shapes legitimately differ until _updateInternal reconciles). swift test: 398 tests, 4 issues — all pre-existing Gemma4 float-tolerance cases (identical on main before this merge). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Satchitananda
added a commit
that referenced
this pull request
Jul 26, 2026
Sync with upstream main: Gemma4 video input coexisting with our audio port
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges the 24 upstream commits our lineage was missing (upstream tip
3cbf928). The two stacks now coexist:Headline reconciliations
getInputEmbeddingsruns the audio scatter block then the video scatter block;prepare(input:)performs both placeholder expansions;LMInputcarries image + video + audio. Upstream's generalized per-layer-inputmultimodalMask(image/audio/video) supersedes our image+audio version, and their unifiedmultimodalTokenCountMismatcherror replacesimageTokenCountMismatch.verify: [.all]so mismatched quantized shapes fail loudly (upstream's new regression test passes), relaxing to.noUnusedKeysonly when an mxfp8 layer was actually built (its shapes legitimately differ until_updateInternalreconciles the real checkpoint data).effectiveEOSTokenIds(Load EOS token IDs from nested text configs ml-explore/mlx-swift-lm#449) adopted in both factories, keeping our sampling-params surfacing via ModelContext.Also inherited: upstream MTP enablement for 12B unified (ml-explore#415), Gemma4Text MoE (ml-explore#364), Hunyuan V1, DeepSeek-V2, TurboQuant KV compression (ml-explore#232), Qwen3.5 M-RoPE optimization, FoundationModels fixes, integration-test infra.
Verification
swift buildclean.swift test: 398 tests / 51 suites, 5 issues — the same pre-existing Gemma4 chunked-prefill float-tolerance cases that fail identically on main before this merge. The new upstreamMixedPrecisionQuantLoadTestssuite passes with the verification resolution above.swift buildclean; SightRoll dev app BUILD SUCCEEDED against this branch.Post-merge
3.31.4-fork.2(or wait for upstream's next release tag to align exactly).