[Feature]: LLM modelConfig strict schema rejects threads / override-tensor / n-cpu-moe / fit-target / batch-size that the addon already supports (4.7 vs 22 tok/s on a 103 GB MoE)

[yuranich]

Problem

loadModel() / qvac serve validate the LLM modelConfig with a strict zod schema (packages/inference/src/schemas/llamacpp-config.ts, applied with .strict() in schemas/load-model.ts). Allowed keys today:

ctx_size temp top_p top_k seed gpu_layers lora device predict system_prompt load_mode verbosity presence_penalty frequency_penalty repeat_penalty stop_sequences parallel tools cache-type-k cache-type-v main-gpu split-mode flash-attn tensor-split reasoning_budget image_tile_mode image_no_upscale mmproj-use-gpu

Everything else is rejected with Unrecognized key, although @qvac/llm-llamacpp forwards unknown keys as --<key> <value> to fabric's own argument parser (LoadFitNormalization.cpp) and its LlamaConfig type has an open [key: string] index signature for exactly that. The keys that matter for running large MoE models on a desktop GPU are all on the rejected side:

Key (fabric flag)	Why it matters
threads / threads-batch	CPU threads for the expert path; default heuristic gives 4 on an 8-P-core Arrow Lake
override-tensor, cpu-moe, n-cpu-moe, n-cpu-ffn	keep routed experts in RAM, everything else on GPU — the only way to run a >VRAM MoE model at speed
fit, fit-target, fit-ctx	automatic version of the above
batch-size, ubatch-size	prompt-processing throughput
moe-cache-mib, prefetch-weights, tensor-read-lazy	qvac-fabric's own features are unreachable from the SDK
no-kv-offload, numa, cpu-mask	placement details

Measured impact (Intel Core Ultra 7 265KF 8P+12E, 128 GB DDR5-4800, RTX 3090 24 GB, Ubuntu 26.04, Vulkan; Qwen3.8-Flash-Next UD-Q4_K_XL, 103.7 GiB MoE, ctx 16384; 44-token prompt → generation tok/s):

Config	gen tok/s
qvac serve, best reachable config (gpu_layers: 12)	4.7
@qvac/llm-llamacpp 0.49.2 direct from Bare, gpu_layers: 99 + override-tensor: "blk\.(1[0-9]|[2-4][0-9])\.ffn_(up|down|gate)_exps=CPU" (default threads = 4)	15
same + threads: 8	22
llama.cpp b10927 CUDA --fit-target 512 -t 8 (reference)	24–26

The same 22 tok/s is what fabric's --fit would choose automatically, but fit does not run in the addon either (#4432), so today the SDK has no route to it at all.

Two smaller papercuts in the same schema: values must be numbers in qvac.config.json ("ctx_size": "16384" → expected number, received string) while the @qvac/llm-llamacpp README says "all parameters must be strings"; and qvac configure has no notion of expert placement.

Proposal

• Make the LLM modelConfig schema .passthrough() for string/number values (the addon already validates unknown flags against fabric's parser and returns InvalidArgument for unknown ones), or explicitly add threads, threads-batch, override-tensor, cpu-moe, n-cpu-moe, n-cpu-ffn, fit, fit-target, fit-ctx, batch-size, ubatch-size, moe-cache-mib, prefetch-weights, tensor-read-lazy.
• Mirror the same keys in the serve.models[].config docs and in qvac configure's guided fields.

Likely component

@qvac/inference (schemas/llamacpp-config.ts, schemas/load-model.ts) and @qvac/cli (qvac configure, HTTP-server docs). No native change needed — the addon already accepts these.

SDK 0.19.1 / CLI 0.13.0 / @qvac/llm-llamacpp 0.49.2 (fabric v10297.1.1) and 0.52.0 (fabric v10549.0.0).