GB10 real-time serving: opt-in quantization + serve-path optimizations (2.2x: 101.7 -> 46.5 ms/frame) - #103

Open

jethac wants to merge 11 commits into

NVIDIA:mainfrom

jethac:pr-gb10-realtime

Commits on Jul 24, 2026

GB10 installability: relax torch pin; zero-init checkpoint-missing keys in meta-device load

committed
fb791db
fix: cast voice clone audio to model dtype before Mimi encode

amarrmb
authored and
jethac
committed
b97e02b
feat: add FP8 quantization, skip other_mimi, pinned memory, mimi compile

amarrmb
authored and
jethac
committed
3f70528
gate: put fp8 / skip-other-mimi / pinned-memory / mimi-compile behind opt-in flags

jethac
and
claude
committed
a587d9a
LMGen: optional depformer early exit (--dep-q-exit); output-invariant in serve flow

jethac
and
claude
committed
d41edaa
offline: --skip-other-mimi and --mimi-fp16 flags

committed
270ad6f
quantization scaffolding + --fp8 flag wiring

committed
055b3f5
w8a16: weight-only 8-bit LM quantization with Triton dequant-in-register GEMV + --w8a16/--fast presets

jethac
and
claude
committed
e8f33a6
GB10 discoverability hint: one info log when no perf flag is active on sm_121

jethac
and
claude
committed
7e2de4a

Commits on Aug 3, 2026