Add support for GLM-4.7-Flash (glm4_moe_lite)

[Knightkolla]

Closes #850.

Summary

Adds QEfficient support for GLM-4.7-Flash (HF architecture glm4_moe_lite). This ships natively in transformers==5.5.4, already pinned in this repo, so no upstream dependency bump is needed.

glm4_moe_lite is, per HF's own modular source (transformers/models/glm4_moe_lite/modular_glm4_moe_lite.py), a bare pass-through subclass of glm4_moe (GLM-4.5) for rotary embedding, MLP, TopkRouter, RMSNorm, Experts, MoE, decoder layer, Model, and ForCausalLM. The one real architectural difference is attention: Glm4MoeLiteAttention subclasses DeepseekV3Attention upstream, meaning GLM-4.7-Flash uses Multi-head Latent Attention (MLA), the same mechanism DeepSeek-V3 uses, with GLM's interleaved RoPE convention layered on top.

Changes

• New model file: QEfficient/transformers/models/glm4_moe_lite/modeling_glm4_moe_lite.py, following this repo's existing per-family convention (self-contained, mirrors glm4_moe/deepseek_v3's wrapper files, no cross-file class inheritance). Reuses glm4_moe's decoder-loop/MoE/model logic where behavior is identical; writes QEffGlm4MoeLiteAttention fresh, combining DeepSeek's MLA structure with GLM's interleaved RoPE and this repo's precomputed-sin/cos + cache-update conventions.
• Transform wiring: pytorch_transforms.py now includes the new classes in CustomOpsTransform, KVCacheTransform, and OptimizedMoEMapperTransform, plus an opt-out in BlockingAttentionTransform._skip_classes since blocked decode isn't implemented for this attention yet.
• MLA export-shape fix in modeling_auto.py: generalizes a previously DeepSeek-V3-only, exact-architecture-string check (governing the retained-state past_key_values export shape override) to the repo's existing is_mla_model utility.
  • get_padding_shape_from_config can't derive MLA's asymmetric key/value head_dim from a single head_dim config field.
  • DeepSeek-V3 already needed get_dummy_pkv_cache to work around this, and glm4_moe_lite needs the same, but the consuming check was hardcoded to DeepSeek's architecture name only.
  • Generalized to the existing MLA-detection utility instead (verified behavior-preserving for DeepSeek, see Test Plan).
• Test coverage: extends tests/unit_test/models/test_model_quickcheck.py's TINY_MOE_PREFILL_SUBFUNCTION_CONFIGS with a glm4_moe_lite entry (config-derived tiny model, no Hub checkpoint exists for this architecture yet), picked up automatically by the existing test_moe_prefill_subfunction_export_uses_einsum_reductions test.

Scope / What's Deferred

This covers the baseline full-KV decode path only, per this repo's MoE onboarding guidance (start with the plain path, add optimizations only after baseline parity and export stability are proven). Deliberately not included:

• Blocked/chunked decode attention for MLA (opted out via BlockingAttentionTransform._skip_classes so it warns instead of silently ignoring attn_blocking_config, rather than being silently wrong).
• The compressed-cache "absorption" MLA optimization that QEffDeepseekV3Attention.fused_forward* implements for DeepSeek.
• Disaggregated/chunked-prefill serving (SPECIALIZED_DISAGG_SERVING_MODEL_ARCH is not extended).
• Full ORT/ApiRunner-based runtime-parity testing. No stable CPU runtime-parity lane exists yet for MLA/MoE families in this repo's quickcheck (neither DeepSeek-V3 nor plain glm4_moe has one either), so this PR's coverage tier (export-smoke) matches existing precedent. ApiRunner currently can't build correct MLA cache shapes (same root cause as the modeling_auto.py fix above, but in QEfficient/utils/generate_inputs.py's InputHandler, which wasn't touched here to keep this change's blast radius scoped to the new model).

Test Plan

Check	Result
pre-commit run (ruff + ruff-format) on changed/new files	Clean
HF PyTorch vs. QEff PyTorch parity (single-shot forward + 4-step greedy decode vs. HF .generate(), config-derived tiny model, manual script)	Pass
ONNX export smoke (qeff_model.export(), correct retained-state I/O names/shapes)	Pass
pytest test_model_quickcheck.py -k glm4_moe_lite	Pass
pytest test_model_quickcheck.py (full suite)	184 passed, 17 skipped, 1 failed, 1 deselected (~21 min)
pytest test_blocking_transform.py	17/17 pass
pytest test_transform_accuracy.py	176/176 pass

Notes on the full quickcheck run:

• 1 failure (test_causal_subfunction_export_uses_semantic_weight_and_node_names): pre-existing, unrelated to this change. Reproduced identically on main with this PR's changes fully removed (git stash -u), using a plain LlamaConfig, no glm4_moe_lite/MLA involvement. Not fixed here; flagging for a maintainer.
• 1 deselected (test_kimi_k25_quickcheck_hf_qeff_vision_logits_parity): consistently stalls on network I/O for moonshotai/Kimi-K2.5 (trust_remote_code=True remote-code fetch) in this environment, 9+ minutes wall-clock with ~15s of actual CPU time (blocked, not computing). Unrelated to glm4_moe_lite; excluded from this run rather than left to hang. Worth a maintainer re-checking in an environment with better connectivity to that repo.

Known Issues Found (Out of Scope for This PR)

• Latent bug in DeepSeek attention: QEffDeepseekV3Attention's forward/forward_full_kv* methods (QEfficient/transformers/models/deepseek_v3/modeling_deepseek.py) reference self.q_head_dim and self.softmax_scale, which don't exist on transformers==5.5.4's DeepseekV3Attention (renamed to qk_head_dim and scaling upstream). Looks pre-existing, predating the transformers pin moving to 5.5.4. Not touched here since it's orthogonal to this change, but worth a maintainer's attention.
• test_causal_subfunction_export_uses_semantic_weight_and_node_names fails on main today, unrelated to this PR (see Test Plan).
• test_kimi_k25_quickcheck_hf_qeff_vision_logits_parity stalls on network I/O fetching moonshotai/Kimi-K2.5 remote code in this environment, unrelated to this PR (see Test Plan).
• macOS dev environment: torchvision can't install locally on macOS, it's pinned in pyproject.toml to Linux-x86_64-only prebuilt wheel URLs (manylinux_2_28_x86_64), so it silently doesn't install on Darwin. CI (ubuntu-latest) is unaffected. Unrelated to this PR; flagging since it affects anyone developing on a Mac.

AI Assistance Disclosure

This change was developed with Claude Code assistance (exploration, code authoring, and the validation described above). I have reviewed every changed line and can defend this change end-to-end.