add moh

[wangna11BD]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

New features

PR changes

Models

Description

为DSV4 CSA index增加moh结构

[risemeup1111]

序号	位置	优先级	状态
1	paddleformers/transformers/deepseek_v4/modeling.py:526		🚧

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[risemeup1111]

indexer_moh_bias 是可持久化且会在每个 optimizer step 更新的 buffer，但这里仅在 HF -> Fleet 的正向 AOA 中加入 _ -> ...indexer_moh_bias。_gen_inv_aoa_config 的 decoder 和 MTP indexer 区块都没有把该 buffer 导出到 HF，因此 save_pretrained 不会保存训练后的 bias；下次从 HF 加载时又会执行这个 add primitive，把 bias 重新初始化为 0，丢失负载均衡状态。请在两个逆向 indexer 映射中补上 Fleet -> HF 的对应语句，并增加保存/加载 round-trip 回归测试。