[Feat] Add Sampled-token OPD Support to RL Training - #1977

Open

YanhuiDua wants to merge 16 commits into

InternLM:mainfrom

YanhuiDua:dev-opd

Commits on Aug 17, 2026

[Refactor] Move rollout filtering into agent loops
YanhuiDua
committed
3a0f1e8
[Feat] Add Teacher logprob client to AgentLoop
YanhuiDua
committed
5ea5763
[Feat] Add PG-OPD token advantages
YanhuiDua
committed
e307f86
[Feat] Propagate PG-OPD config to training agent loops
YanhuiDua
committed
8c6f4dd
[Fix] use old_logprobs instead of rollout_olgprobs to compute reverse kl
YanhuiDua
committed
f24a42c
add eval in opd config
YanhuiDua
committed
961e243
add lmdeploy backend
YanhuiDua
committed
b6c002e
add OPDTeacherLaunchConfig, multi-teacher opd config and scripts
YanhuiDua
committed
ad31104
refactor cuda_visible_devices to global_device_rank to support multi-node
YanhuiDua
committed
b769e75
fix mm input
YanhuiDua
committed
8c02bad
support teacher replicas with round-robin scoring
YanhuiDua
committed
1d21b83
fix when reward is None
YanhuiDua
committed
d635683
support prefix cache
YanhuiDua
committed
646d9a0
add log_level in OPDTeacherLaunchConfig
YanhuiDua
committed
d5e8972
add teacher session router
YanhuiDua
committed
c77eddc

Commits on Aug 21, 2026

refactor to distillation loss and compute teacher logprob on training worker
YanhuiDua
committed
0eea4a9