feature(zsh): migrate URSA-MATH stage3 training to LightRFT - #53

Open

HansBug wants to merge 40 commits into

opendilab:mainfrom

HansBug:dev/math_prm_train

Commits on Mar 20, 2026

feature(math_prm): keep minimal upstream stage3 path
HansBug
committed
fec2744
fix(math_prm): sync stage3 rollout updates from working branch
HansBug
committed
d8590af

Commits on Mar 22, 2026

fix(wandb): remove live heartbeat logging
HansBug
committed
3ff0caf

Commits on Mar 26, 2026

fix(math_prm): sync runtime eval updates from working branch
HansBug
committed
6ae4d56
style(math_prm): remove trailing whitespace from stage3 files
HansBug
committed
3f5470a

Commits on Mar 31, 2026

fix(strategy): reload keep-on-gpu rollout actor after sync
HansBug
committed
a81a817

Commits on Apr 27, 2026

fix(math_prm): sync stage3 training path from working branch

HansBug
and
claude
committed
902050a
fix(math_prm): bring stage3 doc/training/runtime updates from working branch

HansBug
and
claude
committed
a36c860
Merge remote-tracking branch 'main/main' into dev/math_prm_train
HansBug
committed
d25d64e
fix(math_prm): address PR #53 review comments

HansBug
and
claude
committed
34e97b1
fix(profile): restore profile_recorder.py and re-export StepProfileRecorder

HansBug
and
claude
committed
ceb4441

Commits on Apr 29, 2026

fix(math_prm): K1 KL estimator + freeze fix + ratio diagnostics

HansBug
and
claude
committed
b905c23

Commits on Apr 30, 2026

fix(math_prm): align URSA log_probs with VLM forward and revert KL hacks

HansBug
and
claude
committed
67c1d31

Commits on May 5, 2026

fix(math_prm): protect PRM from actor-leaked image tokens + add resume support

HansBug
and
claude
committed
cce5ae5

Commits on May 7, 2026

fix(math_prm): align rollout/eval with bare HF generate; gate EOS patch behind opt-in flag

HansBug
and
claude
committed
3c54da8

Commits on May 8, 2026

feat(math_prm): add per-step PRM reward path (URSA paper variant 2)

HansBug
and
claude
committed
f23e687
fix(math_prm): URSA-native step-boundary alignment for per-step PRM reward

HansBug
and
claude
committed
95ee722
feat(math_prm): add --per_step_reward_mode {raw,group_norm} for variant 2

HansBug
and
claude
committed
2220049
chore(math_prm): wire .env auto-source + PER_STEP_REWARD_MODE into official shell

HansBug
and
claude
committed
f6987cf
chore(math_prm): map LIGHTRFT_WANDB_API_KEY -> WANDB_API_KEY in launch shell

HansBug
and
claude
committed
4eda088

Commits on May 11, 2026

fix(math_prm): protect actor.forward from actor-leaked <|image|> tokens

HansBug
and
claude
committed
9e5cbd5
fix(math_prm): cap eval DataLoader bs by local_hf_generate_max_batch_size

HansBug
and
claude
committed
0fca7c1

Commits on May 18, 2026

chore: route math prm outputs to configured root
HansBug
committed
3b45ea9
fix(math_prm): keep URSA logprob forward cache-free
HansBug
committed
f901761

Commits on May 25, 2026

Commits on Jun 3, 2026

Merge upstream/main into dev/math_prm_train
HansBug
committed
230977b
math_prm: address Agent Review #1 — clean debug artifacts + README results
HansBug
committed
956a850
math_prm: address Agent Review #2 — clean stale docs + explicit register
HansBug
committed
215ba1a
math_prm: drop inert --per_step_reward_mode CLI arg from variant 2 launcher
HansBug
committed
4b7ab05
style(lightrft): yapf reformat 7 files flagged by format-check

HansBug
and
claude
committed
95cb755
fix(lightrft): drop merge leftovers that broke flake8

HansBug
and
claude
committed
11c3b4e
fix(lightrft): restore generate_fn def + suppress zero general_model_reward print

HansBug
and
claude
committed
bbfdaa8
fix(lightrft): narrow abs-zero skip set to general_model_reward only

HansBug
and
claude
committed
a6202bb
docs(math_prm): R5 polish — drop unused deps, fix README step labels, add trainer docstrings

HansBug
and
claude
committed
ca33772

Commits on Jul 1, 2026

fix(math-prm): address review compatibility issues
HansBug
committed
0b219aa

Commits on Jul 2, 2026

fix(math-prm): penalize post-answer continuation
HansBug
committed
b5be0ce
style: apply CI formatting fixes
HansBug
committed
f085791