Skip to content

[DO NOT MERGE] Opt-in sharded GRPO log-prob head: gather the vocab table once, scan local tokens (-11.8% step time, v7x 397B bench) - #2873

Closed
parambole wants to merge 4 commits into
atwigg/mlperffrom
parambole/logps-sharded-head
Closed

[DO NOT MERGE] Opt-in sharded GRPO log-prob head: gather the vocab table once, scan local tokens (-11.8% step time, v7x 397B bench)#2873
parambole wants to merge 4 commits into
atwigg/mlperffrom
parambole/logps-sharded-head

Forward TUNIX_LOGPS_SHARDED_HEAD to the trainer in k8s_launcher.sh

6d2fb66
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar