feat: support RAO grouped agent rollouts - #1610
Open
MaxLEAF3824 wants to merge 3 commits into
Open
Conversation
Carry recursive rollout metadata through the OpenAI proxy and training data path so downstream RAO workloads can preserve node identity and depth. Add the FA4 backend required by the team's B200 environment. Key changes: - Preserve session, parent, depth, and interaction metadata - Export recursive rollout structure for downstream training - Register FlashAttention 4 as a supported FSDP backend - Refresh generated CLI references Co-Authored-By: Claude <noreply@anthropic.com>
Allow agent workflows to attach node-level rollout metadata and process complete rollout groups before flattening, enabling strict root LOO and zero-variance rejection without changing PPO.\n\nCo-Authored-By: Claude <noreply@anthropic.com>
Keep valid siblings when one rollout fails, propagate stable group and rejection metadata, and bind Agent proxy limits to the active train or eval generation context. Key changes: - isolate per-child grouped rollout failures - reject invalid RAO exports without aborting the worker - preserve group metadata through tensor export - apply role-specific Agent token ceilings Co-Authored-By: Claude <noreply@anthropic.com>
MaxLEAF3824
requested review from
HwVanICI,
fishcrap,
garrett4wade,
geshi001,
guozhihao-224,
nuzant,
rchardx and
sitabulaixizawaluduo
as code owners
August 17, 2026 06:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tests
python -m pytest -q tests/test_eval_agent_token_limit.py tests/test_grouped_rollout_workflow.py— 6 passedpython -m ruff check ...on the six changed/new files — passedpython -m ruff format --check ...on the six changed/new files — passedpython -m pre_commit run --all-files— formatting, whitespace, YAML/JSON, large-file, private-key, lockfile, SPDX, CODEOWNERS, and notebook hooks passed; two environment hooks could not run because the hook environment lackstomlliband an executable namedpythonHardware
The added tests are CPU unit tests. No distributed or GPU integration run was performed for this PR.
🤖 Generated with Claude Code