feat: add Qwen3.8-27B (dense) support - #3319
Open
samsja wants to merge 1 commit into
Open
Conversation
- Extend the Qwen3.5-family patch gate to cover Qwen3.8 by name and the qwen3_5 model_type fallback. - Bump deps/renderers to the Qwen3.8 renderer (PrimeIntellect-ai/renderers#132). - Bump deps/verifiers to the Qwen3.8 renderer test coverage (PrimeIntellect-ai/verifiers#2416). - Document Qwen3.8 in README and advanced.md.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6e8a5af. Configure here.
| "Qwen3.5" in config.name | ||
| or "Qwen3.8" in config.name | ||
| or "qwen3_5" in config.name.lower() | ||
| ): |
There was a problem hiding this comment.
Missing KL mismatch table
Medium Severity
This PR adds custom-trainer support for Qwen3.8-27B, but it does not include the required mean KL mismatch table across 20 steps on the math environment with batch_size=64 (all entries < 0.015). That validation is required when introducing a new custom model so trainer and inference stay aligned.
Additional Locations (1)
Triggered by project rule: BugBot Instructions
Reviewed by Cursor Bugbot for commit 6e8a5af. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


What
Adds support for Qwen3.8-27B (the new dense 27B model built on the Qwen3.5 architecture) to prime-rl.
Qwen3.8-27B uses the
qwen3_5/qwen3_5_textmodel types, so it already resolves to the existing custom Qwen3.5 dense implementation. This PR makes sure the Qwen3.5-family trainer patches run for it and bumps the renderers/verifiers submodules to the new Qwen3.8 renderer + test coverage.Changes
src/prime_rl/trainer/model.py— extend the Qwen3.5-family patch gate to also match"Qwen3.8"by name, and broaden the model_type fallback fromqwen3_5_moeto anyqwen3_5*(coversqwen3_5_textfor renamed Qwen3.8 checkpoints). This ensures the position-ids / linear-attention / conversion patches apply to Qwen3.8.deps/renderers→5520d1e— the newqwen3.8renderer (feat(renderers): add Qwen3.8-27B (dense) renderer renderers#132).deps/verifiers→2c93838a— Qwen3.8 added to renderer e2e/client test matrices (test(verifiers): add Qwen3.8-27B to renderer test matrices verifiers#2416).README.md,docs/advanced.md— document Qwen3.8 in the supported-model tables.Notes
Qwen3_5ForConditionalGenerationarchitecture (incl. the XML tool parser) natively, so no inference-side change is needed.mtp.*weights; these are ignored by the non-strict DCP load, same as Qwen3.5-MoE.Note
Low Risk
Small change to when existing Qwen3.5 monkeypatches run; no new modeling or load path. Wrong gating could skip or extra-apply patches, but the logic is a straightforward name/
model_typeexpansion.Overview
Enables Qwen3.8-27B (dense, same stack as Qwen3.5) by running the existing Qwen3.5 trainer patches for it.
In
get_model, the name gate now matches"Qwen3.8", and the config fallback is broadened fromqwen3_5_moe*to anymodel_typestarting withqwen3_5(coversqwen3_5,qwen3_5_text, and renamed checkpoints). Docs list Qwen3.8 as reusing theqwen3_5VLM/custom path.Reviewed by Cursor Bugbot for commit 6e8a5af. Bugbot is set up for automated code reviews on this repo. Configure here.