Skip to content

feat: add Qwen3.8-27B (dense) support - #3319

Open
samsja wants to merge 1 commit into
mainfrom
feat/qwen38-support
Open

feat: add Qwen3.8-27B (dense) support#3319
samsja wants to merge 1 commit into
mainfrom
feat/qwen38-support

Conversation

@samsja

@samsja samsja commented Aug 20, 2026

Copy link
Copy Markdown
Member

What

Adds support for Qwen3.8-27B (the new dense 27B model built on the Qwen3.5 architecture) to prime-rl.

Qwen3.8-27B uses the qwen3_5 / qwen3_5_text model types, so it already resolves to the existing custom Qwen3.5 dense implementation. This PR makes sure the Qwen3.5-family trainer patches run for it and bumps the renderers/verifiers submodules to the new Qwen3.8 renderer + test coverage.

Changes

Notes

  • vLLM already serves the Qwen3_5ForConditionalGeneration architecture (incl. the XML tool parser) natively, so no inference-side change is needed.
  • Qwen3.8 checkpoints carry trained mtp.* weights; these are ignored by the non-strict DCP load, same as Qwen3.5-MoE.

Note

Low Risk
Small change to when existing Qwen3.5 monkeypatches run; no new modeling or load path. Wrong gating could skip or extra-apply patches, but the logic is a straightforward name/model_type expansion.

Overview
Enables Qwen3.8-27B (dense, same stack as Qwen3.5) by running the existing Qwen3.5 trainer patches for it.

In get_model, the name gate now matches "Qwen3.8", and the config fallback is broadened from qwen3_5_moe* to any model_type starting with qwen3_5 (covers qwen3_5, qwen3_5_text, and renamed checkpoints). Docs list Qwen3.8 as reusing the qwen3_5 VLM/custom path.

Reviewed by Cursor Bugbot for commit 6e8a5af. Bugbot is set up for automated code reviews on this repo. Configure here.

- Extend the Qwen3.5-family patch gate to cover Qwen3.8 by name and
  the qwen3_5 model_type fallback.
- Bump deps/renderers to the Qwen3.8 renderer (PrimeIntellect-ai/renderers#132).
- Bump deps/verifiers to the Qwen3.8 renderer test coverage (PrimeIntellect-ai/verifiers#2416).
- Document Qwen3.8 in README and advanced.md.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6e8a5af. Configure here.

"Qwen3.5" in config.name
or "Qwen3.8" in config.name
or "qwen3_5" in config.name.lower()
):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing KL mismatch table

Medium Severity

This PR adds custom-trainer support for Qwen3.8-27B, but it does not include the required mean KL mismatch table across 20 steps on the math environment with batch_size=64 (all entries < 0.015). That validation is required when introducing a new custom model so trainer and inference stay aligned.

Additional Locations (1)
Fix in Cursor Fix in Web

Triggered by project rule: BugBot Instructions

Reviewed by Cursor Bugbot for commit 6e8a5af. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant