feat: build KeptTokens from native sampling_mask - #2460
Draft
mikasenghaas wants to merge 1 commit into
Draft
Conversation
vLLM 0.28 returns kept-set sampling masks natively (sampling_mask, one list of surviving vocab ids per completion token) instead of the custom base64 kept_tokens payload. Convert to the flat int32 ids/counts arrays on the train client; graph attribution validates alignment and attaches the arrays as-is.
mikasenghaas
added a commit
to PrimeIntellect-ai/prime-rl
that referenced
this pull request
Aug 29, 2026
Truncated train sampling (top_p < 1, top_k) renormalizes the rollout distribution over the surviving kept set; rollout logprobs reflect that (processed_logprobs) while the trainer normalizes over the full vocab, biasing every importance ratio. Record the kept set at sampling time and renormalize trainer logprobs over the same set (DeepSeek V3.2's Keep Sampling Mask, arXiv:2512.02556 3.1). Same user API as #3235: [orchestrator.train.sampling] top_p/top_k, no replay flags. Truncating policy sampling auto-enables inference.enable_return_sampling_mask, bounds top_k to 512 (trainer mask tensors pad to the largest kept set), and rejects opd/opsd and temperature 0. Unlike #3235 the capture is vLLM's native --return-sampling-mask (>= 0.28, V2 model runner) instead of custom engine patches: the /generate response carries sampling_mask natively, renderers parse it (PrimeIntellect-ai/renderers#144) and verifiers carry it as KeptTokens arrays (PrimeIntellect-ai/verifiers#2460). Capture is engine-wide: vLLM rejects requests with temperature <= 0 or top_k <= 0 while it is on, and it is incompatible with router replay (V1-only).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
KeptTokensfrom vLLM's nativesampling_maskpayload (one list of surviving vocab ids per completion token; native since vLLM 0.28 via--return-sampling-mask), replacing the custom base64kept_tokenspayload from prime-rl's capture patches (superseded by native support).KeptTokens.from_sampling_maskconverts to the flat int32ids/countsarrays; graph attribution validates alignment and attaches the arrays as-is (no base64 decode/copy).Branch.kept_tokensis unchanged for downstream consumers.Pairs with PrimeIntellect-ai/renderers#144 and prime-rl's native sampling-replay PR.
🤖 Generated with Claude Code
Note
Build
KeptTokensfrom nativesampling_maskinstead ofkept_tokensdictKeptTokens.from_sampling_maskclassmethod in types.py to constructcountsandidsas flat int32 numpy arrays from a list-of-listssampling_mask.response_from_generatein train.py now readssampling_maskfrom the generate result and callsfrom_sampling_maskinstead of unpacking akept_tokensdict._attribute_kept_tokensin graph.py no longer base64-decodespayload.counts/payload.ids; it validates lengths and attaches the payload directly to the node.response_from_generateno longer accepts akept_tokenspayload; callers must providesampling_maskinstead.Macroscope summarized f0d4154.