Skip to content

feat: build KeptTokens from native sampling_mask - #2460

Draft
mikasenghaas wants to merge 1 commit into
mainfrom
feat/native-sampling-mask
Draft

feat: build KeptTokens from native sampling_mask#2460
mikasenghaas wants to merge 1 commit into
mainfrom
feat/native-sampling-mask

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Aug 29, 2026

Copy link
Copy Markdown
Member

Summary

  • Build KeptTokens from vLLM's native sampling_mask payload (one list of surviving vocab ids per completion token; native since vLLM 0.28 via --return-sampling-mask), replacing the custom base64 kept_tokens payload from prime-rl's capture patches (superseded by native support).
  • KeptTokens.from_sampling_mask converts to the flat int32 ids/counts arrays; graph attribution validates alignment and attaches the arrays as-is (no base64 decode/copy). Branch.kept_tokens is unchanged for downstream consumers.

Pairs with PrimeIntellect-ai/renderers#144 and prime-rl's native sampling-replay PR.

🤖 Generated with Claude Code

Note

Build KeptTokens from native sampling_mask instead of kept_tokens dict

  • Adds KeptTokens.from_sampling_mask classmethod in types.py to construct counts and ids as flat int32 numpy arrays from a list-of-lists sampling_mask.
  • response_from_generate in train.py now reads sampling_mask from the generate result and calls from_sampling_mask instead of unpacking a kept_tokens dict.
  • _attribute_kept_tokens in graph.py no longer base64-decodes payload.counts/payload.ids; it validates lengths and attaches the payload directly to the node.
  • Behavioral Change: response_from_generate no longer accepts a kept_tokens payload; callers must provide sampling_mask instead.

Macroscope summarized f0d4154.

vLLM 0.28 returns kept-set sampling masks natively (sampling_mask,
one list of surviving vocab ids per completion token) instead of the
custom base64 kept_tokens payload. Convert to the flat int32 ids/counts
arrays on the train client; graph attribution validates alignment and
attaches the arrays as-is.
mikasenghaas added a commit to PrimeIntellect-ai/prime-rl that referenced this pull request Aug 29, 2026
Truncated train sampling (top_p < 1, top_k) renormalizes the rollout
distribution over the surviving kept set; rollout logprobs reflect that
(processed_logprobs) while the trainer normalizes over the full vocab,
biasing every importance ratio. Record the kept set at sampling time
and renormalize trainer logprobs over the same set (DeepSeek V3.2's
Keep Sampling Mask, arXiv:2512.02556 3.1).

Same user API as #3235: [orchestrator.train.sampling] top_p/top_k, no
replay flags. Truncating policy sampling auto-enables
inference.enable_return_sampling_mask, bounds top_k to 512 (trainer
mask tensors pad to the largest kept set), and rejects opd/opsd and
temperature 0.

Unlike #3235 the capture is vLLM's native --return-sampling-mask
(>= 0.28, V2 model runner) instead of custom engine patches: the
/generate response carries sampling_mask natively, renderers parse it
(PrimeIntellect-ai/renderers#144) and verifiers carry it as KeptTokens
arrays (PrimeIntellect-ai/verifiers#2460). Capture is engine-wide:
vLLM rejects requests with temperature <= 0 or top_k <= 0 while it is
on, and it is incompatible with router replay (V1-only).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant