[trainer] fix: make the multi-trajectory padding row survive THD context parallel

[gaohongkui]

What does this PR do?

construct_minimal_padding_template builds the synthetic divisibility row as 2 tokens
(one prompt + one response). That is below what the THD context-parallel split can handle.

preprocess_packed_seqs pads each row to align_size = tp * cp * 2 and then hands CP
rank r the slice d[half * r : half * (r + 1)], where half is derived from the
padded length while d = input_ids[i, attention_mask[i]] holds only the valid
tokens. For a 2-token row at tp=cp=2 the row pads to 8, half is 2, and rank 1 asks for
d[2:4] of a 2-element tensor — an empty slice, which raises on assignment:

RuntimeError: The expanded size of the tensor (2) must match the existing size (0)
at non-singleton dimension 0.  Target sizes: [2].  Tensor sizes: [0]

This is not a claim that verl's own preprocess_packed_seqs is broken — #6001 clamped
that slice and this repo carries the fix. The problem is that the megatron engine dispatches
the forward to the bridge, and the vendored copies there never picked up #6001. Real
traceback from Qwen3.6-35B-A3B GRPO, THD + TP2/CP2 on H20-141G:

File "verl/models/mcore/model_forward.py", line 345, in gptmodel_forward_model_engine
File "mbridge/models/qwen3_5/model.py", line 478, in forward
    input_ids_thd, _ = preprocess_packed_seqs(
File "mbridge/core/util.py", line 665, in preprocess_packed_seqs
RuntimeError: The expanded size of the tensor (2) must match the existing size (0) ...

Inspecting the loaded modules at runtime confirmed it, rather than inferring it: verl's copy
has the clamp, both vendored copies do not.

I have filed the clamp for the bridges as well — ISEEKYAN/mbridge#157 and
NVIDIA-NeMo/Megatron-Bridge#5614 (two vendored copies there). This PR is the verl half of
the same fix, not a substitute for them: sizing the synthetic row to the alignment means
verl no longer depends on which bridge version happens to be installed, and it is the only
half that helps users on an already-released bridge.

Scope: only batches whose size is not already divisible create padding rows, so a fixed
train_batch_size * rollout.n with one trajectory per prompt never hits this. It appears
once the trajectory count per prompt varies — which is exactly what padding_utils exists
for (see the module docstring).

On the constant

Measured smallest valid length at which the unclamped split stops raising:

topology	align_size	crashes at valid length	safe from
TP1/CP2	4	1	2
TP2/CP2	8	1, 2	3
TP2/CP4	16	1–6	8
TP4/CP4	32	1–8	16

So 128 total tokens is not a minimum requirement — it is a value that is a multiple of
tp * cp * 2 for every tp * cp up to 64, so the row never needs alignment padding on any
realistic topology and the constant does not have to be revisited when the parallel layout
changes. The cost is 128 no-op tokens per padding row against a per-GPU token budget in the
tens of thousands. Happy to derive it from the config instead if you would prefer that —
padding_utils does not currently see tp/cp, which is why I used a constant.

The row still has response_mask all zero and is_padding set, so it contributes no
gradient and stays out of metrics. The one place that reads tag["seq_len"] without
filtering is_padding is global_seqlen_lst in _compute_metrics, feeding
get_seqlen_balanced_partitions and log_seqlen_unbalance; there the true length is
strictly better information, and the effect on the balance metric is ~126 tokens against
hundreds of thousands per rank.

Checklist Before Starting

• Search for similar PRs. Queries run: padding_utils, construct_minimal_padding_template,
  upsample_batch_to_divisible, preprocess_packed_seqs, context parallel padding (PRs, all
  states). The related prior work is [bug] Bug: CP preprocess mixes rmpad tokens with padded layout — crashes on short / heavily padded sequences #5981 / [data] fix: pad data in preprocess_packed_seqs if shorter than align_size #6001 / [data] fix: pad data in preprocess_packed_seqs if shorter than align_size #6035 / fix(mcore): pad valid tokens to padded length in CP preprocess to avoid OOB indexing #6088, which fixed the clamp in
  verl's own copy — this PR fixes the producer so the vendored bridge copies stop being load
  bearing. No open PR touches padding_utils.py.
• Format the PR title as [{modules}] {type}: {description}

Test

New CPU test tests/trainer/test_padding_utils_on_cpu.py (17 cases):

$ python3 -m pytest tests/trainer/test_padding_utils_on_cpu.py -q
.................                                                        [100%]
17 passed in 4.31s

It pins the property that makes the row safe (its length is a multiple of the alignment for
every tp * cp up to 64), that the row still contributes no gradient and no reward, the
exact failure at 2 tokens on the unclamped path, and — via torch.equal — that clamping is a
pure guard for rows of length 128 / 4096 / 14000 at both CP ranks.

Repo checks:

$ PR_TITLE="$(git log -1 --format=%s)" python3 tests/special_sanity/check_pr_title.py
✅ PR title is valid: ..., modules: ['trainer'], type: fix

$ python3 tests/special_sanity/check_license.py -d tests/trainer verl/trainer
(exit 0)

$ pre-commit run --files verl/trainer/ppo/padding_utils.py tests/trainer/test_padding_utils_on_cpu.py
ruff (legacy alias) .... Passed      mypy ................... Passed
ruff format ............ Passed      Check license .......... Passed
Validate test structure  Passed      Check naming conventions Passed
(all 14 hooks Passed)

End to end on a cluster: with the longer row, THD + CP=2 completes 4 training steps and
reward magnitude matches an otherwise identical CP=1 run (0.797/0.697/0.473/0.472 vs
0.847/0.415/0.483/0.457 — GRPO rollout is sampled, so only magnitude and trend are
comparable). Peak reserved memory 99.2 GB (CP=2) vs 113.9 GB (CP=1) of 141 GB. Causality was
established by single-variable isolation: reverting only the row length, with CP and the
token budget unchanged, reproduces the crash above.

API and Usage Example

No API change. _PADDING_TOKENS_PER_SIDE is module-private and affects only the synthetic
rows appended for divisibility.

AI assistance

This change was prepared with AI assistance (Claude). I reviewed every changed line, ran the
tests and checks above myself, and can defend the change end-to-end.