Skip to content

[reward] fix: keep boxed answers within the documented length budget - #7440

Open
JimmyWang0417 wants to merge 1 commit into
verl-project:mainfrom
JimmyWang0417:codex/fix-math-dapo-boxed-window
Open

[reward] fix: keep boxed answers within the documented length budget#7440
JimmyWang0417 wants to merge 1 commit into
verl-project:mainfrom
JimmyWang0417:codex/fix-math-dapo-boxed-window

Conversation

@JimmyWang0417

Copy link
Copy Markdown

What does this PR do?

compute_score() already limits math-DAPO solutions to their final 300
characters, matching the documented maximum MATH-500 answer length of 159
characters. is_correct_strict_box() then applies another 100-character slice,
which can remove the opening \boxed{ marker from a valid answer and silently
score it as incorrect.

This change keeps the existing 300-character bound as the single window. Strict
boxed-answer extraction receives the full bounded string while preserving the
existing four-element validation for pause_tokens_index.

Checklist Before Starting

Test

Before the fix, the documented-length strict, fallback, and pause-index cases
failed because the second 100-character slice removed the box opener:

PYTHONDONTWRITEBYTECODE=1 python /tmp/verl_w9_test_runner.py \
  '/home/jimmywang0417/Git Repositories/cv/.two-agent-cooperation/verl-w9'
3 failed, 2 passed

After removing only the redundant slice assignments:

PYTHONDONTWRITEBYTECODE=1 python /tmp/verl_w9_test_runner.py \
  '/home/jimmywang0417/Git Repositories/cv/.two-agent-cooperation/verl-w9'
5 passed
PYTHONDONTWRITEBYTECODE=1 python -m compileall -q \
  verl/utils/reward_score/math_dapo.py \
  tests/utils/reward_score/test_math_dapo_on_cpu.py
Passed
uvx --with hydra-core pre-commit run --all-files \
  --show-diff-on-failure --color=never
All 13 configured hooks passed

The full dependency test environment and remote CI were not run locally.

API and Usage Example

No signature, return-schema, configuration, or score-value changes. Existing
callers continue to use compute_score() unchanged.

Design & Code Changes

  • Keep compute_score() as the sole owner of the 300-character efficiency
    window.
  • Remove the two inner 100-character slice assignments before strict box
    extraction.
  • Preserve the four-element pause_tokens_index validation without inferring an
    unproven token-to-character coordinate conversion.
  • Add focused CPU coverage for strict mode, fallback mode, final-box selection,
    pause-index compatibility, and unchanged no-box failure behavior.

AI Assistance

OpenAI Codex was used for repository exploration, implementation assistance,
test orchestration, and review. I reviewed every changed line, can explain the
change end-to-end, and personally ran the focused regression tests.

Checklist Before Submitting

  • Read the Contribute Guide and AGENTS.md.
  • Applied the full-repository pre-commit checks.
  • Added a CPU regression test collected by cpu_unit_tests.yml.
  • Documentation is not required because no public interface or configuration
    changes.
  • Request project CI in Slack or Feishu after the PR is available.
  • The recipe submodule is not involved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant