Skip to content

Preserve input validation under Python optimization - #202

Open
Mr-Neutr0n wants to merge 1 commit into
deepseek-ai:mainfrom
Mr-Neutr0n:fix/runtime-input-validation
Open

Preserve input validation under Python optimization#202
Mr-Neutr0n wants to merge 1 commit into
deepseek-ai:mainfrom
Mr-Neutr0n:fix/runtime-input-validation

Conversation

@Mr-Neutr0n

Copy link
Copy Markdown

Problem

The public Python wrappers use assert for required input validation. Python removes those statements under python -O, so invalid inputs can reach the CUDA extension.

A reproduced example is sparse decode with causal=True and is_fp8_kvcache=False: current main calls the backend in optimized mode instead of rejecting the unsupported combination. Reuse-consistency checks, the legacy num_splits guard, and unsupported prefill options are removed in the same way.

Invalid first calls can also set have_initialized before the later mode check fails, leaving scheduler state behind for a request that never ran.

Fix

  • replace public assert statements with explicit argument exceptions that survive optimization
  • use TypeError for the scheduler object type, ValueError for unsupported arguments and reuse mismatches, and RuntimeError for an impossible internal scheduler state
  • validate dense/sparse mode requirements before committing first-call scheduler state
  • preserve the accepted inputs and backend call signatures

Regression coverage

The new CPU-only tests isolate the wrapper from the CUDA extension and verify under both regular and optimized Python that:

  • invalid sparse options never reach the backend and do not initialize scheduler state
  • a non-None legacy num_splits placeholder is rejected
  • nonzero dropout is rejected by all three prefill entry points

Validation

  • uv run --with torch --with numpy python -m unittest tests/test_flash_mla_input_validation.py
  • uv run --with torch --with numpy python -O -m unittest tests/test_flash_mla_input_validation.py
  • uvx ruff check tests/test_flash_mla_input_validation.py
  • python3 -m compileall -q flash_mla tests/test_flash_mla_input_validation.py
  • verified no public assert statements remain in flash_mla_interface.py
  • git diff --check

This is independent of #201: that PR makes scheduler initialization transactional when the backend itself raises; this PR ensures invalid arguments cannot bypass validation or commit state before a backend call.

Prepared with OpenAI Codex assistance; I reproduced the optimized-mode failure and reviewed the change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant