Skip to content

Cap raw-frame reads at the video frame-pixel budget - #807

Closed
cursor[bot] wants to merge 2 commits into
mainfrom
cursor/cap-raw-frame-buffers-4635
Closed

cursor[bot] wants to merge 2 commits into
mainfrom
cursor/cap-raw-frame-buffers-4635

Conversation

@cursor

@cursor cursor Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Summary

Camera-motion streaming and measurement, raw RGB sampling (used by mediapipe_hands), and the opt-in camera_stability check now refuse a video whose frames exceed 33,554,432 pixels. That is the default VideoLimits.maximum_frame_pixels that probe_video and video import already enforce. The refusal is a RawFrameError that names the budget, raised before any raw decode starts. In-budget videos decode exactly as before.

Why

_probe_frame_shape returned whatever width and height ffprobe reported. _raw_frame_stream then multiplied them into a per-frame read() size. A container header declaring an enormous frame made the measurement process allocate a buffer of that size for every frame, and ffmpeg allocate the decoded frame too.

The fix binds the budget to the values actually used:

  • _probe_frame_shape refuses probed dimensions over the budget, before any ffmpeg decode is spawned.
  • _raw_frame_stream checks the frame shape it is given (pixels and at most 3 bytes per pixel) before it starts ffmpeg. The read size can never exceed the budget, whatever produced the shape.
  • rgb_frames checks the scaled sample shape before the digest trace. An upscaling long_edge_pixels can't make ffmpeg build oversized frames.
  • Every raw decode (rawvideo reader, framemd5 trace, window-measurement decode) passes -max_pixels to ffmpeg's decoder. The probe and the decode open the file separately, so a file swapped in between, or a stream that changes resolution mid-stream, is refused by the decoder itself. ffmpeg checks the stride-aligned width (up to 64 padding columns) against -max_pixels, so the decoder cap is twice the budget: a cap at exactly the budget rejected legal frames such as 1000×33554.
  • window_measurements (_Y4mLumaFrames) applies the caller's limits.maximum_frame_pixels to the Y4M header it reads from ffmpeg. Previously only the earlier, separate probe_video call checked it.

One source for the default budget

The default lives in a new dependency-free module, hflow._frame_budget:

  • DEFAULT_MAXIMUM_FRAME_PIXELS: the single source for the default budget.
  • ffmpeg_decoder_maximum_pixels(): derives ffmpeg's -max_pixels value from a budget (twice the budget, capped at the option's maximum).

VideoLimits, the raw-frame readers and window measurements all use it. luma_frames and rgb_frames take a maximum_frame_pixels keyword that defaults to it.

hflow._video_measurements is kept extractable by tests/test_module_boundaries.py. That test now allows hflow._frame_budget beside hflow._field_guards, and a new boundary rule keeps _frame_budget itself free of HFlow imports. I chose this over having hflow.media import from the measurement package because that package's __init__ loads numpy and the camera-motion code. Plain import hflow.media still doesn't load numpy.

No transform output changes, so TRANSFORM_BEHAVIOR_VERSION is not bumped.

Other ffmpeg decodes I reviewed and left unchanged: Episode.frames and Episode.frames_at_indices (src/hflow/episode.py). They have ffmpeg write JPEGs and a framecrc trace to disk, and never read a raw frame into the Python process.

Validation

  • New regression test: tests/test_ffmpeg.py::test_raw_frames_refuse_frames_over_the_pixel_limit_before_decoding.
    • Setup: parametrized over luma_frames and rgb_frames. It uses the ordinary 160×120 black_tail_video fixture, a limit of 160 * 120 - 1, the real ffprobe, and a stand-in ffmpeg script that leaves a marker file if started.
    • It expects a RawFrameError naming the limit, and no marker.
    • With the fix: 2 passed.
    • With the budget check disabled: 2 failed, because ffmpeg was started (decode failed, ffmpeg failed to trace decoded frames).
  • Tests from Add failing tests for uncapped camera-motion frame buffers #803 (Add failing tests for uncapped camera-motion frame buffers #803), not included in this PR:
    • Without the fix: 11 failed, 2 passed.
    • With this branch's head: 13 passed, with the file copied in temporarily.
  • Variants of the same technique, run against the first commit in scratch scripts outside the repo:
    • RGB upscale (long_edge_pixels=20000 on a 160×120 source): refused before the digest trace.
    • Small-then-16384×8192 Annex B stream: refused at the probe.
    • Probe sees 160×120 but decode reads 16384×8192: ffmpeg refuses through -max_pixels, and the Python peak is 0.1 MB.
    • Over-budget or 4-bytes-per-pixel shape passed straight to the reader: refused without spawning ffmpeg.
    • Window measurement with a swapped source:
      • With the fix: MeasurementFailure('MediaToolError'), peak 1.2 MB.
      • Without the fix: peak 101.8 MB.
  • Related suites, uv run pytest -q -n 4 on test_camera_shake_summary, test_checks, test_episode, test_ffmpeg, test_mediapipe_hands, test_module_boundaries, test_motion, test_motion_stream, test_testing, test_video_measurement_settings, test_window_measurements, test_video_import and test_identity_stability: 520 passed, 6 skipped.
  • Full suite, uv run pytest -q -n 4: 2603 passed, 8 skipped, 37 failed.
    • 36 failures are in tests/test_packaging.py and tests/test_packaging_manifest_parsing.py. The native-overlay compile fails because this VM lacks Python 3.12 headers, and they fail identically on unchanged main.
    • One is a test_verification.py test that also fails intermittently on main.
  • uv run ruff check --fix, uv run ruff format, uv run ty check: all pass.

Checklist

  • I added or updated outcome-focused tests for changed business logic.
  • I updated documentation for changed behavior, flags, formats, or requirements.
  • I ran uv run ruff check --fix, uv run ruff format, and uv run ty check.
  • I ran the relevant pytest suite.
  • I did not add recordings, generated media, credentials, private URLs, or runtime artifacts.
  • I preserved stored-data compatibility or documented an explicit version change.
Open in Web Open in Cursor 

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Note

Medium Risk
Touches all raw-frame and window-decode paths; misconfigured limits could reject valid large frames or leave edge cases, but behavior aligns with existing probe_video limits.

Reviewed by Cursor Bugbot for commit 1ba7769. Bugbot is set up for automated code reviews on this repo. Configure here.

Camera-motion decoding, RGB sampling, and the camera_stability check sized
each raw-frame read from the dimensions ffprobe reported, so a header that
declared an enormous frame made the measurement process allocate it.

Refuse probed dimensions over the 33554432-pixel budget that probe_video
and video import already enforce, refuse any raw-frame read shape over it
inside the reader, check scaled RGB samples before decoding, and pass
-max_pixels to every raw decode so a stream that changes between probe and
decode cannot push FFmpeg past the bound. The window-measurement luma
reader applies the caller's frame limit to the Y4M header it reads.
@greptile-apps

greptile-apps Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge; no blocking issue was found.

Summary

This PR caps raw-frame reads and adds FFmpeg decoder limits.

  • Raw video readers stop before oversized frames are decoded.
  • Window camera-shake reads stay within the caller’s frame budget.

No actionable new issues were found. Tests were not run during this review.

Reviews (2) · Last reviewed commit: "Share the default frame-pixel budget wit..." · Reviewed by Greptile

greptile-apps[bot]
greptile-apps Bot previously approved these changes Nov 20, 2026

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you include a regression test in this PR that checks an over-budget frame is rejected before decoding, and use one source for the default pixel budget shared with VideoLimits?

@kstonekuan

Copy link
Copy Markdown
Contributor

@cursor Could you update this PR to share the default pixel budget with VideoLimits and add a small test of rejection before decode using an ordinary fixture and a low test limit?

@cursor

cursor Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Author

I've updated PR #807. VideoLimits and the raw-frame readers now take their default pixel budget from one place, and there is a small test showing that over-limit frames are refused before ffmpeg starts.

Shared budget: the default lives in a new module, src/hflow/_frame_budget.py. It also holds the helper that turns a budget into ffmpeg's decoder cap. VideoLimits, the raw-frame readers and window measurements all use it. luma_frames and rgb_frames now accept a maximum_frame_pixels limit, which defaults to it.

One trade-off you should know about: the measurement package isn't allowed to import hflow.media, because tests/test_module_boundaries.py keeps it extractable. So I added the new module to that test's allowed list, next to hflow._field_guards, and added a rule keeping the module itself free of HFlow imports. The alternative, having hflow.media import from the measurement package, would make import hflow.media load numpy and the camera-motion code.

Test: test_raw_frames_refuse_frames_over_the_pixel_limit_before_decoding in tests/test_ffmpeg.py, run once each for luma_frames and rgb_frames.

  • Setup: the ordinary 160×120 black_tail_video fixture, a limit one pixel below that size, the real ffprobe, and a stand-in ffmpeg script that leaves a marker file if it's ever started.
  • It checks: the reader raises an error naming the limit, and the marker never appears.
  • Results: it passes with the fix, and fails in both cases when I disable the check, because ffmpeg gets started.

Other results:

  • The 13 tests from Add failing tests for uncapped camera-motion frame buffers #803 still pass.
  • The related suites pass: 520 passed, 6 skipped.
  • ruff and ty are clean.
  • The full suite has 37 failures, the same ones main has on this VM. 36 are packaging tests, which fail here because the native build lacks the Python 3.12 headers. The other is a verification test that fails intermittently on main too.
Open in Web Open in Cursor 

Move the 33554432-pixel default into the dependency-free hflow._frame_budget
module, used by VideoLimits, the raw-frame readers, and the FFmpeg decoder
cap. luma_frames and rgb_frames take a maximum_frame_pixels limit, and a
regression test with an ordinary 160x120 fixture and a lower limit checks
that the readers refuse before FFmpeg starts a decode.
@greptile-apps
greptile-apps Bot dismissed their stale review November 20, 2026 13:23

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared limit and refusal tests are in place. The full local suite passed on Python 3.11 and 3.14.

@kstonekuan

Copy link
Copy Markdown
Contributor

Landed on main in c3ee742.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants