Repository navigation
fix(lerobot): open dataset videos only as local MP4 files - #805
cursor[bot] wants to merge 2 commits into
Conversation
The LeRobot importer passed each downloaded camera file to ffmpeg and ffprobe with FFmpeg's default protocols and demuxers. A dataset publisher controls the file name and bytes, so a playlist or concat script made the import open other local files (and reach URLs) named inside it, and could transcode another local video into the published episode. Open those files with a file-only protocol whitelist and a format whitelist limited to the ISO BMFF (MP4) demuxer, so FFmpeg refuses a probed playlist or script before it reads its header.
[Medium impact] The PR appears safe to merge; no blocking issues were found.
|
kstonekuan
left a comment
There was a problem hiding this comment.
Could you include a regression test in this PR that uses a benign non-MP4 camera fixture and checks that import rejects it without publishing an episode?
|
@cursor Could you update this PR with a regression test using a benign non-MP4 camera fixture? Check that import rejects it without publishing an episode. |
|
I added the regression test you asked for and pushed it to the PR. With the fix it passes; with the importer reverted to The test is
Results:
I also updated the PR description to describe the new test and its results. The bot footers that had been added below my section are unchanged. |
A decodable Matroska camera video, served under an .mkv name and under the default .mp4 template, must make the import fail without writing landing/ or prepared-manifest.json, on both the whole-file and windowed transcode paths.
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
kstonekuan
left a comment
There was a problem hiding this comment.
The new regression passes, and Ruff, ty, and the full suite passed locally.
|
Landed on main in b9f3e35. |


Summary
The LeRobot importer now opens each downloaded camera file with
-protocol_whitelist fileand-format_whitelist mov,mp4,m4a,3gp,3g2,mj2, placed before-i(which now namesfile:<resolved path>). This applies to both the transcode (_transcode_mp4_to_h264) and the packet-time probe (_get_video_pts_times, which is also used by_relative_video_pts_times). A dataset whose camera file is an HLS playlist, a concat script, or anything else that is not ISO BMFF is refused. ffmpeg refuses it before that demuxer reads its header, so no segment it names is opened or requested. The import fails with the existingRuntimeErrorfrom the ffmpeg step, and nothing is published.Legitimate LeRobot MP4 videos decode exactly as before, so the converter version is unchanged.
Why
The dataset publisher controls both
video_pathinmeta/info.jsonand the downloaded bytes. ffmpeg and ffprobe used to run with their default protocols and demuxers. A playlist under that name made the import host open other local files or request URLs listed inside it, and another local video could be transcoded into the published episode.A protocol whitelist alone, like the
filewhitelist that video import uses, does not stop this. Playlist segments can themselves befile:URLs or relative paths. The demuxer whitelist is what binds the check to the downloaded object: ffmpeg probes that file's bytes and refuses any other demuxer in the same process that would act on them. Nothing runs between a separate check and the open. The existing relative-path check on the Hub filename in_download_fileis unchanged.Scope: I searched every other ffmpeg/ffprobe call (
importers/video.py,media.py,source_sampling.py,blur.py,episode.py,window_measurements.py,_video_measurements/*,ffmpeg/_contact_sheet.py,video.py). They open files the operator supplies locally, or MP4s and H.264 that HFlow writes itself, not objects a remote publisher controls. The LeRobot importer is the only path where remote content reaches ffmpeg. The MP4 demuxer's external data references stay off (itsenable_drefsdefault), and only thefileprotocol is allowed in any case.Regression test
tests/test_lerobot_converter.py::test_import_refuses_a_non_mp4_camera_file_without_publishingrunsimport_lerobot_datasetend to end against a benign, decodable Matroska camera video. It serves the video under an.mkvvideo_pathand under the default.mp4template, and covers both the whole-file and windowed transcode paths. It asserts the import raises and writes neitherlanding/norprepared-manifest.json.Validation
The playlist tests that prove the fix are in #802. They are not included in this PR.
uv run pytest -q tests/test_lerobot_playlist_segments.py(from test: refuse LeRobot playlist segments that point outside the cache #802)file-urlandrelativesegments, with and without a video window.uv run pytest -q tests/test_lerobot_converter.py::test_import_refuses_a_non_mp4_camera_file_without_publishingsrc/hflow/importers/lerobot.pyreverted tomain: 4 failed (DID NOT RAISE). The Matroska file was converted and published.file:URL and absolute-path segments, and anffconcatscript, each under.mp4,.m3u8,.txtand.ffconcatnames, with and without a window: no outside-file open, import refused, nolanding/. An HLS playlist with anhttp://127.0.0.1segment got no request to the listener. Run directly, both the managed ffmpeg 8.1.2 and system ffmpeg 6.1 reportFormat not on whitelistfor the hls and concat demuxers and still open a real MP4.uv run pytest -q -n 4 tests/test_lerobot*.py tests/test_ffmpeg_standard_input.py tests/test_cli.pyon this branch: 181 passed.uv run pytest -q -n 4(full suite) on the fix commit: 2602 passed, 8 skipped, 36 failed. All 36 failures are intests/test_packaging.pyandtests/test_packaging_manifest_parsing.py, where the native-overlay C compile fails in this VM. The identical 36 fail on unmodifiedmain.uv run ruff check,uv run ruff format --check,uv run ty check: all pass.Checklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.Note
High Risk
Hardens the only path where remote Hugging Face dataset bytes reach FFmpeg, closing a vector where malicious camera media could open arbitrary local files or trigger network fetches during import.
Reviewed by Cursor Bugbot for commit 728a1a5. Bugbot is set up for automated code reviews on this repo. Configure here.