Repository navigation
fix(lerobot): compare source video times in presentation order - #782
kstonekuan merged 1 commit into
Conversation
ffprobe lists packets in decode order, so an H.264 source with B-frames (libx264's default) reports non-monotonic pts_time values. The importer compares that list index by index with the parquet timestamps, so every such episode was refused with a timestamp disagreement at frame 1.
|
👋 Hi @albertlieyingadrian — thank you so much for your first contribution to HFlow! A maintainer will review your pull request as soon as possible. In the meantime:
💡 Tip: one open pull request per contributor at a time. Before starting an issue, check its sidebar for an assignee or a linked pull request. Either one means somebody is already on it; everything else is fair game. We are excited to have you here and appreciate your help making the project better! 🙌 |
[Medium impact] This PR appears safe to merge.
|
kstonekuan
left a comment
There was a problem hiding this comment.
Thanks for the fix! Ruff, formatting, ty, and the full suite pass locally (2,631 passed, 47 skipped).
Summary
hflow import lerobotnow accepts LeRobot v3 sources whose video carries B-frames. An H.264 dataset (lerobot ... --dataset.vcodec=h264, libx264 defaults) was refused on every episode with:Why
_get_video_pts_timesreadspacket=pts_timefrom ffprobe. ffprobe lists packets in decode order, which for a B-frame stream is not presentation order:Both callers pair these times index by index with frames in presentation order: the parquet rows, and the access units from
_transcode_mp4_to_h264, which re-encodes withbframes=0. Sorting the times restores that order. On the windowed path, it also makeswindowed[:frame_count]take the episode's first frames rather than the first packets decoded.Sources without B-frames already have monotonic times, so sorting leaves them unchanged.
Validation
New
test_video_pts_times_are_in_presentation_order_for_b_frame_sources:-bf 3clip;_get_video_pts_timesand_relative_video_pts_timesagainst the 30 fps grid.It fails on
mainand passes here.An end-to-end repro (
_convert_single_episodeon a 60-frame libx264-default source, set up liketest_converter_slices_exactly_the_declared_frame_count) fails onmainwith the error above and landslerobot_episode_0001.mcaphere.uv run pytest -q tests -k lerobot: 152 passed.uv run ruff check,uv run ruff format --checkanduv run ty checkpass (macOS, ffmpeg 8).Checklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.@kstonekuan, would you take a look when you have a chance? Happy to adjust.
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.