Skip to content

fix(lerobot): compare source video times in presentation order - #782

Merged
kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
albertlieyingadrian:fix/lerobot-bframe-pts-order
Nov 13, 2026
Merged

kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
albertlieyingadrian:fix/lerobot-bframe-pts-order

Conversation

@albertlieyingadrian

@albertlieyingadrian albertlieyingadrian commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

hflow import lerobot now accepts LeRobot v3 sources whose video carries B-frames. An H.264 dataset (lerobot ... --dataset.vcodec=h264, libx264 defaults) was refused on every episode with:

ValueError: camera observation.images.up timestamp disagreement at frame 1: parquet=0.033333s video_pts=0.133333s

Why

_get_video_pts_times reads packet=pts_time from ffprobe. ffprobe lists packets in decode order, which for a B-frame stream is not presentation order:

-bf 0 : [0.0, 0.033, 0.067, 0.1, 0.133, 0.167]
default: [0.0, 0.133, 0.067, 0.033, 0.1, 0.267]

Both callers pair these times index by index with frames in presentation order: the parquet rows, and the access units from _transcode_mp4_to_h264, which re-encodes with bframes=0. Sorting the times restores that order. On the windowed path, it also makes windowed[:frame_count] take the episode's first frames rather than the first packets decoded.

Sources without B-frames already have monotonic times, so sorting leaves them unchanged.

Validation

  • New test_video_pts_times_are_in_presentation_order_for_b_frame_sources:

    • builds a 2 s libx264 -bf 3 clip;
    • asserts that the raw ffprobe order is non-monotonic;
    • checks _get_video_pts_times and _relative_video_pts_times against the 30 fps grid.

    It fails on main and passes here.

  • An end-to-end repro (_convert_single_episode on a 60-frame libx264-default source, set up like test_converter_slices_exactly_the_declared_frame_count) fails on main with the error above and lands lerobot_episode_0001.mcap here.

  • uv run pytest -q tests -k lerobot: 152 passed.

  • uv run ruff check, uv run ruff format --check and uv run ty check pass (macOS, ffmpeg 8).

Checklist

  • I added or updated outcome-focused tests for changed business logic.
  • I updated documentation for changed behavior, flags, formats, or requirements. (No doc change needed: the import guide already says output video is re-encoded without B-frames.)
  • I ran uv run ruff check --fix, uv run ruff format, and uv run ty check.
  • I ran the relevant pytest suite.
  • I did not add recordings, generated media, credentials, private URLs, or runtime artifacts.
  • I preserved stored-data compatibility or documented an explicit version change.

@kstonekuan, would you take a look when you have a chance? Happy to adjust.


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

ffprobe lists packets in decode order, so an H.264 source with B-frames
(libx264's default) reports non-monotonic pts_time values. The importer
compares that list index by index with the parquet timestamps, so every
such episode was refused with a timestamp disagreement at frame 1.
@github-actions

Copy link
Copy Markdown

👋 Hi @albertlieyingadrian — thank you so much for your first contribution to HFlow!

A maintainer will review your pull request as soon as possible. In the meantime:

💡 Tip: one open pull request per contributor at a time. Before starting an issue, check its sidebar for an assignee or a linked pull request. Either one means somebody is already on it; everything else is fair game.

We are excited to have you here and appreciate your help making the project better! 🙌

@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge.

Summary

Fixes LeRobot imports for videos with B-frames by sorting source presentation times before comparing them with data rows.

  • LeRobot video imports match packet times to frames in viewing order.

Reviews (1) · Last reviewed commit: "fix(lerobot): compare source video times..." · Reviewed by Greptile

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! Ruff, formatting, ty, and the full suite pass locally (2,631 passed, 47 skipped).

@kstonekuan
kstonekuan merged commit fa97bc1 into Hebbian-Robotics:main Nov 13, 2026
5 checks passed
@kstonekuan kstonekuan mentioned this pull request Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants