Repository navigation
fix(lerobot): declare every exported feature in info.json and check exported schemas - #781
Merged
kstonekuan merged 1 commit intoNov 13, 2026
Conversation
…xported schemas The exported feature registry kept only float32 declarations, so integer and boolean features written to the data files (episode_index, frame_index, index, task_index, next.done) were missing from info.json. Declare every written non-video feature with its source declaration, and check each written data file's schema against its source so a silent type or metadata change fails the export. EXPORT_VERSION moves to v3. Refs Hebbian-Robotics#776
Contributor
[Medium impact] The PR appears safe to merge; no actionable issues were found.
|
kstonekuan
approved these changes
Nov 13, 2026
kstonekuan
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the fix! Ruff, formatting, ty, and CI pass. The local suite had one intermittent runtime redirect test failure; it passed on rerun.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #776 (the second finding; #777 fixed the first). The exported
meta/info.jsonnow declares every non-video feature that is written to the data files, including integer indexes and boolean flags such asepisode_index,frame_index,index,task_indexandnext.done, with the source declaration unchanged. Declarations for columns that are not written are left out. Each written data file's schema is also checked against its source, so a silent type, nullability or metadata change now fails the export with an error naming the file and the first differing field.Why
The registry filter kept only features whose dtype was
float32. Integer and boolean columns stayed in the exported Parquet but disappeared fromfeatures, and a declaredfloat32feature with no output column would still have been listed. The new check enforces the guarantee from #777 on every export rather than only in tests: the earlier float64 widening passed the workflow and the verifier silently.EXPORT_VERSIONmoves fromlerobot-export-v2tolerobot-export-v3becausemeta/info.jsonchanges for the same source and selection (the schema check alone would not need a bump).examples/README.mddescribes the registry and the check, says to regenerate v1/v2 exports, and drops the line saying the exporter runs official-API validation whenlerobotis installed:_validate_v3does structural checks only and never importslerobot.Validation
Rebased on
mainafter #777 (93ad1b5); run in a cleanpython:3.12-bookwormcontainer.test_export_registry_matches_written_features(exact registry/column relationship, unchanged dtype/shape/names, selected cameras) andtest_export_refuses_written_schema_mismatch(type, nullability, field metadata, schema metadata, missing, extra and reordered fields, including a failure in the second data file with the existing destination preserved). All 8 cases fail withexport.pyfrommainand pass with this change.uv run pytest -q tests/test_lerobot_export.py: 40 passed.uv run pytest -q -n 4: 2669 passed, 13 skipped. The twotests/test_ffmpeg.pyfailures also fail onmainin this container (no system FFmpeg).test_keyframe_sampling_reads_a_fraction_of_a_long_recordingfailed only inside the parallel full run; it passes on its own on both this branch andmain, andtests/test_byte_range_source.pypasses in full (18) on this branch.uv run ruff check .,uv run ruff format --check .,uv run ty check: all pass.Checklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.