Add OGLO tactile glove device (plugin, tracker, schema, example)#691
Add OGLO tactile glove device (plugin, tracker, schema, example)#691styu12 wants to merge 1 commit into
Conversation
|
📝 Docs preview is not auto-deployed for fork PRs. A maintainer with write access to |
📝 WalkthroughWalkthroughThis PR introduces the complete OGLO tactile glove BLE plugin for the IsaacTeleop system. It adds a FlatBuffers schema ( Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Pull request overview
Adds OGLO tactile gloves as a first-class IsaacTeleop device by introducing a new FlatBuffer schema, a BLE-based plugin with two output modes (local MCAP or OpenXR SchemaPusher), a live tracker implementation for DeviceIO sessions/MCAP recording, plus end-to-end examples and docs.
Changes:
- Add
core.OgloGloveSample*FlatBuffer schema with C++/Python bindings and a schema unit test. - Add
oglo_tactileplugin (packet parser, BLE client seam with stub + SimpleBLE backend, MCAP/SchemaPusher sinks) and a standalone C++ parser unit test. - Add
OgloTactileTracker+ live tracker integration (factory wiring, recording traits), plus an end-to-end Quest+gloves recording + heatmap overlay example and docs.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/plugins/oglo_tactile/tests/test_oglo_packet_parser.cpp |
Standalone C++ unit test validating packed12-v5 decoding + fallback/robustness cases. |
src/plugins/oglo_tactile/README.md |
Plugin build/usage docs, licensing note, and test instructions. |
src/plugins/oglo_tactile/plugin.yaml |
Registers OGLO plugin + left/right device paths for the plugin manager. |
src/plugins/oglo_tactile/oglo_tactile_plugin.hpp |
Plugin driver interface (options, queueing, device-clock extension). |
src/plugins/oglo_tactile/oglo_tactile_plugin.cpp |
Connect/subscribe loop, notify parsing, queue drain, stall watchdog + reconnect. |
src/plugins/oglo_tactile/oglo_packet_parser.hpp |
Wire-format parser API/types for tactile+IMU notify payloads. |
src/plugins/oglo_tactile/oglo_packet_parser.cpp |
Implementation for packed12-v5 and schema-4 Method B/C decoding. |
src/plugins/oglo_tactile/oglo_glove_sink.hpp |
Sink abstraction + factory for local MCAP vs OpenXR SchemaPusher modes. |
src/plugins/oglo_tactile/oglo_glove_sink.cpp |
MCAP writer sink + SchemaPusher sink; metadata recording for device config. |
src/plugins/oglo_tactile/oglo_config.hpp |
Device Config JSON model (schema/version/geometry/channel order/etc.). |
src/plugins/oglo_tactile/oglo_config.cpp |
Tolerant JSON parsing with defaults + basic validation. |
src/plugins/oglo_tactile/main.cpp |
CLI entrypoint for running one glove in either output mode. |
src/plugins/oglo_tactile/CMakeLists.txt |
Builds OGLO plugin, selects BLE backend, fetches json/SimpleBLE, adds parser test. |
src/plugins/oglo_tactile/ble/oglo_ble_client.hpp |
Backend-agnostic BLE transport interface + UUIDs + factory function. |
src/plugins/oglo_tactile/ble/oglo_ble_client.cpp |
Common BLE helper (advertised name mapping). |
src/plugins/oglo_tactile/ble/oglo_ble_client_stub.cpp |
Default stub backend that errors clearly when no transport is enabled. |
src/plugins/oglo_tactile/ble/oglo_ble_client_simpleble.cpp |
Optional SimpleBLE backend implementation (opt-in build). |
src/core/schema/python/schema_module.cpp |
Registers OGLO schema bindings in the Python _schema module. |
src/core/schema/python/schema_init.py |
Re-exports OGLO schema Python symbols. |
src/core/schema/python/oglo_tactile_bindings.h |
Pybind11 bindings for OgloGloveSample* object API types. |
src/core/schema/python/CMakeLists.txt |
Adds OGLO bindings header to the schema Python module build. |
src/core/schema/fbs/oglo_tactile.fbs |
Defines OgloGloveSample, ...Tracked, and ...Record root type. |
src/core/schema_tests/python/test_oglo_tactile.py |
Python unit tests covering OGLO schema bindings basic behavior. |
src/core/python/deviceio_init.py |
Backward-compatible deviceio re-exports include OGLO tracker + sample type. |
src/core/mcap/cpp/inc/mcap/recording_traits.hpp |
Adds OgloRecordingTraits (schema name + recording/replay channel suffixes). |
src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.hpp |
Live tracker impl declaration for reading OGLO SchemaPusher tensors + MCAP. |
src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.cpp |
Live tracker impl wiring: SchemaTracker config + MCAP channel creation + update. |
src/core/live_trackers/cpp/live_deviceio_factory.cpp |
Factory dispatch wiring + creator for OGLO live tracker impl. |
src/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hpp |
Adds OGLO tracker forward decl + factory method declaration. |
src/core/live_trackers/cpp/CMakeLists.txt |
Adds OGLO live tracker sources to live_trackers library build. |
src/core/deviceio_trackers/python/tracker_bindings.cpp |
Exposes OgloTactileTracker to Python and its get_glove_data() accessor. |
src/core/deviceio_trackers/python/deviceio_trackers_init.py |
Re-exports OgloTactileTracker in the Python trackers module. |
src/core/deviceio_trackers/cpp/oglo_tactile_tracker.cpp |
Implements tracker facade and session dispatch to the live impl. |
src/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hpp |
Declares OgloTactileTracker facade API and defaults. |
src/core/deviceio_trackers/cpp/CMakeLists.txt |
Adds OGLO tracker sources/headers to deviceio_trackers library build. |
src/core/deviceio_base/cpp/inc/deviceio_base/oglo_tactile_tracker_base.hpp |
Adds base interface IOgloTactileTrackerImpl for live/replay implementations. |
examples/oglo_tactile/README.md |
End-to-end Quest + two gloves + synced MCAP + overlay demo instructions. |
examples/oglo_tactile/oglo_teleop_record.py |
Orchestrates plugins + DeviceIOSession recording and optional XR heatmap overlay. |
examples/oglo_tactile/oglo_heatmap.py |
Heatmap renderer + normalization and a standalone PNG demo mode. |
docs/source/device/oglo.rst |
Device docs page for OGLO plugin/tracker usage and build instructions. |
docs/source/device/index.rst |
Adds OGLO to the device docs toctree. |
CMakeLists.txt |
Adds BUILD_PLUGIN_OGLO option and includes the plugin subdirectory when enabled. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/plugins/oglo_tactile/CMakeLists.txt (1)
36-43: 🧹 Nitpick | 🔵 TrivialConsider pinning to commit hashes instead of semantic version tags.
While
v3.11.3andv0.7.3are stable release tags, commit hashes provide stronger immutability guarantees against hypothetical upstream retags. This hardening is optional but improves supply-chain reproducibility.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/plugins/oglo_tactile/CMakeLists.txt` around lines 36 - 43, The FetchContent_Declare block for nlohmann_json is using a semantic version tag (v3.11.3) in the GIT_TAG parameter, which can be vulnerable to upstream retags. Replace the semantic version tag with the specific commit hash that corresponds to the v3.11.3 release to ensure supply-chain reproducibility and immutability. Look up the commit hash for the v3.11.3 release on the nlohmann/json repository and update the GIT_TAG value accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/plugins/oglo_tactile/main.cpp`:
- Around line 72-90: The std::stoi function in the --scan-timeout-ms argument
parsing silently accepts numeric prefixes and ignores trailing characters,
allowing invalid inputs like "15000ms" to be parsed as 15000. In the try block
where std::stoi(val) is called, modify it to use the optional third parameter
(size_t* idx) to capture the number of characters actually parsed. After
parsing, validate that the parsed character count equals the full length of the
val string; if not, set ms to -1 to trigger the validation error below, ensuring
full string consumption is enforced.
In `@src/plugins/oglo_tactile/oglo_tactile_plugin.cpp`:
- Around line 114-152: When the stop flag becomes true, the while loop exits
immediately leaving any samples already in m_queue unprocessed, causing data
loss. After the while loop ends (after the condition
`!stop.load(std::memory_order_relaxed)` becomes false), add a final drain
operation that swaps any remaining samples from m_queue into a batch and
processes them through the same sample processing logic (the for loop that calls
m_sink->on_sample for each sample in the batch) to ensure all queued samples are
persisted before the thread exits.
---
Nitpick comments:
In `@src/plugins/oglo_tactile/CMakeLists.txt`:
- Around line 36-43: The FetchContent_Declare block for nlohmann_json is using a
semantic version tag (v3.11.3) in the GIT_TAG parameter, which can be vulnerable
to upstream retags. Replace the semantic version tag with the specific commit
hash that corresponds to the v3.11.3 release to ensure supply-chain
reproducibility and immutability. Look up the commit hash for the v3.11.3
release on the nlohmann/json repository and update the GIT_TAG value
accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a598f383-709c-44c7-8184-8fa1ba896a0d
📒 Files selected for processing (42)
CMakeLists.txtdocs/source/device/index.rstdocs/source/device/oglo.rstexamples/oglo_tactile/README.mdexamples/oglo_tactile/oglo_heatmap.pyexamples/oglo_tactile/oglo_teleop_record.pysrc/core/deviceio_base/cpp/inc/deviceio_base/oglo_tactile_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hppsrc/core/deviceio_trackers/cpp/oglo_tactile_tracker.cppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.cppsrc/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.hppsrc/core/mcap/cpp/inc/mcap/recording_traits.hppsrc/core/python/deviceio_init.pysrc/core/schema/fbs/oglo_tactile.fbssrc/core/schema/python/CMakeLists.txtsrc/core/schema/python/oglo_tactile_bindings.hsrc/core/schema/python/schema_init.pysrc/core/schema/python/schema_module.cppsrc/core/schema_tests/python/test_oglo_tactile.pysrc/plugins/oglo_tactile/CMakeLists.txtsrc/plugins/oglo_tactile/README.mdsrc/plugins/oglo_tactile/ble/oglo_ble_client.cppsrc/plugins/oglo_tactile/ble/oglo_ble_client.hppsrc/plugins/oglo_tactile/ble/oglo_ble_client_simpleble.cppsrc/plugins/oglo_tactile/ble/oglo_ble_client_stub.cppsrc/plugins/oglo_tactile/main.cppsrc/plugins/oglo_tactile/oglo_config.cppsrc/plugins/oglo_tactile/oglo_config.hppsrc/plugins/oglo_tactile/oglo_glove_sink.cppsrc/plugins/oglo_tactile/oglo_glove_sink.hppsrc/plugins/oglo_tactile/oglo_packet_parser.cppsrc/plugins/oglo_tactile/oglo_packet_parser.hppsrc/plugins/oglo_tactile/oglo_tactile_plugin.cppsrc/plugins/oglo_tactile/oglo_tactile_plugin.hppsrc/plugins/oglo_tactile/plugin.yamlsrc/plugins/oglo_tactile/tests/test_oglo_packet_parser.cpp
65e0224 to
6d622df
Compare
Integrate OGLO tactile gloves (80 taxels + 6-axis IMU per hand, 100 Hz over BLE) as an IsaacTeleop device, following docs/source/device/add_device.rst. This enables synchronized MetaQuest (hand/head) + OGLO data collection into a single MCAP, with an optional in-headset tactile heatmap overlay (TeleViz). Contents: - Schema: src/core/schema/fbs/oglo_tactile.fbs (OgloGloveSample / Tracked / Record) plus C++ and Python bindings and a schema pytest. - Plugin: src/plugins/oglo_tactile - BLE -> config-driven packed12-v5 parser -> SchemaPusher (Mode 2) or local MCAP (Mode 1). The transport is isolated behind the OgloBleClient interface; a dependency-free unit test validates the wire decode against the firmware packing reference. - Tracker: OgloTactileTracker + LiveOgloTactileTrackerImpl (SchemaTracker reader, mirroring the generic_3axis_pedal pattern), factory registration, and recording traits. - Example: examples/oglo_tactile - end-to-end record demo, the heatmap renderer, and an offline Foxglove (CompressedImage) exporter. - Docs: docs/source/device/oglo.rst (registered in the device toctree). BLE backend: the plugin talks to BlueZ directly over libdbus (AFL-2.1, permissive), so it carries no copyleft dependency and connects out of the box on any Linux host with the BlueZ daemon. The transport sits behind the OgloBleClient interface (make_ble_client()), so an alternative backend can be dropped in without touching the parser, schema, or tracker. Testing: - ctest -R oglo_packet_parser (wire decode vs firmware packing reference) - ctest -R schema_test_oglo_tactile (schema bindings) - Manual: MetaQuest + two gloves -> one synchronized MCAP (hands/head/oglo_left/oglo_right) on a shared DeviceDataTimestamp clock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jerry Han <jerry@opengraphlabs.com>
Summary
Adds OGLO tactile gloves as a first-class IsaacTeleop device, following the
Add a New Device
pattern (FlatBuffer schema → plugin
SchemaPusher→ tracker → MCAP), modeled on thegeneric_3axis_pedalreference.Each glove streams 80 taxels (5 fingers × 4×4) + a 6-axis IMU at 100 Hz over BLE.
A MetaQuest (hand/head) and both gloves record into one time-synchronized MCAP on a shared
DeviceDataTimestampclock, with an optional in-headset tactile heatmap overlay (TeleViz).What's included
oglo_tactile.fbs(OgloGloveSample/…Tracked/…Record) + C++/Python bindings + schema pytest.src/plugins/oglo_tactile: BLE read → config-driven parser → push (Mode 2) or local MCAP (Mode 1).OgloTactileTracker+LiveOgloTactileTrackerImpl, factory registration, recording traits.docs/source/device/oglo.rst.BLE backend
The plugin talks to BlueZ directly over libdbus (AFL-2.1, permissive) — no copyleft dependency,
and it connects out of the box on any Linux host with the BlueZ daemon (
apt install libdbus-1-dev).The transport is isolated behind the
OgloBleClientinterface, so an alternative backend can bedropped in via
make_ble_client()without touching the parser, schema, or tracker.Verified on hardware — both gloves connect and stream at 100 Hz into a synchronized MCAP
(the screenshot above is a live capture).
Scope: BLE-only. A USB-C serial transport could slot into the same factory as future work.
Type of change
Testing
g++ -std=c++20 … && ./t→OK: all 350 checks passed).hands/{left,right},head/head,oglo_{left,right}, all onsample_time_local_common_clock. Steps inexamples/oglo_tactile/README.md.clang-format-14,ruff, and REUSE/SPDX all clean.Checklist
ruffandclang-format-14(CI's version)Plugin is off by default (
-DBUILD_PLUGIN_OGLO=ONto enable), Linux-only. Happy to open atracking issue or adjust the review shape if maintainers prefer.