Skip to content

Add OGLO tactile glove device (plugin, tracker, schema, example)#691

Open
styu12 wants to merge 1 commit into
NVIDIA:mainfrom
styu12:feat/oglo-device
Open

Add OGLO tactile glove device (plugin, tracker, schema, example)#691
styu12 wants to merge 1 commit into
NVIDIA:mainfrom
styu12:feat/oglo-device

Conversation

@styu12

@styu12 styu12 commented Jun 22, 2026

Copy link
Copy Markdown

Summary

Adds OGLO tactile gloves as a first-class IsaacTeleop device, following the
Add a New Device
pattern (FlatBuffer schema → plugin SchemaPusher → tracker → MCAP), modeled on the
generic_3axis_pedal reference.

Each glove streams 80 taxels (5 fingers × 4×4) + a 6-axis IMU at 100 Hz over BLE.
A MetaQuest (hand/head) and both gloves record into one time-synchronized MCAP on a shared
DeviceDataTimestamp clock, with an optional in-headset tactile heatmap overlay (TeleViz).

In-headset CloudXR view: OGLO gloves on the operator's hands with the live two-hand taxel heatmap (bottom-right) composited into the Quest view

What's included

  • Schemaoglo_tactile.fbs (OgloGloveSample / …Tracked / …Record) + C++/Python bindings + schema pytest.
  • Pluginsrc/plugins/oglo_tactile: BLE read → config-driven parser → push (Mode 2) or local MCAP (Mode 1).
  • TrackerOgloTactileTracker + LiveOgloTactileTrackerImpl, factory registration, recording traits.
  • Example + docs — end-to-end record demo, heatmap renderer, Foxglove exporter, and docs/source/device/oglo.rst.

BLE backend

The plugin talks to BlueZ directly over libdbus (AFL-2.1, permissive) — no copyleft dependency,
and it connects out of the box on any Linux host with the BlueZ daemon (apt install libdbus-1-dev).
The transport is isolated behind the OgloBleClient interface, so an alternative backend can be
dropped in via make_ble_client() without touching the parser, schema, or tracker.

Verified on hardware — both gloves connect and stream at 100 Hz into a synchronized MCAP
(the screenshot above is a live capture).

Scope: BLE-only. A USB-C serial transport could slot into the same factory as future work.

Type of change

  • New feature (non-breaking)
  • Documentation update

Testing

ctest --test-dir build -R oglo_packet_parser --output-on-failure   # wire decode vs firmware packing ref
ctest --test-dir build -R schema_test_oglo_tactile --output-on-failure  # schema bindings
  • The parser test is dependency-free (g++ -std=c++20 … && ./tOK: all 350 checks passed).
  • Manual: MetaQuest (CloudXR) + two gloves → one MCAP with hands/{left,right}, head/head,
    oglo_{left,right}, all on sample_time_local_common_clock. Steps in examples/oglo_tactile/README.md.
  • Tooling matched to CI: clang-format-14, ruff, and REUSE/SPDX all clean.

Checklist

  • Read the contribution guidelines
  • Ran ruff and clang-format-14 (CI's version)
  • Updated documentation
  • Added tests
  • Signed off all commits (DCO)

Plugin is off by default (-DBUILD_PLUGIN_OGLO=ON to enable), Linux-only. Happy to open a
tracking issue or adjust the review shape if maintainers prefer.

Copilot AI review requested due to automatic review settings June 22, 2026 10:10
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview is not auto-deployed for fork PRs.

A maintainer with write access to NVIDIA/IsaacTeleop can deploy a preview by
commenting /preview-docs on this PR. Once deployed, the preview
will live at:

https://nvidia.github.io/IsaacTeleop/preview/pr-691/

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR introduces the complete OGLO tactile glove BLE plugin for the IsaacTeleop system. It adds a FlatBuffers schema (OgloGloveSample, OgloGloveSampleTracked, OgloGloveSampleRecord) with pybind11 Python bindings and exports. A core::IOgloTactileTrackerImpl abstract interface and OgloTactileTracker facade integrate into the existing tracker hierarchy. LiveOgloTactileTrackerImpl handles MCAP recording via OgloRecordingTraits. The oglo_tactile_plugin executable (Linux-only) abstracts BLE via OgloBleClient with a SimpleBLE backend (opt-in) and a stub; a PacketParser decodes packed12-v5 and schema-4 Method B/C wire formats. OgloTactilePlugin manages BLE connection, 32-bit µs → 64-bit ns wrap extension, batched notification queuing, and stall watchdog reconnect. An example demo adds a heatmap renderer and a CloudXR+MetaQuest orchestration script.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title directly and accurately summarizes the primary change: adding a complete OGLO tactile glove device implementation including plugin, tracker, schema, and example code.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds OGLO tactile gloves as a first-class IsaacTeleop device by introducing a new FlatBuffer schema, a BLE-based plugin with two output modes (local MCAP or OpenXR SchemaPusher), a live tracker implementation for DeviceIO sessions/MCAP recording, plus end-to-end examples and docs.

Changes:

  • Add core.OgloGloveSample* FlatBuffer schema with C++/Python bindings and a schema unit test.
  • Add oglo_tactile plugin (packet parser, BLE client seam with stub + SimpleBLE backend, MCAP/SchemaPusher sinks) and a standalone C++ parser unit test.
  • Add OgloTactileTracker + live tracker integration (factory wiring, recording traits), plus an end-to-end Quest+gloves recording + heatmap overlay example and docs.

Reviewed changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/plugins/oglo_tactile/tests/test_oglo_packet_parser.cpp Standalone C++ unit test validating packed12-v5 decoding + fallback/robustness cases.
src/plugins/oglo_tactile/README.md Plugin build/usage docs, licensing note, and test instructions.
src/plugins/oglo_tactile/plugin.yaml Registers OGLO plugin + left/right device paths for the plugin manager.
src/plugins/oglo_tactile/oglo_tactile_plugin.hpp Plugin driver interface (options, queueing, device-clock extension).
src/plugins/oglo_tactile/oglo_tactile_plugin.cpp Connect/subscribe loop, notify parsing, queue drain, stall watchdog + reconnect.
src/plugins/oglo_tactile/oglo_packet_parser.hpp Wire-format parser API/types for tactile+IMU notify payloads.
src/plugins/oglo_tactile/oglo_packet_parser.cpp Implementation for packed12-v5 and schema-4 Method B/C decoding.
src/plugins/oglo_tactile/oglo_glove_sink.hpp Sink abstraction + factory for local MCAP vs OpenXR SchemaPusher modes.
src/plugins/oglo_tactile/oglo_glove_sink.cpp MCAP writer sink + SchemaPusher sink; metadata recording for device config.
src/plugins/oglo_tactile/oglo_config.hpp Device Config JSON model (schema/version/geometry/channel order/etc.).
src/plugins/oglo_tactile/oglo_config.cpp Tolerant JSON parsing with defaults + basic validation.
src/plugins/oglo_tactile/main.cpp CLI entrypoint for running one glove in either output mode.
src/plugins/oglo_tactile/CMakeLists.txt Builds OGLO plugin, selects BLE backend, fetches json/SimpleBLE, adds parser test.
src/plugins/oglo_tactile/ble/oglo_ble_client.hpp Backend-agnostic BLE transport interface + UUIDs + factory function.
src/plugins/oglo_tactile/ble/oglo_ble_client.cpp Common BLE helper (advertised name mapping).
src/plugins/oglo_tactile/ble/oglo_ble_client_stub.cpp Default stub backend that errors clearly when no transport is enabled.
src/plugins/oglo_tactile/ble/oglo_ble_client_simpleble.cpp Optional SimpleBLE backend implementation (opt-in build).
src/core/schema/python/schema_module.cpp Registers OGLO schema bindings in the Python _schema module.
src/core/schema/python/schema_init.py Re-exports OGLO schema Python symbols.
src/core/schema/python/oglo_tactile_bindings.h Pybind11 bindings for OgloGloveSample* object API types.
src/core/schema/python/CMakeLists.txt Adds OGLO bindings header to the schema Python module build.
src/core/schema/fbs/oglo_tactile.fbs Defines OgloGloveSample, ...Tracked, and ...Record root type.
src/core/schema_tests/python/test_oglo_tactile.py Python unit tests covering OGLO schema bindings basic behavior.
src/core/python/deviceio_init.py Backward-compatible deviceio re-exports include OGLO tracker + sample type.
src/core/mcap/cpp/inc/mcap/recording_traits.hpp Adds OgloRecordingTraits (schema name + recording/replay channel suffixes).
src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.hpp Live tracker impl declaration for reading OGLO SchemaPusher tensors + MCAP.
src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.cpp Live tracker impl wiring: SchemaTracker config + MCAP channel creation + update.
src/core/live_trackers/cpp/live_deviceio_factory.cpp Factory dispatch wiring + creator for OGLO live tracker impl.
src/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hpp Adds OGLO tracker forward decl + factory method declaration.
src/core/live_trackers/cpp/CMakeLists.txt Adds OGLO live tracker sources to live_trackers library build.
src/core/deviceio_trackers/python/tracker_bindings.cpp Exposes OgloTactileTracker to Python and its get_glove_data() accessor.
src/core/deviceio_trackers/python/deviceio_trackers_init.py Re-exports OgloTactileTracker in the Python trackers module.
src/core/deviceio_trackers/cpp/oglo_tactile_tracker.cpp Implements tracker facade and session dispatch to the live impl.
src/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hpp Declares OgloTactileTracker facade API and defaults.
src/core/deviceio_trackers/cpp/CMakeLists.txt Adds OGLO tracker sources/headers to deviceio_trackers library build.
src/core/deviceio_base/cpp/inc/deviceio_base/oglo_tactile_tracker_base.hpp Adds base interface IOgloTactileTrackerImpl for live/replay implementations.
examples/oglo_tactile/README.md End-to-end Quest + two gloves + synced MCAP + overlay demo instructions.
examples/oglo_tactile/oglo_teleop_record.py Orchestrates plugins + DeviceIOSession recording and optional XR heatmap overlay.
examples/oglo_tactile/oglo_heatmap.py Heatmap renderer + normalization and a standalone PNG demo mode.
docs/source/device/oglo.rst Device docs page for OGLO plugin/tracker usage and build instructions.
docs/source/device/index.rst Adds OGLO to the device docs toctree.
CMakeLists.txt Adds BUILD_PLUGIN_OGLO option and includes the plugin subdirectory when enabled.

Comment thread src/plugins/oglo_tactile/oglo_tactile_plugin.hpp
Comment thread src/plugins/oglo_tactile/README.md Outdated
Comment thread examples/oglo_tactile/oglo_teleop_record.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/plugins/oglo_tactile/CMakeLists.txt (1)

36-43: 🧹 Nitpick | 🔵 Trivial

Consider pinning to commit hashes instead of semantic version tags.

While v3.11.3 and v0.7.3 are stable release tags, commit hashes provide stronger immutability guarantees against hypothetical upstream retags. This hardening is optional but improves supply-chain reproducibility.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/plugins/oglo_tactile/CMakeLists.txt` around lines 36 - 43, The
FetchContent_Declare block for nlohmann_json is using a semantic version tag
(v3.11.3) in the GIT_TAG parameter, which can be vulnerable to upstream retags.
Replace the semantic version tag with the specific commit hash that corresponds
to the v3.11.3 release to ensure supply-chain reproducibility and immutability.
Look up the commit hash for the v3.11.3 release on the nlohmann/json repository
and update the GIT_TAG value accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/plugins/oglo_tactile/main.cpp`:
- Around line 72-90: The std::stoi function in the --scan-timeout-ms argument
parsing silently accepts numeric prefixes and ignores trailing characters,
allowing invalid inputs like "15000ms" to be parsed as 15000. In the try block
where std::stoi(val) is called, modify it to use the optional third parameter
(size_t* idx) to capture the number of characters actually parsed. After
parsing, validate that the parsed character count equals the full length of the
val string; if not, set ms to -1 to trigger the validation error below, ensuring
full string consumption is enforced.

In `@src/plugins/oglo_tactile/oglo_tactile_plugin.cpp`:
- Around line 114-152: When the stop flag becomes true, the while loop exits
immediately leaving any samples already in m_queue unprocessed, causing data
loss. After the while loop ends (after the condition
`!stop.load(std::memory_order_relaxed)` becomes false), add a final drain
operation that swaps any remaining samples from m_queue into a batch and
processes them through the same sample processing logic (the for loop that calls
m_sink->on_sample for each sample in the batch) to ensure all queued samples are
persisted before the thread exits.

---

Nitpick comments:
In `@src/plugins/oglo_tactile/CMakeLists.txt`:
- Around line 36-43: The FetchContent_Declare block for nlohmann_json is using a
semantic version tag (v3.11.3) in the GIT_TAG parameter, which can be vulnerable
to upstream retags. Replace the semantic version tag with the specific commit
hash that corresponds to the v3.11.3 release to ensure supply-chain
reproducibility and immutability. Look up the commit hash for the v3.11.3
release on the nlohmann/json repository and update the GIT_TAG value
accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a598f383-709c-44c7-8184-8fa1ba896a0d

📥 Commits

Reviewing files that changed from the base of the PR and between e017437 and 3771f95.

📒 Files selected for processing (42)
  • CMakeLists.txt
  • docs/source/device/index.rst
  • docs/source/device/oglo.rst
  • examples/oglo_tactile/README.md
  • examples/oglo_tactile/oglo_heatmap.py
  • examples/oglo_tactile/oglo_teleop_record.py
  • src/core/deviceio_base/cpp/inc/deviceio_base/oglo_tactile_tracker_base.hpp
  • src/core/deviceio_trackers/cpp/CMakeLists.txt
  • src/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hpp
  • src/core/deviceio_trackers/cpp/oglo_tactile_tracker.cpp
  • src/core/deviceio_trackers/python/deviceio_trackers_init.py
  • src/core/deviceio_trackers/python/tracker_bindings.cpp
  • src/core/live_trackers/cpp/CMakeLists.txt
  • src/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hpp
  • src/core/live_trackers/cpp/live_deviceio_factory.cpp
  • src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.cpp
  • src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.hpp
  • src/core/mcap/cpp/inc/mcap/recording_traits.hpp
  • src/core/python/deviceio_init.py
  • src/core/schema/fbs/oglo_tactile.fbs
  • src/core/schema/python/CMakeLists.txt
  • src/core/schema/python/oglo_tactile_bindings.h
  • src/core/schema/python/schema_init.py
  • src/core/schema/python/schema_module.cpp
  • src/core/schema_tests/python/test_oglo_tactile.py
  • src/plugins/oglo_tactile/CMakeLists.txt
  • src/plugins/oglo_tactile/README.md
  • src/plugins/oglo_tactile/ble/oglo_ble_client.cpp
  • src/plugins/oglo_tactile/ble/oglo_ble_client.hpp
  • src/plugins/oglo_tactile/ble/oglo_ble_client_simpleble.cpp
  • src/plugins/oglo_tactile/ble/oglo_ble_client_stub.cpp
  • src/plugins/oglo_tactile/main.cpp
  • src/plugins/oglo_tactile/oglo_config.cpp
  • src/plugins/oglo_tactile/oglo_config.hpp
  • src/plugins/oglo_tactile/oglo_glove_sink.cpp
  • src/plugins/oglo_tactile/oglo_glove_sink.hpp
  • src/plugins/oglo_tactile/oglo_packet_parser.cpp
  • src/plugins/oglo_tactile/oglo_packet_parser.hpp
  • src/plugins/oglo_tactile/oglo_tactile_plugin.cpp
  • src/plugins/oglo_tactile/oglo_tactile_plugin.hpp
  • src/plugins/oglo_tactile/plugin.yaml
  • src/plugins/oglo_tactile/tests/test_oglo_packet_parser.cpp

Comment thread src/plugins/oglo_tactile/main.cpp
Comment thread src/plugins/oglo_tactile/oglo_tactile_plugin.cpp
@styu12 styu12 force-pushed the feat/oglo-device branch 4 times, most recently from 65e0224 to 6d622df Compare June 22, 2026 13:20
Integrate OGLO tactile gloves (80 taxels + 6-axis IMU per hand, 100 Hz over BLE)
as an IsaacTeleop device, following docs/source/device/add_device.rst. This
enables synchronized MetaQuest (hand/head) + OGLO data collection into a single
MCAP, with an optional in-headset tactile heatmap overlay (TeleViz).

Contents:
- Schema: src/core/schema/fbs/oglo_tactile.fbs (OgloGloveSample / Tracked /
  Record) plus C++ and Python bindings and a schema pytest.
- Plugin: src/plugins/oglo_tactile - BLE -> config-driven packed12-v5 parser ->
  SchemaPusher (Mode 2) or local MCAP (Mode 1). The transport is isolated behind
  the OgloBleClient interface; a dependency-free unit test validates the wire
  decode against the firmware packing reference.
- Tracker: OgloTactileTracker + LiveOgloTactileTrackerImpl (SchemaTracker reader,
  mirroring the generic_3axis_pedal pattern), factory registration, and recording
  traits.
- Example: examples/oglo_tactile - end-to-end record demo, the heatmap renderer,
  and an offline Foxglove (CompressedImage) exporter.
- Docs: docs/source/device/oglo.rst (registered in the device toctree).

BLE backend: the plugin talks to BlueZ directly over libdbus (AFL-2.1,
permissive), so it carries no copyleft dependency and connects out of the box on
any Linux host with the BlueZ daemon. The transport sits behind the OgloBleClient
interface (make_ble_client()), so an alternative backend can be dropped in
without touching the parser, schema, or tracker.

Testing:
- ctest -R oglo_packet_parser  (wire decode vs firmware packing reference)
- ctest -R schema_test_oglo_tactile  (schema bindings)
- Manual: MetaQuest + two gloves -> one synchronized MCAP
  (hands/head/oglo_left/oglo_right) on a shared DeviceDataTimestamp clock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jerry Han <jerry@opengraphlabs.com>
@styu12 styu12 force-pushed the feat/oglo-device branch from 6d622df to d9cbfaf Compare June 22, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants