Add OGLO tactile glove device (plugin, tracker, schema, example)

CMakeLists.txt

@@ -59,6 +59,7 @@ endif()
 # Options
 option(BUILD_PLUGINS "Build plugins" ON)
 option(BUILD_PLUGIN_OAK_CAMERA "Build OAK camera plugin (requires vcpkg for DepthAI v3.x)" OFF)
+option(BUILD_PLUGIN_OGLO "Build OGLO tactile glove plugin (BLE, Linux only; fetches SimpleBLE + nlohmann/json)" OFF)
 option(BUILD_EXAMPLES "Build examples" ON)
 option(BUILD_EXAMPLE_TELEOP_ROS2 "Build only the teleop_ros2 example (e.g. for Docker)" OFF)
 option(BUILD_TESTING "Build unit tests" ON)
@@ -169,6 +170,9 @@ if(BUILD_PLUGINS)
     if(BUILD_PLUGIN_OAK_CAMERA)
         add_subdirectory(src/plugins/oak)
     endif()
+    if(BUILD_PLUGIN_OGLO)
+        add_subdirectory(src/plugins/oglo_tactile)
+    endif()
 endif()
 
 # Formatting enforcement (runs on Linux by default)

docs/source/device/index.rst

@@ -23,3 +23,4 @@ See the `Plugins directory <https://github.com/NVIDIA/IsaacTeleop/tree/main/src/
    haptic_feedback
    manus
    oak
+   oglo

docs/source/device/oglo.rst

@@ -0,0 +1,94 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+OGLO Tactile Glove Plugin
+=========================
+
+C++ plugin that streams **OGLO** tactile gloves over BLE and records them through
+the standard DeviceIO path. Each glove (one BLE device per hand) provides **80
+tactile taxels** (5 fingers × 4×4) plus a **6-axis IMU** at 100 Hz. Source and
+plugin README: :code-file:`src/plugins/oglo_tactile/README.md`.
+
+This plugin follows the :doc:`add_device` pattern (FlatBuffer schema → plugin
+``SchemaPusher`` → tracker → MCAP) and is modeled on the OAK camera plugin,
+including its two recording modes.
+
+.. contents:: On this page
+   :local:
+   :depth: 2
+
+Components
+----------
+
+- **Schema** — :code-file:`src/core/schema/fbs/oglo_tactile.fbs`
+  (``OgloGloveSample`` / ``OgloGloveSampleTracked`` / ``OgloGloveSampleRecord``).
+- **Plugin** — :code-dir:`src/plugins/oglo_tactile` (BLE read → parse → push/record).
+- **Tracker** — ``OgloTactileTracker``
+  (:code-file:`src/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hpp`)
+  with live backend ``LiveOgloTactileTrackerImpl``
+  (:code-file:`src/core/live_trackers/cpp/live_oglo_tactile_tracker_impl.cpp`).
+
+Build
+-----
+
+Linux only (BlueZ). The plugin is off by default.
+
+.. code-block:: bash
+
+   sudo apt install libdbus-1-dev          # BlueZ daemon + libdbus for the BLE backend
+   cmake -B build -DBUILD_PLUGIN_OGLO=ON
+   cmake --build build --target oglo_tactile_plugin --parallel
+
+``nlohmann/json`` (MIT) is fetched automatically.
+
+.. note::
+
+   **BLE backend.** The plugin talks to BlueZ directly over `libdbus
+   <https://www.freedesktop.org/wiki/Software/dbus/>`_ (AFL-2.1, permissive), so
+   the build carries **no copyleft dependency** and works out of the box. The
+   transport is isolated behind the ``OgloBleClient`` interface
+   (:code-file:`src/plugins/oglo_tactile/ble/oglo_ble_client.hpp`), so an
+   alternative backend can be dropped in via ``make_ble_client()`` without
+   touching the parser, schema, or tracker.
+
+Usage
+-----
+
+The plugin streams one glove (``--side``) and either records a local MCAP
+(**Mode 1**) or pushes via OpenXR for a host tracker (**Mode 2**):
+
+.. code-block:: bash
+
+   # Mode 1 — standalone local MCAP (no OpenXR / TeleopSession)
+   ./build/src/plugins/oglo_tactile/oglo_tactile_plugin --side right --mcap-filename=right.mcap
+
+   # Mode 2 — push for a host OgloTactileTracker into a shared session MCAP
+   ./build/src/plugins/oglo_tactile/oglo_tactile_plugin --side right --collection-prefix=oglo
+
+The packet parser reads the device **Config characteristic** first and branches
+on the notify ``flags`` byte (``0x04`` packed12 v5 — primary; ``0x02``/``0x01``
+legacy schema-4 fallback), so payload sizes are never hardcoded.
+
+Recorded data
+-------------
+
+Channels ``oglo_left`` / ``oglo_right`` carry ``core.OgloGloveSampleRecord``:
+``seq``, ``device_time_us``, ``taxels[80]`` (raw 12-bit, ``finger,row,col``),
+and a 6-axis IMU, each with a ``DeviceDataTimestamp`` whose
+``sample_time_local_common_clock`` is on the shared host monotonic clock — so
+OGLO aligns in time with hand/head streams.
+
+A complete data-collection demo (MetaQuest hand/head + both gloves + a live
+in-headset tactile heatmap) lives at
+:code-dir:`examples/oglo_tactile`; see its ``README.md``.
+
+Tests
+-----
+
+``test_oglo_packet_parser`` validates the wire decode against the firmware's own
+12-bit packing reference, plus the schema-4 fallback and malformed-packet
+rejection:
+
+.. code-block:: bash
+
+   ctest --test-dir build -R oglo_packet_parser --output-on-failure

examples/oglo_tactile/README.md

@@ -0,0 +1,131 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# OGLO + MetaQuest Data-Collection Demo
+
+End-to-end demo on a Linux host:
+
+- **MetaQuest** connects over the same WiFi via **CloudXR** → streams hand + head pose.
+- **OGLO gloves** (2) connect over **BLE** → 80 taxels + 6-axis IMU per hand at 100 Hz.
+- One **time-synced MCAP** records Quest hand/head + both gloves.
+- A live **tactile heatmap** is composited into the operator's Quest view (TeleViz).
+
+```
+ MetaQuest ──CloudXR/WiFi──► laptop OpenXR runtime ──► HandTracker/HeadTracker ─┐
+ OGLO L/R  ──BLE──► oglo_tactile plugins ──SchemaPusher──► OgloTactileTracker ──┤
+                                                                                ├─► MCAP (synced)
+   oglo_heatmap ──► QuadLayer ──► TeleViz compositor ──CloudXR──► Quest screen ─┘
+```
+
+`oglo_teleop_record.py` orchestrates everything (launches the plugins, runs the
+session, records, and draws the overlay). `oglo_heatmap.py` is the renderer.
+
+---
+
+## A. One-time setup
+
+1. **System deps**
+
+   ```bash
+   sudo apt install libdbus-1-dev          # BlueZ for the BLE plugin
+   pip install pillow numpy                 # heatmap renderer
+   pip install cupy-cuda12x                 # headset overlay GPU upload (match your CUDA)
+   ```
+   `cupy` is only needed for the in-headset overlay; recording works without it.
+
+2. **Build** (plugin + viz + python wheel)
+
+   ```bash
+   cd IsaacTeleop
+   cmake -B build -DBUILD_PLUGIN_OGLO=ON -DBUILD_VIZ=ON
+   cmake --build build --parallel
+   cmake --install build           # builds the isaacteleop python wheel/package
+   ctest --test-dir build -R oglo_packet_parser --output-on-failure   # parser sanity
+   ```
+   The BLE backend (BlueZ via libdbus, permissive) connects out of the box.
+
+3. **Gloves**: confirm both run firmware `0.7.1` (schema 5), are charged, and
+   advertise `OGLO LEFT` / `OGLO RIGHT`. Validate each glove standalone first:
+
+   ```bash
+   ./build/src/plugins/oglo_tactile/oglo_tactile_plugin --side right --mcap-filename=right.mcap
+   # press taxels; Ctrl+C; expect schema_ver:5, ~100Hz, seq continuous
+   ```
+
+4. **CloudXR**: install per the IsaacTeleop Quick Start (`pip install 'isaacteleop[cloudxr]'`).
+
+---
+
+## B. Demo scenario (run in order)
+
+**Terminal 1 — CloudXR runtime**
+```bash
+cd ~/Documents/IsaacTeleop
+source scripts/setup_cloudxr_env.sh
+python -m isaacteleop.cloudxr
+# Note the printed web-client URL, e.g.  https://<laptop-ip>:48322/
+```
+
+**MetaQuest — connect** (same WiFi)
+- Open the headset browser → the CloudXR web client at
+  `https://nvidia.github.io/IsaacTeleop/client` → enter the laptop IP → accept the
+  self-signed certificate → **CONNECT**. (Or run the server with `--host-client`
+  and open `https://<laptop-ip>:48322/client/` instead.)
+- You should now be in the CloudXR view; hand/head tracking is live.
+
+**Terminal 2 — gloves + recording + overlay**
+```bash
+cd ~/Documents/IsaacTeleop
+source scripts/setup_cloudxr_env.sh
+source ~/.cloudxr/run/cloudxr.env          # path printed by step above
+cd examples/oglo_tactile
+python oglo_teleop_record.py
+```
+The script: connects both gloves over BLE, starts recording to
+`oglo_teleop_<timestamp>.mcap`, and shows the **two-hand tactile heatmap** in the
+headset. **Keep hands relaxed for ~1 s** at start so the baseline tares.
+
+Press a glove → the matching finger/taxel lights up (YlOrRd) on the Quest screen.
+
+**Stop**: `Ctrl+C` in Terminal 2 (flushes + closes the MCAP), then `Ctrl+C` in Terminal 1.
+
+---
+
+## C. Verify the recording
+
+```bash
+mcap info oglo_teleop_*.mcap
+# channels: hands/{left,right}_hand, head/head, oglo_{left,right}/oglo(+_tracked)
+```
+All streams share `sample_time_local_common_clock`, so Quest pose and OGLO
+tactile align in time.
+
+---
+
+## Useful flags (`oglo_teleop_record.py`)
+
+| Flag | Purpose |
+|------|---------|
+| `--plugin-bin PATH` | Path to `oglo_tactile_plugin` (auto-detected under `build/`). |
+| `--mcap PATH` | Output file (default timestamped). |
+| `--duration S` | Auto-stop after S seconds (0 = until Ctrl+C). |
+| `--raw` | Raw-ADC heatmap (no baseline subtraction). |
+| `--no-overlay` | Record only (skip headset heatmap). |
+| `--overlay-fullscreen` | Fullscreen overlay (facing-safe fallback). |
+| `--panel-dist / --panel-right / --panel-drop / --panel-w` | Head-locked HUD placement/size (meters); default is a small bottom-right panel. |
+| `--no-headlock` (`--panel-y / --panel-z`) | Use a fixed stage-space panel instead of head-locked. |
+| `--plugin-stagger S` | Delay between launching the two gloves (avoids BLE scan contention; default 8 s). |
+| `--no-plugins` | Don't auto-launch plugins (you start them yourself). |
+
+---
+
+## Troubleshooting
+
+- **Glove not found**: `bluetoothctl scan on` should list `OGLO LEFT/RIGHT`. Power-cycle the glove; ensure it isn't already connected elsewhere.
+- **Quest view but no heatmap**: `cupy` missing/mismatched CUDA → see warning in Terminal 2; install the `cupy` build matching your CUDA. Panel off-view → adjust `--panel-right/--panel-drop/--panel-dist`, or use `--overlay-fullscreen`.
+- **One hand not connecting**: usually BLE scan contention — increase `--plugin-stagger` (e.g. 12). Confirm the glove advertises via `bluetoothctl scan on`.
+- **No tactile in MCAP**: confirm the plugins connected (Terminal 2 logs `Connected: side=… schema_ver:5`) and that the tracker `collection_id` (`oglo/left`,`oglo/right`) matches the plugin `--collection-prefix=oglo`.
+- **Quest won't connect**: same WiFi/subnet, firewall allows the CloudXR ports, accept the self-signed cert in the headset browser.
+- **Stutter**: the overlay is auxiliary — drop it with `--no-overlay` to confirm recording is unaffected; lower overlay rate if needed.