RUM Phase 1: stats collection + viewer consent/upload

.github/skills/testing.md

@@ -13,9 +13,14 @@
 
 ## Test Framework
 
-- librealsense uses a **custom Python-based test framework**
-- The test orchestrator is `unit-tests/run-unit-tests.py`
-- Tests must be run **from the `unit-tests/` directory**
+Two frameworks, split by test language:
+
+- **C++ tests** (`test-*.cpp`) are orchestrated by the legacy runner `unit-tests/run-unit-tests.py`.
+  Keeping that runner for C++ testing is its only remaining purpose.
+- **Python tests** (`pytest-*.py`) are pytest only. See `.github/skills/pytest-infra.md`.
+  No legacy `test-*.py` scripts remain, and `run-unit-tests.py` never collects `pytest-*.py`.
+
+Either way, run **from the `unit-tests/` directory**.
 
 ## Test Categories
 
@@ -31,7 +36,21 @@
 | `unit-tests/log/` | Logging | Logging infrastructure tests |
 | `unit-tests/types/` | Types | Type system tests |
 
-## Running All Tests
+## Running the pytest Tests
+
+```bash
+cd unit-tests
+python3 -m pytest -v                        # all collectable tests
+python3 -m pytest -v live/frames            # one directory
+python3 -m pytest -v pytest-fw-update.py    # one file
+python3 -m pytest -v -k "hdr and not preset"   # filter by test name
+```
+
+Flags are registered in `conftest.py`: `--live`, `--not-live`, `--device`, `--exclude-device`,
+`--context`, `--tag`, `--repeat`, `--reruns`, `--debug`, `--rslog`, `--test-dir`. Note that `-s`
+disables the per-test log files. See `.github/skills/pytest-infra.md` for fixtures and markers.
+
+## Running the C++ Tests (legacy runner)
 
 Navigate to the `unit-tests/` directory and run:
 
@@ -57,21 +76,23 @@ For full usage and all available flags:
 python3 run-unit-tests.py --help
 ```
 
-## Running Specific Tests
+## Running Specific C++ Tests
+
+The flags below belong to `run-unit-tests.py`, so they select among the C++ tests.
+For pytest use `-k`, `-m` and the flags listed above.
 
 ### By Name (Regex)
 
 Use `-r` / `--regex` to run tests whose names match a regular expression:
 
 ```bash
-python3 run-unit-tests.py -s -r "test-hdr"
-python3 run-unit-tests.py -s -r "test-metadata"
-python3 run-unit-tests.py -s --regex "test-stream.*"
+python3 run-unit-tests.py -s -r "test-log-vs-LOG"
+python3 run-unit-tests.py -s --regex "rsutils-string.*"
 ```
 
 **Test name derivation**: the orchestrator builds a test's name from its path relative to `unit-tests/`, replacing directory separators with `-` and stripping the leading `test-` from the filename. For example:
-- `live/hw-reset/test-stress.py` → `test-live-hw-reset-stress`
-- `func/test-hdr.py` → `test-func-hdr`
+- `rsutils/string/test-hexarray.cpp` becomes `test-rsutils-string-hexarray`
+- `log/test-vs-LOG-shared.cpp` becomes `test-log-vs-LOG-shared`
 
 So when using `-r`, omit the `test-` filename prefix and join subdirectories with `-`.
 
@@ -80,18 +101,18 @@ So when using `-r`, omit the `test-` filename prefix and join subdirectories wit
 Use `--skip-regex` to exclude tests whose names match:
 
 ```bash
-python3 run-unit-tests.py -s --skip-regex "test-fw-update"
+python3 run-unit-tests.py -s --skip-regex "test-rsutils-string-hexarray"
 ```
 
 ### By Tag
 
-Use `-t` / `--tag` to run tests with a specific tag. Tags are assigned automatically based on:
-- File type: `exe` (C++ binaries) or `py` (Python scripts)
+Use `-t` / `--tag` to run tests with a specific tag. Tags come from:
+- File type: `exe` (C++ binaries). The `py` tag matches nothing now that no legacy `test-*.py` remain.
 - Directory location: e.g., tests in `unit-tests/live/` get the `live` tag
+- An explicit `//#test:tag <name>` directive in the source file
 
 ```bash
 python3 run-unit-tests.py -s -t live          # run only live tests
-python3 run-unit-tests.py -s -t py            # run only Python tests
 python3 run-unit-tests.py -s -t exe           # run only compiled C++ tests
 python3 run-unit-tests.py -s -t live -t exe   # run tests that have BOTH tags
 ```
@@ -136,31 +157,34 @@ python3 run-unit-tests.py --rslog          # enable LibRS debug logging in tests
 python3 run-unit-tests.py --debug          # enable framework debug output; also prints "test took X seconds" per test
 ```
 
-## Running Nightly-Only Tests
+## Running Nightly-Only and Weekly Tests
 
-Some tests are marked `# test:donotrun:!nightly` and are **skipped by default**. Pass `--context nightly` to enable them:
+Both runners take `--context`, which accepts a **space-separated list**.
+
+pytest tests gated with `@pytest.mark.context("nightly")` are skipped unless the context is passed:
 
 ```bash
-python3 run-unit-tests.py --context nightly -r hw-reset-stress ../build/Release
+python3 -m pytest -v live/hw-reset/pytest-stress.py --context nightly
 ```
 
-## Running Weekly Tests
-
-Weekly tests use a higher iteration count / longer timeout (controlled by `'weekly' in test.context` inside the test). The `--context` flag accepts a **space-separated list**, so to run a nightly-guarded test with weekly behaviour pass **both** contexts:
+A test that also scales its iteration count or timeout for weekly runs reads the context through the
+`test_context_var` fixture, so pass both words to get nightly collection plus weekly behaviour:
 
 ```bash
-# 'nightly' satisfies the test:donotrun:!nightly guard
-# 'weekly' activates higher iteration counts and longer timeouts inside the test
-python3 run-unit-tests.py --context "nightly weekly" -r hw-reset-stress ../build/Release
+python3 -m pytest -v live/hw-reset/pytest-stress.py --context "nightly weekly"
 ```
 
-Passing `--context weekly` alone is **not sufficient** — the `test:donotrun:!nightly` directive will still filter the test out.
+C++ tests use a `//#test:donotrun:<context>` directive in the source file, evaluated by
+`run-unit-tests.py` against the same `--context` list.
 
 ## Repeating and Retrying
 
 ```bash
 python3 run-unit-tests.py --repeat 3       # repeat each test 3 times
 python3 run-unit-tests.py --retry 2        # retry failed tests up to 2 times
+
+python3 -m pytest --repeat 3               # pytest: repeat each file's tests 3 times
+python3 -m pytest --reruns 2               # pytest: retry a failed test up to 2 times
 ```
 
 ## Recording and Playback (Mock Hardware)
@@ -185,7 +209,7 @@ This is useful for:
 The `UNIT_TESTS_ARGS` CMake variable passes arguments to `unit-test-config.py` during configuration:
 
 ```bash
-cmake .. -DBUILD_UNIT_TESTS=ON -DUNIT_TESTS_ARGS="-t live -r test-streaming"
+cmake .. -DBUILD_UNIT_TESTS=ON -DUNIT_TESTS_ARGS="-t live -r rsutils-string.*"
 ```
 
 ## Using a Custom Test Directory
@@ -196,17 +220,16 @@ python3 run-unit-tests.py --test-dir /path/to/custom/tests
 
 ## Custom Firmware for Testing
 
-The SDK no longer ships a bundled firmware blob, so `test-fw-update` **requires** a custom firmware path for the device under test. Without one it logs a warning and skips. Download a signed `.bin` from <https://dev.realsenseai.com/docs/firmware-updates>, then:
+The SDK no longer ships a bundled firmware blob, so `pytest-fw-update` **requires** a custom firmware path for the device under test. Without one it skips. Download a signed `.bin` from <https://dev.realsenseai.com/docs/firmware-updates>, then:
 
 ```bash
-python3 run-unit-tests.py --custom-fw-d400 /path/to/firmware.bin
-python3 run-unit-tests.py --custom-fw-d555 /path/to/firmware.bin
-python3 run-unit-tests.py --custom-fw-d585 /path/to/firmware.bin
+python3 -m pytest pytest-fw-update.py --custom-fw-d400 /path/to/firmware.bin
+python3 -m pytest pytest-fw-update.py --custom-fw-d555 /path/to/firmware.bin
+python3 -m pytest pytest-fw-update.py --custom-fw-d585 /path/to/firmware.bin
 ```
 
-`--custom-fw-d585` is only ever flashed onto a device whose name contains "D585" but not "D585S"
-(e.g. "D585 Prototype") -- the safety SKU D585S is never updated with it, even though the
-`#test:device each(D585)` directive dispatches the test for both (D585S just logs a skip).
+`--custom-fw-d585` targets the non-safety D585 only (e.g. "D585 Prototype"). The safety SKU is kept
+out of the test by `pytest.mark.device_exclude("D585S")`, so it is never collected and never flashed.
 
 ## Troubleshooting
 

.github/workflows/buildsCI.yaml

@@ -36,7 +36,7 @@ jobs:
 
 
   #--------------------------------------------------------------------------------
-  Win_SH_EX_CfU:  # Windows, shared, with Examples & Tools, and Check for Updates
+  Win_SH_EX_CfU_Stats:  # Windows, shared, with Examples & Tools, Check for Updates, and RUM stats (ENABLE_STATS)
     runs-on: windows-2025
     timeout-minutes: 60
     steps:
@@ -68,7 +68,7 @@ jobs:
         cd ${{env.WIN_BUILD_DIR}}
         pwd
         ls        
-        cmake ${LRS_SRC_DIR} -A x64 -DBUILD_SHARED_LIBS=true -DBUILD_EXAMPLES=true -DBUILD_TOOLS=true -DCHECK_FOR_UPDATES=true
+        cmake ${LRS_SRC_DIR} -A x64 -DBUILD_SHARED_LIBS=true -DBUILD_EXAMPLES=true -DBUILD_TOOLS=true -DCHECK_FOR_UPDATES=true -DENABLE_STATS=ON
 
     - name: Build
       # Build your program with the given configuration
@@ -368,7 +368,7 @@ jobs:
 
 
   #--------------------------------------------------------------------------------
-  U22_SH_Py_CI:  # Ubuntu 2020, Shared, Python, LibCI with executables
+  U22_SH_Py_CI:  # Ubuntu 2020, Shared, Python, LibCI with executables, and RUM stats (ENABLE_STATS)
     runs-on: ubuntu-22.04
     timeout-minutes: 60
     steps:
@@ -397,7 +397,7 @@ jobs:
       shell: bash
       run: |
         cd build
-        cmake .. -DCMAKE_BUILD_TYPE=${{env.LRS_RUN_CONFIG}} -DBUILD_SHARED_LIBS=true -DBUILD_EXAMPLES=false -DBUILD_TOOLS=true -DBUILD_UNIT_TESTS=true -DUNIT_TESTS_ARGS="--not-live --context=linux" -DCHECK_FOR_UPDATES=false -DBUILD_WITH_DDS=false -DBUILD_PYTHON_BINDINGS=true -DPYTHON_EXECUTABLE=$(which python3)
+        cmake .. -DCMAKE_BUILD_TYPE=${{env.LRS_RUN_CONFIG}} -DBUILD_SHARED_LIBS=true -DBUILD_EXAMPLES=false -DBUILD_TOOLS=true -DBUILD_UNIT_TESTS=true -DUNIT_TESTS_ARGS="--not-live --context=linux" -DCHECK_FOR_UPDATES=false -DBUILD_WITH_DDS=false -DBUILD_PYTHON_BINDINGS=true -DPYTHON_EXECUTABLE=$(which python3) -DENABLE_STATS=ON
         cmake --build . -- -j4  
 
     - name: LibCI
@@ -413,8 +413,8 @@ jobs:
         python3 -m pytest unit-tests/ --color=no --debug -s --not-live --context "linux"
 
   #--------------------------------------------------------------------------------
-  U24_SH_Py_CI_SYS_JSON:  # Ubuntu 24.04, Shared, Python, LibCI with executables and system provided nlohmann_json library
-    runs-on: ubuntu-24.04
+  U26_SH_Py_CI_SYS_JSON:  # Ubuntu 26.04, Shared, Python, LibCI with executables and system provided nlohmann_json library
+    runs-on: ubuntu-26.04
     timeout-minutes: 60
     steps:
     - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd #v6.0.2

.github/workflows/pre-release.yml

@@ -58,6 +58,10 @@ jobs:
       ROS_DISTRO: ${{ matrix.ros_distro }}
       PRERELEASE: true
       BASEDIR: ${{ github.workspace }}/.work
+      # industrial_ci defaults the prerelease host container to ros:noetic-ros-core
+      # (focal, EOL). Must stay a ros:* image — setting DOCKER_IMAGE disables
+      # industrial_ci's ROS apt setup, so colcon must be installable already.
+      DOCKER_IMAGE: ros:jazzy-ros-core
 
     steps:
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 #v4

CMake/external_libcurl.cmake

@@ -1,4 +1,4 @@
-if(CHECK_FOR_UPDATES)
+if(CHECK_FOR_UPDATES OR ENABLE_STATS)
 
     pop_security_flags() # remove security flags
 
@@ -65,4 +65,4 @@ if(CHECK_FOR_UPDATES)
     endif()
 
     push_security_flags()
-endif() #CHECK_FOR_UPDATES
+endif() #CHECK_FOR_UPDATES OR ENABLE_STATS

CMake/global_config.cmake

@@ -70,6 +70,10 @@ macro(global_set_flags)
         add_definitions(-DBUILD_SHARED_LIBS)
     endif()
 
+    if (ENABLE_STATS)
+        add_definitions(-DENABLE_STATS)
+    endif()
+
     if (BUILD_WITH_CUDA)
         include(CMake/cuda_config.cmake)
     endif()
@@ -87,10 +91,17 @@ macro(global_set_flags)
             message(STATUS "CHECK_FOR_UPDATES depends on BUILD_GRAPHICAL_EXAMPLES flag, turning it off..")
             set(CHECK_FOR_UPDATES false)
         else()
-            include(CMake/external_libcurl.cmake)
             add_definitions(-DCHECK_FOR_UPDATES)
         endif()
     endif()
+
+    # libcurl is needed by sw-update (CHECK_FOR_UPDATES) and RUM cloud upload (ENABLE_STATS).
+    # BUILD_WITH_LIBCURL is the derived "curl is linked" guard - gates the shared "Online Services"
+    # viewer tab that hosts both features.
+    if(CHECK_FOR_UPDATES OR ENABLE_STATS)
+        include(CMake/external_libcurl.cmake)
+        add_definitions(-DBUILD_WITH_LIBCURL)
+    endif()
 
     add_definitions(-D${BACKEND} -DUNICODE)
 endmacro()

CMake/lrs_options.cmake

@@ -68,4 +68,5 @@ option(USE_EXTERNAL_LZ4 "Use externally build LZ4 library instead of building an
 option(USE_EXTERNAL_NLOHMANN_JSON "Use an externally built nlohmann-json development package instead of downloading it as part of this build" OFF)
 option(BUILD_ASAN "Enable AddressSanitizer" OFF)
 option(BUILD_ROSBAG2 "Build and use rosbag2 recording system" ON) # temporary flag, should be removed when deprecated ROSBAG1 recording system is removed
+option(ENABLE_STATS "Enable RUM (Real User Monitoring) usage-statistics collection" OFF)
 mark_as_advanced(BUILD_ASAN)

CMake/unix_config.cmake

@@ -18,6 +18,11 @@ macro(os_set_flags)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pedantic -Wno-missing-field-initializers")
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-switch -Wno-multichar -Wsequence-point -Wformat -Wformat-security")
 
+    # Default -ffp-contract=fast fuses a*b+c*d into an FMA wherever the ISA has one (x86-64-v3
+    # on Ubuntu 26.04, always on aarch64), dropping a rounding and shifting filter output 1 LSB.
+    set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS}   -ffp-contract=off")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -ffp-contract=off")
+
     execute_process(COMMAND ${CMAKE_C_COMPILER} -dumpmachine OUTPUT_VARIABLE MACHINE)
     if(${MACHINE} MATCHES "arm64-*" OR ${MACHINE} MATCHES "aarch64-*")
         if(BUILD_WITH_NEON)

common/CMakeLists.txt

@@ -79,9 +79,15 @@ set(COMMON_SRC
     "${CMAKE_CURRENT_LIST_DIR}/textual-icons.h"
     )
 
+set(HTTP_FILES
+    "${CMAKE_CURRENT_LIST_DIR}/http/curl-wrapper.h"
+    "${CMAKE_CURRENT_LIST_DIR}/http/curl-wrapper.cpp"
+    "${CMAKE_CURRENT_LIST_DIR}/http/http-downloader.h"
+    "${CMAKE_CURRENT_LIST_DIR}/http/http-downloader.cpp"
+    "${CMAKE_CURRENT_LIST_DIR}/http/http-uploader.h"
+    )
+
 set(SW_UPDATE_FILES
-    "${CMAKE_CURRENT_LIST_DIR}/sw-update/http-downloader.h"
-    "${CMAKE_CURRENT_LIST_DIR}/sw-update/http-downloader.cpp"
     "${CMAKE_CURRENT_LIST_DIR}/sw-update/dev-updates-profile.h"
     "${CMAKE_CURRENT_LIST_DIR}/sw-update/dev-updates-profile.cpp"
     "${CMAKE_CURRENT_LIST_DIR}/sw-update/versions-db-manager.h"
@@ -98,10 +104,17 @@ set(UTILITIES_FILES
     "${CMAKE_CURRENT_LIST_DIR}/utilities/imgui/wrap.cpp"
     )
 
+set(RUM_UPLOADER_FILES
+    "${CMAKE_CURRENT_LIST_DIR}/rum-uploader/rum-uploader.h"
+    "${CMAKE_CURRENT_LIST_DIR}/rum-uploader/rum-uploader.cpp"
+    )
+
 set(COMMON_SRC
     ${COMMON_SRC}
+    ${HTTP_FILES}
     ${SW_UPDATE_FILES}
     ${REFLECTIVITY_FILES}
     ${UTILITIES_FILES}
+    ${RUM_UPLOADER_FILES}
     )
 

common/d500-on-chip-calib.cpp

@@ -170,8 +170,8 @@ namespace rs2
 
     bool d500_on_chip_calib_manager::uses_interactive_triggered_calibration() const
     {
-        // Mirrors ds::d5x5_interactive_triggered_calibration_pids in src/ds/d500/d500-private.h —
-        // the viewer cannot include SDK-internal headers. Keep the two lists in sync when adding new PIDs.
+        // Mirrors ds::d5x5_family_pids in src/ds/d500/d500-private.h — the viewer cannot include
+        // SDK-internal headers. Keep the two lists in sync when adding new PIDs.
         static const std::set< std::string > interactive_triggered_calibration_pids = {
             "0C01", "0C02", "0C03", "0C04", "0C05", "0C06", "0C07", "0C08"
         };