Skip to content

CI: give GPU unit-test containers a working NVIDIA Vulkan driver - #813

Merged
swahtz merged 1 commit into
openvdb:mainfrom
swahtz:ci/vulkan-in-gpu-tests
Oct 9, 2026
Merged

swahtz merged 1 commit into
openvdb:mainfrom
swahtz:ci/vulkan-in-gpu-tests

Conversation

@swahtz

@swahtz swahtz commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Summary

tests/unit/test_viz_shutdown.py::test_viewer_shutdown_with_gpu_buffers, added in #810, needs a Vulkan device. On the GPU runners it fails with Failed to create Vulkan instance. VkResult: -9 (VK_ERROR_INCOMPATIBLE_DRIVER). #810's unit-test jobs were skipped, so #812 is the first PR to hit it. Every PR based on current main will fail the same way.

This PR gives the three GPU unit-test jobs a working NVIDIA Vulkan driver.

Cause

The runner AMI is fine. Its 580.65.06 driver includes the Vulkan driver, and vulkaninfo on the host lists the L4. The unit-test containers could not load that driver, for two reasons:

  1. The CUDA images set NVIDIA_DRIVER_CAPABILITIES=compute,utility. The container toolkit therefore does not mount the Vulkan driver (libGLX_nvidia and /etc/vulkan/icd.d/nvidia_icd.json).
  2. The driver depends on libraries the slim images lack.
    • libGLX_nvidia links against libXext.so.6.
    • It also opens libEGL.so.1 at run time.
    • Without either one, the driver loads but returns no vkCreateInstance, and instance creation fails with -9.

Change

The fvdb Unit Tests jobs in tests.yml (conda), cu130.yml and cu132.yml now set up Vulkan in their containers:

  • All three jobs: set NVIDIA_DRIVER_CAPABILITIES: all.
  • pip jobs: install libvulkan1 libxext6 libegl1.
  • conda job:
    • install libXext libglvnd-egl;
    • set VK_DRIVER_FILES=/etc/vulkan/icd.d/nvidia_icd.json, in case the conda-provided Vulkan loader doesn't search /etc/vulkan.

The gtest and docs jobs are unchanged.

Test plan

  • Diagnosed by hand on a runner instance from the CI AMI (Ubuntu 24.04, driver 580.65.06, L4, container toolkit 1.17.8):
    • The stock nvidia/cuda:13.0.2-cudnn-runtime-ubuntu22.04 container has no NVIDIA Vulkan driver files.
    • With NVIDIA_DRIVER_CAPABILITIES=all but without libxext6, the loader skips the driver (libXext.so.6: cannot open).
    • With libxext6 but without libegl1, the driver loads but returns no vkCreateInstance.
    • With libvulkan1 libxext6 libegl1, vulkaninfo --summary lists NVIDIA L4.
  • CI: these workflows run on pull_request_target, which uses main's workflow definitions. This PR's own unit-test jobs will therefore still fail on the viewer test. The fix takes effect once it merges; the next PR, or a rerun of Support empty JaggedTensor structures and validate raw constructors #812, should then pass test_viz_shutdown.
  • The conda job's VK_DRIVER_FILES setting is untested. If that job still fails after merge, its loader output will show whether the variable is needed or not enough.

🤖 Generated with Claude Code

test_viz_shutdown (added in openvdb#810) creates a Vulkan device, and on the
GPU runners it fails with VkResult -9 (VK_ERROR_INCOMPATIBLE_DRIVER).
The runner host's driver includes Vulkan, but the unit-test containers
could not load it:

- The CUDA images request NVIDIA_DRIVER_CAPABILITIES=compute,utility, so
  the container toolkit does not mount the Vulkan driver.
- libGLX_nvidia links against libXext and opens libEGL at run time, and
  neither is in the slim CUDA images.

Set NVIDIA_DRIVER_CAPABILITIES=all in the three unit-test jobs and
install the Vulkan loader, libXext, and libEGL. The conda job also
points its conda-provided loader at the mounted driver manifest.

Verified by hand on the runner AMI (Ubuntu 24.04, driver 580.65.06,
L4): with these settings vulkaninfo inside
nvidia/cuda:13.0.2-cudnn-runtime-ubuntu22.04 lists the NVIDIA L4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
@swahtz
swahtz requested a review from a team as a code owner October 9, 2026 01:54
@swahtz
swahtz requested review from phapalova and sifakis October 9, 2026 01:54
@swahtz swahtz added the CI Issues related to the Github actions CI/CD. For build issues use CMake/Build label Oct 9, 2026
@swahtz swahtz self-assigned this Oct 9, 2026
@swahtz
swahtz merged commit f655b75 into openvdb:main Oct 9, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Issues related to the Github actions CI/CD. For build issues use CMake/Build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant