moq-video: remaining work (codecs, platforms, decode, HW validation)

[kixelated]

Tracking issue for the gaps in moq-video after the ffmpeg removal + native-codec rework on dev. Audit done against rs/moq-video on the dev branch.

Current state:

• Encode: H.264 via openh264 (software, all platforms), VideoToolbox (macOS), Media Foundation (Windows), NVENC + VAAPI (Linux, feature-gated, unverified). H.265 via VideoToolbox (macOS), Media Foundation (Windows), and NVENC (Linux, feature-gated, unverified).
• Decode: H.264 via VideoToolbox (macOS), Media Foundation/DXVA (Windows), openh264 (software). H.265 via Media Foundation/DXVA (Windows) and VideoToolbox (macOS, feat(moq-video): H.265 hardware decode on macOS (VideoToolbox) #1859); no software fallback. In [Unreleased].
• Capture: camera on all three platforms (AVFoundation / V4L2 / Media Foundation); screen capture on macOS (ScreenCaptureKit) and Windows (DXGI Desktop Duplication).

Decode

• H.264 hardware decode on Windows (Media Foundation) - done in code (DXVA: NVDEC / Intel / AMD; GPU-less host falls back to openh264). In [Unreleased]; HW-validation pending.
• H.264 hardware decode on Linux (NVENC/NVDEC, VAAPI, or V4L2)
• H.265 hardware decode on macOS (VideoToolbox) - done in feat(moq-video): H.265 hardware decode on macOS (VideoToolbox) #1859 (verified end-to-end on Apple M4).
• H.265 hardware decode on Windows (Media Foundation) - done in code (DXVA). In [Unreleased]; HW-unverified (test box had no HEVC decoder MFT installed).
• H.265 hardware decode on Linux (NVDEC / VAAPI)
• H.265 software decode - only if a backend with acceptable real-time performance and no licensing concerns exists; otherwise skip (do NOT add software HEVC decode just to fill the matrix)

Encode

• Linux NVENC H.265 - done in feat(moq-video): NVENC H.265 encode + refresh README #1840 (codec GUID selected from Codec; HW-unverified)
• Linux VAAPI H.265 - VAAPI backend advertises H.264 only today. Update: moq-vaapi 0.0.2 vendors HEVC buffer types (src/buffer/hevc.rs) but its Encoder is hardcoded H.264 (VAProfileH264Main / VAEntrypointEncSlice), so exposing an HEVC encoder is a moq-dev/vaapi change, not just a moq-video flag.
• Software H.265 encode - explicitly out of scope (no software HEVC encode)

Robustness

• NVENC driver-probe (avoid abort on GPU-less box) - done in fix(moq-video): probe NVIDIA driver libs before NVENC init (avoid abort on GPU-less box) #1844. Auto aborted under panic = "abort" when libcuda was missing because cudarc/the SDK panic! on a failed dlopen instead of erroring; Nvenc::open now probes libcuda / libnvidia-encode first and falls back cleanly.
• VAAPI: make moq-vaapi dlopen libva (not a driver-probe) - the original premise was wrong. moq-vaapi 0.0.2 links libva (binary carries NEEDED libva.so.2 / libva-drm.so.2, verified via readelf), it does not dlopen it, and its Encoder::new returns a Result rather than panicking. So unlike NVENC there is no panic-on-miss to guard: a libva-less box fails at process load (before any Rust probe could run), and the no-GPU case (libva present, no usable VA driver) already falls back cleanly via Encoder::new → Err → backend::open → openh264. A probe in open() can't help. Mitigated in feat(moq-video): opt-out nvenc/vaapi features (default-on) + correct libva docs #1860 (merged): corrected the docs that wrongly claimed dlopen, and made nvenc/vaapi default-on opt-out features so a self-compiler can drop the libva/CUDA deps (cargo build -p moq-cli --no-default-features --features "iroh quinn websocket capture"). Still open: the real fix is restoring the documented vaapi_dlopen design (no cargo:rustc-link-lib, no DT_NEEDED) in the separate moq-dev/vaapi crate, then adding the NVENC-style probe — only then does an always-on VAAPI binary load on a libva-less host.

Hardware validation (written but never run on real HW)

• NVENC encode on a Linux + NVIDIA box - confirm pitch alignment, forced-IDR, the chosen cudarc/dlopen feature set, and H.265 VPS/SPS/PPS landing inline ahead of each IDR
• VAAPI encode on a Linux + Intel/AMD box - low-power vs full entrypoint, NV12 upload round-trip, cargo deny license resolution
• VAAPI zero-copy dmabuf input path - currently uses an NV12 surface-upload; add the Frame::DmaBuf + V4L2 VIDIOC_EXPBUF path for NV12-capable sources
• Windows Media Foundation capture on real HW - on-demand open/close (LED off when unwatched), NV12 delivery for MJPEG/YUY2 cameras
• Live camera run per platform - capture needs device permission a headless/agent process can't get

Capture

• Screen/display capture on Linux (PipeWire/portal, or X11/DRM)
• Screen/display capture on Windows (Desktop Duplication) - done in code (DXGI Desktop Duplication, whole-monitor). In [Unreleased]; compile-verified, HW-validation pending.

Codecs (new)

• AV1 - returns when a hardware backend lands (NVENC/VAAPI AV1 on Linux). No software AV1 encode (rav1e too slow for real-time). Software AV1 decode only if perf + licensing are acceptable. Codec enum is #[non_exhaustive] so this is additive.
• VP8 / VP9 - the hang catalog models them but moq-video has no path either way; track here, low priority

Other platform coverage (noted in DESIGN, lower priority)

• Intel QSV-specific encode path (Intel currently goes through VAAPI)
• Raspberry Pi v4l2m2m stateful encoder
• iOS (VideoToolbox is available; not wired)

Docs

• Fix the stale rs/moq-video/README.md - done in feat(moq-video): NVENC H.265 encode + refresh README #1840 (removed the ffmpeg / system-FFmpeg / "decode not implemented" claims; added a capability matrix).

---

Constraints captured from the request: do not add software H.265/AV1 encode at all; only add software H.265/AV1 decode if a backend has good enough real-time performance and no licensing concerns.

(Written by Claude)

[coderabbitai]

🔗 Related PRs

#1767 - feat(hang): add Catalog.Producer/Consumer wrapping @moq/json [merged]
#1781 - fix(hang): omit empty HEVC constraint component in codec string [merged]
#1782 - fix(ci): unbreak the self-hosted A1 runner (libmoq clippy + ffmpeg bindgen headers) [merged]
#1812 - fix(moq-mux): carry all distinct SPS/PPS/VPS through transmux, not just the last seen [merged]
#1815 - feat(moq-mux): generic verbatim MPEG-TS carriage (mpegts catalog section) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[kixelated]

Audited the open boxes against dev and updated the list. Summary:

Checked off (done in code, merged to dev in [Unreleased]):

• H.264 hardware decode on Windows — Media Foundation / DXVA (decode/backend/mediafoundation.rs).
• H.265 hardware decode on Windows — same backend (HW-unverified: the test box had no HEVC decoder MFT).
• Screen/display capture on Windows — DXGI Desktop Duplication (capture/desktopduplication.rs).

These are wired in the selectors and documented in the CHANGELOG [Unreleased]; the only caveat is on-hardware validation.

H.265 hardware decode on macOS (VideoToolbox) — implemented in #1859 and verified end-to-end on an Apple M4 (HEVC encode → decode round-trip). Left unchecked pending merge.

VAAPI "driver-probe" box — premise was wrong, reframed. I went to mirror the NVENC driver_available() probe (#1844) and found it doesn't apply to the published moq-vaapi 0.0.2 we depend on. Verified in a rust:latest container:

• Building a crate that depends on moq-vaapi 0.0.2 without libva-dev fails at pkg-config ... libva ("system library libva required by crate moq-vaapi was not found").
• With libva-dev, the linked binary carries NEEDED libva.so.2 and NEEDED libva-drm.so.2 (readelf -d).

So moq-vaapi links libva (its build.rs uses pkg_config::probe), it does not dlopen it, and Encoder::new returns a Result rather than panicking. Consequences:

• A libva-less box fails at process load, before any Rust probe could run — so an open()-time probe can't help (unlike NVENC, where cudarc/the SDK dlopen + panic! at runtime).
• The no-GPU case (libva present, no usable VA driver) already falls back cleanly: Encoder::new → Err → backend::open → openh264.

The dlopen design documented for the old cros-codecs vaapi_dlopen git dep was lost when moq-vaapi was extracted to a published crate. The real fix is restoring dlopen in the moq-dev/vaapi crate (then the NVENC-style probe becomes meaningful). #1860 corrects the moq-video comments + DESIGN doc that wrongly claimed dlopen.

Bonus finding (Linux VAAPI H.265): moq-vaapi 0.0.2 vendors HEVC buffer types (src/buffer/hevc.rs) but its Encoder is hardcoded H.264 (VAProfileH264Main), so that box is also a moq-dev/vaapi change, not just a moq-video flag.

(Written by Claude)

[kixelated]

#1859 and #1860 merged to dev.

• ✅ H.265 hardware decode on macOS (VideoToolbox) — done in feat(moq-video): H.265 hardware decode on macOS (VideoToolbox) #1859, verified end-to-end on Apple M4.
• VAAPI box — feat(moq-video): opt-out nvenc/vaapi features (default-on) + correct libva docs #1860 mitigated it: corrected the docs that wrongly claimed dlopen, and reintroduced nvenc/vaapi as default-on opt-out features (a self-compiler can now drop the libva/CUDA deps with --no-default-features). Left open, since the real fix — making moq-vaapi actually dlopen libva so an always-on VAAPI binary loads on a libva-less host — still lives in the moq-dev/vaapi crate.

(Written by Claude)

[kixelated]

NVENC encode validated on a Linux + NVIDIA box (RTX 3070 Ti, driver 595.71.05). It turned out NVENC encode had never actually worked on hardware, only compile-verified, so validation surfaced three correctness bugs. All are now fixed and covered by hardware-gated tests (no-ops on a GPU-less host).

• ✅ NVENC encode on a Linux + NVIDIA box — H.264 and H.265, end-to-end.

Bugs found and fixed:

1. Forced keyframes were silently ignored. enable_picture_type_decision makes NVENC pick frame types itself and ignore the per-frame pictureType, so encode(..., keyframe=true) mid-stream produced a P frame, not an IDR. Fixed by using the NV_ENC_PIC_FLAG_FORCEIDR picture flag, which applies with picture-type decision on. (Masked before because the capture producer only forces a keyframe on the first frame, which is an IDR regardless.)
2. Only the first IDR carried parameter sets. Periodic/GOP IDRs had no SPS/PPS (nor VPS for HEVC), so a subscriber joining mid-stream couldn't decode at any later keyframe — exactly what avc3/hev1 need. Fixed with repeatSPSPPS + idrPeriod = gop.
3. Every frame was corrupted. NVENC's input-buffer pitch exceeds the width (512 for a 320-wide IYUV buffer, 256-aligned), but the write was flat, shearing the image. The old width % 64 == 0 guard's "aligned is safe" premise was wrong. Fixed with a pitched, plane-by-plane copy at the buffer's real stride; verified by an in-crate encode→openh264-decode round-trip (MAE ~0.3 vs the input) plus an ffmpeg decode. This also drops the width-multiple-of-64 restriction (any even width works now).

Fixes 1 and 3 needed small additions to the nvidia-video-codec-sdk fork (the safe wrapper exposed neither FORCEIDR nor a pitched write — it had a literal TODO: Write pitched?): a force_idr flag on EncodePictureParams and BufferLock::write_rows / pitch(), pushed to the dynamic-loading branch.

Environment note (not a code bug): on a Nix dev-shell the runtime driver probe returns "not found" even with the driver present, because the nix loader doesn't search /usr/lib64 by soname (no ldconfig cache). All validation ran with LD_LIBRARY_PATH=/usr/lib64; a normal glibc distro resolves it via ldconfig, but it may matter for the Nix packaging of moq-cli.

Still open on Linux: NVENC/NVDEC decode (not implemented), VAAPI H.265 encode + the moq-vaapi dlopen work, and screen capture (PipeWire) — none validatable on this box (no Intel/AMD GPU, VAAPI/PipeWire not installed).

(Written by Claude Opus 4.8)