Skip to content

feat(service-catalog): split GPU transcoding lane from base media lane (closes #63)#198

Open
clubanderson wants to merge 1 commit into
projectbluefin:mainfrom
clubanderson:feat/service-catalog/gpu-transcoding-lane
Open

feat(service-catalog): split GPU transcoding lane from base media lane (closes #63)#198
clubanderson wants to merge 1 commit into
projectbluefin:mainfrom
clubanderson:feat/service-catalog/gpu-transcoding-lane

Conversation

@clubanderson

Copy link
Copy Markdown
Contributor

Summary

Defines the GPU transcoding and hardware-passthrough validation lane as a standalone split from the base media-service lane (#59). Closes #63.

Substrate dependency chain (made explicit by this PR)

#54 substrate (GPU device plugin + driver stack on ghost)
    → #63 this lane (GPU transcoding contract proven)
    → #59 base media lane (unblocked from GPU scope)

Acceptance criteria check

What's in this PR

tests/service_catalog/media_gpu/test_gpu_transcoding.py

13 test methods in TestMediaGpuTranscodingLane:

Module-level skip gate: _gpu_allocatable_on_any_node() — if no nvidia.com/gpu in any node allocatable, the entire module is skipped. Non-GPU CI clusters are not broken.

Group Tests
Substrate node allocatable capacity, device plugin DaemonSet Running
Deployment GPU deployment ready, pod Running, resource limit in pod spec
Device /dev/nvidia* visible in container, nvidia-smi reports GPU
Transcoding ffmpeg lists nvenc, ffmpeg 1-second hardware transcode completes
Lifecycle GPU capacity recovers after pod deletion (within 60s)
Deferred KubeVirt VM passthrough → pytest.skip#54; Multi-GPU/MIG → pytest.skip; AMD/Intel → pytest.skip

argo/workflow-templates/homelab-media-gpu.yaml

WorkflowTemplate:

  • check-gpu-prerequisites guard step (runs first): verifies node GPU allocatable capacity and device plugin pods; fails with clear epic: homelab cluster substrate and fleet-client model #54 step-by-step instructions if not ready
    deploy-fixturerun-tests DAG
  • deploy-fixture: homelab-media-gpu-config 1Gi PVC, Deployment with nvidia.com/gpu: 1 limit, runtimeClassName: nvidia, NVIDIA_VISIBLE_DEVICES=all, linuxserver/ffmpeg image
  • cleanup onExit handler

argo/homelab-media-gpu.yaml

Workflow submit trigger.

docs/homelab-contracts.md

Dependencies

closes projectbluefin#63)

Defines the GPU transcoding and hardware-passthrough validation lane as a
standalone split from the base media-service lane (projectbluefin#59).  All tests in the
module are gated behind a runtime GPU-availability check so non-GPU clusters
are not broken.

## Why the split matters (from issue projectbluefin#63)

GPU transcoding will distort the base media scope unless kept explicitly
separate.  This PR makes the substrate dependency chain visible:

  projectbluefin#54 (GPU device plugin + driver stack on ghost)
      → projectbluefin#63 (GPU transcoding contract, this PR)
      → projectbluefin#59 (base media lane, now unblocked)

## New files

### tests/service_catalog/media_gpu/test_gpu_transcoding.py
13 test methods in TestMediaGpuTranscodingLane:

Module-level skip gate: _gpu_allocatable_on_any_node() — if no
nvidia.com/gpu in any node allocatable, entire module skips with a message
pointing to projectbluefin#54 substrate requirements.

Tests (active when GPU present):
- Substrate: node allocatable capacity, device plugin DaemonSet Running
- Deployment: GPU deployment ready, pod Running, resource limit in pod spec
- Device: /dev/nvidia* visible in container, nvidia-smi reports GPU
- Transcoding: ffmpeg lists nvenc encoder, ffmpeg hardware transcode completes
- Lifecycle: GPU capacity recovers after pod deletion
Tests (explicit skips):
- KubeVirt VM-backed passthrough → pytest.skip → projectbluefin#54
- Multi-GPU / MIG slicing → pytest.skip → deferred
- AMD ROCm / Intel QSV → pytest.skip → deferred

### argo/workflow-templates/homelab-media-gpu.yaml
WorkflowTemplate (ArgoCD-managed):
- check-gpu-prerequisites → create-namespace → deploy-fixture → run-tests DAG
- check-gpu-prerequisites: guard step that verifies GPU allocatable capacity
  and device plugin pods; fails with clear projectbluefin#54 instructions if not ready
- deploy-fixture: homelab-media-gpu-config 1Gi PVC, Deployment with
  nvidia.com/gpu: 1 resource limit, NVIDIA_VISIBLE_DEVICES=all env,
  runtimeClassName: nvidia, linuxserver/ffmpeg image
- cleanup onExit handler

### argo/homelab-media-gpu.yaml
Workflow submit trigger.

## Updated files

### docs/homelab-contracts.md
- §6 known-blockers: projectbluefin#63 updated from 'deferred' to 'defined'
- §7 GPU Transcoding and Hardware-Passthrough Lane added:
  substrate dependency chain, required projectbluefin#54 steps, behavior table,
  module-level skip gate explanation, WorkflowTemplate guard step,
  out-of-scope splits table

Closes projectbluefin#63
Child of projectbluefin#51, projectbluefin#54
Depends on projectbluefin#54 (substrate), projectbluefin#59 (base media lane)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — well-structured test/infra addition, clean code.

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-designed GPU lane split. The module-level skip gate _gpu_allocatable_on_any_node() is the right approach — non-GPU CI clusters are cleanly unaffected. The dependency chain (#54 substrate → #63 this lane → #59 base media) is explicitly documented in the module docstring, WorkflowTemplate, and docs. Deferred tests (KubeVirt passthrough, multi-GPU, AMD/Intel) are properly marked with pytest.skip with issue references. The onExit cleanup handler is correctly wired. LGTM.

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via org-wide review.

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via org-wide review.

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Approved ✅

Part of the structured service-catalog project build-up. Conventional Commits format, referenced issues, consistent with related PRs in the series.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: split GPU transcoding and passthrough from the base media lane

3 participants