Private AI Code Assistant runs on OpenShift: developers use OpenShift Dev Spaces and an OpenAI-compatible MaaS / RHCL gateway in front of vLLM (llm-d). Inference stays on-cluster unless external model backends are configured.
flowchart TD
IDE[DevSpaces]
MaaS["maas-default-gateway RHCL plus MaaS keys and quota"]
GR[guardrails-proxy plus TrustyAI]
SR[Semantic Router]
LLMD[llm-d]
VLLM[vLLM]
EXT[external provider]
IDE -->|"chat one virtual model"| MaaS
MaaS -->|"/v1/chat/completions"| GR
GR --> SR
SR --> LLMD --> VLLM
SR --> EXT
IDE -->|"tab always local"| MaaS
MaaS -->|"tab and non-chat skip GR and SR"| LLMD
- Developer workspace — Continue, Cline, Roo Code, or OpenCode in a Dev Spaces pod call
maas-default-gatewayover cluster-internal HTTPS with a per-namespace API key. - MaaS / RHCL — AuthPolicy validates the API key. HTTPRoute
pca-maas-front-doorsends chat to guardrails (when enabled), tab/local/v1and other/v1to llm-d. RHCL is the policy engine on this Gateway, not a second hop. - Guardrails — TrustyAI detectors, then llm-d HTTP :80 (or Semantic Router when
semanticRouter.enabled). Streaming chat is an input-only probe then llm-d SSE; output secret scans run on non-stream JSON only. - Semantic Router — optional official vLLM SR + Envoy behind Service
pca-semantic-router. Helm injects local Qwen; extras are optional OpenAI-compatible backends. Keywordcodeor complexityhard→ strongest extra; easy non-code → local Qwen. Not an OpenShift AI component. - llm-d / vLLM —
LLMInferenceService(KServe). EPP is off by default.
See maas-attachment.md for why PCA owns the HTTPRoute instead of MaaSModelRef.endpointOverride.
Illustrative inference traffic flow:
| Layer | Typical components |
|---|---|
| IDE | OpenShift Dev Spaces (che-code), AI extensions / OpenCode |
| Front door | maas-default-gateway in openshift-ingress + AuthPolicy on pca-maas-front-door |
| Routing | llm-d gateway, Gateway API HTTPRoute, optional Semantic Router |
| Serving | vLLM via LLMInferenceService (KServe RawDeployment) |
| Platform | RHOAI (MaaS Managed), NVIDIA GPU Operator, NFD, cert-manager, LWS, RHCL |
| Cloud | ROSA (AWS), ARO (Azure), or an existing OpenShift cluster |
Cloud-specific versions and GPU SKUs are documented in the deploy guides:
ArgoCD (ROSA/ARO) syncs charts from charts/ via pca-app-of-apps:
| Chart | Wave | Role |
|---|---|---|
pca-app-of-apps |
root | AppProject + child Applications |
pca-operators |
1 | Operator Subscriptions (RHOAI, GPU, Dev Spaces, NFD, RHCL, …) |
pca-platform-config |
2 | Namespaces, DSC/DSCI (modelsAsService: Managed), secrets, CheCluster, locked MaaS Gateway + TLS |
pca-ai-serving |
3 | PVC, HardwareProfile, LLMInferenceService, llm-d, MaaS HTTPRoute, optional guardrails/SR, observability |
pca-devspaces |
4 | Per-developer DevWorkspaces + extension ConfigMaps |
pca-benchmarks |
5 | Optional GuideLLM Job (opt-in) |
Existing OpenShift deploys only pca-platform-config, pca-ai-serving, and pca-devspaces via Helm. Patch DSC kserve.modelsAsService to Managed. pca-platform-config owns Gateway+TLS when maas.gateway.create is true (chart default; independent of clusterConfig.enabled). See the existing-OpenShift README.
Open each file in a browser or IDE for full resolution.
SVG sources for the same views live alongside the PNGs in docs/images/.



