You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0f1d431
Browse filesBrowse the repository at this point in the historyBrowse files
docs: align guides and inference architecture with the code
Correct assistant choices, gateway authentication, EPP defaults, guardrails behavior, and deployment guidance. Refresh the inference diagram and its content source, with larger text and a taller layout.
Copy file name to clipboardExpand all lines: PCA_Deployment_ROSA/README.md
+23-23Lines changed: 23 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,9 +3,9 @@
3
3
Fully reproducible deployment of an enterprise-grade private AI code assistant on Red Hat OpenShift (ROSA HCP), using **Terraform** for infrastructure provisioning and **ArgoCD (GitOps)** for all on-cluster components.
4
4
5
5
The end result is a ROSA HCP cluster running:
6
-
-**Qwen/Qwen3.6-35B-A3B-FP8** served via KServe + llm-d with intelligent EPP routing
7
-
-**OpenShift Dev Spaces** with pre-configured VS Code extensions (Roo Code, Continue, Cline) consuming the self-hosted model
-**PVC**: 100Gi `model-cache` on `gp3-csi` (persists model weights across restarts). Chart always sets `HF_HUB_OFFLINE=1` for LLMIS local-path load (storage-initializer populates the PVC before vLLM starts).
659
659
-**LLMInferenceService** (`qwen3-coder`): `Qwen/Qwen3.6-35B-A3B-FP8` via `vllm.image` (API name matches `model.id`) with vLLM args:
Each extension connects to the self-hosted model endpoint. Here is exactly what is pre-populated:
686
+
OpenCode is the default; `type: continue` installs all three extensions below per workspace. IDEs use MaaS with per-namespace API keys. Direct llm-d access is an escape hatch (`aiGateway.escapeHatchToLlmd=true`).
|**Roo Code**| Yes |ConfigMap auto-mounted into workspace | Profile named **"Private AI Assistant"**is pre-selected with all modes (architect, code, ask, debug) pointed at the model. Streaming is disabled (required for tool calling). |
697
-
|**Continue**| Yes (dev1 only) |`~/.continue/config.yaml`written by postStart script | Model named **"Private AI Assistant"**appears in the chat panel. Tab autocomplete (named **"Private AI Autocomplete"**) is also pre-configured. |
698
-
|**Cline**|Installed only (dev1 only) | Extension is installed but requires **manual UI setup** (Cline stores settings in VS Code `globalState`, not in files) | User must open Cline settings and enter: Provider = OpenAI Compatible, Base URL / Model ID / API Key from the table above. |
696
+
|**Roo Code**| Yes |Provider profiles ConfigMap | Profile **"llm-d-qwen3-coder"**selects the model for all configured modes; streaming is disabled. |
697
+
|**Continue**| Yes |`~/.continue/config.yaml`ConfigMap |**"Private model (llm-d)"**and **"Private model Autocomplete (llm-d)"** are configured for chat and tab autocomplete. |
698
+
|**Cline**|Provider files supplied | Provider ConfigMaps + postStart copy |OpenAI-compatible URL, model ID, and API key are supplied for each workspace. |
699
699
700
700
---
701
701
@@ -725,10 +725,10 @@ Each extension connects to the self-hosted model endpoint. Here is exactly what
Copy file name to clipboardExpand all lines: README.md
+11-9Lines changed: 11 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Private AI Code Assistant on Red Hat OpenShift
2
2
3
-
Deploy a private self-hosted AI coding assistant on OpenShift so developers get AI-powered IDEs while no code leaves your environment
3
+
Deploy a self-hosted AI coding assistant on OpenShift, with private inference unless you enable external model backends.
4
4
5
5
## Table of Contents
6
6
@@ -30,26 +30,26 @@ Teams want IDE-integrated AI coding help without sending proprietary source to p
30
30
31
31
This quickstart deploys:
32
32
33
-
-**OpenShift Dev Spaces** workspaces with OpenCode (Continue, Cline, Roo Code optional)
34
-
-**On-cluster inference**with vLLM behind **llm-d**intelligent routing — default model `Qwen/Qwen3.6-35B-A3B-FP8` (you can change it to any open-source model you want)
35
-
-An optional **RHCL AI Gateway**front door with per-developer API keys
33
+
-**OpenShift Dev Spaces** workspaces offering OpenCode (default), Continue, Cline, or Roo Code
34
+
-**On-cluster vLLM inference**through **llm-d** — default model `Qwen/Qwen3.6-35B-A3B-FP8`, configurable for compatible models
35
+
-**MaaS / RHCL AI Gateway** with per-developer API keys, enabled by default
36
36
- GitOps Helm charts (ROSA/ARO) or Helm-only install on an existing cluster
37
37
38
38
### Features
39
39
40
40
- Source and inference stay inside your OpenShift network boundary when configured that way
41
41
- OpenAI-compatible `/v1` API for IDE extensions
42
-
-Prefix-cache–aware routing via llm-d Endpoint Picker (EPP)
42
+
-vLLM prefix caching; EPP off by default
43
43
- Multi-developer namespaces with ConfigMap-prewired extensions
44
-
-Optional observability (Grafana; Langfuse/OTel), guardrails, and MCP
44
+
-Grafana/guardrails by default; ROSA/ARO Langfuse/OTel; optional MCP
45
45
46
46
### Solution stack
47
47
48
48
| Layer | Technology |
49
49
|-------|------------|
50
50
| IDE | OpenShift Dev Spaces |
51
51
| Auth front door | MaaS / RHCL (`maas-default-gateway`) |
52
-
| Routing | llm-d + Gateway API + EPP |
52
+
| Routing | llm-d + Gateway API; optional EPP |
53
53
| Serving | vLLM / KServe `LLMInferenceService`|
54
54
| Platform | Red Hat OpenShift AI, NVIDIA GPU Operator, NFD |
55
55
| Delivery | Unified Helm charts in `charts/` (ArgoCD or `make`) |
@@ -58,6 +58,8 @@ This quickstart deploys:
58
58
59
59
Developers → Dev Spaces → maas-default-gateway → (optional guardrails / Semantic Router) → llm-d → vLLM on NVIDIA GPU.
60
60
61
+
Illustrative inference traffic flow:
62
+
61
63

62
64
63
65
Full diagrams: [docs/architecture.md](docs/architecture.md).
@@ -158,7 +160,7 @@ make ai-serving-undeploy-existing-openshift DELETE_NAMESPACE=1
|[docs/semantic-router.md](docs/semantic-router.md)| Official vLLM SR hop, extras, inject |
164
166
|[docs/models-and-hardware.md](docs/models-and-hardware.md)| Changing models and accelerators |
@@ -177,7 +179,7 @@ make ai-serving-undeploy-existing-openshift DELETE_NAMESPACE=1
177
179
## Tags
178
180
179
181
-**Title:** Private AI Code Assistant on Red Hat OpenShift
180
-
-**Description:** Deploy a private self-hosted AI coding assistant on OpenShift so developers get AI-powered IDEs while no code leaves your environment
182
+
-**Description:** Deploy a self-hosted AI coding assistant on OpenShift, with private inference unless you enable external model backends.
181
183
-**Industry:** Media and IT services
182
184
-**Product:** OpenShift AI
183
185
-**Use case:** AI coding assistant, private inference
If you previously deployed guardrails under `pca-platform-config`, upgrade **ai-serving before platform-config** so guardrails are recreated in the AI namespace before the old platform release prunes them. ArgoCD wave order (platform wave 2, ai-serving wave 3) already does this; for `make ai-serving-deploy-existing-openshift`, run that target before any platform-config upgrade that removes the old subchart.
103
+
ArgoCD and `make ai-serving-deploy-existing-openshift` deploy **platform-config before ai-serving**. Migrating guardrails from an older platform-config release requires a separate staged upgrade; the normal deployment order does not perform that migration.
104
104
105
105
### Demo DevSpaces (`dev-user1..N`)
106
106
@@ -126,7 +126,7 @@ make devspace-deploy-existing-openshift DEV_USER=dev-user2
126
126
127
127
> **OpenCode build:** `opencode-build` is cluster-singleton infrastructure owned by the first Helm release. The Makefile auto-detects whether it already exists and suppresses `opencodeBuild` for subsequent runs. If deploying manually with `helm upgrade --install`, pass `--set opencodeBuild.enabled=false` from the second developer onward.
128
128
>
129
-
> **Do not uninstall** the release that owns `opencode-build` while other OpenCode workspaces exist — doing so deletes the BuildConfig and ImageStream (the Namespace is kept), breaking image pulls for all running workspaces. Recovery: delete the empty namespace (`oc delete namespace opencode-build`), then re-run any opencode devspace deploy without `--set opencodeBuild.enabled=false` to recreate the full build infrastructure.
129
+
> **Shared build resources:** Helm uninstall preserves the `opencode-build` Namespace, BuildConfig, ImageStream, and image-puller RoleBindings (`helm.sh/resource-policy: keep`).
130
130
131
131
## Parameters
132
132
@@ -151,8 +151,7 @@ make devspace-deploy-existing-openshift DEV_USER=dev-user2
151
151
| `--set opencodeBuild.enabled=false` | 2nd+ opencode developer — avoids Helm ownership conflict on the shared `opencode-build` namespace (Makefile detects this automatically) |
152
152
| `--set pca-observability.langfuse.enabled=true` | Opt in Langfuse (+ OTel) with AI serving |
153
153
| `--set guardrails.enabled=false` | Disable guardrails subchart on ai-serving (default is enabled) |
| `--set maas.hostname=<Gateway/Route host>` | Pin HTTPRoute, AuthConfig, and EnvoyFilter to the public MaaS host (`oc get gateway maas-default-gateway -n openshift-ingress`). Empty default is Istio catch-all `*:443`. Do not commit the cluster hostname in values files. |
157
156
| `--set semanticRouter.enabled=true,global.semanticRouter.enabled=true` | Deploy the Semantic Router hop (pins local Qwen until extras exist). Prefer `SEMANTIC_ROUTER_ENABLED=true` so Make sets both flags |
158
157
@@ -198,7 +197,7 @@ Grafana (boards B/C — latency, KV/GPU) deploys by default with AI serving via
198
197
199
198
| Flag | Default | What you get |
200
199
|------|---------|--------------|
201
-
| `grafana.enabled` | `true` | 1-pod Grafana + boards B/C. Boards A/D when Langfuse is on |
200
+
| `pca-observability.grafana.enabled` | `true` | 1-pod Grafana + boards B/C. Boards A/D when Langfuse is on |
202
201
| `pca-observability.langfuse.enabled` | `false` | Langfuse + OTel Collector; wires vLLM OTLP in the same release |
203
202
| `pca-observability.langfuse.ioCapture` | `full` | When Langfuse is on: store full prompt/completion via vLLM middleware (async). Set `metadata` for tokens/latency only |
make devspace-deploy-existing-openshift N=2 MCP_ENABLED=true
224
+
make devspace-deploy-existing-openshift N=2 TYPE=continue MCP_ENABLED=true
226
225
```
227
226
228
227
---
@@ -233,15 +232,17 @@ MCP gives AI coding extensions (Continue, Roo Code) live read-only access to clu
233
232
234
233
### Deploy with MCP enabled from the start
235
234
236
-
Pass `MCP_ENABLED=true` to both the AI serving and devspace make targets:
235
+
Pass `MCP_ENABLED=true` to both targets and `TYPE=continue` for automatic Continue/Roo Code wiring. The default OpenCode configuration has no MCP entry.
237
236
238
237
```bash
239
238
make ai-serving-deploy-existing-openshift HF_TOKEN=hf_xxx MCP_ENABLED=true
240
-
make devspace-deploy-existing-openshift N=2 MCP_ENABLED=true
239
+
make devspace-deploy-existing-openshift N=2 TYPE=continue MCP_ENABLED=true
241
240
```
242
241
243
242
### Enable MCP on an already-running deployment
244
243
244
+
Automatic IDE wiring applies to `type: continue` workspaces:
245
+
245
246
```bash
246
247
make mcp-enable AI_NAMESPACE=<ai-ns> DEV_USER=dev-user1
247
248
```
@@ -298,10 +299,12 @@ This target:
298
299
299
300
The DevWorkspace is created with `started: false`.
300
301
301
-
### Step 2 — Trigger the image build (first time only)
302
+
### Step 2 — Wait for the image build (first time only)
303
+
304
+
The BuildConfig's `ConfigChange` trigger starts the build automatically. Follow its logs:
The build installs OpenCode CLI (version pinned via `opencodeBuild.opencodeVersion` in values) and stubs `xdg-open` to prevent crashes in headless environments.
0 commit comments