diff --git a/docs/en/administer/secure_profile/install_secure_dependencies.mdx b/docs/en/administer/secure_profile/install_secure_dependencies.mdx index 07792336..d2afc28f 100644 --- a/docs/en/administer/secure_profile/install_secure_dependencies.mdx +++ b/docs/en/administer/secure_profile/install_secure_dependencies.mdx @@ -8,20 +8,10 @@ The [secure profile](./enable_secure_profile.mdx) requires **cert-manager**, **S ## Common installation pattern -Each component is delivered as a platform package — an OLM operator or a Cluster Plugin. The steps are the same for all of them: +Each component is delivered as a platform package — an OLM operator or a Cluster Plugin. For the common package download, `violet` setup, and upload procedure, see [Upload Packages](../../installation/package-upload.mdx). The pattern is the same for all of them: -1. **Download** the component package from the Customer Portal — `https://cloud.alauda.cn` (China) or `https://cloud.alauda.io` (international). -2. **Upload** it to the platform with `violet`: - - ```bash - violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - .tgz - ``` - - `violet` relocates the package's images into the platform registry, so the components install on **air-gapped** clusters. +1. **Download** the component package from the Customer Portal. +2. **Upload** it to the platform following [Upload Packages](../../installation/package-upload.mdx). `violet` relocates the package's images into the platform registry, so the components install on **air-gapped** clusters. 3. **Install** from the web console — **Administrator → Marketplace / OperatorHub**, selecting the target cluster — or with `kubectl` (a `Subscription` for OLM operators, a `ModuleInfo` for Cluster Plugins). The sections below cover the per-component configuration. @@ -30,7 +20,7 @@ The sections below cover the per-component configuration. SPIRE issues the workload SVIDs that AuthBridge uses for mTLS. It is delivered as a **Cluster Plugin**. -Upload the SPIRE package with `violet` and install it on the **workload cluster** (Marketplace → install on cluster, or create a `ModuleInfo` from the global cluster). Configure the install: +Upload the SPIRE package following [Upload Packages](../../installation/package-upload.mdx) and install it on the **workload cluster** (Marketplace → install on cluster, or create a `ModuleInfo` from the global cluster). Configure the install: | Setting | Notes | |---------|-------| @@ -59,7 +49,7 @@ Keycloak issues the OAuth2/JWT tokens that AuthBridge validates. It is delivered ### 1. Install the operator -Upload the Keycloak operator package with `violet`, then install it from **OperatorHub**. Platform operators require the **`Manual`** upgrade approval — approve the generated `InstallPlan` to complete the install. +Upload the Keycloak operator package following [Upload Packages](../../installation/package-upload.mdx), then install it from **OperatorHub**. Platform operators require the **`Manual`** upgrade approval — approve the generated `InstallPlan` to complete the install. ### 2. Deploy a Keycloak instance @@ -117,7 +107,7 @@ The value you set as `keycloak.publicUrl` becomes the JWT issuer (`iss`) claim t The secure profile uses Istio **ambient** mesh (ztunnel) for namespace-level mTLS, delivered as ** Service Mesh v2** (the `servicemesh-operator2` / Sail operator). -Install the **Alauda Service Mesh v2** operator from the Marketplace (download → `violet` → install), version **2.1.1 or later** (ambient support). Ambient on ACP also requires the **Multus** CNI plugin and `kube-ovn` ≥ v4.1.5. +Install the **Alauda Service Mesh v2** operator from the Marketplace (download → upload via [Upload Packages](../../installation/package-upload.mdx) → install), version **2.1.1 or later** (ambient support). Ambient on ACP also requires the **Multus** CNI plugin and `kube-ovn` ≥ v4.1.5. Then enable ambient by creating the `IstioCNI`, `Istio`, and `ZTunnel` resources with `profile: ambient`. Create the `istio-cni`, `istio-system`, and `ztunnel` namespaces first, each labeled `istio-discovery=enabled`. The full, authoritative procedure (prerequisites, namespace labels, verification) is in the **Alauda Service Mesh v2** documentation — *Installing Istio ambient mode* ([`alauda/servicemesh2-docs`](https://github.com/alauda/servicemesh2-docs/blob/master/docs/en/installing/ambient-mode/installing-ambient-mode.mdx)). In summary: diff --git a/docs/en/ai_applications/components/dify/install.mdx b/docs/en/ai_applications/components/dify/install.mdx index ca564727..b7ecc044 100644 --- a/docs/en/ai_applications/components/dify/install.mdx +++ b/docs/en/ai_applications/components/dify/install.mdx @@ -12,7 +12,7 @@ For what Dify is and the components it deploys, see [Introduction](./intro). | You need | Why | |---|---| -| The Dify operator package (`.tgz`) | Pushed via `violet` so the operator appears in OperatorHub | +| The Dify operator package | Downloaded and uploaded with the common package procedure so the operator appears in OperatorHub (see [Upload Packages](../../../installation/package-upload.mdx)) | | External **PostgreSQL 12+** | Dify's main database (only PostgreSQL is supported) | | External **Redis 6+** (standalone or Sentinel) | Cache + Celery broker; Cluster mode is **not** supported | | External **pgvector** *(optional)* | Vector store for RAG; disable if not using RAG | @@ -22,15 +22,7 @@ For what Dify is and the components it deploys, see [Introduction](./intro). ## Step 1 — Publish the Operator Package -Download the Dify operator package (e.g. `dify-operator.alpha.ALL.v.tgz`) from Customer Support / Portal, then push it to the platform repository so it appears in **Marketplace / OperatorHub**: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - dify-operator.alpha.ALL.v.tgz -``` +Download the Dify operator package from Customer Support / Portal, then upload the package following [Upload Packages](../../../installation/package-upload.mdx) so it appears in **Marketplace / OperatorHub**. ## Step 2 — Install the Dify Operator diff --git a/docs/en/ai_applications/components/kagenti/install.mdx b/docs/en/ai_applications/components/kagenti/install.mdx index 9a526e72..680463ad 100644 --- a/docs/en/ai_applications/components/kagenti/install.mdx +++ b/docs/en/ai_applications/components/kagenti/install.mdx @@ -2,7 +2,7 @@ weight: 20 --- -# Installation +# Install Kagenti Kagenti Operator is delivered as an **OLM Operator** and installed from the platform **OperatorHub**. @@ -13,23 +13,15 @@ Kagenti Operator is delivered as an **OLM Operator** and installed from the plat - Operator Lifecycle Manager (OLM) available on the target cluster (provided by ACP). - **cert-manager** installed on the target cluster. The operator's admission webhooks get their serving certificate from cert-manager. -## Upload Operator \{#upload-operator} +## Upload the Operator Package -Download the `Kagenti Operator` bundle from the Customer Portal / Marketplace, then publish it to the platform repository with the `violet` command-line tool: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - kagenti-operator.ALL.xxxx.tgz -``` +Download the `Kagenti Operator` bundle from the Customer Portal / Marketplace, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). :::info The operator bundle records its runtime images in the CSV `relatedImages`, so a `violet` release relocates them into the platform registry. This makes the operator installable on **air-gapped** clusters without reaching `docker.io` or `ghcr.io`. ::: -## Install Operator +## Install the Operator In the **Administrator** view: diff --git a/docs/en/ai_applications/components/llama_stack/install.mdx b/docs/en/ai_applications/components/llama_stack/install.mdx index 26ca6550..767a3813 100644 --- a/docs/en/ai_applications/components/llama_stack/install.mdx +++ b/docs/en/ai_applications/components/llama_stack/install.mdx @@ -4,27 +4,42 @@ weight: 20 # Install Llama Stack -This document describes how to install and deploy Llama Stack Server on Kubernetes using the Llama Stack Operator. +In Alauda AI 2.8.x, **Alauda Build of Llama Stack** is installed and managed by Alauda AI. The `llamaStack` component in the cluster-scoped `default` `AmlCluster` defaults to `Unmanaged`; set it to `Managed`, and you do not need to install the Operator from OperatorHub manually. After the component is reconciled, deploy Llama Stack Server with a `LlamaStackDistribution` custom resource as described below. -## Upload Operator +## Upload the Operator Package -Download the Llama Stack Operator installation file (e.g., `llama-stack-operator.alpha.ALL.xxxx.tgz`). +Download the Llama Stack Operator package, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). -Use the violet command to publish to the platform repository: +## Enable Llama Stack -```bash -violet push --platform-address=platform-access-address --platform-username=platform-admin --platform-password=platform-admin-password llama-stack-operator.alpha.ALL.xxxx.tgz +In **Administrator** view: + +1. Open **Marketplace** > **OperatorHub**. +2. Select the target cluster and open **Alauda AI**. +3. Open the **All Instances** tab and edit the `default` `AmlCluster` instance. +4. Set `spec.components.llamaStack.managementState` to `Managed`. +5. Save the `AmlCluster` instance and wait for the Llama Stack Operator to become ready. + +You can apply the component setting in the `AmlCluster` YAML: + +```yaml +spec: + components: + llamaStack: + managementState: Managed ``` -## Install Operator +## Verify the Installation -1. Go to the `Administrator` view in the Alauda Container Platform. +Check the `AmlCluster` status and the Llama Stack Operator: -2. In the left navigation, select `Marketplace` / `Operator Hub`. +```bash +kubectl get amlcluster default -3. In the right panel, find `Alauda build of Llama Stack` and click `Install`. +kubectl get pods -n llama-stack-k8s-operator-system +``` -4. Keep all parameters as default and complete the installation. +The `AmlCluster` should report `Phase=Ready`, and the Llama Stack Operator controller pod should be in `Running` state. The `LlamaStackDistribution` CRD is registered once the operator is ready. ## Deploy Llama Stack Server diff --git a/docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx b/docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx index c48868fb..a195d26a 100644 --- a/docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx +++ b/docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx @@ -2,27 +2,19 @@ weight: 20 --- -# Installation +# Install MCP Lifecycle Operator Alauda Build of MCP Lifecycle Operator is delivered as an **OLM Operator** and installed from the platform **OperatorHub**. -## Upload Operator +## Upload the Operator Package -Download the Alauda Build of MCP Lifecycle Operator package from the Customer Portal / Marketplace, then publish it to the platform repository with the `violet` command-line tool: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - mcp-lifecycle-operator.ALL.xxxx.tgz -``` +Download the Alauda Build of MCP Lifecycle Operator package from the Customer Portal / Marketplace, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). :::info The operator package includes the OLM bundle and the related runtime images required by the operator. For air-gapped clusters, make sure the MCP server images used by your `MCPServer` resources are also mirrored to a registry that the cluster can pull from. ::: -## Install Operator +## Install the Operator In the **Administrator** view: @@ -55,7 +47,7 @@ You should see the `mcp-lifecycle-operator-controller-manager` pod in `Running` ## Upgrade Operator -1. Upload the new Alauda Build of MCP Lifecycle Operator package with `violet push`. +1. Upload the new Alauda Build of MCP Lifecycle Operator package following [Upload Packages](../../../installation/package-upload.mdx). 2. Go to **Administrator** -> **Marketplace** -> **OperatorHub**. 3. Find **Alauda Build of MCP Lifecycle Operator** and apply the new version according to the configured upgrade strategy. 4. Verify the CSV, controller pod, and existing `MCPServer` resources after the upgrade. diff --git a/docs/en/deploy/components/envoy_ai_gateway/install.mdx b/docs/en/deploy/components/envoy_ai_gateway/install.mdx deleted file mode 100644 index 26a09adc..00000000 --- a/docs/en/deploy/components/envoy_ai_gateway/install.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -weight: 20 ---- - -# Install Envoy AI Gateway - -## Download the Operator Package - -:::info - -`Alauda Build of Envoy AI Gateway` Operator package can be retrieved from Customer Portal. - -Please contact Customer Support for more information. - -::: - -## Upload the Operator Package - -Download the Alauda Build of Envoy AI Gateway Operator installation file, for example `envoy-ai-gateway.alpha.ALL.v0.4.x.tgz`. - -Use `violet` to upload the package to the platform repository: - -```bash -violet push --platform-address= --platform-username= --platform-password= envoy-ai-gateway.alpha.ALL.v0.4.x.tgz -``` - -:::warning -Before installing the Operator, uninstall the existing **Alauda Build of Envoy AI Gateway** cluster plugin from the target cluster if it was previously installed as a cluster plugin. Go to `Administrator` -> `Marketplace` -> `Cluster Plugins`, find **Alauda Build of Envoy AI Gateway**, and uninstall it from the target cluster. -::: - -## Installing Alauda Build of Envoy AI Gateway - -In **Administrator** view: - -1. Go to `Marketplace` / `OperatorHub`. -2. Select the destination cluster. -3. Search for **Alauda Build of Envoy AI Gateway**. -4. Click **Install**. -5. Keep the default installation settings unless the environment requires a different namespace. -6. Complete the installation. - -## Verify the Installation - -After installation, verify that the operator is running: - -```bash -kubectl get pods -n envoy-gateway-system | grep "ai-gateway" -``` - -The Envoy AI Gateway pods should be in `Running` state. - -## Upgrading Alauda Build of Envoy AI Gateway - -1. Upload the new version of the **Alauda Build of Envoy AI Gateway** Operator package using the `violet` tool. -2. Go to the `Administrator` -> `Marketplace` -> `OperatorHub` page, find **Alauda Build of Envoy AI Gateway**, and click **Confirm** to apply the new version. diff --git a/docs/en/deploy/components/envoy_ai_gateway/intro.mdx b/docs/en/deploy/components/envoy_ai_gateway/intro.mdx index 41e5fa42..20ac1ca7 100644 --- a/docs/en/deploy/components/envoy_ai_gateway/intro.mdx +++ b/docs/en/deploy/components/envoy_ai_gateway/intro.mdx @@ -20,7 +20,7 @@ Main components and capabilities include: Envoy AI Gateway is a required dependency of **Alauda Build of KServe** for exposing inference services. -For installation on the platform, see [Install Envoy AI Gateway](./install). +Envoy AI Gateway is installed and managed by Alauda AI through the `envoyAIGateway` component in the `default` `AmlCluster`. For the Alauda AI installation and component configuration, see [Install Alauda AI](../../../installation/ai-cluster.mdx). ## Guides diff --git a/docs/en/deploy/components/index.mdx b/docs/en/deploy/components/index.mdx index 883753d3..40c84dbb 100644 --- a/docs/en/deploy/components/index.mdx +++ b/docs/en/deploy/components/index.mdx @@ -1,5 +1,5 @@ --- -weight: 60 +weight: 70 i18n: title: en: Components diff --git a/docs/en/deploy/components/infernex_bridge/install.mdx b/docs/en/deploy/components/infernex_bridge/install.mdx index 581b04a0..19128ea3 100644 --- a/docs/en/deploy/components/infernex_bridge/install.mdx +++ b/docs/en/deploy/components/infernex_bridge/install.mdx @@ -19,7 +19,7 @@ Before installing **Alauda Build of InferNex Bridge**, ensure the target cluster | KServe | Operator | Required when using the KServe `LLMInferenceService` entry point. The declared upstream compatibility range is KServe v0.17.0-v0.19.0. Alauda Build of KServe v0.16.1 has also been validated. | | Envoy Gateway and Gateway API | Operator / CRDs | Required when exposing inference services through Gateway API resources. | | Gateway API Inference Extension | CRDs | Required for `InferencePool` based intelligent routing. | -| Alauda Build of LeaderWorkerSet | Cluster Plugin | Install it before InferNex Bridge. The controller watches `LeaderWorkerSet` resources and uses them for multi-node inference workloads. | +| Alauda Build of LeaderWorkerSet | Operator (managed by AmlCluster) | Required for InferNex Bridge. The controller watches `LeaderWorkerSet` resources and uses them for multi-node inference workloads. It is normally enabled through the `lws` component in the `default` `AmlCluster`; see [Install Alauda AI](../../../installation/ai-cluster.mdx). | | Inference runtime prerequisites | Runtime | Prepare NPU nodes, model storage, runtime templates, runtime images, and network access required by the selected inference engine. | :::info @@ -42,7 +42,7 @@ The following CRDs are not installed by this OLM bundle. Install them separately | CRD | Source Project | When Required | How to Install | | ------------------------------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -| `leaderworkersets.leaderworkerset.x-k8s.io` | [Alauda Build of LeaderWorkerSet](../lws/index.mdx) | InferNex Bridge controller and multi-node inference workloads | Follow the [LeaderWorkerSet installation guide](../lws/install.mdx) before installing InferNex Bridge. | +| `leaderworkersets.leaderworkerset.x-k8s.io` | [Alauda Build of LeaderWorkerSet](../lws/index.mdx) | InferNex Bridge controller and multi-node inference workloads | Ensure LeaderWorkerSet is installed and running. It is normally managed through the `lws` component in the `default` `AmlCluster`; see [Install Alauda AI](../../../installation/ai-cluster.mdx). | | `rolebasedgroups.workloads.x-k8s.io` | [RoleBasedGroup (RBG)](https://github.com/sgl-project/rbg) | ResourceScalingGroup targets that explicitly use `RoleBasedGroup` | Install from the upstream [RBG v0.7.0 release manifest](https://raw.githubusercontent.com/sgl-project/rbg/v0.7.0/deploy/kubectl/manifests.yaml). | The `rolebasedgroups.workloads.x-k8s.io` CRD is provided by the upstream [RoleBasedGroup (RBG) project](https://github.com/sgl-project/rbg). It is a conditional dependency and is not required for Deployment or LeaderWorkerSet targets. To install the upstream RBG v0.7.0 controller and CRDs, use the versioned release manifest: @@ -242,18 +242,11 @@ kubectl get deployment,daemonset,service -n \ Expected workloads include the EagleEye hardware diagnosis Deployment, hardware monitor DaemonSet, and network performance exporter DaemonSet. If EagleEye is not required, keep all three child switches set to `false`. -## Upload Operator \{#upload-operator} +## Upload the Operator Package -Obtain the arm64 installation package from your Alauda product delivery channel: -`infernex-bridge.alpha.arm64.v26.6.0.tgz`. +Obtain the arm64 installation package from your Alauda product delivery channel, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). -Use the `violet` command to publish it to the platform repository: - -```bash -violet push --platform-address= --platform-username= --platform-password= infernex-bridge.alpha.arm64.v26.6.0.tgz -``` - -## Install Operator +## Install the Operator In **Administrator** view: @@ -302,7 +295,7 @@ For community-maintained examples, see [InferNex Bridge v26.6.0 examples](https: kubectl get infernexservice,infernexserviceconfig -A -o yaml > infernex-bridge-backup.yaml ``` -2. Upload the new version of the **Alauda Build of InferNex Bridge** operator package using the `violet` tool. +2. Upload the new version of the **Alauda Build of InferNex Bridge** operator package following [Upload Packages](../../../installation/package-upload.mdx). 3. Go to the `Administrator` -> `Marketplace` -> `OperatorHub` page, find **Alauda Build of InferNex Bridge**, review the target version, and click **Confirm** to apply it. ### Verification @@ -324,7 +317,7 @@ kubectl get crd \ Use a previous package only when its CRDs are compatible with the custom resources already stored in the cluster. In OperatorHub, select the previous available version and confirm the manual upgrade action. If the previous version -is not available, upload its installation package with `violet` first. +is not available, upload its installation package first (see [Upload Packages](../../../installation/package-upload.mdx)). After rollback, verify the controller, CRDs, inference services, runtime image IDs, and one representative inference request. Do not delete the five bundled diff --git a/docs/en/deploy/components/kserve/install.mdx b/docs/en/deploy/components/kserve/install.mdx deleted file mode 100644 index b190454c..00000000 --- a/docs/en/deploy/components/kserve/install.mdx +++ /dev/null @@ -1,182 +0,0 @@ ---- -weight: 20 ---- - -# Install KServe - -## Prerequisites - -Before installing **Alauda Build of KServe**, you need to ensure the following dependencies are installed: - -### Required Dependencies - -| Dependency | Type | Description | -|------------|------|-------------| -| Alauda build of Envoy Gateway | Operator | Provides the underlying gateway functionality for AI services. **Only v1.5.x is supported; other versions do not match this KServe dependency.** | -| [Alauda Build of Envoy AI Gateway](../envoy_ai_gateway/install.mdx) | Operator | Provides AI-specific gateway capabilities | -| [Alauda Build of LeaderWorkerSet](../lws/install.mdx) | Cluster Plugin | Provides leader-worker set functionality for AI workloads | -| GIE (gateway-api-inference-extension) | Built-in | Bundled with Alauda Build of KServe by default. If GIE is already installed in the cluster, the built-in installation can be disabled via the `gie.builtIn` parameter during operator configuration. | - -:::info -`Alauda build of Envoy Gateway` is natively integrated into ACP 4.2. For environments running earlier versions (including ACP 4.0 and 4.1), please contact Customer Support for compatibility and installation guidance. -::: - -### Installation Notes - -1. **Required Dependencies**: All required dependencies must be installed before installing Alauda Build of KServe. -2. **GIE Integration**: GIE is bundled and enabled by default. If your environment already has GIE installed separately, set `gie.builtIn` to `false` in the operator configuration to disable the built-in installation. - -## Upload Operator \{#upload-operator} - -Download the Alauda Build of KServe Operator installation file (e.g., `kserve-operator.ALL.xxxx.tgz`). - -Use the `violet` command to publish it to the platform repository: - -```bash -violet push --platform-address= --platform-username= --platform-password= kserve-operator.ALL.xxxx.tgz -``` - -## Install Operator - -In **Administrator** view: - -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install the KServe Operator. -3. Search for and select **Alauda Build of KServe**, then click **Install**. - - **Install Alauda Build of KServe** window will pop up. - -4. Leave **Channel** unchanged. -5. Check whether the **Version** matches the **Alauda Build of KServe** version you want to install. -6. Leave **Installation Location** unchanged, it should be `kserve-operator` by default. -7. Select **Manual** for **Upgrade Strategy**. -8. Click **Install**. - -### Verification - -Confirm that the **Alauda Build of KServe** tile shows one of the following states: - -- `Installing`: installation is in progress; wait for this to change to `Installed`. -- `Installed`: installation is complete. - -## Create KServe Instance - -After the operator is installed, create a `KServe` custom resource to deploy the KServe instance. - -Switch to **YAML view** and apply the following configuration, then adjust the callout fields for your environment: - -```yaml -apiVersion: components.aml.dev/v1alpha1 -kind: KServe -metadata: - name: default-kserve -spec: - namespace: kserve # [!code callout] - values: - global: - clusterName: # [!code callout] - deployFlavor: single-node # [!code callout] - platformAddress: # [!code callout] - preset: - GIE: # [!code callout] - enabled: true - envoy_ai_gateway: # [!code callout] - port: 1063 - service: ai-gateway-controller - envoy_gateway: # [!code callout] - create_instance: true - deploy_type: ControllerNamespace - instance_name: aieg - sa_namespace: envoy-gateway-system - service_account: envoy-gateway - kserve_gateway: # [!code callout] - enabled: true - gateway_class: "" - name: kserve-ingress-gateway - namespace: kserve - port: 80 - registry: - address: # [!code callout] - kserve: - controller: - deploymentMode: Knative # [!code callout] - gateway: - domain: # [!code callout] - storage: - caBundleConfigMapName: aml-global-ca-bundle # [!code callout] -``` - - - -1. `spec.namespace` — Kubernetes namespace where KServe components are deployed. Default: `kserve`. -2. `global.clusterName` — Cluster name as registered in the platform. Example: `business-1`. -3. `global.deployFlavor` — `single-node` for non-HA, `ha-cluster` for production HA. -4. `global.platformAddress` — Alauda Container Platform management endpoint address. Example: `https://`. -5. `preset.GIE` — Built-in Gateway API Inference Extension for enhanced AI capabilities. See [GIE Configuration](#gie-gateway-api-inference-extension-configuration). -6. `preset.envoy_ai_gateway` — AI-specific gateway for intelligent routing and policy enforcement. See [Envoy AI Gateway Configuration](#envoy-ai-gateway-configuration). -7. `preset.envoy_gateway` — Underlying Envoy-based gateway infrastructure. See [Envoy Gateway Configuration](#envoy-gateway-configuration). -8. `preset.kserve_gateway` — Ingress gateway for KServe inference services. See [KServe Gateway Configuration](#kserve-gateway-configuration). -9. `global.registry.address` — The container registry endpoint used by the target cluster (`global.clusterName`) to pull KServe infrastructure and runtime images. -Example: `registry.alauda.cn:60070`. -10. `kserve.controller.deploymentMode` — Set to `Knative` for serverless features like scale-to-zero, or `Standard` for native Kubernetes deployments -11. `kserve.controller.gateway.domain` — Domain for the ingress gateway to expose inference service endpoints. Use a wildcard domain, e.g., `*.example.com`. -12. `kserve.storage.caBundleConfigMapName` — ConfigMap name containing the CA bundle for storage connections. - - - - -### Verification - -Check the status of the `KServe` resource: - -```bash -kubectl get kserve default-kserve -n kserve-operator -``` - -The instance is ready when the status shows `DEPLOYED: True`. - -### Envoy Gateway Configuration \{#envoy-gateway-configuration} - -| Field | Description | Default | -|-------|-------------|---------| -| `preset.envoy_gateway.service_account` | Service account name used by Envoy Gateway. | `envoy-gateway` | -| `preset.envoy_gateway.sa_namespace` | Namespace where the Envoy Gateway service account is located. | `envoy-gateway-system` | -| `preset.envoy_gateway.create_instance` | Create an Envoy Gateway instance to manage inference traffic with bundled extensions. | `true` | -| `preset.envoy_gateway.instance_name` | Name of the Envoy Gateway instance to create. | `aieg` | - -### Envoy AI Gateway Configuration \{#envoy-ai-gateway-configuration} - -| Field | Description | Default | -|-------|-------------|---------| -| `preset.envoy_ai_gateway.service` | Kubernetes service name for Envoy AI Gateway. | `ai-gateway-controller` | -| `preset.envoy_ai_gateway.port` | Port number used by Envoy AI Gateway. | `1063` | - -### KServe Gateway Configuration \{#kserve-gateway-configuration} - -| Field | Description | Default | -|-------|-------------|---------| -| `preset.kserve_gateway.enabled` | Deploy a KServe Gateway instance for InferenceService traffic. | `true` | -| `preset.kserve_gateway.name` | Name of the KServe Gateway. | `kserve-ingress-gateway` | -| `preset.kserve_gateway.namespace` | Namespace where the KServe Gateway is deployed. | `kserve` | -| `preset.kserve_gateway.gateway_class` | Optional custom GatewayClass name. If empty, derived as `{namespace}-{name}`. | `""` | -| `preset.kserve_gateway.port` | Port number used by the KServe Gateway. | `80` | - -### GIE (gateway-api-inference-extension) Configuration \{#gie-gateway-api-inference-extension-configuration} - -| Field | Description | Default | -|-------|-------------|---------| -| `preset.GIE.enabled` | Enable the bundled Gateway API Inference Extension. Set to `false` if GIE is already installed separately in the cluster. | `true` | - - -## Upgrading Alauda Build of KServe - -1. Upload the new version of the **Alauda Build of KServe** operator package using the `violet` tool. -2. Go to the `Administrator` -> `Marketplace` -> `OperatorHub` page, find **Alauda Build of KServe**, and click **Confirm** to apply the new version. - -### Verification - -After upgrading, confirm that the **Alauda Build of KServe** tile shows `Installed` and verify the KServe instance status: - -```bash -kubectl get kserve default-kserve -n kserve-operator -``` diff --git a/docs/en/deploy/components/kserve/intro.mdx b/docs/en/deploy/components/kserve/intro.mdx index 86f913fa..a504cbfb 100644 --- a/docs/en/deploy/components/kserve/intro.mdx +++ b/docs/en/deploy/components/kserve/intro.mdx @@ -29,7 +29,7 @@ Predictive AI covers traditional machine learning model serving with high throug - **Inference Graph**: The **InferenceGraph** CRD enables composing multiple models into a pipeline, including pre/post-processing nodes, routing logic, and ensemble patterns. - **Autoscaling**: Scale-to-zero and scale-from-zero support via KEDA or Kubernetes HPA, with policies based on request rate, queue depth, or custom metrics. -For installation on the platform, see [Install KServe](./install). +KServe is installed and managed by Alauda AI through the `kserve` component in the `default` `AmlCluster`. For the KServe configuration, see [Install Alauda AI](../../../installation/ai-cluster.mdx). ## Documentation diff --git a/docs/en/deploy/components/lws/install.mdx b/docs/en/deploy/components/lws/install.mdx deleted file mode 100644 index 2e7cb617..00000000 --- a/docs/en/deploy/components/lws/install.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -weight: 20 ---- - -# Install LeaderWorkerSet - -## Downloading Cluster plugin - -:::info - -`Alauda Build of LeaderWorkerSet` cluster plugin can be retrieved from Customer Portal. - -Please contact Consumer Support for more information. - -::: - -## Uploading the Cluster plugin - -For more information on uploading the cluster plugin, please refer to - -## Installing Alauda Build of LeaderWorkerSet - -1. Go to the `Administrator` -> `Marketplace` -> `Cluster Plugin` page, switch to the target cluster, and then deploy the `Alauda Build of LeaderWorkerSet` Cluster plugin. - :::info - **Note: Deploy form parameters can be kept as default or modified after knowing how to use them.** - ::: - -2. Verify result. You can see the status of "Installed" in the UI or you can check the pod status: - ```bash - kubectl get pods -n cpaas-system | grep "lws" - ``` - -## Upgrading Alauda Build of LeaderWorkerSet - -1. Upload the new version for package of **Alauda Build of LeaderWorkerSet** plugin to ACP. -2. Go to the `Administrator` -> `Clusters` -> `Target Cluster` -> `Functional Components` page, then click the `Upgrade` button, and you will see the `Alauda Build of LeaderWorkerSet` can be upgraded. diff --git a/docs/en/deploy/components/lws/intro.mdx b/docs/en/deploy/components/lws/intro.mdx index d4644704..e9b6dd0c 100644 --- a/docs/en/deploy/components/lws/intro.mdx +++ b/docs/en/deploy/components/lws/intro.mdx @@ -18,7 +18,7 @@ Main components and capabilities include: - **Rolling Updates & Failure Recovery**: Supports rolling restarts and automatic pod replacement at the group level, ensuring the entire Leader/Worker group is recycled consistently when a failure or update occurs. - **Startup Sequencing**: The leader pod can act as the entry point and coordinator, with worker pods starting after the leader is ready — enabling frameworks that require a master process to be initialized before workers connect. -For installation on the platform, see [Install LeaderWorkerSet](./install). +LWS is installed and managed by Alauda AI through the `lws` component in the `default` `AmlCluster`. For the Alauda AI installation and component configuration, see [Install Alauda AI](../../../installation/ai-cluster.mdx). ## Documentation diff --git a/docs/en/deploy/gateway/identity_authentication.mdx b/docs/en/deploy/gateway/identity_authentication.mdx index 77c912e0..bd842139 100644 --- a/docs/en/deploy/gateway/identity_authentication.mdx +++ b/docs/en/deploy/gateway/identity_authentication.mdx @@ -18,7 +18,7 @@ This turns a per-consumer credential, such as an SSO (Single Sign-On) token or a ## Prerequisites -1. Envoy AI Gateway is installed. See [Install Envoy AI Gateway](../components/envoy_ai_gateway/install). +1. Envoy AI Gateway is installed. It is managed by Alauda AI through the `envoyAIGateway` component in the `default` `AmlCluster`; see [Install Alauda AI](../../../installation/ai-cluster.mdx) and the [Envoy AI Gateway introduction](../components/envoy_ai_gateway/intro). 2. An `AIGatewayRoute` already routes requests to one or more backends. 3. For the OIDC/JWT path: an OIDC issuer with a reachable JWKS endpoint. The platform's built-in identity provider, Dex, is the default; any other OIDC issuer (Keycloak, Auth0, Okta, GitHub OIDC, an enterprise Entra ID tenant) also works as long as the gateway can reach its `/.well-known/openid-configuration` and JWKS URL. 4. For the API-key path: cluster permission to create `Secret` objects in the gateway's namespace. diff --git a/docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx b/docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx index 41728512..7dfdeb65 100644 --- a/docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx +++ b/docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx @@ -13,7 +13,7 @@ Use this page when your team already works in the Kubeflow dashboard. For the pl ## Prerequisites - The [Kubeflow operators](../../../develop/components/kubeflow/install.mdx) are installed, and you can reach the Kubeflow central dashboard. -- [Alauda Build of KServe](../../components/kserve/install.mdx) is installed. +- [Alauda Build of KServe](../../components/kserve/intro.mdx) is installed. It is managed by Alauda AI through the `kserve` component in the `default` `AmlCluster`; see [Install Alauda AI](../../../installation/ai-cluster.mdx). - The model is reachable at a `storageUri` the cluster can read. See [Model Storage](../../model_management/model_storage.mdx) for the supported storage types. ## Access the Endpoints UI diff --git a/docs/en/deploy/maas.mdx b/docs/en/deploy/maas.mdx deleted file mode 100644 index b9a9ee5f..00000000 --- a/docs/en/deploy/maas.mdx +++ /dev/null @@ -1,31 +0,0 @@ ---- -weight: 30 ---- - -# Model as a Service (MaaS) - -Model as a Service (MaaS) provides a managed model-serving gateway for Alauda AI. It gives administrators a central place to publish models, manage access, configure token quotas, and monitor usage, while application developers call subscribed models through API keys. - -MaaS is based on **Alauda Build of Envoy Gateway** and **Alauda Build of Envoy AI Gateway**. Envoy Gateway provides the Kubernetes Gateway API data-plane foundation, while Envoy AI Gateway adds model-aware routing and OpenAI-compatible inference traffic handling. MaaS uses **Authorino** for authentication. - -## Model Sources - -MaaS can publish inference services created with **llm-d**. An `LLMInferenceService` can be added to MaaS as a managed model and then referenced by subscriptions. - -Administrators can also connect external model services. These external services are managed as model sources in MaaS and can be referenced by subscriptions in the same way as platform-hosted models. - -## Administrator Experience - -In the **Administrator** view, MaaS provides a dedicated management entry. Administrators can create subscriptions and control which users are allowed to use them. - -Each subscription can bind multiple models. For every model in a subscription, administrators can configure multiple token limits with different time ranges, allowing quota policies such as short-term burst limits and longer-term monthly limits. - -The MaaS management view also provides traffic dashboards for subscriptions. Administrators can inspect metrics such as request count, token usage, and response latency. - -MaaS provides usage reports. Administrators can view how much each user consumed by day or by month. - -## User Experience - -Users access MaaS through **My Subscriptions**. In each subscription assigned to them, users can create, revoke, and delete API keys. - -Users can also view their own usage, including request and token consumption, so they can track how their applications consume subscribed model services. diff --git a/docs/en/deploy/maas/administrator.mdx b/docs/en/deploy/maas/administrator.mdx new file mode 100644 index 00000000..5f150979 --- /dev/null +++ b/docs/en/deploy/maas/administrator.mdx @@ -0,0 +1,134 @@ +--- +weight: 20 +--- + +# MaaS Administrator Guide + +This guide explains how platform administrators manage Model as a Service (MaaS): publish model services, create subscriptions, enforce token quotas, configure optional guardrails, and review usage. + +For the product overview, see [Model as a Service (MaaS)](intro.mdx). For the developer workflow, see [MaaS User Guide](user.mdx). + +## Publish platform inference services + +A model owner publishes a platform-hosted inference service from the service's management page. The service must be an `LLMInferenceService`. + +1. Open the project or namespace that contains the inference service. +2. Open the service management page. +3. From the service actions, select **Publish to MaaS**. +4. Enter an optional display name and description. +5. Click **Publish**. + +Publishing creates a MaaS model reference. The model then appears in the administrator's **MaaS** → **Models** → **Published Models** list and can be added to subscriptions. + +Unpublishing removes the model from the MaaS catalog. Existing subscriptions that reference the model no longer resolve it, so remove or replace the model in those subscriptions before unpublishing it. + +## Import an external model service + +Use an external model service when the model is hosted outside the platform or is not represented by an `LLMInferenceService`. + +1. In the **Administrator** view, open **MaaS** → **Models**. +2. On the **Model Services** tab, click **Import External Model Service**. +3. Enter a lowercase name for the service. +4. Enter the provider's OpenAI-compatible endpoint, including its API base path when required. For example, use `https://host/v1` when the provider expects `/v1` requests. +5. Enter an upstream API key if the provider requires one. Leave it empty for an endpoint that does not require credentials. +6. Click **Create**. + +The API key is stored in a Kubernetes Secret and injected by the gateway when it calls the provider. The consumer's MaaS API key and the provider's upstream key are separate credentials: developers receive only the MaaS key. + +### Publish models from an external service + +After importing a service: + +1. On the **Model Services** tab, open the service action menu. +2. Select **Publish Models**. +3. Add, rename, or remove the model names exposed by that service. +4. Click **Save**. + +Each published model becomes a separate subscribable model. One external service can therefore expose multiple model names, and those models can use the same endpoint and upstream credential. + +The **Published Models** tab shows the model name, namespace, reference target, and readiness phase. Use the type filter to distinguish **External** models from models backed by an **Inference Service**. + +## Create a subscription + +A subscription is the unit of model access and token quota. A user must be a subscriber of a subscription that contains a model before they can call that model. + +1. In the **Administrator** view, open **MaaS** → **Subscriptions**. +2. Click **Create Subscription**. +3. Enter a subscription name. +4. Add one or more subscribers by username. +5. Add one or more published models. +6. For each model, configure at least one token limit and time window. +7. Optionally add more quota tiers for the same model. +8. Click **Save**. + +:::info +Users can see a subscription in **My Subscriptions** only when they are included as subscribers. Add each user's username to **Subscribers** to grant access to that user. To make the subscription available to all authenticated users, add `*` as a subscriber. The `*` value is a wildcard, not a literal username; users still need a valid MaaS API key to call the models. +::: + +For each model, the form accepts a limit in millions of tokens and a window such as **Minute**, **Hour**, or **Day**. For example, you can set `100M` tokens per hour and `1000M` tokens per day. All configured tiers are enforced. + +A subscription can contain several models, and each model can have different limits. This lets you provide different access tiers without creating a separate gateway for each model. + +The subscription list shows the subscription name, models, subscribers, and current phase. Use the row action menu to view, edit, or delete a subscription. + +## Configure guardrails + +MaaS can optionally check model requests and responses with configured content guardrails. + +### Enable guardrails + +1. Open **MaaS** → **Guardrails**. +2. Click **Edit settings**. +3. Enable **guardrails**. +4. Select a failure policy: + - **End request on guardrails failure** (`FailClosed`) rejects or ends the request when the guardrails engine cannot complete the check. + - **Ignore guardrails on failure** (`FailOpen`) allows the request to continue when the guardrails engine fails. +5. Save the settings. + +Disabling guardrails removes the data-plane integration. Guardrail settings that remain on subscriptions do not take effect while the global feature is disabled. + +### Create a guardrail configuration + +1. On the **Guardrails** page, click **Create Config**. +2. Enter a configuration ID using lowercase letters, numbers, and hyphens. +3. Configure the guardrail models, including model type, model name, and base URL. +4. Add an optional API key for a guardrail model. Credentials are stored as Kubernetes Secrets and exposed to the guardrails engine through environment variables. +5. Edit the `config.yaml` content. +6. Edit the `rails.co` content when your configuration requires Colang flows. +7. Click **Save**. + +A configuration can contain multiple guardrail models, but each model type must be unique within the configuration. The **Guardrails** list shows whether a configuration is attached to a subscription. + +### Attach guardrails to a subscription model + +When creating or editing a subscription, enable guardrails for a model and select one of the available guardrail configurations. The setting is applied per model, so different models in the same subscription can use different configurations or no guardrails. + +## Review usage and reports + +### Subscription management usage + +The **Subscriptions** page shows the subscriptions and their configured limits. Use the subscription's usage view, where available, to review token consumption for the selected subscription and model. + +### Usage report + +Open **MaaS** → **Report** to review administrator-level usage: + +1. Select a time range: **Last 1 day**, **Last 7 days**, **Last 30 days**, **Last 90 days**, or **Last 1 year**. +2. Optionally select a subscription. +3. Search by username. +4. Click refresh to reload the data. + +The report displays: + +- Username +- Subscription +- Total tokens +- Input tokens +- Cached tokens +- Output tokens +- Reasoning tokens + +Click **Export CSV** to export the filtered report. The export dialog lets you group data by subscription, user, API key, or model, and group time by day or month. + +Usage data is intended for operational visibility, quota observation, and showback. It is not a billing settlement record. + diff --git a/docs/en/deploy/maas/index.mdx b/docs/en/deploy/maas/index.mdx new file mode 100644 index 00000000..53a95048 --- /dev/null +++ b/docs/en/deploy/maas/index.mdx @@ -0,0 +1,7 @@ +--- +weight: 60 +--- + +# Model as a Service + + diff --git a/docs/en/deploy/maas/intro.mdx b/docs/en/deploy/maas/intro.mdx new file mode 100644 index 00000000..6918a3b0 --- /dev/null +++ b/docs/en/deploy/maas/intro.mdx @@ -0,0 +1,57 @@ +--- +weight: 10 +--- + +# Introduction + +Model as a Service (MaaS) provides a managed gateway for serving large language models through an OpenAI-compatible API. It gives platform teams a consistent way to publish models, grant access, enforce token quotas, and understand usage, while application developers call the models they are subscribed to with API keys. + +MaaS is based on **Alauda Build of Envoy Gateway**, **Alauda Build of Envoy AI Gateway**, and **Authorino**. Envoy Gateway provides the Kubernetes Gateway API foundation, Envoy AI Gateway provides model-aware routing and token accounting, and Authorino authenticates requests. Alauda AI installs and manages these components through the `envoyGateway`, `envoyAIGateway`, and `authorino` components of the `default` `AmlCluster`; see [Install Alauda AI](../../installation/ai-cluster.mdx). + +## Why use MaaS? + +MaaS separates model serving from model consumption: + +- Model owners can publish an inference service without exposing its internal route details. +- Administrators can combine models into subscriptions and assign access to users. +- Developers use one OpenAI-compatible gateway instead of learning the URL and authentication method of every backend. +- Platform teams can enforce per-model token quotas and review consumption by user, subscription, model, and token type. + +MaaS can expose both platform-hosted inference services and external OpenAI-compatible model providers through the same gateway. + +## Request flow + +For each request, MaaS performs the following checks: + +1. The caller presents an API key and is authenticated. +2. The caller has access to the requested model through an active subscription. +3. The subscription has remaining token quota for that user and model. +4. The gateway routes the request to the selected model and records the actual token usage returned by the model service. + +An unauthenticated or unauthorized request returns `401` or `403`. A request that exceeds its subscription quota returns `429`. + +Quotas are enforced per user, not per API key. Multiple API keys belonging to the same user share the user's allowance for a subscription and model. + +## Main concepts + +| Concept | Purpose | +| --- | --- | +| **Model** | A published model that can be selected by a subscription and called through the MaaS gateway. | +| **Model source** | A platform inference service or an external model service that supplies one or more models. | +| **Subscription** | The access and quota unit. It binds users to models and defines token limits for each model. | +| **API key** | A developer credential used to call subscribed models. The plaintext key is shown only once when it is created. | +| **Usage** | Token and request consumption shown to administrators and to developers for their own API keys. | +| **Guardrails** | Optional content checks that can be attached to models in a subscription. | + +## Choose a guide + +- [MaaS Administrator Guide](administrator.mdx) — publish and import models, create subscriptions, configure quotas and guardrails, and review reports. +- [MaaS User Guide](user.mdx) — create an API key, call a subscribed model, and view personal quota and usage. + +## Related topics + +- [Inference Service](../inference_service/inference_service) — create the `LLMInferenceService` models that can be published to MaaS. +- [Authenticating Consumers](../gateway/identity_authentication) — authenticate callers and propagate identity to gateway policies. +- [Configuring Token Quotas](../gateway/token_rate_limiting) — configure token-based quota enforcement at the inference gateway. +- [Metering Token Usage](../gateway/usage_metering) — understand how token consumption is reported. +- [Routing to LLM Providers](../gateway/external_provider_routing) — route external providers through a controlled gateway. diff --git a/docs/en/deploy/maas/user.mdx b/docs/en/deploy/maas/user.mdx new file mode 100644 index 00000000..a5c8e8aa --- /dev/null +++ b/docs/en/deploy/maas/user.mdx @@ -0,0 +1,148 @@ +--- +weight: 30 +--- + +# MaaS User Guide + +This guide explains how developers and application users call models through Model as a Service (MaaS). You will learn how to create an API key, find the models available to you, send an OpenAI-compatible request, and review your own quota and usage. + +For the product overview, see [Model as a Service (MaaS)](intro.mdx). For administration tasks, see [MaaS Administrator Guide](administrator.mdx). + +## Open My Subscriptions + +1. Open the account menu in the top-right corner of the Alauda AI console. +2. Select **My Subscriptions**. + +MaaS opens in a side panel over the current page. It is tied to your user identity rather than to a project namespace, so you can open it from any view. + +The panel contains two tabs: + +- **API Keys** — create and manage your MaaS credentials and find an access example. +- **Usage** — review quota status and personal usage for a subscription. + +## Find your models and endpoint + +On the **API Keys** tab, the **Access** section lists the models currently available to you. Each row shows the model name and endpoint. + +Select a model to update the example request. Use the copy control next to the endpoint or example request to copy it to your clipboard. + +The endpoint normally has this form: + +```text +https:///v1 +``` + +Use the model name shown in the access list as the `model` value in your request. Do not use the name of the underlying Kubernetes resource unless it is also the published model name. + +## Create an API key + +1. On the **API Keys** tab, click **Create API Key**. +2. Enter a descriptive key name, such as `my-chatbot-dev`. +3. Select the subscription to use. +4. Select a validity period. Depending on the platform policy, you can choose a preset such as 30 or 90 days, a custom duration, or a permanent key. +5. Click **Create**. +6. Copy the plaintext key and store it securely. + +The plaintext key is shown **only once** and cannot be retrieved later. The dialog also provides a ready-to-run request with the new key filled in. Treat the key like a password: + +- Do not commit it to source control. +- Do not include it in browser code or public notebooks. +- Store it in a secret manager or an environment variable. +- Create separate keys for separate applications when possible. +- Revoke a key immediately if it may have been exposed. + +The key list shows the key name, status, subscription, creation time, and last-used time. A key can have one of the following statuses: + +- `active` — can be used to call subscribed models. +- `expired` — its validity period has ended. +- `revoked` — it has been disabled and cannot be used. + +You can **Revoke** an active key. After it is revoked, you can delete it from the list. + +## Call a model + +MaaS exposes an OpenAI-compatible API. Set the gateway URL and API key as environment variables: + +```bash +export MAAS_ENDPOINT="https:///v1" +export MAAS_API_KEY="" +``` + +Send a chat completion request with `curl`: + +```bash +curl "${MAAS_ENDPOINT}/chat/completions" \ + -H "Authorization: Bearer ${MAAS_API_KEY}" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "", + "messages": [ + {"role": "user", "content": "Hello"} + ] + }' +``` + +You can also use an OpenAI-compatible Python client: + +```python +from openai import OpenAI + +client = OpenAI( + base_url="https:///v1", + api_key="", +) + +response = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "Hello"}], +) + +print(response.choices[0].message.content) +``` + +Replace `` with the model name shown in your **Access** list. The MaaS gateway chooses the backend associated with that published model name; your application does not need to know the backend service URL. + +## Understand quota behavior + +MaaS quotas are configured by the administrator for each model in your subscription. A subscription can have multiple time windows, such as an hourly limit and a daily limit. + +Quota consumption is tracked per user, subscription, and model. Your API keys share your user quota: + +- Creating a second API key does not reset or increase your quota. +- Requests made with your different keys count toward the same user allowance. +- Other users have separate quota buckets. + +When a quota window is close to its limit, reduce request volume or ask your administrator to review the subscription. When the quota is exhausted, the gateway returns `429 Too Many Requests`. The quota panel shows the configured limit, current usage when available, and the reset time. + +## View personal usage + +Open the **Usage** tab in **My Subscriptions**: + +1. Select a subscription. +2. Review the per-model quota bars. +3. Choose a time range: **Last 24 hours**, **Last 7 days**, or **Last 30 days**. +4. Optionally filter the chart by model or API key. +5. Click refresh to load the latest data. + +The usage summary includes request count, total tokens, and cached tokens. The chart shows the token series over time, including uncached input, cached input, and output tokens when the backend provides those values. + +This is a personal view: it includes only requests made with API keys that belong to you. It does not show another user's usage, even if you are an administrator in another part of the platform. + +## Understand request errors + +| Status | Meaning | What to do | +| --- | --- | --- | +| `401 Unauthorized` | The API key is missing, invalid, expired, or revoked. | Check the `Authorization` header and create a new key if necessary. | +| `403 Forbidden` | You do not have access to the requested model or do not have an applicable subscription. | Confirm the model name and contact your MaaS administrator. | +| `429 Too Many Requests` | The subscription quota for the user, model, or time window is exhausted. | Wait for the quota to reset or ask the administrator to review the limit. | +| `5xx` | The gateway or model backend encountered an error. | Retry when appropriate and contact the platform administrator if the problem continues. | + +A model can also be visible in the access list but temporarily unavailable while its inference service is not ready. If requests continue to fail after the service should be ready, contact the administrator with the model name and request time. + +## Security recommendations + +- Keep API keys out of source code, Git repositories, shell history, and client-side applications. +- Use the shortest validity period that meets your needs. +- Use one key per application or environment so a compromised key can be revoked without interrupting unrelated applications. +- Revoke unused or exposed keys from **My Subscriptions**. +- Never share the MaaS API key with the external provider. MaaS handles the upstream provider credential on the gateway side. diff --git a/docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx b/docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx index 473bbe02..5d7275aa 100644 --- a/docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx +++ b/docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx @@ -56,7 +56,7 @@ For the YAML fields and platform-specific labels/annotations the agent needs to ## Manage gateways: authentication and rate limits \{#manage-gateways} -Alauda Build of Envoy AI Gateway is a required dependency of Alauda Build of KServe and fronts inference traffic with an OpenAI-compatible API surface, AI-aware routing, and per-model policies (see [Envoy AI Gateway introduction](../../deploy/components/envoy_ai_gateway/intro.mdx) and [installation](../../deploy/components/envoy_ai_gateway/install.mdx)). The agent is well-suited to author its CRDs, which are otherwise verbose: +Alauda Build of Envoy AI Gateway is a required dependency of Alauda Build of KServe and fronts inference traffic with an OpenAI-compatible API surface, AI-aware routing, and per-model policies (see [Envoy AI Gateway introduction](../../deploy/components/envoy_ai_gateway/intro.mdx)). Envoy AI Gateway is managed by Alauda AI through the `envoyAIGateway` component in the `default` `AmlCluster`; see [Install Alauda AI](../../installation/ai-cluster.mdx). The agent is well-suited to author its CRDs, which are otherwise verbose: | Concern | CRD / Resource | Where it comes from | |---|---|---| @@ -260,7 +260,6 @@ Each step is a separate prompt with its own diff to review. The agent is the typ - [Speculative Decoding for vLLM Inference Services](../../deploy/inference_service/guides/vllm_speculative_decoding.mdx) - [Extend Inference Runtimes](../../deploy/inference_service/guides/custom_inference_runtime.mdx) - [Envoy AI Gateway — introduction](../../deploy/components/envoy_ai_gateway/intro.mdx) -- [Install Envoy AI Gateway](../../deploy/components/envoy_ai_gateway/install.mdx) - [Hardware Profiles](../../administer/hardware_profile/intro.mdx) - [Fine-Tuning with Kubeflow Trainer v2](../../train/guides/fine-tune-with-trainer-v2.mdx) - [Fine-tuning LLMs with Training Hub](../../train/guides/training-hub-fine-tuning.mdx) diff --git a/docs/en/develop/components/data_science_pipelines/install.mdx b/docs/en/develop/components/data_science_pipelines/install.mdx index 5f0512c2..bbb06825 100644 --- a/docs/en/develop/components/data_science_pipelines/install.mdx +++ b/docs/en/develop/components/data_science_pipelines/install.mdx @@ -2,7 +2,7 @@ weight: 20 --- -# Installation +# Install Data Science Pipelines The Data Science Pipelines Operator is delivered as an **OLM Operator** and installed from the platform **OperatorHub**. @@ -21,23 +21,15 @@ The Data Science Pipelines Operator is delivered as an **OLM Operator** and inst Skip this if the cluster already has `workflows.argoproj.io` (for example, via an existing Argo install). - **Alauda ServiceMesh v2 (Istio)** — only required if you plan to expose the DSPA APIServer through an Istio `VirtualService` (`EXTERNAL_ROUTE_PROVIDER=virtualservice`). `virtualservices.networking.istio.io` must be Established. -## Upload Operator \{#upload-operator} +## Upload the Operator Package -Download the `Data Science Pipelines Operator` bundle from the Customer Portal / Marketplace (e.g. `data-science-pipelines-operator.ALL.xxxx.tgz`), then publish it to the platform repository with the `violet` command-line tool: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - data-science-pipelines-operator.ALL.xxxx.tgz -``` +Download the `Data Science Pipelines Operator` bundle from the Customer Portal / Marketplace, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). :::info The operator bundle records every runtime image (operator, KFP APIServer / driver / launcher / persistence-agent / scheduled-workflow, Argo workflow-controller and argoexec, MLMD, MariaDB, and the pipeline runtime image) in the CSV `relatedImages`, so a `violet` release relocates them into the platform registry. This makes the operator installable on **air-gapped** clusters without reaching `quay.io` / `docker.io`. ::: -## Install Operator +## Install the Operator In the **Administrator** view: diff --git a/docs/en/develop/components/feast/install.mdx b/docs/en/develop/components/feast/install.mdx index 9006b702..77b35521 100644 --- a/docs/en/develop/components/feast/install.mdx +++ b/docs/en/develop/components/feast/install.mdx @@ -4,40 +4,42 @@ weight: 20 # Install Feast -This page describes how to install the Feast Operator on Alauda Container Platform. After installation, `FeatureStore` custom resources can be created to deploy Feast services. +In Alauda AI 2.8.x, **Alauda Build of Feast** is installed and managed by Alauda AI. Set the `feast` component in the cluster-scoped `default` `AmlCluster` to `Managed`; you do not need to install the Operator from OperatorHub or create a custom resource manually. After the component is reconciled, `FeatureStore` custom resources can be created to deploy Feast services. ## Upload the Operator Package -Download the Feast Operator package, for example `feast-operator.ALL.xxxx.tgz`. +Download the Feast Operator package, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). -Use `violet` to upload the package to the platform repository: +## Enable Feast -```bash -violet push --platform-address= --platform-username= --platform-password= feast-operator.ALL.xxxx.tgz -``` +In **Administrator** view: -## Install the Operator +1. Open **Marketplace** > **OperatorHub**. +2. Select the target cluster and open **Alauda AI**. +3. Open the **All Instances** tab and edit the `default` `AmlCluster` instance. +4. Set `spec.components.feast.managementState` to `Managed`. +5. Save the `AmlCluster` instance and wait for the Feast Operator to become ready. -In **Administrator** view: +You can apply the component setting in the `AmlCluster` YAML: -1. Go to `Marketplace` / `Operator Hub`. -2. Select the destination cluster. -3. Search for **Alauda Build of Feast**. -4. Click **Install**. -5. Keep the default installation settings unless the environment requires a different namespace. -6. Complete the installation. +```yaml +spec: + components: + feast: + managementState: Managed +``` ## Verify the Installation -After installation, verify that the operator is running: +Check the `AmlCluster` status and the Feast Operator: ```bash +kubectl get amlcluster default + kubectl get pods -n feast-operator-system ``` -The Feast Operator controller pod should be in `Running` state. - -The `FeatureStore` CRD can also be verified: +The `AmlCluster` should report `Phase=Ready`, and the Feast Operator controller pod should be in `Running` state. The `FeatureStore` CRD is registered once the operator is ready: ```bash kubectl get crd featurestores.feast.dev @@ -45,6 +47,6 @@ kubectl get crd featurestores.feast.dev ## Next Steps -Installing the operator creates only the controller and CRDs. Feast services are created after a `FeatureStore` custom resource is applied. +Enabling the component creates the Operator, controller, and CRDs. Feast services are created after a `FeatureStore` custom resource is applied. Continue with [Quickstart](./quickstart) for `FeatureStore` configuration and deployment examples. diff --git a/docs/en/develop/components/kubeflow/install.mdx b/docs/en/develop/components/kubeflow/install.mdx index 2fbeea73..117e1498 100644 --- a/docs/en/develop/components/kubeflow/install.mdx +++ b/docs/en/develop/components/kubeflow/install.mdx @@ -24,7 +24,7 @@ Before you begin, make sure the following prerequisites are met: 2. Alauda AI is already deployed. Alauda AI 2.3 or later is required for the v26.3.0 operator set. 3. Alauda Build of KServe is installed. 4. ASM is deployed in the business cluster where Kubeflow will run. If ASM is not already installed, deploy it before continuing. ASM v1 is deprecated. Use ASM v2 whenever possible. -5. The LWS plugin, Alauda Build of LeaderWorkerSet, is installed if you plan to deploy `kubeflow-trainer-operator`. +5. LeaderWorkerSet is installed and ready. LWS is normally managed by Alauda AI through the `lws` component in the `default` `AmlCluster`; see [Install Alauda AI](../../../installation/ai-cluster.mdx). It is required only if you plan to deploy `kubeflow-trainer-operator`. 6. The `oauth2-proxy` plugin is configured as described below. ### Configure Dex Redirection @@ -132,15 +132,7 @@ spec: ## Component Onboarding -Download the operator bundle packages for the following operators and upload them with `violet`. The bundles register the operators with the ACP OperatorHub. - -```bash -# Replace the platform address, username, password, and bundle package path. -violet push --platform-address="https://" \ - --platform-username="admin@cpaas.io" \ - --platform-password="" \ - -``` +Download the operator bundle packages for the following operators, then upload them following [Upload Packages](../../../installation/package-upload.mdx). The bundles register the operators with the ACP OperatorHub. - `kfbase-operator`: Kubeflow base functionality (owns `KubeflowBase` CR). - `kfp-operator`: Kubeflow Pipelines (owns `KubeflowPipelines` CR). amd64-only. @@ -286,7 +278,7 @@ When creating the instance, configure the following fields as needed: > **Note:** If the deprecated `kftraining` Cluster Plugin is still installed (from a pre-v26.3.0 cluster), uninstall it before installing `kubeflow-trainer-operator`. > -> **Note:** Install the LWS plugin before deploying `kubeflow-trainer-operator`, because LWS is a dependency of `kubeflow-trainer-operator`. +> **Note:** LeaderWorkerSet is a dependency of `kubeflow-trainer-operator`. Verify that LWS is enabled and ready through the `lws` component in the `default` `AmlCluster` before deploying `kubeflow-trainer-operator`; see [Install Alauda AI](../../../installation/ai-cluster.mdx). > > **Note:** v26.3.0 of `kubeflow-trainer-operator` aligns with upstream `kubeflow/manifests` 26.03 and ships **Trainer v2.1.0 + JobSet v0.10.1**. For clusters where an OLM CatalogSource already advertises a higher trainer version (`>=2.2.0`), install with `installPlanApproval: Manual` and `startingCSV: kubeflow-trainer-operator.v2.1.0` to prevent OLM from auto-upgrading past the 26.03 pin. diff --git a/docs/en/develop/components/kuberay/install.mdx b/docs/en/develop/components/kuberay/install.mdx index c9faa6e7..5cda27bd 100644 --- a/docs/en/develop/components/kuberay/install.mdx +++ b/docs/en/develop/components/kuberay/install.mdx @@ -2,22 +2,14 @@ weight: 20 --- -# Installation +# Install KubeRay Operator ## Prerequisites - **ACP version: v4.0 or later** ### Downloading Cluster Plugin -:::info - -Download `Alauda Build of KubeRay Operator` package from the Customer Portal or contact support. - -::: - -### Uploading the Cluster Plugin - -The platform provides the **`violet`** command-line tool for uploading packages downloaded from the Customer Portal Marketplace. +Download the `Alauda Build of KubeRay Operator` package from the Customer Portal or contact support, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). ### Installing Alauda Build of KubeRay Operator diff --git a/docs/en/develop/components/label_studio/install.mdx b/docs/en/develop/components/label_studio/install.mdx index d53b5756..7fa73bd6 100644 --- a/docs/en/develop/components/label_studio/install.mdx +++ b/docs/en/develop/components/label_studio/install.mdx @@ -8,12 +8,7 @@ This document provides detailed instructions on how to deploy Label Studio to a ## Downloading -Download the Label Studio installation file: `label-studio.ALL.xxxx.tgz` - -Use the violet command to publish to the platform repository: -```bash -violet push --platform-address=platform-access-address --platform-username=platform-admin --platform-password=platform-admin-password label-studio.ALL.xxxx.tgz -``` +Download the Label Studio installation package, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). ## Deployment diff --git a/docs/en/develop/components/mlflow/install.mdx b/docs/en/develop/components/mlflow/install.mdx index b3e53b12..06a134c6 100644 --- a/docs/en/develop/components/mlflow/install.mdx +++ b/docs/en/develop/components/mlflow/install.mdx @@ -2,9 +2,9 @@ weight: 20 --- -# Installation +# Install MLflow -MLflow is delivered as an **OLM Operator** (the *MLflow Operator*) and installed from the platform **OperatorHub**. Installing the tracking server is a two-step flow: install the operator, then create one `MLflow` custom resource that reconciles the tracking-server stack. +In Alauda AI 2.8.x, **MLflow** is installed and managed by Alauda AI. The `mlflow` component in the cluster-scoped `default` `AmlCluster` defaults to `Unmanaged`; set it to `Managed`, and Alauda AI installs the **MLflow Operator** automatically — you do not need to install the Operator from OperatorHub manually. Enabling the component installs the Operator only; you still create an `MLflow` custom resource to deploy the tracking-server stack, as described in [Create the MLflow tracking server](#create-the-mlflow-tracking-server). ## Prerequisites @@ -26,38 +26,38 @@ MLflow is delivered as an **OLM Operator** (the *MLflow Operator*) and installed kubectl label namespace finetune mlflow-enabled=true ``` -## Upload Operator \{#upload-operator} +## Upload the Operator Package -Download the `MLflow Operator` bundle from the Customer Portal / Marketplace (e.g. `mlflow.ALL.xxxx.tgz`), then publish it to the platform repository with the `violet` command-line tool: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - mlflow.ALL.xxxx.tgz -``` +Download the `MLflow Operator` bundle from the Customer Portal / Marketplace, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). :::info The operator bundle records every runtime image (operator, MLflow tracking server, and the `oauth2-proxy`) in the CSV `relatedImages`, so a `violet` release relocates them into the platform registry. This makes the operator installable on **air-gapped** clusters without reaching `build-harbor.alauda.cn` / `docker.io`. ::: -## Install Operator +## Enable MLflow + +In **Administrator** view: + +1. Open **Marketplace** > **OperatorHub**. +2. Select the target cluster and open **Alauda AI**. +3. Open the **All Instances** tab and edit the `default` `AmlCluster` instance. +4. Set `spec.components.mlflow.managementState` to `Managed`. +5. Save the `AmlCluster` instance and wait for the MLflow Operator to become ready. -In the **Administrator** view: +You can apply the component setting in the `AmlCluster` YAML: -1. Click **Marketplace** / **OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown, select the destination cluster. -3. Search for and select **MLflow Operator**, then click **Install**. -4. Leave **Channel** unchanged (`stable`). -5. Check that the **Version** is `v3.13.0`. -6. Leave **Installation Location** unchanged — it defaults to the `mlflow-operator` namespace. -7. Choose an **Upgrade Strategy** (`Manual` is recommended for production). -8. Click **Install**. +```yaml +spec: + components: + mlflow: + managementState: Managed +``` Confirm the operator is ready before continuing: ```bash +kubectl get amlcluster default + # the CSV reports Succeeded kubectl get csv -n mlflow-operator | grep mlflow-operator @@ -65,7 +65,9 @@ kubectl get csv -n mlflow-operator | grep mlflow-operator kubectl get pods -n mlflow-operator ``` -## Create the MLflow tracking server +The `AmlCluster` should report `Phase=Ready`. Enabling the component installs the **MLflow Operator**; the tracking server is deployed only after you create an `MLflow` custom resource as described below. + +## Create the MLflow tracking server \{#create-the-mlflow-tracking-server} The operator does nothing until you create an `MLflow` custom resource (`mlflow.alauda.io/v1alpha1`). You can create it two ways: diff --git a/docs/en/develop/components/spark_operator/install.mdx b/docs/en/develop/components/spark_operator/install.mdx index d7782c28..9341d36a 100644 --- a/docs/en/develop/components/spark_operator/install.mdx +++ b/docs/en/develop/components/spark_operator/install.mdx @@ -2,7 +2,7 @@ weight: 20 --- -# Installation +# Install Spark Operator Alauda Build of Spark Operator is delivered as an **OLM Operator** and installed from the platform **OperatorHub**. @@ -12,23 +12,15 @@ Alauda Build of Spark Operator is delivered as an **OLM Operator** and installed - Target cluster architecture `linux/amd64` or `linux/arm64` (the operator ships multi-arch images). - Operator Lifecycle Manager (OLM) available on the target cluster (provided by ACP). -## Upload Operator \{#upload-operator} +## Upload the Operator Package -Download the `Alauda Build of Spark Operator` bundle from the Customer Portal / Marketplace (e.g. `spark-operator.ALL.xxxx.tgz`), then publish it to the platform repository with the `violet` command-line tool: - -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - spark-operator.ALL.xxxx.tgz -``` +Download the `Alauda Build of Spark Operator` bundle from the Customer Portal / Marketplace, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). :::info The operator bundle records its runtime images (the operator image and the Apache Spark runtime image) in the CSV `relatedImages`, so a `violet` release relocates them into the platform registry. This makes the operator installable on **air-gapped** clusters without reaching `docker.io`. ::: -## Install Operator +## Install the Operator In the **Administrator** view: diff --git a/docs/en/evaluate_safety/components/trustyai/install.mdx b/docs/en/evaluate_safety/components/trustyai/install.mdx index 6ae8f703..93fdbb09 100644 --- a/docs/en/evaluate_safety/components/trustyai/install.mdx +++ b/docs/en/evaluate_safety/components/trustyai/install.mdx @@ -4,30 +4,45 @@ weight: 20 # Install TrustyAI -This document describes how to install TrustyAI operator. +In Alauda AI 2.8.x, **Alauda Build of TrustyAI** is installed and managed by Alauda AI. The `trustyAI` component in the cluster-scoped `default` `AmlCluster` defaults to `Unmanaged`; set it to `Managed`, and you do not need to install the Operator from OperatorHub manually. After the component is reconciled, TrustyAI custom resources can be created to use its capabilities. -## Upload Operator +## Upload the Operator Package -Download the TrustyAI operator installation file (e.g., `trustyai-service-operator.alpha.ALL.xxxx.tgz`). +Download the TrustyAI Operator package, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). -Use the violet command to publish to the platform repository: +## Enable TrustyAI -```bash -violet push --platform-address=platform-access-address --platform-username=platform-admin --platform-password=platform-admin-password trustyai-service-operator.alpha.ALL.xxxx.tgz +In **Administrator** view: + +1. Open **Marketplace** > **OperatorHub**. +2. Select the target cluster and open **Alauda AI**. +3. Open the **All Instances** tab and edit the `default` `AmlCluster` instance. +4. Set `spec.components.trustyAI.managementState` to `Managed`. +5. Save the `AmlCluster` instance and wait for the TrustyAI Operator to become ready. + +You can apply the component setting in the `AmlCluster` YAML: + +```yaml +spec: + components: + trustyAI: + managementState: Managed ``` -## Install Operator +## Verify the Installation -1. Go to the `Administrator` view in the Alauda Container Platform. +Check the `AmlCluster` status and the TrustyAI Operator: -2. In the left navigation, select `Marketplace` / `Operator Hub`. +```bash +kubectl get amlcluster default -3. In the right panel, find `Alauda Build of TrustyAI` and click `Install`. +kubectl get pods -n trustyai-operator-system +``` -4. Keep all parameters as default and complete the installation. +The `AmlCluster` should report `Phase=Ready`, and the TrustyAI Operator controller pod should be in `Running` state. The TrustyAI CRDs are registered once the operator is ready. ## Next steps -After the operator is installed, users can use CRDs such as **TrustyAIService**, **LMEvalJob**, **GuardrailsOrchestrator**, and **NemoGuardrails** to deploy inference monitoring (TrustyAI Service), run model evaluation tasks, and manage AI guardrails orchestration (both FMS Guardrails and NeMo Guardrails). +After the component is enabled, users can use CRDs such as **TrustyAIService**, **LMEvalJob**, **GuardrailsOrchestrator**, and **NemoGuardrails** to deploy inference monitoring (TrustyAI Service), run model evaluation tasks, and manage AI guardrails orchestration (both FMS Guardrails and NeMo Guardrails). See [Deploy TrustyAI Service](./deploy_tas) and [Bias and Drift Monitoring](../../../monitor/bias_drift), [Evaluate LLM](../../lm_eval), [AI Guardrails for LLM safety](../../ai_guardrails), and [NeMo Guardrails](../../nemo_guardrails) for details. diff --git a/docs/en/installation/ai-cluster.mdx b/docs/en/installation/ai-cluster.mdx index 17b07d61..43bc02fb 100644 --- a/docs/en/installation/ai-cluster.mdx +++ b/docs/en/installation/ai-cluster.mdx @@ -1,14 +1,12 @@ --- -weight: 40 +weight: 30 --- # Install Alauda AI -**Alauda AI** now offers flexible deployment options. Starting with **Alauda AI** `1.4`, the **Knative** capability is an optional feature, allowing for a more streamlined installation if it's not needed. +**Alauda AI** provides model management, inference, training, and MLOps capabilities through the Alauda AI Operator. The automatically created `default` `AmlCluster` instance controls the components enabled in the cluster. By default, Alauda AI uses the **KServe** `Standard` mode for the inference backend, which is particularly recommended for resource-intensive generative workloads. This mode provides a straightforward way to deploy models and offers robust, customizable deployment capabilities by leveraging foundational Kubernetes functionalities. -To begin, you will need to deploy the **Alauda AI Operator**. This is the core engine for all Alauda AI products. By default, it uses the **KServe** `Standard` mode for the inference backend, which is particularly recommended for resource-intensive generative workloads. This mode provides a straightforward way to deploy models and offers robust, customizable deployment capabilities by leveraging foundational Kubernetes functionalities. - -If your use case requires `Knative` functionality, which enables advanced features like **scaling to zero on demand** for cost optimization, you can optionally install the **Knative Operator**. This operator is not part of the default installation and can be added at any time to enable `Knative` functionality. +Optional capabilities, including **Knative** functionality for scaling inference services to zero, are enabled by updating the component switches in the `AmlCluster` instance. The required operators and operands are installed automatically according to those settings. @@ -16,124 +14,36 @@ If your use case requires `Knative` functionality, which enables advanced featur [Recommended deployment option](https://kserve.github.io/website/docs/admin-guide/overview#generative-inference): For generative inference workloads, the **Standard** approach (previously known as RawKubernetes Deployment) is recommended as it provides the most control over resource allocation and scaling. ::: -## Downloading \{#downloading} - -**Operator Components**: - -- **Alauda AI Operator** - - Alauda AI Operator is the main engine that powers Alauda AI products. It focuses on two core functions: model management and inference services, and provides a flexible framework that can be easily expanded. +## Downloading packages - _Download package: aml-operator.xxx.tgz_ +For the common procedure for downloading installation packages and the `violet` tool, see [Upload Packages](./package-upload.mdx). -- **Knative Operator** +The package required for Alauda AI is: - Knative Operator provides serverless model inference. +- **Alauda AI** — the main platform component for model management, inference services, and component lifecycle management. - _Download package: knative-operator.ALL.v1.x.x-yymmdd.tgz_ - -:::info -You can download the app named 'Alauda AI' and 'Knative Operator' from the Marketplace on the Customer Portal website. -::: +Download the package version that matches the Alauda AI release and the target cluster architecture. ## Prerequisites -Install the following dependent components on the target cluster before installing Alauda AI. They provide the authentication, metadata storage, and cache services required by [Model as a Service (MaaS)](../deploy/maas.mdx): +The dependent components listed below are delivered as packages. You do not need to install them on the target cluster separately — upload their packages to the platform repository before installing Alauda AI, and the `AmlCluster` installs and manages the components after Alauda AI is installed. + +The following components are required by default: -- **Authorino Operator** - **PostgreSQL** - **Alauda Cache Service for Redis OSS** +- **Alauda Build of Authorino** +- **Alauda Build of Envoy AI Gateway** +- **Alauda Build of KServe** +- **Alauda Build of LeaderWorkerSet** -## Uploading \{#uploading} - -We need to upload both `Alauda AI` and `Knative Operator` to the cluster where Alauda AI is to be used. - - - -### Downloading the violet tool - -First, we need to download the `violet` tool if not present on the machine. - -Log into the Web Console and switch to the **Administrator** view: - -1. Click **Marketplace / Upload Packages**. -2. Click **Download Packaging and Listing Tool**. -3. Locate the right OS / CPU architecture under **Execution Environment**. -4. Click **Download** to download the `violet` tool. -5. Run `chmod +x ${PATH_TO_THE_VIOLET_TOOL}` to make the tool executable. - -### Uploading package - -Save the following script in `uploading-ai-cluster-packages.sh` first, then read the comments below to update environment variables for configuration in that script. - -```bash title=uploading-ai-cluster-packages.sh -#!/usr/bin/env bash -export PLATFORM_ADDRESS=https://platform-address # [!code callout] -export PLATFORM_ADMIN_USER= # [!code callout] -export PLATFORM_ADMIN_PASSWORD= # [!code callout] -export CLUSTER= # [!code callout] - -export AI_CLUSTER_OPERATOR_NAME= # [!code callout] -export KNATIVE_OPERATOR_PKG_NAME= # [!code callout] - -VIOLET_EXTRA_ARGS=() -IS_EXTERNAL_REGISTRY= - -# If the image registry type of destination cluster is not platform built-in (external private or public repository). -# Additional configuration is required (uncomment following line): -# IS_EXTERNAL_REGISTRY=true -if [[ "${IS_EXTERNAL_REGISTRY}" == "true" ]]; then - REGISTRY_ADDRESS= # [!code callout] - REGISTRY_USERNAME= # [!code callout] - REGISTRY_PASSWORD= # [!code callout] - - VIOLET_EXTRA_ARGS+=( - --dst-repo "${REGISTRY_ADDRESS}" - --username "${REGISTRY_USERNAME}" - --password "${REGISTRY_PASSWORD}" - ) -fi - -# [!code highlight] -# Push **Alauda AI Cluster** operator package to destination cluster -violet push \ - ${AI_CLUSTER_OPERATOR_NAME} \ - --platform-address=${PLATFORM_ADDRESS} \ - --platform-username=${PLATFORM_ADMIN_USER} \ - --platform-password=${PLATFORM_ADMIN_PASSWORD} \ - --clusters=${CLUSTER} \ - ${VIOLET_EXTRA_ARGS[@]} - -# [!code highlight] -# Push **Knative Operator** package to destination cluster -violet push \ - ${KNATIVE_OPERATOR_PKG_NAME} \ - --platform-address=${PLATFORM_ADDRESS} \ - --platform-username=${PLATFORM_ADMIN_USER} \ - --platform-password=${PLATFORM_ADMIN_PASSWORD} \ - --clusters=${CLUSTER} \ - ${VIOLET_EXTRA_ARGS[@]} -``` - - - -1. `${PLATFORM_ADDRESS}` is your ACP platform address. -2. `${PLATFORM_ADMIN_USER}` is the username of the ACP platform admin. -3. `${PLATFORM_ADMIN_PASSWORD}` is the password of the ACP platform admin. -4. `${CLUSTER}` is the name of the cluster to install the Alauda AI components into. -5. `${AI_CLUSTER_OPERATOR_NAME}` is the path to the Alauda AI Cluster Operator package tarball. -6. `${KNATIVE_OPERATOR_PKG_NAME}` is the path to the Knative Operator package tarball. -7. `${REGISTRY_ADDRESS}` is the address of the external registry. -8. `${REGISTRY_USERNAME}` is the username of the external registry. -9. `${REGISTRY_PASSWORD}` is the password of the external registry. +## Uploading packages - - -After configuration, execute the script file using `bash ./uploading-ai-cluster-packages.sh` to upload both `Alauda AI` and `Knative Operator`. +Upload the **Alauda AI** package to the cluster where Alauda AI will run. Follow [Upload Packages](./package-upload.mdx) to prepare `violet`, configure platform or external-registry credentials, and upload the package. - +After uploading the package, install Alauda AI from **OperatorHub** as described below. Optional capabilities are enabled after installation through the `AmlCluster` component configuration. -## Installing Alauda AI Operator +## Installing Alauda AI @@ -163,414 +73,462 @@ Confirm that the **Alauda AI** tile shows one of the following states: -## Installing Alauda Build of KServe Operator +## Configuring Alauda AI Instance -For detailed installation steps, see [Install KServe](../deploy/components/kserve/install.mdx) in Alauda Build of KServe. +Once Alauda AI is installed, the operator automatically creates the cluster-scoped `default` `AmlCluster` instance. You do not need to create the `default` instance manually. Review and update the automatically created instance according to your environment. -## Enabling Knative Functionality \{#enabling-knative-functionality} + +### Deploy Alauda AI -Knative functionality is an optional capability that requires an additional operator and instance to be deployed. +In **Administrator** view: -:::warning -If you plan to use Knative functionality, you **MUST** install the Knative Operator and create the Knative Serving instance **BEFORE** configuring the Alauda AI instance to ensure the required CRDs are available in the cluster. -::: +1. Click **Marketplace / OperatorHub**. +2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install the Alauda AI Operator. +3. Select **Alauda AI**, then click it. +4. In the **Alauda AI** page, click **All Instances** from the tab. +5. Wait for the `default` `AmlCluster` instance to appear, then edit it. +6. Select **Deploy Flavor** from dropdown: + 1. `single-node` for non HA deployments. + 2. `ha-cluster` for HA cluster deployments (**Recommended** for production). +7. Input a valid domain for **Domain** field. + + :::info + This domain is used by ingress gateway for exposing model serving services. + Most likely, you will want to use a wildcard name, like *.example.com. -### 1. Installing the Knative Operator + You can specify the following certificate types by updating the **Domain Certificate Type** field: + - `Provided` + - `SelfSigned` + - `ACPDefaultIngress` - + By default, the configuration uses `SelfSigned` certificate type for securing ingress traffic to your cluster, the certificate is + stored in the `knative-serving-cert` secret that is specified in the **Domain Certificate Secret** field. + ::: -:::info -Starting from **Knative Operator**, the Knative networking layer switches to **Kourier**, so installing **Istio** is no longer required. -::: +### Component management states \{#component-management-states} -#### Procedure +Alauda AI 2.8 automatically installs and manages the components selected by the `AmlCluster` configuration. The following values show the default component settings. Merge the entries you need into the existing `default` `AmlCluster`; do not create a second instance. -In **Administrator** view: +```yaml +spec: + components: + authorino: + managementState: Managed + envoyAIGateway: + managementState: Unmanaged + envoyGateway: + managementState: SharedManaged + feast: + managementState: Unmanaged + knativeServing: + managementState: Unmanaged + providerType: Operator + llamaStack: + managementState: Unmanaged + lws: + managementState: Managed + mlflow: + managementState: Unmanaged + npuOperator: + managementState: Unmanaged + values: + clusterd: + enabled: false + devicePlugin: + enabled: true + driver: + enabled: true + recoveryPolicy: + autoRecover: false + upgradePolicy: + autoUpgrade: false + version: 25.5.0 + exporter: + enabled: true + mindioacp: + enabled: false + mindiotft: + enabled: false + nodeD: + enabled: false + ociRuntime: + enabled: true + rscontroller: + enabled: false + trainer: + enabled: false + postgres: + managementState: SharedManaged + redis: + managementState: SharedManaged + servingRuntimeOperator: + managementState: Unmanaged + trustyAI: + managementState: Unmanaged + workbench: + managementState: Unmanaged + values: + global: + istio: + enabled: false +``` -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install. -3. Search for and select **Knative Operator**, then click **Install**. +The `spec.components` field uses configuration keys that correspond to the following products and Operators: + +| Component key | Product or Operator | +|:--|:--| +| `authorino` | **Alauda Build of Authorino** | +| `envoyAIGateway` | **Alauda Build of Envoy AI Gateway** | +| `envoyGateway` | **Alauda Build of Envoy Gateway** (SharedManaged) | +| `feast` | **Alauda Build of Feast** | +| `knativeServing` | **Knative Serving** | +| `llamaStack` | **Alauda Build of Llama Stack** | +| `lws` | **Alauda Build of LeaderWorkerSet** | +| `mlflow` | **MLflow** | +| `npuOperator` | **Alauda Build of NPU Operator** | +| `postgres` | **PostgreSQL** (SharedManaged) | +| `redis` | **Alauda Cache Service for Redis OSS** (SharedManaged) | +| `servingRuntimeOperator` | **Alauda Build of Serving Runtime** | +| `trustyAI` | **Alauda Build of TrustyAI** | +| `workbench` | **Alauda AI Workbench** | + +The management states have the following meanings: + +| Value | Meaning | +|:--|:--| +| `Managed` | Alauda AI manages the component and its lifecycle. For normal installations, use `Managed` so Alauda AI installs and reconciles the component. | +| `Unmanaged` | Alauda AI does not install or reconcile the component. Use this only for temporary workarounds, compatibility requirements, or advanced deployments; manage the component separately when required. | +| `SharedManaged` | Alauda AI deploys the component for shared use, but does not lock the Operator version. | +| `Removed` | Alauda AI removes the component and its managed resources. This value cannot be used for components with the `SharedManaged` state; set those components to `Unmanaged` instead. | + + + +For common 2.8 deployments, set the following component switches to `Managed` when the corresponding capabilities are required. The `servingRuntimeOperator` setting depends on the accelerator and serving-runtime requirements of your environment. - **Install Knative Operator** window will pop up. +```yaml +spec: + components: + envoyAIGateway: + managementState: Managed + servingRuntimeOperator: + managementState: Managed # [!code callout] + workbench: + managementState: Managed +``` -4. Then in the **Install Knative Operator** window. -5. Leave **Channel** unchanged. -6. Check whether the **Version** matches the **Knative Operator** version you want to install. -7. Leave **Installation Location** unchanged. -8. Select **Manual** for **Upgrade Strategy**. -9. Click **Install**. + -#### Verification +1. `servingRuntimeOperator` controls **Alauda Build of Serving Runtime**, which contains vLLM and llm-d serving runtimes for Nvidia GPUs and is available only for x86. Enable it for Nvidia/x86 deployments that require these runtimes. If your environment uses Ascend or another non-Nvidia accelerator, do not enable or install this component; use the serving runtime appropriate for that hardware instead. -Confirm that the **Knative Operator** tile shows one of the following states: + -- `Installing`: installation is in progress; wait for this to change to `Installed`. -- `Installed`: installation is complete. +:::info +**Enable Knative functionality** - +Set `spec.components.knativeServing.managementState` to `SharedManaged` in the existing `default` `AmlCluster`: +```yaml +spec: + components: + knativeServing: + managementState: SharedManaged +``` +::: -### 2. Creating Knative Serving Instance +### Configure KServe -Once **Knative Operator** is installed, you need to create the `KnativeServing` instance manually. +**Alauda Build of KServe** is installed and managed by Alauda AI. The `kserve` component in the `default` `AmlCluster` instance defaults to `Managed`, so Alauda AI installs the KServe Operator and automatically creates the `KServe` instance. You do not need to upload the KServe Operator package, install the Operator from OperatorHub, or create a `KServe` custom resource manually. - +The KServe parameters are exposed through `spec.components.kserve.values`. Merge the settings you need into the existing `default` `AmlCluster`: -#### Procedure - -1. Create the `knative-serving` namespace. - - ```bash - kubectl create ns knative-serving - ``` - -2. In the **Administrator** view, navigate to **Operators** -> **Installed Operators**. -3. Select the **Knative Operator**. -4. Under **Provided APIs**, locate **KnativeServing** and click **Create Instance**. -5. Switch to **YAML view**. -6. Replace the content with the following YAML: -7. Click **Create**. - - - ```yaml - apiVersion: operator.knative.dev/v1beta1 - kind: KnativeServing - metadata: - name: knative-serving - namespace: knative-serving - spec: - # For ACP 4.0, use version 1.18.1 - # For ACP 4.1 and above, use version 1.19.6 - version: "1.19.6" # [!code callout] - config: - deployment: - registries-skipping-tag-resolving: kind.local,ko.local,dev.local,private-registry # [!code callout] - domain: - example.com: "" - features: - kubernetes.podspec-affinity: enabled - kubernetes.podspec-hostipc: enabled - kubernetes.podspec-hostnetwork: enabled - kubernetes.podspec-init-containers: enabled - kubernetes.podspec-nodeselector: enabled - kubernetes.podspec-persistent-volume-claim: enabled - kubernetes.podspec-persistent-volume-write: enabled - kubernetes.podspec-securitycontext: enabled - kubernetes.podspec-tolerations: enabled - kubernetes.podspec-volumes-emptydir: enabled - queueproxy.resource-defaults: enabled - network: - domain-template: '{{.Name}}.{{.Namespace}}.{{.Domain}}' - ingress-class: kourier.ingress.networking.knative.dev - ingress: - kourier: - enabled: true - ``` -:::warning -- For ACP 4.0, use version **1.18.1** -- For ACP 4.1 and above, use version **1.19.6** -::: +```yaml +spec: + components: + kserve: + managementState: Managed + values: + defaultDeploymentMode: RawDeployment + domain: example.com + ingressGateway: knative-serving/knative-ingress-gateway + kserve: + storage: + uidModelcar: 1000 + preset: + envoyAIGateway: + port: 1063 + service: ai-gateway-controller + envoyGateway: + createInstance: true + deployType: ControllerNamespace + instanceName: aieg + saNamespace: envoy-gateway-system + serviceAccount: envoy-gateway + gie: + enabled: true + kserveGateway: + enabled: true + name: kserve-ingress-gateway + namespace: kserve + port: 80 + service_type: NodePort +``` - +The main `kserve` component settings: + +| Field | Description | Default | +|-------|-------------|---------| +| `managementState` | Whether Alauda AI manages the KServe Operator and instance. Use `Managed` for normal installations. | `Managed` | +| `values.defaultDeploymentMode` | Deployment mode for inference services: `RawDeployment` (`Standard`) or `Knative` for serverless features like scale-to-zero. | `RawDeployment` | +| `values.domain` | Domain used by the ingress gateway to expose inference-service endpoints. Use a wildcard domain, e.g., `*.example.com`. | None | +| `values.ingressGateway` | Ingress gateway bound to KServe, in the form `/`. | `knative-serving/knative-ingress-gateway` | +| `values.kserve.storage.uidModelcar` | UID used for Modelcar workloads. Set to `0` when you plan to use `vLLM-ascend`; the default is `1000`. | `1000` | + +The gateway presets configure the AI gateway stack that fronts inference traffic: + +| Field | Description | Default | +|-------|-------------|---------| +| `preset.envoyGateway.createInstance` | Create an Envoy Gateway instance to manage inference traffic with bundled extensions. | `true` | +| `preset.envoyGateway.instanceName` | Name of the Envoy Gateway instance to create. | `aieg` | +| `preset.envoyGateway.saNamespace` | Namespace where the Envoy Gateway service account is located. | `envoy-gateway-system` | +| `preset.envoyGateway.serviceAccount` | Service account name used by Envoy Gateway. | `envoy-gateway` | +| `preset.envoyAIGateway.service` | Kubernetes service name for Envoy AI Gateway. | `ai-gateway-controller` | +| `preset.envoyAIGateway.port` | Port number used by Envoy AI Gateway. | `1063` | +| `preset.gie.enabled` | Enable the bundled Gateway API Inference Extension. Set to `false` if GIE is already installed separately in the cluster. | `true` | +| `preset.kserveGateway.enabled` | Deploy a KServe Gateway instance for InferenceService traffic. | `true` | +| `preset.kserveGateway.name` | Name of the KServe Gateway. | `kserve-ingress-gateway` | +| `preset.kserveGateway.namespace` | Namespace where the KServe Gateway is deployed. | `kserve` | +| `preset.kserveGateway.port` | Port number used by the KServe Gateway. | `80` | +| `preset.kserveGateway.service_type` | Service type for the KServe Gateway. | `NodePort` | + +Verify that the `AmlCluster` reconciles and the `KServe` instance is created by Alauda AI: -1. Specify the version of Knative Serving to be deployed. +```bash +kubectl get amlcluster default -2. `private-registry` is a placeholder for your private registry address. You can find this in the **Administrator** view, then click **Clusters**, select `your cluster`, and check the **Private Registry** value in the **Basic Info** section. +kubectl get kserve default-kserve -n kserve-operator +``` +The `AmlCluster` should report `Phase=Ready` and `Reason=Reconciled`, and the `KServe` instance is ready when the status shows `DEPLOYED: True`. +### Configure a custom OIDC provider (Optional) - +By default, Alauda AI uses ACP Dex as the OIDC provider. In this default +setup, no additional `spec.oidc` configuration is required in the +`AmlCluster` instance. - +If you want Alauda AI to use another OIDC provider, register an OAuth2/OIDC +client in that provider, allow the Alauda AI callback URL, and then update +`spec.oidc` in the `AmlCluster` YAML. The callback URL is: -## Configuring Alauda AI Instance +```text +https:///clusters//aml/oauth2/callback +``` -Once Alauda AI Operator (and optionally, Knative Operator) is installed, the operator automatically creates the `default` `AmlCluster` instance. You do not need to create the `default` instance manually. Review and update the automatically created instance according to your environment. +Alauda AI reads the OIDC client secret from a Kubernetes Secret in the +`kubeflow` namespace of the Alauda AI installation cluster. The default +Secret name is `aml-oidc-secret`, and the Secret key must be +`client-secret`. Update this Secret with the client secret from your OIDC +provider: - -### Procedure +```bash +kubectl create secret generic aml-oidc-secret \ + -n kubeflow \ + --from-literal=client-secret='' \ + --dry-run=client -o yaml | kubectl apply -f - +``` -In **Administrator** view: +Then configure `spec.oidc`: -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install the Alauda AI Operator. -3. Select **Alauda AI**, then click it. -4. In the **Alauda AI** page, click **All Instances** from the tab. -5. Wait for the `default` `AmlCluster` instance to appear, then edit it. -6. Select **Deploy Flavor** from dropdown: - 1. `single-node` for non HA deployments. - 2. `ha-cluster` for HA cluster deployments (**Recommended** for production). -7. Set **KServe Mode** to **Managed**. -8. Input a valid domain for **Domain** field. +```yaml +spec: + oidc: + # OIDC issuer URL. This must match the issuer value advertised by the + # provider, for example a Keycloak realm URL. + issuerURL: https:// + # OAuth2/OIDC client ID registered in the external provider. + clientID: + # Kubernetes Secret name in the kubeflow namespace. The Secret must + # contain a client-secret key. Default: aml-oidc-secret. + clientSecretName: aml-oidc-secret + # OAuth2 scopes requested during login. Keep this minimal to avoid + # oversized ID/access tokens and oversized login cookies. + # Default: openid profile email. + scope: openid profile email + # Whether to use the preferred_username claim as the email value. + # Default: true. + preferredUsernameAsEmail: true + main: + # OIDC authorization endpoint. If you use discovery, this maps from + # authorization_endpoint. + loginURL: https:// + # OAuth2 callback URL registered in the external provider. + redirectURL: https:///clusters//aml/oauth2/callback +``` - :::info - This domain is used by ingress gateway for exposing model serving services. - Most likely, you will want to use a wildcard name, like *.example.com. +If the provider exposes a standard OIDC discovery document at +`/.well-known/openid-configuration`, Alauda AI automatically +fills `redeemURL`, `jwksURL`, and `profileURL` from discovery when these +fields are not set. If discovery is unavailable, configure these fields +explicitly: - You can specify the following certificate types by updating the **Domain Certificate Type** field: +| Discovery field | `spec.oidc` field | +| --- | --- | +| `authorization_endpoint` | `loginURL` | +| `token_endpoint` | `redeemURL` | +| `userinfo_endpoint` | `profileURL` | +| `jwks_uri` | `jwksURL` | - - `Provided` - - `SelfSigned` - - `ACPDefaultIngress` +Use the mapped `loginURL` value for `main.loginURL`, or for +`secondary.loginURL` if you configure a secondary endpoint. - By default, the configuration uses `SelfSigned` certificate type for securing ingress traffic to your cluster, the certificate is - stored in the `knative-serving-cert` secret that is specified in the **Domain Certificate Secret** field. - ::: +```yaml +spec: + oidc: + redeemURL: https:// + jwksURL: https:// + profileURL: https:// +``` -9. **(Optional)** Configure a custom OIDC provider. - - By default, Alauda AI uses ACP Dex as the OIDC provider. In this default - setup, no additional `spec.oidc` configuration is required in the - `AmlCluster` instance. - - If you want Alauda AI to use another OIDC provider, register an OAuth2/OIDC - client in that provider, allow the Alauda AI callback URL, and then update - `spec.oidc` in the `AmlCluster` YAML. The callback URL is: - - ```text - https:///clusters//aml/oauth2/callback - ``` - - Alauda AI reads the OIDC client secret from a Kubernetes Secret in the - `kubeflow` namespace of the Alauda AI installation cluster. The default - Secret name is `aml-oidc-secret`, and the Secret key must be - `client-secret`. Update this Secret with the client secret from your OIDC - provider: - - ```bash - kubectl create secret generic aml-oidc-secret \ - -n kubeflow \ - --from-literal=client-secret='' \ - --dry-run=client -o yaml | kubectl apply -f - - ``` - - Then configure `spec.oidc`: - - ```yaml - spec: - oidc: - # OIDC issuer URL. This must match the issuer value advertised by the - # provider, for example a Keycloak realm URL. - issuerURL: https:// - # OAuth2/OIDC client ID registered in the external provider. - clientID: - # Kubernetes Secret name in the kubeflow namespace. The Secret must - # contain a client-secret key. Default: aml-oidc-secret. - clientSecretName: aml-oidc-secret - # OAuth2 scopes requested during login. Keep this minimal to avoid - # oversized ID/access tokens and oversized login cookies. - # Default: openid profile email. - scope: openid profile email - # Whether to use the preferred_username claim as the email value. - # Default: true. - preferredUsernameAsEmail: true - main: - # OIDC authorization endpoint. If you use discovery, this maps from - # authorization_endpoint. - loginURL: https:// - # OAuth2 callback URL registered in the external provider. - redirectURL: https:///clusters//aml/oauth2/callback - ``` - - If the provider exposes a standard OIDC discovery document at - `/.well-known/openid-configuration`, Alauda AI automatically - fills `redeemURL`, `jwksURL`, and `profileURL` from discovery when these - fields are not set. If discovery is unavailable, configure these fields - explicitly: - - | Discovery field | `spec.oidc` field | - | --- | --- | - | `authorization_endpoint` | `loginURL` | - | `token_endpoint` | `redeemURL` | - | `userinfo_endpoint` | `profileURL` | - | `jwks_uri` | `jwksURL` | - - Use the mapped `loginURL` value for `main.loginURL`, or for - `secondary.loginURL` if you configure a secondary endpoint. - - ```yaml - spec: - oidc: - redeemURL: https:// - jwksURL: https:// - profileURL: https:// - ``` - - Example Keycloak configuration: - - - In the target realm, create an **OpenID Connect** client. - - Set **Client ID** to the value used in `spec.oidc.clientID`, for example - `aml`. - - Turn on **Client authentication**. - - Under **Authentication flow**, select **Standard flow**. - - Turn on **Require PKCE** and set **PKCE Method** to `S256`. - - Set **Valid redirect URIs** to - `https:///clusters//aml/*`. - - Copy the client secret from the Keycloak client **Credentials** tab and - update the `aml-oidc-secret` Secret shown above. - - In the client **Client scopes** settings, set `basic`, `email`, and - `profile` to **Default**, and set other scopes to **Optional** unless your - environment explicitly needs them. Avoid adding large claim mappers such - as groups, realm roles, client roles, address, phone, offline access, and - other application-specific claims unless required. Large tokens can make - the oauth2-proxy cookie exceed browser or ingress header size limits and - cause login loops or HTTP 431/400 errors. - -10. **(Optional)** Configure the languages available in the Alauda AI console. - - Alauda AI reads the available console languages from - `spec.i18n.languages` in the `AmlCluster` instance. The default value is - `en`. - - Supported language codes are: - - | Code | Language | - | --- | --- | - | `zh` | Chinese (Simplified) | - | `en` | English | - | `fr` | French | - | `ru` | Russian | - | `ja` | Japanese | - | `ko` | Korean | - | `de` | German | - | `es` | Spanish | - | `it` | Italian | - | `pt-BR` | Portuguese (Brazil) | - | `zh-TW` | Chinese (Traditional) | - - `en` and `zh` are built-in languages. Other languages only add the - language as a selectable entry. To switch to those languages successfully, - you must also provide the corresponding translation file. - - You can use the built-in English and Chinese translation files as - references when preparing translation files for other languages: - - ```text - /clusters//aml/console-aml/assets/i18n/en/lich-single.json - /clusters//aml/console-aml/assets/i18n/zh/lich-single.json - ``` - - Generate the target-language translation from these files, or contact - Alauda to obtain the latest translation content for the required language. - - For example, to make English, Chinese, and French selectable in the - console, update the `AmlCluster` YAML: - - ```yaml - spec: - i18n: - languages: - - en - - zh - - fr - ``` - - Then create a translation file `ConfigMap` for French in the - `cpaas-system` namespace: - - ```yaml - apiVersion: v1 - kind: ConfigMap - metadata: - labels: - image-load-config: "true" - name: aml-i18n-fr - namespace: cpaas-system - data: - overrides: | - [{ - "dest": "/aml-i18n/fr/lich-single.json", - "configMap": { - "name": "aml-i18n-fr", - "key": "lich-single.json" - } - }] - lich-single.json: | - { - "nav_model_repo": "Referentiel de modeles", - "nav_infer_svc": "Service d'inference", - "workbench": "Workbench", - "tool": "Outils" - } - ``` - - Use the same pattern for other non-built-in languages by replacing `fr` - with the target language code and providing the translated - `lich-single.json` content. - -11. **(Optional)** If you want to enable Knative functionality, update the `AmlCluster` YAML to reference the `KnativeServing` instance: +Example Keycloak configuration: + +- In the target realm, create an **OpenID Connect** client. +- Set **Client ID** to the value used in `spec.oidc.clientID`, for example + `aml`. +- Turn on **Client authentication**. +- Under **Authentication flow**, select **Standard flow**. +- Turn on **Require PKCE** and set **PKCE Method** to `S256`. +- Set **Valid redirect URIs** to + `https:///clusters//aml/*`. +- Copy the client secret from the Keycloak client **Credentials** tab and + update the `aml-oidc-secret` Secret shown above. +- In the client **Client scopes** settings, set `basic`, `email`, and + `profile` to **Default**, and set other scopes to **Optional** unless your + environment explicitly needs them. Avoid adding large claim mappers such + as groups, realm roles, client roles, address, phone, offline access, and + other application-specific claims unless required. Large tokens can make + the oauth2-proxy cookie exceed browser or ingress header size limits and + cause login loops or HTTP 431/400 errors. + +### Configure the console languages (Optional) + +Alauda AI reads the available console languages from +`spec.i18n.languages` in the `AmlCluster` instance. The default value is +`en`. + +Supported language codes are: + +| Code | Language | +| --- | --- | +| `zh` | Chinese (Simplified) | +| `en` | English | +| `fr` | French | +| `ru` | Russian | +| `ja` | Japanese | +| `ko` | Korean | +| `de` | German | +| `es` | Spanish | +| `it` | Italian | +| `pt-BR` | Portuguese (Brazil) | +| `zh-TW` | Chinese (Traditional) | + +`en` and `zh` are built-in languages. Other languages only add the +language as a selectable entry. To switch to those languages successfully, +you must also provide the corresponding translation file. + +You can use the built-in English and Chinese translation files as +references when preparing translation files for other languages: + +```text +/clusters//aml/console-aml/assets/i18n/en/lich-single.json +/clusters//aml/console-aml/assets/i18n/zh/lich-single.json +``` - :::info - Configure this only if you installed the Knative Operator and created the - `KnativeServing` instance in the previous steps. If you are not using - Knative functionality, leave this configuration unset. - ::: +Generate the target-language translation from these files, or contact +Alauda to obtain the latest translation content for the required language. - ```yaml - spec: - components: - knativeServing: - externalCRRef: - apiVersion: operator.knative.dev/v1beta1 - kind: KnativeServing - name: knative-serving - namespace: knative-serving - managementState: Unmanaged - ``` +For example, to make English, Chinese, and French selectable in the +console, update the `AmlCluster` YAML: -12. Under **Model Catalog** section, configure the following parameters: +```yaml +spec: + i18n: + languages: + - en + - zh + - fr +``` - - **Model OCI Registry Address**: Registry address hosting model OCI artifacts for Model Catalog. This field has no default value and must be configured for your environment. +Then create a translation file `ConfigMap` for French in the +`cpaas-system` namespace: - This registry stores the model OCI images used by Model Catalog. Use Harbor or another production-mode OCI registry with HTTPS access enabled. Model Catalog does not support configuring `imagePullSecret` for pulling model OCI images, so the Harbor project or repository used for Model Catalog must allow anonymous pull access from inference cluster nodes. In Harbor, set the project that stores Model Catalog images to **Public**. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + labels: + image-load-config: "true" + name: aml-i18n-fr + namespace: cpaas-system +data: + overrides: | + [{ + "dest": "/aml-i18n/fr/lich-single.json", + "configMap": { + "name": "aml-i18n-fr", + "key": "lich-single.json" + } + }] + lich-single.json: | + { + "nav_model_repo": "Referentiel de modeles", + "nav_infer_svc": "Service d'inference", + "workbench": "Workbench", + "tool": "Outils" + } +``` - If you cannot deploy a registry with HTTPS in the target environment, you can use an HTTP registry as a fallback. Configure the container runtime on every node in the inference cluster before deploying models. For containerd, add an insecure registry mirror for the registry address, for example by creating `/etc/containerd/certs.d//hosts.toml`: +Use the same pattern for other non-built-in languages by replacing `fr` +with the target language code and providing the translated +`lich-single.json` content. - ```toml - server = "http://" +### Configure Model Catalog - [host."http://"] - capabilities = ["pull", "resolve"] - ``` +- **Model OCI Registry Address**: Registry address hosting model OCI artifacts for Model Catalog. This field has no default value and must be configured for your environment. - Then restart containerd or apply the equivalent node-runtime configuration through your cluster management system. This configuration must exist on the nodes where inference service pods are scheduled; otherwise the pod image pull will fail even if Model Catalog can list the model. The exact containerd configuration path can vary by Kubernetes distribution; after applying the configuration, verify that the node can pull a Model Catalog image, for example with `crictl pull /:`. + This registry stores the model OCI images used by Model Catalog. Use Harbor or another production-mode OCI registry with HTTPS access enabled. Model Catalog does not support configuring `imagePullSecret` for pulling model OCI images, so the Harbor project or repository used for Model Catalog must allow anonymous pull access from inference cluster nodes. In Harbor, set the project that stores Model Catalog images to **Public**. - - **Source of PVC**: Choose whether to reuse an existing PVC or create a new one. Use `CreateNew` to let the installation create the PVC. - - **StorageClass Name**: StorageClass used when creating a new PVC. + If you cannot deploy a registry with HTTPS in the target environment, you can use an HTTP registry as a fallback. Configure the container runtime on every node in the inference cluster before deploying models. For containerd, add an insecure registry mirror for the registry address, for example by creating `/etc/containerd/certs.d//hosts.toml`: -13. If you plan to use `llm-d` or `vLLM-ascend`, set KServe Modelcar UID to `0`. The default value is `1000`. + ```toml + server = "http://" - ```yaml - spec: - components: - kserve: - values: - kserve: - storage: - uidModelcar: 0 - ``` + [host."http://"] + capabilities = ["pull", "resolve"] + ``` - This setting is cluster-level and affects all Modelcar workloads in the Alauda AI installation cluster. + Then restart containerd or apply the equivalent node-runtime configuration through your cluster management system. This configuration must exist on the nodes where inference service pods are scheduled; otherwise the pod image pull will fail even if Model Catalog can list the model. The exact containerd configuration path can vary by Kubernetes distribution; after applying the configuration, verify that the node can pull a Model Catalog image, for example with `crictl pull /:`. -14. Review the configuration and save the `default` `AmlCluster` instance. +- **Source of PVC**: Choose whether to reuse an existing PVC or create a new one. Use `CreateNew` to let the installation create the PVC. +- **StorageClass Name**: StorageClass used when creating a new PVC. ### Verification -Check the status field from the `AmlCluster` resource named `default`: +Review the configuration and save the `default` `AmlCluster` instance, then check its status: ```bash kubectl get amlcluster default ``` -It should return `Ready`: +The resource should be `Ready`: ``` -NAME READY REASON -default True Succeeded +NAME PHASE READY REASON +default Ready True Reconciled ``` + ## Importing Built-in Model Images for Catalog \{#importing-built-in-model-images-for-catalog} The **Catalog** feature in Alauda AI ships with a set of built-in model OCI images that users can deploy as inference services from the Web Console. These images **must be imported into the OCI registry configured by Model Catalog before the Catalog can serve them**. Without this step, the installation completes successfully, but deploying a built-in model from the Catalog will later fail with `ImagePullBackOff`. @@ -734,3 +692,19 @@ spec: :::note The specific path should be consistent with the collection configuration of `Alauda Container Platform Log Collector`. ::: + +### 2. Set KServe Modelcar UID for vLLM-ascend + +If you plan to use `vLLM-ascend`, set KServe Modelcar UID to `0` (the default is `1000`) in the `default` `AmlCluster`: + +```yaml +spec: + components: + kserve: + values: + kserve: + storage: + uidModelcar: 0 +``` + +This setting is cluster-level and affects all Modelcar workloads in the Alauda AI installation cluster. diff --git a/docs/en/installation/ai-essentials.mdx b/docs/en/installation/ai-essentials.mdx index 38a2b99a..6554a92c 100644 --- a/docs/en/installation/ai-essentials.mdx +++ b/docs/en/installation/ai-essentials.mdx @@ -1,5 +1,5 @@ --- -weight: 30 +weight: 50 --- # Configure Alauda AI Entry and Branding diff --git a/docs/en/installation/package-upload.mdx b/docs/en/installation/package-upload.mdx new file mode 100644 index 00000000..4bc9db4d --- /dev/null +++ b/docs/en/installation/package-upload.mdx @@ -0,0 +1,129 @@ +--- +weight: 20 +--- + +# Upload Packages + +Alauda AI components are delivered as platform packages. Before installing an operator or Cluster Plugin, download the package from the Customer Portal and upload it to the target cluster with the `violet` packaging tool. + +This page describes the common package workflow. For component-specific prerequisites, installation settings, and custom resources, continue to the installation page for the component after uploading its package. + +## Downloading packages + +Download the package for the required component and version from the **Marketplace** on the Customer Portal: + +- Download the package that matches the Alauda AI release and the target architecture. +- Use an **OLM operator bundle** for components installed from **OperatorHub**. +- Use a **Cluster Plugin** package for components installed from **Cluster Plugins**. +- Keep the downloaded package available on the machine where you will run `violet`. + +The exact package filename and version are supplied with the corresponding product release. Do not substitute a package from another Alauda AI release. + +## Downloading the `violet` tool + +Download `violet` if it is not already available on the machine used to upload packages. + +In the Web Console, switch to the **Administrator** view: + +1. Click **Marketplace** > **Upload Packages**. +2. Click **Download Packaging and Listing Tool**. +3. Locate the correct operating system and CPU architecture under **Execution Environment**. +4. Click **Download**. +5. Make the downloaded file executable: + + ```bash + chmod +x + ``` + +## Uploading a package + +Use the following command to upload an operator or Cluster Plugin package to the target cluster: + +```bash +violet push \ + \ + --platform-address= \ + --platform-username= \ + --platform-password= \ + --clusters= +``` + +Replace the placeholders with the platform address, platform administrator credentials, target cluster name, and local package path. The package is uploaded to the platform repository and becomes available for installation on the selected cluster. + +:::info +For a platform-built registry, no additional registry arguments are required. `violet` relocates the images included in the package into the platform registry so the operator or Cluster Plugin can be installed without pulling those images from their original registries. +::: + +### Uploading to an external registry + +If the target cluster uses an external private or public registry instead of the platform-built registry, provide the destination registry and its credentials: + +```bash +violet push \ + \ + --platform-address= \ + --platform-username= \ + --platform-password= \ + --clusters= \ + --dst-repo= \ + --username= \ + --password= +``` + +Use a registry address that the target cluster can reach and pull from. Confirm that all images required by the package are available in the destination registry before installing the component. + +### Uploading multiple packages + +You can upload several packages with a shell script. Set the platform variables and add each downloaded package to the `PACKAGES` array: + +```bash title=uploading-packages.sh +#!/usr/bin/env bash +set -euo pipefail + +PLATFORM_ADDRESS=https:// +PLATFORM_ADMIN_USER= +PLATFORM_ADMIN_PASSWORD= +CLUSTER= + +PACKAGES=( + + +) + +VIOLET_EXTRA_ARGS=() +# Uncomment and set these values when the target cluster uses an external registry. +# VIOLET_EXTRA_ARGS+=( +# --dst-repo= +# --username= +# --password= +# ) + +for PACKAGE in "${PACKAGES[@]}"; do + violet push \ + "$PACKAGE" \ + --platform-address="${PLATFORM_ADDRESS}" \ + --platform-username="${PLATFORM_ADMIN_USER}" \ + --platform-password="${PLATFORM_ADMIN_PASSWORD}" \ + --clusters="${CLUSTER}" \ + "${VIOLET_EXTRA_ARGS[@]}" +done +``` + +Run the script with: + +```bash +bash ./uploading-packages.sh +``` + +:::warning +Protect platform and registry passwords. Do not commit a script containing real credentials to source control or share it outside the authorized installation team. +::: + +## Install the uploaded component + +After the upload finishes, open the target cluster in the Web Console: + +- For an OLM operator, go to **Administrator** > **Marketplace** > **OperatorHub**, select the uploaded component, and install it. Then follow the component documentation to create its custom resource, if required. +- For a Cluster Plugin, go to **Administrator** > **Marketplace** > **Cluster Plugins** and install the uploaded component according to the platform documentation. + +If the component or version does not appear, verify that the package upload completed successfully, that the selected cluster is correct, and that the package matches the target platform architecture and release. diff --git a/docs/en/installation/pre-configuration.mdx b/docs/en/installation/pre-configuration.mdx index ab157577..55aa947d 100644 --- a/docs/en/installation/pre-configuration.mdx +++ b/docs/en/installation/pre-configuration.mdx @@ -4,100 +4,6 @@ weight: 10 # Pre-installation Configuration -## **Preparing the GitLab Service (Optional)** - -GitLab-backed model storage is deprecated. You do not need to prepare GitLab for a new Alauda AI installation unless you still plan to use the legacy GitLab-backed **Model Management** workflow. - -:::warning -GitLab-backed model storage is deprecated. It remains available for compatibility only and is planned for removal in a future Alauda AI release. For built-in models and new model delivery workflows, use **Model Catalog** with OCI model artifacts instead. -::: - -### **Deployment Options** - -#### **1. GitLab service requirements** - -Regardless of deployment method, all GitLab instances must satisfy: - -- **Version**: Must be **v15 or later**. -- **Protocol**: Must use **HTTPS**. For setup instructions, refer to . -- **Git LFS**: Must be **enabled**. For setup instructions, refer to . -- **Hosting**: Must be **self-hosted** (public cloud-hosted GitLab services are **not supported**). -- **Access Tokens**: **Disable expiration dates** for access tokens. - -#### **2. Use the platform-provided plugin** - -Deploy a new GitLab service using the **'Alauda Build of GitLab'** plugin. -For instructions, refer to: . - -#### **3. Use your own GitLab service** - -Alternatively, you can use a **self-managed GitLab instance**, but it **must meet** the **GitLab service requirements**. - -### **GitLab Configuration** - -If you still use the legacy GitLab-backed workflow, perform these GitLab configuration steps after service acquisition. - -#### **1. Disable expiration dates for access tokens** - -If GitLab is running `v17.0` or greater, we need to **disable** expiration dates for access tokens. - -:::warning - -If expiration date for access token keeps enabled, we have to refresh admin token manually at least yearly, or Alauda AI may stop functionally. - -::: - -To **disable** expiration dates for new access tokens: - -1. On the left sidebar, at the bottom, select **Admin**. -2. Select **Settings > General**. -3. Expand **Account and limit**. -4. **Uncheck** the **Personal / Project / Group access token expiration** checkbox. -5. Select **Save changes**. - -#### **2. Generate new token** - -To **generate** impersonation token for admin: - -1. On the left sidebar, at the bottom, select **Admin**. -2. Select **Overview > Users**. -3. Select the admin user (Administrator for example). -4. On the top navigation bar, select **Impersonation Tokens**. -5. Select **Add new token**. -6. In the popup form: - 1. Input a **Token name** for Alauda AI (`aml-root`, for example). - 2. Remove **Expiration Date** (select "x" icon to remove expiration date). - 3. Check **ALL** scopes (especially **api** scope) for **Select scopes**. -7. Select **Create impersonation token**. -8. **Save** the newly generated token under **Your new impersonation token**, we need to use it later. - -:::warning - -Make sure you save the newly generate token - you won't be able to access it again. - -::: - -#### **3. Create kubernetes secret for admin token** - -Then we create secret for gitlab admin token named **aml-gitlab-admin-token** under **cpaas-system** namespace: - -```bash -# [!code highlight] -# Please replace ${TOKEN} with real token saved previously -# [!code callout:3] -kubectl create secret generic aml-gitlab-admin-token \ - --from-literal="password=${TOKEN}" \ - -n cpaas-system -``` - - - -1. Create a gitlab admin token secret named **aml-gitlab-admin-token** -2. The token is saved under **password** key, please replace `${TOKEN}` with the real token saved previously. -3. The secret is created under **cpaas-system** namespace. - - - ## **Preparing the Harbor Service** If you plan to use the **Model Catalog** feature, prepare an **Alauda Build of Harbor** service before installing Alauda AI. The registry must meet the following requirements: @@ -108,122 +14,3 @@ If you plan to use the **Model Catalog** feature, prepare an **Alauda Build of H If you cannot deploy an HTTPS registry in the target environment, you can use an HTTP registry as a fallback, but you must configure the inference cluster container runtime before deploying models. The detailed containerd configuration is covered when setting **Model OCI Registry Address** during Alauda AI instance creation. For the installation procedure, see . - -## **Frequently Asked Questions (FAQ)** - -### **1. How to optimize GitLab 18.5 and later configuration for large LFS objects?** - -**Problem:** -When pushing large LFS objects to GitLab 18.5 and later, you may encounter an HTTP 413 error. For AI model management, you often need to upload large model files via LFS, which exceed the default `proxy-body-size` limit (typically 512M) in the Nginx ingress controller. - -The following is authentic diagnostic output from the Git LFS client; the `%!!(string=...)` fragments are raw Go-formatting artifacts and can be ignored—focus on the `HTTP 413` response as the actionable error. - -```bash -# [!code highlight] -❯ git push origin main -Locking support detected on remote "origin". Consider enabling it with: - $ git config lfs.https://gitlab-18-5-aml.alaudatech.net/mlops-demo-ai-test/amlmodels/qa.git/info/lfs.locksverify true -LFS: Client error &{%!!(string=https) %!!(string=) %!!(*url.Userinfo=) %!!(string=gitlab-18-5-aml.alaudatech.net) %!!(string=/mlops-demo-ai-test/amlmodels/qa.git/gitlab-lfs/objects/fdf756fa7fcbe7404d5c60e26bff1a0c8b8aa1f72ced49e7dd0210fe288fb7fe/988097824) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=) %!!(string=) %!!(string=)}s(MISSING) from HTTP 413 -Uploading LFS objects: 0% (0/1), 0 B | 0 B/s, done. -error: failed to push some refs to 'https://gitlab-18-5-aml.alaudatech.net/mlops-demo-ai-test/amlmodels/qa.git' -``` - -**Solution:** -To handle large file uploads and improve overall performance, configure the GitLab webservice ingress body size and read timeout through `spec.helmValues.gitlab.webservice.ingress`, and configure the remaining Nginx Ingress behavior through annotations on your GitLab service. - -#### **Webservice Ingress Settings** - -Configure the following fields under `spec.helmValues.gitlab.webservice.ingress`: - -| Field | Recommended Value | Description | -|-------|-------------------|-------------| -| `proxyBodySize` | `"0"` | Disables the client request body size limit, allowing large AI model uploads through Git LFS. | -| `proxyReadTimeout` | `"3600"` | Increases the timeout (in seconds) for reading a response from the proxied server to 1 hour, preventing timeouts during long-running operations. | - -#### **Ingress Annotation Parameters** - -Below is a list of recommended Ingress annotations and their functionality: - -| Parameter | Recommended Value | Description | -|-----------|-------------------|-------------| -| `nginx.ingress.kubernetes.io/proxy-buffering` | `"off"`| Disables proxy buffering, improving response times for large requests and allowing data to stream directly to the client/server. | -| `nginx.ingress.kubernetes.io/proxy-request-buffering`| `"off"` | Disables buffering of the client request body, passing data directly to the upstream server to reduce memory usage on the ingress controller. | -| `nginx.ingress.kubernetes.io/proxy-send-timeout` | `"3600"` | Increases the timeout (in seconds) for transmitting a request to the proxied server to 1 hour, supporting prolonged uploads. | - -#### **Configuration Steps** - -You can apply these optimizations by updating the `GitLabOfficial` Custom Resource (CR). - -**1. Apply via `kubectl patch` command** - -Use the following command to directly update the GitLab webservice ingress settings and ingress annotations in your GitLabOfficial CR: - -```bash -# [!code highlight] -# Update GitLabOfficial CR with optimized ingress settings -# [!code callout:1,2] -kubectl patch gitlabofficial your-instance-name -n your-instance-namespace --type=merge -p '{ - "spec": { - "helmValues": { - "gitlab": { - "webservice": { - "ingress": { - "proxyBodySize": "0", - "proxyReadTimeout": "3600" - } - } - }, - "global": { - "ingress": { - "annotations": { - "nginx.ingress.kubernetes.io/proxy-buffering": "off", - "nginx.ingress.kubernetes.io/proxy-request-buffering": "off", - "nginx.ingress.kubernetes.io/proxy-send-timeout": "3600" - } - } - } - } - } -}' -``` - - - -1. Replace `your-instance-name` with the name of your GitLabOfficial instance (e.g., `gitlab-aml`). -2. Replace `your-instance-namespace` with the namespace where your GitLabOfficial instance is deployed (e.g., `gitlab-system-aml`). - - - -**2. YAML Hierarchy Reference** - -For reference, the hierarchical structure within the `GitLabOfficial` CR `spec` is as follows: - -```yaml -# [!code highlight] -apiVersion: operator.alaudadevops.io/v1alpha1 -kind: GitLabOfficial -metadata: - name: gitlab-aml - namespace: gitlab-system-aml -spec: - # ... other specs ... - helmValues: - gitlab: - webservice: - ingress: - proxyBodySize: "0" - proxyReadTimeout: "3600" - global: - ingress: - annotations: - nginx.ingress.kubernetes.io/proxy-buffering: "off" - nginx.ingress.kubernetes.io/proxy-request-buffering: "off" - nginx.ingress.kubernetes.io/proxy-send-timeout: "3600" -``` - - - -1. These optimizations ensure GitLab 18.5 can seamlessly handle large AI model uploads via Git LFS and improve overall data transfer stability. -2. We highly recommend applying these configurations during the initial GitLab deployment to prevent post-deployment operational issues. - - diff --git a/docs/en/installation/tools.mdx b/docs/en/installation/tools.mdx index 43537161..0286112c 100644 --- a/docs/en/installation/tools.mdx +++ b/docs/en/installation/tools.mdx @@ -1,5 +1,5 @@ --- -weight: 50 +weight: 60 --- # Tools Menu Configuration diff --git a/docs/en/installation/workbench.mdx b/docs/en/installation/workbench.mdx index e4dd56fa..8e2c9d43 100644 --- a/docs/en/installation/workbench.mdx +++ b/docs/en/installation/workbench.mdx @@ -1,97 +1,65 @@ --- -weight: 20 +weight: 40 --- # Install Workbench -Starting with v0.2.0, Workbench is delivered as an **OLM Helm operator**. Install the operator from **OperatorHub**, then create a `Workbench` custom resource that deploys and manages `workspace-controller` and the built-in JupyterLab and code-server `WorkspaceKind` resources. +In Alauda AI 2.8.x, Workbench is normally installed and managed by Alauda AI. Enable the `workbench` component in the cluster-scoped `default` `AmlCluster`; you do not need to upload the Workbench Operator package, install the Operator separately, or create a `Workbench` custom resource manually. ## Prerequisites -- Alauda AI is installed on the target cluster. -- Operator Lifecycle Manager (OLM) is available through the platform. -- cert-manager is installed for the workspace-controller webhook certificate. -- The separate `aml-skipper` component is installed for Workbench routing. -- **Optional:** ASM (Istio) is required only for the Elyra Kubeflow Pipelines run-URL redirect. Istio integration is disabled by default, so Workbench can install without Istio CRDs. +- Alauda AI is installed on the target cluster. See [Install Alauda AI](./ai-cluster.mdx). +- The target cluster has sufficient resources for Workbench workloads. +- **Optional:** Alauda Service Mesh (Istio) is required only for the Elyra Kubeflow Pipelines run-URL redirect. Istio integration is disabled by default. -## Upload the operator +## Enable Workbench -Download the **Alauda AI Workbench Operator** bundle from the Customer Portal / Marketplace. If it is not already available in OperatorHub, publish the downloaded bundle to the platform repository: +In **Administrator** view: -```bash -violet push \ - --platform-address= \ - --platform-username= \ - --platform-password= \ - -``` - -## Install the Workbench operator - -### Procedure - -1. Go to the **Administrator** page. -2. Click **Marketplace** > **OperatorHub**. -3. From the **Cluster** dropdown, select the business cluster where Workbench will run. -4. Find **Alauda AI Workbench Operator** and click **Install**. -5. Keep the `stable` channel and verify that the version is `v0.2.0`. -6. Leave the installation location at its default and choose an upgrade strategy. -7. Click **Install** and wait for the ClusterServiceVersion (CSV) status to become `Succeeded`. +1. Open **Marketplace** > **OperatorHub**. +2. Select the target cluster and open **Alauda AI**. +3. Open the **All Instances** tab and edit the `default` `AmlCluster` instance. +4. Set `spec.components.workbench.managementState` to `Managed`. +5. Save the `AmlCluster` instance and wait for Workbench to become ready. -## Create a Workbench instance - -The operator does not deploy `workspace-controller` until you create a `Workbench` custom resource. - -1. Open the installed **Alauda AI Workbench Operator**. -2. Select the **All instances** tab and click **Create**. -3. Keep **Enable Istio integration** disabled unless the cluster runs ASM and you need Elyra-generated KFP run URLs to redirect through the Kubeflow Istio ingress gateway. -4. Click **Create**. - -You can create the same resource with `kubectl`: +You can apply the component setting in the `AmlCluster` YAML: ```yaml -apiVersion: workbench.alauda.io/v1alpha1 -kind: Workbench -metadata: - name: workbench spec: - global: - istio: - enabled: false + components: + workbench: + managementState: Managed + values: + global: + istio: + enabled: false ``` -Save the manifest as `workbench.yaml`, then apply it: +Set `spec.components.workbench.values.global.istio.enabled` to `true` only when the target cluster runs Alauda Service Mesh and you need Elyra-generated KFP run URLs to redirect through the Kubeflow Istio ingress gateway. + +:::note +The default `workbench` state is `Unmanaged`, so Workbench is not installed until you explicitly set the component to `Managed`. For normal deployments, use `Managed` and let Alauda AI install and reconcile Workbench. +::: + +## Verification + +Check the `AmlCluster` status: ```bash -kubectl apply -f workbench.yaml +kubectl get amlcluster default ``` -The operator reads the cluster name, ingress class, and image registry from the `kube-public/global-info` `ConfigMap`. You normally do not need to set those values in the `Workbench` resource. - -## Verification +The resource should report `Phase=Ready` and `Reason=Reconciled`. You can also verify the Workbench resources created by Alauda AI: ```bash -# The Workbench custom resource exists. +# The Workbench instance is ready. kubectl get workbench -A -# The workspace-controller operand is available. +# The workspace controller is available. kubectl -n kubeflow get deploy workspace-controller-controller-manager # Built-in IDE templates are registered. kubectl get workspacekind ``` -After the deployment is ready, confirm that **Workbench** appears in Alauda AI and that you can open the Workbench list. - -## Feature Gate Configuration - -### Procedure - -#### Accessing the Feature Gate Page - -Access the feature gates at `{platform-access-address}/console-platform/feature-gate`. For example, if your platform access address is `https://demo.example.com/console-portal/`, the feature gate page would be `https://demo.example.com/console-platform/feature-gate`. - -#### Enabling Workbench Features - -1. Find the gate list with the display name of your target which is `ai-workbench`. -2. Turn On the switch button of it. +After the deployment is ready, open **Alauda AI** and confirm that **Workbench** appears and that you can open the Workbench list. \ No newline at end of file diff --git a/docs/en/overview/release_notes.mdx b/docs/en/overview/release_notes.mdx index 4365bf99..464fedd9 100644 --- a/docs/en/overview/release_notes.mdx +++ b/docs/en/overview/release_notes.mdx @@ -4,32 +4,92 @@ weight: 40 # Release Notes -## AI 2.7.0 +## Alauda AI 2.8.0 -### **New and Optimized Features** +Alauda AI 2.8.x summarizes the major platform, model-serving, training, and MLOps changes introduced after Alauda AI 2.3.x. -#### **Model as a Service (MaaS)** +### **Upgrade Support** -This release adds **Model as a Service (MaaS)**, a managed model-serving gateway based on Alauda Build of Envoy Gateway and Alauda Build of Envoy AI Gateway. MaaS uses Authorino for authentication and lets administrators publish `LLMInferenceService` inference services and external model services through subscriptions. +Alauda AI 2.8.x supports upgrading an existing Alauda AI 2.3.x environment. Review the upgrade documentation and the component-specific migration requirements before starting the upgrade. -Administrators can bind multiple models to each subscription, configure token limits for different time ranges, assign subscriptions to specified users, and view subscription traffic dashboards and usage reports. Users can access **My Subscriptions** to create, revoke, and delete API keys and view their own usage. +### **New and Optimized Features Since Alauda AI 2.3.x** +#### **Platform Architecture and Administration** -### **Breaking Changes** +- **Single-cluster application architecture** simplifies the Alauda AI 2.6.x and later deployment model. The former **Alauda AI Essentials** (`aml-global`) component is removed, and the Alauda AI product entry is configured through the platform's integrated product configuration. +- **Namespace management and permissions** let administrators create or import namespaces and assign namespace-scoped owner, editor, and viewer permissions. The former platform roles `aml-namespace-editor`, `owner`, and `viewer` are deprecated. +- **Authentication and access** are integrated with `oauth2-proxy`, with support for an independent OIDC provider and access entry. +- **Monitoring** is integrated with Prometheus and includes Perses-based dashboards. +- **Alauda AI Platform Control Plane** provides centralized management of supported AI components through the Alauda AI Operator. + +#### **Model Serving and Inference** + +- **Image Based Model Catalog** provides a centralized catalog for deploying ready-to-use model images and supports configurable persistent storage. +- **vLLM-ascend** enables model serving on Ascend NPUs through a custom inference runtime. +- **vLLM Expert Parallel** supports serving compatible Mixture-of-Experts models with expert parallelism. +- **vLLM speculative decoding** supports N-gram and EAGLE-3 configuration paths for reducing inference latency. +- **Alauda Build of InferNex Bridge** provides elastic orchestration capabilities for inference workloads, including `InferNexService` and multi-node inference scenarios. +- **Model as a Service (MaaS)** provides a managed model-serving gateway based on Alauda Build of Envoy Gateway and Alauda Build of Envoy AI Gateway. Administrators can publish `LLMInferenceService` and external model services through subscriptions, configure token limits, assign subscriptions to users, and view traffic and usage reports. + +#### **Training and Distributed Workloads** + +- **Kubeflow components** are upgraded to the Kubeflow 1.11 release line. Kubeflow Trainer v2 supports distributed training workflows on GPU and Ascend NPU environments. +- **JobSet Operator** supports coordinated groups of Kubernetes Jobs for distributed training, batch processing, and HPC workloads. +- **TrainingHub enhancements** add QLoRA and CPT fine-tuning workflows for single-GPU, multi-GPU, multi-node, and Ascend NPU scenarios. +- **Kueue enhancements** add quota management, fair sharing, gang scheduling, preemption, cohorts, and preemptible TrainJobs with checkpoint and resume workflows. +- **Dynamic Resource Allocation (DRA)** enables GPU slicing and shared GPU resource allocation for training workloads. +- **ARM support** is expanded for Kubeflow Pipelines and training runtimes. + +#### **MLOps and Developer Experience** + +- **MLflow enhancements** add AI-agent tracing, external PostgreSQL metadata storage, and object-storage configuration, while retaining platform SSO and Kubernetes RBAC integration. +- **MLflow Operator 3.13.0** is delivered through OperatorHub starting with Alauda AI 2.8.x. Install the Operator and create an `MLflow` custom resource to deploy and manage the tracking server. +- **Alauda AI Workbench v0.2.0** is managed through the `workbench` component in the `default` `AmlCluster` in Alauda AI 2.8.x. Alauda AI installs and reconciles the Workbench Operator and its resources. +- **Elyra Pipeline Editor** enables users to compose Kubeflow Pipelines workflows visually from notebooks and scripts. +- **Reusable pipeline components** and expanded Kubeflow Pipeline mechanisms simplify authoring and reusing pipeline workflows. +- **Llama Stack** adds PGVector and Milvus vector stores for agent and retrieval workflows. +- **Alauda Build of Spark Operator** supports running Spark applications as Kubernetes-native distributed workloads. +- **Coding-agent and agentic MLOps workflows** provide guidance for connecting coding agents to on-premises inference services. + +#### **Hardware and Model Optimization** -#### **Fresh Installation Only** +- **Alauda Build of NPU Operator** is managed through the Alauda AI Operator and includes support for Ascend 910B and immutable operating-system environments. +- **Ascend NPU fine-tuning** supports MindSpeed-LLM and Kubeflow Trainer v2 workflows, including checkpoint conversion, dataset preprocessing, and supervised fine-tuning. +- **ModelSlim integration** provides model compression and quantization workflows for Ascend NPU environments. + +### **Deployment Form Changes in Alauda AI 2.8.x** + +Alauda AI Workbench, MLflow, and Alauda Build of LeaderWorkerSet are delivered through the 2.8.x Operator-based deployment model. Workbench and MLflow are installed as independent Operators from **OperatorHub**, while LWS is installed and reconciled through the `lws` component in the `default` `AmlCluster`: + +| Component | Alauda AI 2.8.x deployment form | Migration note | +|:--|:--|:--| +| **MLflow** | MLflow OLM Operator | Install the MLflow Operator and create an `MLflow` custom resource. | +| **Alauda Build of LeaderWorkerSet (LWS)** | Operator managed through Alauda AI and the `AmlCluster` | Enable the `lws` component in the `default` `AmlCluster`; Alauda AI installs and reconciles LWS. | +| **Alauda AI Workbench** | Workbench OLM Operator | Install the Workbench Operator and create a `Workbench` custom resource. | + +These deployment-form changes do not imply that existing user data should be deleted. Follow the component-specific migration instructions and preserve applicable PVCs and custom resources during migration. + +### **Deprecated Features** + +- The former platform roles `aml-namespace-editor`, `owner`, and `viewer` are deprecated. Use namespace-scoped permissions instead. +- The GitLab-based Model Catalog is deprecated. Use the Image Based Model Catalog with OCI model artifacts for new model delivery workflows. +- The former Alauda AI Model Serving operator is deprecated. Use the Knative Operator for supported serverless inference scenarios. + +### **Breaking Changes** -**AI 2.7.0** does not support upgrade from AI 2.6.x or earlier versions. Existing environment migration is outside the scope of this Fast release and requires a later LTS release with a dedicated upgrade guide. +#### **Deployment and Configuration** +- Alauda AI 2.8.x uses the single-cluster application architecture. The **Alauda AI Essentials** (`aml-global`) component is no longer delivered. +- Project and cluster switching are removed from the AI header. Use namespace switching to change the working context. +- Login and logout are handled through `oauth2-proxy`. +- MLflow, Alauda Build of LeaderWorkerSet, and Alauda AI Workbench change from their previous deployment forms to Operators. Existing installations require component-specific migration instead of an in-place package upgrade. -### Fixed Issues +### **Fixed Issues Since Alauda AI 2.3.x** -- When injecting SecurityContext into the inference service Pod, KServe interferes with the configuration of the sidecar injected by Knative Serving, causing the Pod to fail to run due to configuration conflicts. -- Since model-catalog-ui prioritizes the "email" as the username when extracting a username from a token, if the user's email does not match their username, it may extract an incorrect username, which affects the relevant permission checks. +The following issue was documented as a known issue in Alauda AI 2.3.0 and fixed in a later release: -### Known Issues +- **vLLM Ascend:** Fixed multi-card Ascend inference deployments that could fail to become `Ready` because HCCL initialization did not support both root and non-root modes. -- When using VictoriaMetrics for monitoring data collection of inference services operating in Serverless mode, there is a known issue where the inference services cannot scale down to zero. -- Because the name of the WorkspaceKind resource in a Workbench chart includes ".Release.Revision", when the chart is deployed multiple times, multiple instances of the same WorkspaceKind resource may be deployed due to the incrementing of ".Release.Revision". -- Because the HAMi scheduler component shares the same image as kube-scheduler, it may incorrectly identify the kube-scheduler image registry in the imported cluster, failing to pull the image and thus causing startup failures. +### **Known Issues** +- When VictoriaMetrics is used to collect monitoring data for inference services running in Serverless mode, inference services may not scale down to zero. diff --git a/docs/en/plan/supported_configurations.md b/docs/en/plan/supported_configurations.md index fee91a61..cee92be2 100644 --- a/docs/en/plan/supported_configurations.md +++ b/docs/en/plan/supported_configurations.md @@ -8,97 +8,129 @@ This page lists the currently maintained Alauda AI versions in the component mat ## x86_64 Architecture -| Components | Type | Alauda AI v2.3 Stable | Alauda AI v2.7 Fast | +| Components | Type | Alauda AI v2.3 Stable | Alauda AI v2.8 Stable | | -------------------------------------------- | -------------- | ---------------------------------- | -------------------------------- | -| Alauda Container Platform Supported Versions | | v4.0.x, v4.1.x, v4.2.x, v4.3.x | v4.0.x, v4.1.x, v4.2.x, v4.3.x | -| Alauda AI Essentials (5) | Cluster Plugin | v2.3.0 | Removed | -| Alauda AI | Operator | v2.3.0 | v2.7.0 | -| Alauda AI Workbench | Cluster Plugin | v0.1.7 | v0.1.8 | -| Alauda Build of KServe | Operator | v0.16.0 | v0.16.2 | -| Alauda Build of Serving Runtime (6) | Operator | - | v1.0.0 | +| Alauda Container Platform Supported Versions | | v4.0.x, v4.1.x, v4.2.x, v4.3.x | v4.1.x, v4.2.x, v4.3.x (1) | +| Alauda AI Essentials (2) | Cluster Plugin | v2.3.0 | Removed | +| Alauda AI | Operator | v2.3.0 | v2.8.0 | +| Alauda AI Workbench | Cluster Plugin | v0.1.7 | Replaced (3) | +| Alauda AI Workbench Operator | Operator | - | v0.2.1 | +| Alauda Build of KServe | Operator | v0.16.0 | v0.19.0 | +| Alauda Build of Serving Runtime | Operator | - | v1.1.0 | | Alauda Build of KubeRay Operator | Cluster Plugin | v1.6.0 | v1.6.0 | | Alauda Build of NVIDIA GPU Device Plugin | Cluster Plugin | v0.18.4 | v0.18.4 | | Alauda Build of NVIDIA DRA Driver for GPUs | Cluster Plugin | v25.8.1 | v25.8.1 | | Alauda Build of DCGM-Exporter | Cluster Plugin | v4.2.3-413-1 | v4.2.3-413-1 | -| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 | -| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 | +| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.9.0 | +| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.3 | | Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 | | Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 | -| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 | -| Alauda Build of JobSet (1) | Operator | - | v0.12.0 | +| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | Replaced | +| Alauda Build of LeaderWorkerSet | Operator | - | v0.9.0-1 | +| Alauda Build of JobSet | Operator | - | v0.12.0 | | Volcano | Cluster Plugin | v1.12.4 | v1.12.4 | -| MLFlow | Cluster Plugin | v3.1.5 | v3.10.0 | -| Kubeflow Base | Operator | v1.10.14-1 | v26.3.2 | -| Kubeflow Pipelines | Operator | v1.10.13 | v26.3.2 | -| Kubeflow Trainer v2 (1) | Operator | v1.10.13 | v26.3.2 | -| Data Science Pipeline Operator | Operator | v2.15.1 | v2.15.1 | -| Alauda Build of Kubeflow Model Registry | Operator | v1.10.13 | v0.3.8-2 | +| MLFlow | Cluster Plugin | v3.1.5 | Replaced | +| MLFlow Operator | Operator | - | v3.13.0 | +| Kubeflow Base | Cluster Plugin | v1.10.14-1 | Replaced | +| Kubeflow Base Operator | Operator | - | v26.3.5 | +| Kubeflow Pipelines | Cluster Plugin | v1.10.13 | Replaced | +| Kubeflow Pipelines Operator | Operator | - | v26.3.5 | +| Kubeflow Trainer v2 | Cluster Plugin | v1.10.13 | Replaced | +| Kubeflow Trainer Operator | Operator | - | v26.3.5 | +| Data Science Pipeline Operator | Operator | - | v2.15.1 | +| Kubeflow Model Registry | Operator | v1.10.13 | Replaced | +| Alauda Build of Kubeflow Model Registry | Operator | - | v0.3.8-2 | | Alauda Build of Llama Stack | Operator | v0.8.0 | v0.9.0 | -| Label Studio | Helm Charts | v1.21.0-2 | v1.21.0-2 | -| Alauda Build of Envoy AI Gateway | Cluster Plugin | v0.4.1 | - | +| Label Studio | Helm Chart | v1.21.0-2 | v1.21.0-2 | +| Alauda Build of Envoy AI Gateway | Cluster Plugin | v0.4.1 | Replaced | | Alauda Build of Envoy AI Gateway | Operator | - | v0.6.0-1 | -| Dify | Helm Charts | v1.11.4 | v1.11.4 | -| Langflow | Helm Charts | v1.6.4-1 | v1.6.4-1 | -| Evidently | Helm Charts | v0.7.14-1 | v0.7.14-1 | -| Featureform (2) | Helm Charts | v0.12.1-2 | v0.12.1-2 | +| Dify | Helm Chart | v1.11.4 | Replaced | +| Dify Operator | Operator | - | v1.15.0 | +| Langflow (4) | Helm Chart | v1.6.4-1 | Deprecated | +| Evidently | Helm Chart | v0.7.14-1 | v0.7.14-1 | +| Featureform (4) | Helm Chart | v0.12.1-2 | Deprecated | | Alauda Build of Feast | Operator | v0.61.1 | v0.61.1 | | Knative Operator | Operator | v1.19.3-260213 | v1.19.3-260213 | -| PostgreSQL | Operator | v4.2.0 | v4.2.0 | -| Milvus Operator | Cluster Plugin | v1.3.5 | v1.3.5 | -| Alauda Build of Gitlab | Operator | v18.5.1 | v18.5.1 | -| Alauda Build of TrustyAI | Operator | v3.4.1 | v3.4.1 | +| PostgreSQL (5) | Operator | v4.2.0 | v4.3.3 | +| Alauda Cache Service for Redis OSS (5) | Operator | - | v5.0.2 | +| Alauda Build of Authorino (5) | Operator | - | v0.26.0-1 | +| Milvus Operator | Helm Chart | v1.3.5 | v1.3.5 | +| Alauda Build of Gitlab (6) | Operator | v18.5.1 | - | +| Alauda Build of Harbor (6) | Operator | - | v2.4.14 | +| Alauda Build of TrustyAI | Operator | v3.4.1 | v3.4.2-1 | +| Alauda Build of MCP Lifecycle Operator | Operator | - | v0.2.0-1 | +| Kagenti Operator | Operator | - | v0.3.0-rc.1 | +| Kagenti UI Operator | Operator | - | v0.6.1 | ## ARM Architecture -| Components | Type | Alauda AI v2.3 Stable | Alauda AI v2.7 Fast | + +| Components | Type | Alauda AI v2.3 Stable | Alauda AI v2.8 Stable | | -------------------------------------------- | -------------- | ---------------------------------- | -------------------------------- | -| Alauda Container Platform Supported Versions | | v4.0.x, v4.1.x, v4.2.x, v4.3.x | v4.0.x, v4.1.x, v4.2.x, v4.3.x | -| Alauda AI Essentials (5) | Cluster Plugin | v2.3.0 | Removed | -| Alauda AI | Operator | v2.3.0 | v2.7.0 | -| Alauda AI Workbench | Cluster Plugin | v0.1.7 | v0.1.8 | -| Alauda Build of KServe | Operator | v0.16.0 | v0.16.2 | +| Alauda Container Platform Supported Versions | | v4.0.x, v4.1.x, v4.2.x, v4.3.x | v4.1.x, v4.2.x, v4.3.x | +| Alauda AI Essentials | Cluster Plugin | v2.3.0 | Removed | +| Alauda AI | Operator | v2.3.0 | v2.8.0 | +| Alauda AI Workbench | Cluster Plugin | v0.1.7 | Replaced | +| Alauda AI Workbench Operator | Operator | - | v0.2.1 | +| Alauda Build of KServe | Operator | v0.16.0 | v0.19.0 | | Alauda Build of KubeRay Operator | Cluster Plugin | v1.6.0 | v1.6.0 | | Alauda Build of NVIDIA GPU Device Plugin | Cluster Plugin | v0.18.4 | v0.18.4 | | Alauda Build of NVIDIA DRA Driver for GPUs | Cluster Plugin | v25.8.1 | v25.8.1 | | Alauda Build of DCGM-Exporter | Cluster Plugin | v4.2.3-413-1 | v4.2.3-413-1 | -| Alauda Build of NPU Operator (3) | Cluster Plugin | v1.1.3 | v1.2.4 | -| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 | -| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 | +| Alauda Build of NPU Operator | Cluster Plugin | v1.1.3 | Replaced | +| Alauda Build of NPU Operator | Operator | - | v1.2.4 | +| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.9.0 | +| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.3 | +| Alauda Build of HAMi Ascend Device Plugin | Operator | - | v1.4.0 | +| Alauda Build of InferNex Bridge | Operator | - | v26.6.0 | | Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 | | Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 | -| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 | -| Alauda Build of JobSet (1) | Operator | - | v0.12.0 | +| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | Replaced | +| Alauda Build of LeaderWorkerSet | Operator | - | v0.9.0-1 | +| Alauda Build of JobSet | Operator | - | v0.12.0 | | Volcano | Cluster Plugin | v1.12.4 | v1.12.4 | -| MLFlow | Cluster Plugin | v3.1.5 | v3.10.0 | -| Kubeflow Base | Operator | v1.10.14-1 | v26.3.2 | -| Kubeflow Pipelines | Operator | v1.10.13 | v26.3.2 | -| Kubeflow Trainer v2 (1) | Operator | v1.10.13 | v26.3.2 | -| Data Science Pipeline Operator | Operator | v2.15.1 | v2.15.1 | -| Alauda Build of Kubeflow Model Registry | Operator | v1.10.13 | v0.3.8-2 | -| Alauda build of Llama Stack | Operator | v0.8.0 | v0.9.0 | -| Label Studio | Helm Charts | v1.21.0-2 | v1.21.0-2 | -| Alauda build of Envoy AI Gateway | Cluster Plugin | v0.4.1 | - | -| Alauda build of Envoy AI Gateway | Operator | - | v0.6.4-1 | -| Dify | Helm Charts | v1.11.4 | v1.11.4 | -| Langflow | Helm Charts | v1.6.4-1 | v1.6.4-1 | -| Evidently | Helm Charts | v0.7.14-1 | v0.7.14-1 | +| MLFlow | Cluster Plugin | v3.1.5 | Replaced | +| MLFlow Operator | Operator | - | v3.13.0 | +| Kubeflow Base | Cluster Plugin | v1.10.14-1 | Replaced | +| Kubeflow Base Operator | Operator | - | v26.3.5 | +| Kubeflow Pipelines | Cluster Plugin | v1.10.13 | Replaced | +| Kubeflow Pipelines Operator | Operator | - | v26.3.5 | +| Kubeflow Trainer v2 | Cluster Plugin | v1.10.13 | Replaced | +| Kubeflow Trainer Operator | Operator | - | v26.3.5 | +| Data Science Pipeline Operator | Operator | - | v2.15.1 | +| Kubeflow Model Registry | Operator | v1.10.13 | Replaced | +| Alauda Build of Kubeflow Model Registry | Operator | - | v0.3.8-2 | +| Alauda Build of Llama Stack | Operator | v0.8.0 | v0.9.0 | +| Label Studio | Helm Chart | v1.21.0-2 | v1.21.0-2 | +| Alauda Build of Envoy AI Gateway | Cluster Plugin | v0.4.1 | Replaced | +| Alauda Build of Envoy AI Gateway | Operator | - | v0.6.0-1 | +| Dify | Helm Chart | v1.11.4 | Replaced | +| Dify Operator | Operator | - | v1.15.0 | +| Langflow | Helm Chart | v1.6.4-1 | Deprecated | +| Evidently | Helm Chart | v0.7.14-1 | v0.7.14-1 | | Alauda Build of Feast | Operator | v0.61.1 | v0.61.1 | | Knative Operator | Operator | v1.19.3-260213 | v1.19.3-260213 | -| PostgreSQL | Operator | v4.2.0 | v4.2.0 | -| Milvus Operator | Cluster Plugin | v1.3.5 | v1.3.5 | -| Alauda Build of Gitlab | Operator | v18.5.1 | v18.5.1 | -| Alauda Build of TrustyAI | Operator | v3.4.1 | v3.4.1 | +| PostgreSQL | Operator | v4.2.0 | v4.3.3 | +| Alauda Cache Service for Redis OSS | Operator | - | v5.0.2 | +| Alauda Build of Authorino | Operator | - | v0.26.0-1 | +| Milvus Operator | Helm Chart | v1.3.5 | v1.3.5 | +| Alauda Build of Gitlab | Operator | v18.5.1 | - | +| Alauda Build of Harbor | Operator | - | v2.4.14 | +| Alauda Build of TrustyAI | Operator | v3.4.1 | v3.4.2-1 | +| Alauda Build of MCP Lifecycle Operator | Operator | - | v0.2.0-1 | +| Kagenti Operator | Operator | - | v0.3.0-rc.1 | +| Kagenti UI Operator | Operator | - | v0.6.1 | ## Notes -(1) 'Kubeflow Trainer v2' and 'Alauda Build of JobSet' require Alauda Container Platform 4.1.x or later +(1) Alauda Container Platform v4.0.x is no longer supported in Alauda AI v2.8. -(2) 'Featureform' is only supported on x86_64 architecture +(2) 'Alauda AI Essentials' is removed after Alauda AI v2.6. -(3) 'Alauda Build of NPU Operator' is only supported on ARM architecture +(3) 'Replaced' indicates that the component deployment method has changed; Operator replaces Cluster Plugin or Helm Chart. -(4) 'Alauda build of Envoy AI Gateway' has been refactored from a Cluster Plugin to an Operator in Alauda AI v2.5 +(4) 'Featureform' and 'Langflow' are deprecated and should not be used for new deployments. -(5) 'Alauda AI Essentials' is removed in Alauda AI v2.6 +(5) PostgreSQL, Alauda Cache Service for Redis OSS, and Alauda Build of Authorino are dependencies of MaaS. -(6) 'Alauda Build of Serving Runtime' is split out from 'Alauda AI' in Alauda AI v2.6 and is only supported on x86_64 architecture +(6) Model storage has changed from GitLab to a container registry; Harbor is recommended for storing models. diff --git a/docs/en/train/components/jobset/install.mdx b/docs/en/train/components/jobset/install.mdx index 46e6621f..1527fa85 100644 --- a/docs/en/train/components/jobset/install.mdx +++ b/docs/en/train/components/jobset/install.mdx @@ -8,13 +8,7 @@ This page describes how to install **Alauda Build of JobSet** on Alauda Containe ## Upload the Operator Package -Download the JobSet Operator package, for example `jobset-operator.ALL.xxxx.tgz`. - -Use `violet` to upload the package to the platform repository: - -```bash -violet push --platform-address= --platform-username= --platform-password= jobset-operator.ALL.xxxx.tgz -``` +Download the JobSet Operator package, then upload the package following [Upload Packages](../../../installation/package-upload.mdx). ## Install the Operator diff --git a/docs/en/upgrade/uninstall.mdx b/docs/en/upgrade/uninstall.mdx index a80e085b..0e5686e1 100644 --- a/docs/en/upgrade/uninstall.mdx +++ b/docs/en/upgrade/uninstall.mdx @@ -4,144 +4,108 @@ weight: 40 # Uninstall -This document guides you through the complete uninstallation of Alauda AI and its related components from your cluster. The uninstallation process is divided into two main parts: first, uninstalling the optional **Alauda AI Model Serving** and its instances, and then, uninstalling the core **Alauda AI** instance and Operator. +This document guides you through the complete uninstallation of Alauda AI and its related components from your cluster. -If you don't need the Serverless capability or want to use only the essential core features to save resources, you can choose to perform only the first step to remove the **Alauda AI Model Serving** components. - -**Note:** Before proceeding with the uninstallation, ensure you have backed up all important data and models. The uninstallation process will permanently delete all resources and configurations associated with Alauda AI. - - - -## Uninstalling Alauda AI Model Serving Components - -This section guides you on how to uninstall the components that provide Serverless capabilities. - -### Deleting the Alauda AI Model Serving Instance +In Alauda AI 2.8, components are installed and managed by Alauda AI through the cluster-scoped `default` `AmlCluster` instance. Uninstall the components first, then the Alauda AI core, following the order in this document. Uninstalling Alauda AI and its Operators does not automatically remove custom resources or CRDs; review [General notes](#general-notes) before proceeding. :::warning -Before deleting the **Alauda AI Model Serving** Instance, you must check the **Serverless** parameter configuration. - -**For versions 1.4 and later:** -Ensure the **BuiltIn Knative Serving** field is set to `Unmanaged`. - -**For version 1.3:** -Ensure the **Knative Serving Mode** field is set to `Unmanaged`. +Before uninstalling, back up all important data and models. Data-related resources may remain after uninstallation, and deleting them can permanently delete the associated data. Review [Data remaining after uninstallation](#data-remaining-after-uninstallation) before deleting any resource. ::: - +## Uninstall flow at a glance -#### Procedure +| Component category | How to uninstall | +|:--|:--| +| Components with `Managed` state in the `default` `AmlCluster` (e.g. `authorino`, `lws`, `kserve`) | Set `managementState` to `Removed` | +| Components with `SharedManaged` state (e.g. `envoyGateway`, `postgres`, `redis`) | Set `managementState` to `Unmanaged`, then uninstall the Operator in OperatorHub | +| Components not managed by the `AmlCluster` | Uninstall in OperatorHub, or in Cluster Plugins if installed as a Cluster Plugin | +| Alauda AI core | Delete the `default` `AmlCluster` instance, then uninstall the Alauda AI Operator in OperatorHub | -From the **Administrator** view: +## Uninstall AmlCluster-managed components -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the cluster you want to uninstall from. -3. Find and select **Alauda AI Model Serving**. -4. On the **Alauda AI Model Serving** page, click the **All Instances** tab. -5. Find the instance (commonly named `default-knative-serving`, or your custom instance name). -6. Click the `...` menu on the right side of the instance, then select **Delete**. -7. In the confirmation window, confirm the deletion, then click **Delete**. +### Managed components: set to `Removed` -#### Verification +Components managed by Alauda AI (with `managementState: Managed`) are uninstalled by changing the state to `Removed` in the `default` `AmlCluster`. Alauda AI then removes the component and the resources it manages. -Verify that the instance has been deleted with the following command: +For example, to uninstall KServe: -```bash -kubectl get knativeservings.components.aml.dev -n knative-serving +```yaml +spec: + components: + kserve: + managementState: Removed ``` -If the instance was successfully deleted, the command will return "No resources found in knative-serving namespace." - - - -### Uninstalling the Alauda AI Model Serving Operator +### SharedManaged components: set to `Unmanaged`, then uninstall in OperatorHub - +Components with the `SharedManaged` state (`envoyGateway`, `postgres`, `redis`) do not support the `Removed` value. Set their `managementState` to `Unmanaged` first, then uninstall the corresponding Operator in OperatorHub. -#### Procedure -From the **Administrator** view: +For example, to uninstall PostgreSQL: -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the cluster you want to uninstall from. -3. Find and select **Alauda AI Model Serving**. -4. On the **Alauda AI Model Serving** page, click Uninstall. -5. In the confirmation window, check the installation location of the **Alauda AI Model Serving** Operator (default is kserveless-operator). -6. Click the **Uninstall** button. +```yaml +spec: + components: + postgres: + managementState: Unmanaged +``` -#### Verification -Confirm that the Alauda AI Model Serving tile's status has changed from Installed to Not Installed or has disappeared from the list. +Then, in **Administrator** view: - +1. Click **Marketplace / OperatorHub** and select the target cluster. +2. Find the Operator of the component (for example **PostgreSQL**). +3. Click **Uninstall** and confirm. +For the management states and the component keys, see [Component management states](../installation/ai-cluster.mdx#component-management-states). -## Uninstalling Core Alauda AI Components +## Uninstall components not managed by AmlCluster -After completing the first section, you can proceed to uninstall the core Alauda AI components. +Some components are not managed by the `default` `AmlCluster`. This includes components whose `managementState` is `Unmanaged` and components installed independently, such as **Kagenti**, **Alauda Build of MCP Lifecycle Operator**, **Dify**, and **Alauda Build of InferNex Bridge**. Uninstall these in OperatorHub, or in Cluster Plugins if the component was installed as a Cluster Plugin. -### Deleting the Alauda AI Instance +In **Administrator** view: - +1. Click **Marketplace / OperatorHub** (or **Marketplace / Cluster Plugins**) and select the target cluster. +2. Find the component to uninstall. +3. Click **Uninstall** and confirm. -#### Procedure +## Uninstall Alauda AI core -From the **Administrator** view: +Uninstall the core Alauda AI in two steps: first delete the `default` `AmlCluster` instance, then uninstall the Alauda AI Operator in OperatorHub. -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the cluster you want to uninstall from. -3. Find and select **Alauda AI**. -4. On the **Alauda AI** page, click the **All Instances** tab. -5. Find the instance named `default`. -6. Click the `...` menu on the right side of the instance, then select **Delete**. -7. In the confirmation window, confirm the deletion, then click **Delete**. +### Delete the `default` `AmlCluster` instance -#### Verification +1. In **Administrator** view, click **Marketplace / OperatorHub** and select the target cluster. +2. Select **Alauda AI**, then open the **All Instances** tab. +3. Find the instance named `default`. +4. Click the **...** menu, then select **Delete**, and confirm. -Verify that the instance has been deleted with the following command: +Verify that the instance is deleted: ```bash kubectl get amlcluster default ``` -If the instance was successfully deleted, the command will return No resources found or a similar message. - - - - -### Uninstalling the Alauda AI Operator - - -#### Procedure -From the **Administrator** view: - -1. Click **Marketplace / OperatorHub**. -2. At the top of the console, from the **Cluster** dropdown list, select the cluster you want to uninstall from. -3. Find and select **Alauda AI**. -4. On the **Alauda AI** page, click **Uninstall**. -5. In the confirmation window, check the installation location of the **Alauda AI** Operator (default is aml-operator). -6. Click the **Uninstall** button. +The command returns `No resources found` or a similar message. -#### Verification -Confirm that the Alauda AI tile's status has changed from Installed to Not Installed or has disappeared from the list. +### Uninstall the Alauda AI Operator - +1. In **Administrator** view, click **Marketplace / OperatorHub** and select the target cluster. +2. Select **Alauda AI**, then click **Uninstall**. +3. In the confirmation window, confirm the installation location (default is `aml-operator`), then click **Uninstall**. +Verify that the **Alauda AI** tile changes from `Installed` to `Not Installed`, or disappears from the list. -### Cleaning Up Remaining Resources -CRDs are not removed automatically when an operator is uninstalled. You can manually clean up these remaining resources, and be aware that: +## Data remaining after uninstallation \{#data-remaining-after-uninstallation} :::warning -**Deleting a CRD will permanently delete all associated data.** +After Alauda AI is uninstalled, data-related resources remain in the `kubeflow` and `aml-maas` namespaces. Deleting these resources can cause data loss. Review the resources before deleting them, and delete them only if you are certain the data is no longer needed. +::: -The `amlnamespaces.manage.aml.dev` CRD, which is responsible for managing namespaces, is not included in the deletion commands below. To ensure a complete deletion of resources, you must manually remove the finalizer field after the CRD deletion operation. +## General notes \{#general-notes} -It's generally recommended to keep this CRD unless you are certain you no longer need its management capabilities. +### Delete operator instances before the Operator -::: +For every Operator, if you want to delete the custom-resource instances it manages, delete the instances **before** uninstalling the Operator. Uninstalling an Operator does not delete its instances; the instances and the data they reference remain in the cluster. -```bash -# Delete all control plane CRDs -kubectl delete crd amls.components.aml.dev -kubectl delete crd kserves.components.aml.dev -kubectl delete crd amlclusters.amlclusters.aml.dev -kubectl delete crd knativeservings.components.aml.dev -``` +### CRDs are not removed by uninstallation + +Uninstalling a Cluster Plugin or an Operator from OperatorHub does not delete the CRDs it installed. If you want to delete the CRDs, delete them separately. Note that deleting a CRD also deletes all custom resources of that kind and may remove data. diff --git a/docs/en/upgrade/upgrade-from-previous-version.mdx b/docs/en/upgrade/upgrade-from-previous-version.mdx index f31a5fbc..01e9dd1a 100644 --- a/docs/en/upgrade/upgrade-from-previous-version.mdx +++ b/docs/en/upgrade/upgrade-from-previous-version.mdx @@ -2,270 +2,211 @@ weight: 10 --- -export const prevVersion = '1.5' -export const curVer = '2.3' +export const prevVersion = '2.3' +export const curVer = '2.8' # Upgrade Alauda AI :::info -Upgrade from {prevVersion} to {curVer} +Upgrade from **Alauda AI {prevVersion}.x** to **Alauda AI {curVer}.x**. ::: -## Install Alauda AI Cluster Components - -Please visit [Alauda AI Cluster](../installation/ai-cluster.mdx) for: +This procedure describes the migration from the Alauda AI 2.3.x deployment model to the Alauda AI 2.8.x deployment model. Read the complete procedure before starting. The upgrade changes the deployment form of several components, and some components require migration rather than an in-place upgrade. :::warning -Please ignore `Creating Alauda AI Instance` since we are upgrading **Alauda AI** from a previously managed version. +**Alauda AI 2.8.x does not support ACP 4.0.x.** If the target environment runs ACP 4.0.x, upgrade ACP to a supported version from **ACP 4.1.x through 4.3.x** before upgrading Alauda AI. ::: -1. [Downloading](../installation/ai-cluster.mdx#downloading) operator bundle packages for `Alauda AI` and `Knative Operator` (Optional). -2. [Downloading](../deploy/components/kserve/install.mdx#upload-operator) operator bundle packages for `Alauda Build of KServe`. -3. [Uploading](../installation/ai-cluster.mdx#uploading) operator bundle packages to the destination cluster. -4. To upgrade, follow the process described below. - -## Pre-Upgrade Operations - -### Annotating Stopped Inference Services - -Starting from version {curVer}, the platform adopts the community-native stop capability provided by KServe. To ensure a smooth upgrade, all inference services that are currently in a **stopped** state must be explicitly annotated before upgrading. - :::warning -This step is **required** before upgrading. Failure to annotate stopped inference services may result in unexpected behavior after the upgrade. +Before uninstalling a Cluster Plugin or Operator, review the component-specific migration requirements and preserve important user data and configuration. Export custom resources and configuration where practical, and verify that user PVCs, databases, and object-storage artifacts will be retained. Use storage snapshots or database backups only when they are available and required by your environment. Do not delete a PVC or custom resource unless the relevant migration procedure explicitly requires it. ::: -1. List all inference services that are currently stopped: +## Prepare the Upgrade Packages - ```bash - kubectl get inferenceservices --all-namespaces - ``` +Upload the Alauda AI 2.8.x operator package and all required dependency packages before starting the upgrade. Upload packages to the cluster where the corresponding component will run. For the common package download, `violet` setup, and upload procedure, see [Upload Packages](../installation/package-upload.mdx). -2. For each stopped inference service, add the following annotation: +The required package set depends on the features enabled in the existing environment. At minimum, prepare the following packages for an Alauda AI 2.8.x environment: - ```bash - kubectl annotate inferenceservice -n serving.kserve.io/stop='true' - ``` +| Package or component | Purpose | +|:--|:--| +| **Alauda AI** | Core Alauda AI platform components. | +| **Alauda Build of KServe** | Model-serving components and inference services. | +| **Alauda Build of Serving Runtime** | Serving runtimes for supported inference workloads. | +| **Alauda Build of LeaderWorkerSet** | LeaderWorkerSet workloads. | +| **PostgreSQL** | PostgreSQL services required by supported components. | +| **Alauda Build of Authorino** | Authentication and authorization for MaaS. | +| **Alauda Cache Service for Redis OSS** | Cache services required by supported components. | +| **Alauda AI Workbench Operator** | Workbench environments and workspace management. | +| **MLflow Operator** | MLflow tracking and artifact management. | +| Other operators required by the selected Alauda AI 2.8.x features | For example, operators required by enabled training, gateway, or MLOps capabilities. | - Alternatively, you can edit the resource directly and add the annotation under `metadata.annotations`: +The exact package filenames and versions are delivered with the Alauda AI 2.8.x release. Do not substitute packages from Alauda AI 2.3.x. - ```yaml - apiVersion: serving.kserve.io/v1beta1 - kind: InferenceService - metadata: - name: - annotations: - serving.kserve.io/stop: 'true' #[!code highlight] - ``` +### Uploading operator packages -3. Repeat this step for all stopped inference services across all namespaces. +The common package guide includes the `violet push` command, external-registry options, and a multi-package upload example. Use [Upload Packages](../installation/package-upload.mdx) to upload each package using the filename and target-cluster information supplied with the Alauda AI 2.8.x release. -## Upgrading +## Pre-Upgrade Operations -The following procedure describes how to upgrade from **Alauda AI** {prevVersion} to {curVer}. +Complete the following operations before upgrading **Alauda AI**. -### Upgrade Alauda AI Essentials +### Preserve existing resources -Upgrade **Alauda AI Essentials** in the Functional Components tab of the global cluster. +Review the migration requirements for each component and preserve the resources that must survive the migration. The components in this upgrade are designed to retain their data; keep the data in place rather than recreating it where possible. At a minimum, preserve and record: -In the **Administrator** view, navigate to **Clusters / Cluster** in the left navigation, find the **global** cluster in the right panel, and click the **Upgrade** button on the **Functional Components** tab page to trigger the upgrade. +- `AmlCluster` resources and their current YAML configuration. +- Custom resources managed by Workbench, MLflow, LWS, KServe, and other enabled components. +- Workspaces, `WorkspaceKind` resources, the `aml-workbench-config` `ConfigMap`, and user PVCs. +- MLflow metadata and artifact-storage configuration. +- Inference-service configuration and model-storage configuration. +- Any component-specific secrets, ConfigMaps, RoleBindings, and authorization policies. -:::info -All components that require upgrading, including **Alauda AI Essentials**, will be upgraded together and cannot be upgraded individually. -::: +Use the component-specific backup instructions before uninstalling the old deployment form. For example, the [Workbench upgrade guide](./workbench.mdx) backs up Workbench resources and requires preserving user PVCs. -After the upgrade is complete, please confirm that the status of **Alauda AI Essentials** in the list is **Running** and the version is **v{curVer}.x**. +### Check the target cluster -### Upgrading Alauda AI Operators +Confirm that: -The procedure for upgrading the operator is nearly identical, with only the target component being different. +1. The target cluster is healthy and has sufficient capacity for the new operators and their operands. +2. The target cluster can pull images from the configured registry. +3. The required storage classes and persistent volumes are available. +4. The platform administrator credentials can upload packages and install Operators through OperatorHub. +5. The global cluster can be accessed with `kubectl` for the global migration steps described below. -| Step | Alauda AI Operator | -|:----------------|:--------------------------------------------------------------------------------------------------| -| **1. Navigate** | Log into the Web Console, then go to **Marketplace > OperatorHub** in the **Administrator** view. | -| **2. Select** | Select your target **cluster**. | -| **3. Click** | Click the **Alauda AI** card. | -| **4. Confirm** | Click **Confirm** on the upgrade prompt. | +## Remove Cluster Plugins -:::info -Once the new version is uploaded and recognized by the platform, an upgrade prompt will appear at the top of the operator's page. -::: +Alauda AI 2.8.x changes several components from Cluster Plugins to Operators. These deployment forms do not support an in-place upgrade. Before upgrading **Alauda AI**, remove the old Cluster Plugin versions of the components that are installed in the existing environment: -### Installing Alauda Build of KServe Operator +- **Alauda AI Essentials** from the **global** cluster. +- **Alauda Build of LeaderWorkerSet** from the target cluster. +- **Alauda AI Workbench** from the target cluster. +- **MLflow** from the target cluster. -Starting from version {curVer}, **Alauda Build of KServe** is provided as a separate operator plugin to offer more specialized and flexible model serving capabilities. After completing the core AI operator upgrades, you must install the KServe operator to enable model serving functionality. +Use the following common procedure: -For detailed installation and configuration steps, please refer to the [Alauda Build of KServe Installation Guide](../deploy/components/kserve/install.mdx). +1. Record the existing custom resources and configuration for the components that are installed. +2. Preserve the related user data, including Workbench PVCs, MLflow databases and artifacts, and workloads managed by LeaderWorkerSet. +3. In **Administrator** > **Marketplace** > **Cluster Plugins**, select the relevant cluster: + - Select the **global** cluster and uninstall **Alauda AI Essentials**, if it is installed. + - Select the target cluster and uninstall the installed **Alauda Build of LeaderWorkerSet**, **Alauda AI Workbench**, and **MLflow** Cluster Plugins. +4. Do not delete the preserved PVCs, databases, object-storage artifacts, or custom resources unless the component-specific migration procedure explicitly requires it. +5. After the old Cluster Plugins are removed, enable the replacement components through the Alauda AI 2.8.x deployment model. The upgraded `AmlCluster` installs and reconciles **Alauda Build of LeaderWorkerSet** when `lws.managementState` is `Managed`. Install the replacement Operators and create their custom resources for **Alauda AI Workbench** and **MLflow** as described in their component-specific documentation. -### Upgrading Cluster Plugins +The existing cluster-scoped `AmlCluster` resource named `default` controls the Alauda AI deployment configuration. After upgrading **Alauda AI**, use this resource to configure components managed by Alauda AI, including **Alauda Build of LeaderWorkerSet**. Components delivered as independent Operators, such as Workbench and MLflow, are deployed and configured through their own Operator custom resources. -:::info -This step is **only required if you have deployed** any of the following cluster plugins: **Alauda AI Workbench**, **MLflow**, or **Volcano**. If you have not deployed any of these plugins, you can skip this step. - -For more information about cluster plugins, refer to . +:::warning +Before uninstalling a Cluster Plugin, verify whether its CRDs, custom resources, or dependent resources will be removed. If the uninstall may remove CRDs, confirm that user data is preserved before proceeding. ::: -The procedure to upgrade cluster plugins involves uploading new version packages and then upgrading them from the Web Console. The following plugins require upgrading: - -- Alauda AI Workbench -- MLflow -- Volcano - -#### Uploading Cluster Plugins - -Upload the new cluster plugin packages to the global cluster. For detailed instructions on how to upload cluster plugins, refer to . - -Repeat the upload process for each plugin that you have deployed. - -:::info -After the upload is completed, wait approximately 10–15 minutes for the platform to synchronize the new version information. -::: +For Workbench-specific data-preservation requirements, see [Migrating from the Workbench Cluster Plugin](./workbench.mdx). For the replacement Operator custom resources, see [Install Workbench](../installation/workbench.mdx) and [Install MLflow](../develop/components/mlflow/install.mdx). -#### Verifying the New Version +## Prepare the Global Cluster -Navigate to **Administrator > Marketplace > Upload Packages** and switch to the **Cluster Plugin** tab. Locate each uploaded plugin to verify that the new version is displayed. +Alauda AI 2.8.x uses the single-cluster application architecture. Global resources are no longer supplied by the old **Alauda AI Essentials** (`aml-global`) Cluster Plugin. -#### Troubleshooting +The following commands must run against the **global cluster**, not the cluster where the Alauda AI workloads run. Complete them after uninstalling the old **Alauda AI Essentials** Cluster Plugin. -##### New version not visible after upload (ACP 4.0.x) +### Run `global-install.sh` -On **Alauda Container Platform (ACP) 4.0.x**, the Web Console may fail to show a newly uploaded cluster plugin version even though the upload succeeded. This behavior is a known **ACP** limitation; it has been addressed in newer **ACP** releases. +Download the [`global-install.sh`](/global-install.sh) script supplied with the Alauda AI 2.8.x documentation or release package. On the global cluster, run it with the name of the cluster where Alauda AI is installed: -If the new version does not appear after waiting for synchronization, the following workaround can be used to set the target version on the corresponding `ModuleInfo` and trigger the plugin upgrade directly. +```bash +kubectl config use-context +bash global-install.sh +``` -The following `kubectl` commands must run on the **global** cluster. +The script creates or verifies the global `OAuth2Client`, OIDC Secret, and `ProductEntry` resources used by Alauda AI. Verify the resources after the script completes: -| Plugin (UI / product name) | Plugin Identifier | -|:---------------------------|:------------------| -| Alauda AI Workbench | `workbench` | -| Volcano | `volcano` | -| MLflow | `mlflow` | -| Kubeflow Training Operator | `kftraining` | +```bash +kubectl get oauth2client -n cpaas-system +kubectl get productentry +``` -1. Resolve the `ModuleInfo` resource name. Replace `` with the target cluster name. For ``, use the value in the **Plugin Identifier** column of the table above: +The **Alauda AI** entry should be available in the platform console after the platform synchronizes the new ProductEntry. - ```bash - kubectl get moduleinfoes \ - -l cpaas.io/cluster-name=,cpaas.io/module-name= \ - -o jsonpath='{.items[*].metadata.name}' - ``` +### Run `migrate-roles.sh` -2. Get `` from the `ModulePlugin` status, using the same `` value as in step 1 (see the table above): +Run the `migrate-roles.sh` script supplied with the Alauda AI 2.8.x release on the global cluster after the old Alauda AI Essentials plugin has been removed. - ```bash - kubectl get moduleplugins -o jsonpath='{.status.latestVersion}' - ``` +Use the command-line usage and arguments provided by the delivered script. The exact invocation is intentionally not included until the release script is added to the documentation package. Confirm that the script completes successfully and that the migrated namespace permissions are present before continuing. -3. Patch the `ModuleInfo`. Replace `` with the name from step 1 and `` with the value from step 2: +### Upgrade Alauda AI - ```bash - kubectl patch moduleinfoes --type merge -p '{"spec":{"version":""}}' - ``` +1. Log in to the Web Console and open the **Administrator** view. +2. Go to **Marketplace** > **OperatorHub**. +3. Select the target cluster. +4. Open **Alauda AI**. +5. Review the available version and select the Alauda AI 2.8.x version. +6. Confirm the upgrade and wait for the OperatorHub installation status to become `Installed`. - :::warning - The patch takes effect immediately and **directly triggers the cluster plugin upgrade** to ``, bypassing the Web Console upgrade action. Verify the version string before running the command. - ::: +During the upgrade, the existing cluster-scoped `AmlCluster` resource named `default` is upgraded automatically. The upgraded `AmlCluster` automatically enables the following components: -#### Upgrading from Web Console +- **PostgreSQL** +- **Alauda Cache Service for Redis OSS** +- **Alauda Build of Authorino** +- **Alauda Build of KServe** +- **Alauda Build of LeaderWorkerSet** -1. Navigate to **Administrator > Clusters > Clusters**. -2. Click on the cluster where the plugins are deployed. -3. Switch to the **Functional Components** tab in the cluster details page. -4. Locate the cluster plugins that have available upgrades—an upgrade icon will be displayed next to them. -5. Click the **Upgrade** button for the plugin you want to upgrade. -6. Review the upgrade information in the popup window, confirm the plugin version to be upgraded, and click the **Upgrade** button to start the upgrade process. +You can manually enable the following optional components in the `AmlCluster` configuration when they are required: -Repeat this process for each plugin that you have deployed. +- **MLflow Operator** +- **Alauda AI Workbench Operator** +- **Alauda Build of Serving Runtime** -:::info -Once the new version is uploaded and recognized by the platform, the **Upgrade** button will become available for each plugin on the **Features** tab of the cluster details page. -::: +For the `AmlCluster` configuration, registry settings, component options, and installation details, see [Install Alauda AI](../installation/ai-cluster.mdx). ## Post-Upgrade Operations -### Updating Existing Inference Services - -Due to breaking changes in KServe's product mode definition and the `InferenceService` resource introduced in version {curVer}, all inference services that existed **before the upgrade** must be manually updated. - -:::warning -This step is **required** for all pre-existing inference services. Failure to perform this update may cause inference services to behave unexpectedly. -::: - -For each existing inference service, perform the following steps: - -1. Navigate to the inference service details page. -2. Click **Update Inference Service**. -3. In the update page, click the **YAML** toggle button in the upper-right corner to switch to the YAML view. -4. Locate the `spec.predictor.model.name` field. -5. Delete the `name` field and its value entirely. - - For example, if the YAML contains: - - ```yaml - spec: - predictor: - model: - name: kserve-container #[!code --] - modelFormat: - name: sklearn - ``` - - After deletion, it should look like: +### Verify Alauda AI - ```yaml - spec: - predictor: - model: - modelFormat: - name: sklearn - ``` - -6. Click **Save** to apply the changes. -7. Repeat this process for all inference services that existed before the upgrade. - -## Verification - - -### Alauda AI -Check the status field from the `AmlCluster` resource named `default`: +Check the status of the cluster-scoped `AmlCluster` resource: ```bash kubectl get amlcluster default ``` -Should returns `Ready`: +The resource should be `Ready`: -``` -NAME READY REASON -default True Succeeded +```text +NAME PHASE READY REASON +default Ready True Reconciled ``` -### Alauda Build of KServe +### Verify KServe -Check the status field from the `KServe` resource named `default-kserve`: +If Alauda Build of KServe is installed, verify the KServe instance: ```bash kubectl get kserve default-kserve -n kserve-operator ``` -Should return `DEPLOYED: True`: +The instance should report `DEPLOYED: True`: -``` +```text NAME DEPLOYED REASON -default-kserve True UpgradeSuccessful + default-kserve True UpgradeSuccessful ``` -### Other Cluster Plugins +### Verify Operators and migrated components -In the **Administrator** view, navigate to **Marketplace > Cluster Plugins** and confirm that the following cluster plugins show `Installed` status with the new version: +Verify the OperatorHub and workload status for every installed dependency: -- Alauda AI Workbench (if deployed) -- MLflow (if deployed) -- Volcano (if deployed) +- Confirm each required component is `Installed` and its ClusterServiceVersion reports `Succeeded`. +- Confirm the PostgreSQL, Redis, and Authorino services required by enabled features are ready. +- Confirm the LWS controller is running and a representative distributed workload can start. +- Confirm the MLflow tracking server is ready and can access existing metadata and artifacts. +- Confirm the Workbench custom resource is ready, existing Workspaces and PVCs are present, and a test Workbench can be opened. +- Confirm existing KServe inference services and representative model requests work as expected. +- On the global cluster, confirm the `OAuth2Client` and `ProductEntry` resources created by `global-install.sh`, and verify that users can open Alauda AI through the platform entry. - +### Verify data preservation -## Deprecating Alauda AI Model Serving +Confirm that the following data is still available before declaring the upgrade complete: -Starting from the **Alauda AI 2.x** series, the legacy **Alauda AI Model Serving** operator is deprecated. We strongly recommend that users requiring serverless inference capabilities switch to the **Knative Operator** as soon as possible to ensure long-term support and access to the latest features. +- User PVCs and Workbench home directories. +- Existing Workspaces and retained `WorkspaceKind` resources. +- MLflow metadata and model artifacts. +- Existing inference-service configuration and model data. +- Distributed-workload configuration and checkpoints. -For guidance on how to move your serverless workloads to the new operator, please see the [Migrating to Knative Operator](./migrating-to-knative-operator.mdx) guide. +If a component does not reconcile successfully, stop further migration and use the component-specific backup and rollback procedure. Do not delete CRDs while custom resources still exist; deleting a CRD can also delete the custom resources stored in the cluster. diff --git a/docs/en/upgrade/workbench.mdx b/docs/en/upgrade/workbench.mdx index 92d1f07d..64b06c47 100644 --- a/docs/en/upgrade/workbench.mdx +++ b/docs/en/upgrade/workbench.mdx @@ -20,10 +20,10 @@ Starting with v0.2.0, Workbench is delivered as an OLM Helm operator instead of ``` 2. In **Administrator** > **Marketplace** > **Cluster Plugins**, uninstall the earlier **Workbench** cluster plugin. Preserve user PVCs, retained `WorkspaceKind` resources, and the `aml-workbench-config` `ConfigMap`; the operator adopts or reapplies these resources. -3. Install the **Alauda AI Workbench Operator** and create a `Workbench` custom resource as described in [Install Workbench](../installation/workbench.mdx). +3. Enable Workbench by setting `spec.components.workbench.managementState` to `Managed` in the `default` `AmlCluster`. Alauda AI installs and manages the Workbench Operator and its resources; do not create a `Workbench` custom resource manually. For the normal installation path, see [Install Workbench](../installation/workbench.mdx). 4. Verify the existing Workspaces and their PVCs are still present, then create and connect to a test Workbench. -If the previous installation relied on the Elyra KFP run-URL redirect, set `spec.global.istio.enabled: true` in the `Workbench` custom resource. Istio integration is optional and disabled by default. +If the previous installation relied on the Elyra KFP run-URL redirect, set `spec.components.workbench.values.global.istio.enabled: true` in the `default` `AmlCluster`. Istio integration is optional and disabled by default. ## Migrating from Kubeflow Notebook diff --git a/docs/public/migrate-roles.sh b/docs/public/migrate-roles.sh new file mode 100755 index 00000000..ddef4808 --- /dev/null +++ b/docs/public/migrate-roles.sh @@ -0,0 +1,400 @@ +#!/usr/bin/env bash + +set -euo pipefail + +# Run this script against the global cluster. It migrates the selected cluster's +# legacy UserBindings to namespace RoleBindings in the selected cluster. +# +# Required tools: kubectl, base64, sed, awk, and either shasum or sha256sum. +# The script intentionally does not require jq or yq. +# +# Environment variables: +# KUBECTL kubectl executable (default: kubectl) +# DRY_RUN print manifests without changing the target cluster (default: false) +# +# Command options: +# --keep-source keep source UserBindings after migration +# --backup-dir DIR directory for source UserBinding backups + +kubectl_bin="${KUBECTL:-kubectl}" +dry_run="${DRY_RUN:-false}" +delete_source="true" +backup_dir="" +source_namespace="cpaas-system" + +cluster_name="" +target_kubeconfig="" +declare -a migrated_sources=() + +auto_backup_dir() { + printf '%s/migrate-roles-backup' "$(pwd)" +} + +usage() { + cat <<'EOF' +Usage: + migrate-roles.sh [options] + +Migrates AML namespace UserBindings from the global cluster to namespace +RoleBindings in . + +By default source UserBindings are backed up and deleted after successful +migration. Use --keep-source to retain them. DRY_RUN=true prints manifests +without writing them or deleting source resources. + +Options: + --keep-source keep source UserBindings after migration + --backup-dir DIR directory for source UserBinding backups; when omitted, + ./migrate-roles-backup is created automatically + -h, --help show this help +EOF +} + +fail() { + printf 'error: %s\n' "$*" >&2 + exit 1 +} + +require_command() { + command -v "$1" >/dev/null 2>&1 || fail "$1 is required" +} + +trim() { + printf '%s' "$1" | sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//' +} + +base64_decode() { + if base64 --decode /dev/null 2>&1; then + base64 --decode + else + base64 -D + fi +} + +sha256_prefix() { + local namespace="$1" + local user="$2" + local cluster_role="$3" + + if command -v shasum >/dev/null 2>&1; then + printf '%s\0%s\0%s' "$namespace" "$user" "$cluster_role" \ + | shasum -a 256 | awk '{print substr($1, 1, 6)}' + elif command -v sha256sum >/dev/null 2>&1; then + printf '%s\0%s\0%s' "$namespace" "$user" "$cluster_role" \ + | sha256sum | awk '{print substr($1, 1, 6)}' + else + fail 'shasum or sha256sum is required' + fi +} + +global_kubectl() { + "$kubectl_bin" "$@" +} + +target_kubectl() { + "$kubectl_bin" --kubeconfig "$target_kubeconfig" "$@" +} + +safe_role_binding_user() { + local user="$1" + local safe + + safe="$(printf '%s' "$user" | tr '[:upper:]' '[:lower:]' | sed -E 's/[^a-z0-9]+/-/g; s/^-+//; s/-+$//')" + safe="${safe:0:32}" + safe="$(printf '%s' "$safe" | sed -E 's/-+$//')" + if [[ -z "$safe" ]]; then + safe="u" + fi + printf '%s' "$safe" +} + +role_binding_name() { + local role="$1" + local namespace="$2" + local user="$3" + local cluster_role="aml-namespace-${role}:namespaced-scope" + local safe_user + local suffix + + safe_user="$(safe_role_binding_user "$user")" + suffix="$(sha256_prefix "$namespace" "$user" "$cluster_role")" + printf 'aml-%s-%s-%s' "$role" "$safe_user" "$suffix" +} + +yaml_quote() { + local value="$1" + value="${value//\'/\'\'}" + printf "'%s'" "$value" +} + +role_binding_manifest() { + local name="$1" + local namespace="$2" + local role="$3" + local user="$4" + local cluster_role="aml-namespace-${role}:namespaced-scope" + + cat <"$target_kubeconfig"; then + fail "cannot decode ${source_namespace}/${cluster_name}-kubeconfig" + fi + chmod 600 "$target_kubeconfig" + trap 'rm -f "$target_kubeconfig"' EXIT + + target_kubectl version --request-timeout=15s >/dev/null 2>&1 \ + || fail "cannot connect to target cluster ${cluster_name}" +} + +backup_source_binding() { + local name="$1" + local backup_file="$backup_dir/${name}.yaml" + + global_kubectl get userbindings.auth.alauda.io "$name" -o yaml >"$backup_file" \ + || fail "cannot back up UserBinding ${name}" + printf 'backup: %s\n' "$backup_file" +} + +migrate_bindings() { + local records + local name + local role_ref + local cluster + local constraint_namespace + local label_namespace + local scope + local user + local role + local namespace + local binding_name + + records="$(global_kubectl get userbindings.auth.alauda.io \ + -o 'jsonpath={range .items[*]}{.metadata.name}{"\t"}{.spec.roleRef}{"\t"}{.spec.constraint[0].cluster}{"\t"}{.spec.constraint[0].namespace}{"\t"}{.metadata.labels.cpaas\.io/namespace}{"\t"}{.spec.scope}{"\t"}{.metadata.annotations.auth\.cpaas\.io/user\.email}{"\n"}{end}')" \ + || fail 'cannot list auth.alauda.io/v1 UserBindings from the global cluster' + + while IFS=$'\t' read -r name role_ref cluster constraint_namespace label_namespace scope user; do + [[ -z "$name" ]] && continue + + role_ref="$(trim "$role_ref")" + cluster="$(trim "$cluster")" + constraint_namespace="$(trim "$constraint_namespace")" + label_namespace="$(trim "$label_namespace")" + scope="$(trim "$scope")" + user="$(trim "$user")" + + case "$role_ref" in + aml-namespace-owner) role='owner' ;; + aml-namespace-editor) role='editor' ;; + aml-namespace-viewer) role='viewer' ;; + *) continue ;; + esac + + if [[ "$cluster" != "$cluster_name" ]]; then + continue + fi + if [[ -n "$scope" && "$scope" != 'namespace' ]]; then + printf 'warning: skip UserBinding %s with unsupported scope %s\n' "$name" "$scope" >&2 + continue + fi + + namespace="$constraint_namespace" + if [[ -z "$namespace" ]]; then + namespace="$label_namespace" + fi + if [[ -z "$namespace" || -z "$user" ]]; then + printf 'warning: skip incomplete UserBinding %s\n' "$name" >&2 + continue + fi + + binding_name="$(role_binding_name "$role" "$namespace" "$user")" + printf 'migrate: UserBinding %s -> %s/%s (%s, %s)\n' \ + "$name" "$namespace" "$binding_name" "$role" "$user" + create_role_binding "$binding_name" "$namespace" "$role" "$user" \ + || fail "cannot create RoleBinding ${namespace}/${binding_name} for UserBinding ${name}" + migrated_sources+=("$name") + done <<<"$records" +} + +backup_and_delete_migrated_sources() { + local name + + [[ "$delete_source" == 'true' ]] || return 0 + [[ "$dry_run" == 'true' ]] && { + printf 'dry-run: would back up and delete %d UserBinding(s)\n' "${#migrated_sources[@]}" + return 0 + } + + mkdir -p "$backup_dir" + [[ -d "$backup_dir" && -w "$backup_dir" ]] || fail "backup directory is not writable: $backup_dir" + printf 'backing up and deleting %d successfully migrated UserBinding(s)\n' "${#migrated_sources[@]}" + for name in "${migrated_sources[@]}"; do + backup_source_binding "$name" + global_kubectl delete userbindings.auth.alauda.io "$name" + done +} + +parse_args() { + local arg + + while (($# > 0)); do + arg="$1" + case "$arg" in + --keep-source) + delete_source='false' + shift + ;; + --backup-dir) + (($# >= 2)) || fail '--backup-dir requires a directory' + backup_dir="$2" + shift 2 + ;; + --backup-dir=*) + backup_dir="${arg#*=}" + shift + ;; + -h|--help) + usage + exit 0 + ;; + --) + shift + break + ;; + -* ) + fail "unknown option: $arg" + ;; + *) + [[ -z "$cluster_name" ]] || fail "unexpected argument: $arg" + cluster_name="$arg" + shift + ;; + esac + done + + if (($# > 0)); then + [[ -z "$cluster_name" ]] || fail "unexpected argument: $1" + cluster_name="$1" + shift + fi + [[ -n "$cluster_name" ]] || { usage >&2; exit 1; } +} + +main() { + parse_args "$@" + + require_command "$kubectl_bin" + require_command base64 + require_command sed + require_command awk + require_command tr + if ! command -v shasum >/dev/null 2>&1 && ! command -v sha256sum >/dev/null 2>&1; then + fail 'shasum or sha256sum is required' + fi + + [[ "$dry_run" == 'true' || "$dry_run" == 'false' ]] \ + || fail 'DRY_RUN must be true or false' + if [[ "$delete_source" == 'true' && -z "$backup_dir" ]]; then + backup_dir="$(auto_backup_dir)" + fi + if [[ "$delete_source" == 'false' && -n "$backup_dir" ]]; then + printf 'warning: --backup-dir is ignored with --keep-source\n' >&2 + fi + + load_target_kubeconfig + migrate_bindings + backup_and_delete_migrated_sources +} + +main "$@" diff --git a/llms.txt b/llms.txt index 696fd8a7..4836c9e7 100644 --- a/llms.txt +++ b/llms.txt @@ -31,11 +31,11 @@ - [docs/en/ai_applications/components/kagenti/security_architecture.mdx](docs/en/ai_applications/components/kagenti/security_architecture.mdx): Reference for the Kagenti Operator zero-trust security stack: SPIFFE/SPIRE workload identity, Istio ambient mesh transport security, and OPA plus AuthBridge plus Keycloak authorization with an example global Rego policy. Documents the AuthBridge sidecar, the AgentRuntime security fields, and how to enable the opt-in secure profile. - [docs/en/ai_applications/components/llama_stack/features.mdx](docs/en/ai_applications/components/llama_stack/features.mdx): Enumerates Llama Stack's main capabilities: a centralized server hosting inference/agents/safety/tool runtime/vector I/O/files, remote and inline providers (meta-reference, sqlite-vec, localfs), Kubernetes deployment via `LlamaStackDistribution`, `@client_tool` decorator-based agent creation with streaming sessions, YAML stack configuration with `${env.VAR:~default}` fallbacks, multiple distributions (starter, postgres-demo, meta-reference-gpu), and the `llama-stack-client` Python 3.12+ SDK including PGVector-backed vector store APIs. - [docs/en/ai_applications/components/llama_stack/index.mdx](docs/en/ai_applications/components/llama_stack/index.mdx): Component landing page for Alauda Build of Llama Stack, linking its introduction, installation guide, feature reference, and quickstart. -- [docs/en/ai_applications/components/llama_stack/install.mdx](docs/en/ai_applications/components/llama_stack/install.mdx): Procedure for installing the Llama Stack Operator via Operator Hub and deploying a Llama Stack Server through a `LlamaStackDistribution` custom resource (apiVersion `llamastack.io/v1alpha1`), covering required `VLLM_URL` configuration pointing at a vLLM OpenAI-compatible endpoint, optional `VLLM_API_TOKEN` Secret, PGVector-backed vector stores via ACP PostgreSQL, Hugging Face mirror/offline embedding-model caching (`HF_ENDPOINT`, `HF_HUB_OFFLINE`), and vLLM predictor tool-calling flags (`--enable-auto-tool-choice`, `--tool-call-parser hermes`) needed for agent flows that use client-side or MCP tools. +- [docs/en/ai_applications/components/llama_stack/install.mdx](docs/en/ai_applications/components/llama_stack/install.mdx): Installs Alauda Build of Llama Stack by uploading the Llama Stack Operator package and setting `spec.components.llamaStack.managementState` to `Managed` in the `default` `AmlCluster`; the operator registers the `LlamaStackDistribution` CRD. Then deploys a Llama Stack Server through a `LlamaStackDistribution` custom resource (apiVersion `llamastack.io/v1alpha1`), covering required `VLLM_URL` configuration pointing at a vLLM OpenAI-compatible endpoint, optional `VLLM_API_TOKEN` Secret, PostgreSQL persistence, PGVector-backed vector stores via ACP PostgreSQL, Milvus remote vector stores, Hugging Face mirror/offline embedding-model caching (`HF_ENDPOINT`, `HF_HUB_OFFLINE`), and vLLM predictor tool-calling flags (`--enable-auto-tool-choice`, `--tool-call-parser hermes`) needed for agent flows that use client-side or MCP tools. - [docs/en/ai_applications/components/llama_stack/intro.mdx](docs/en/ai_applications/components/llama_stack/intro.mdx): Introduces Llama Stack as a framework for building AI agents with tools, explaining its core building blocks: the Llama Stack Server deployed via Operator on Kubernetes, the `llama-stack-client` Python SDK, agents and `@client_tool`-decorated tools, YAML configuration registering providers (inference, agents, safety, vector_io, files) and models such as DeepSeek over OpenAI-compatible APIs, and links to upstream documentation at llamastack.github.io. - [docs/en/ai_applications/components/llama_stack/quickstart.mdx](docs/en/ai_applications/components/llama_stack/quickstart.mdx): Walks through running the `llama-stack_quickstart.ipynb` notebook against a deployed Llama Stack Server, demonstrating both `@client_tool` client-side tools and FastMCP-based MCP tools registered with `toolgroups.register`, plus an optional PGVector flow that uploads a file via `client.files.create`, creates a `provider_id="pgvector"` vector store, and runs hybrid search with `search_mode="hybrid"`; also includes an FAQ for installing a Python 3.12 ipykernel from `python-build-standalone` so notebooks can use `llama-stack-client==0.6.0`. - [docs/en/ai_applications/components/mcp_lifecycle_operator/index.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/index.mdx): Component landing page for Alauda Build of MCP Lifecycle Operator, linking its introduction, installation guide, and quickstart. -- [docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx): Installs Alauda Build of MCP Lifecycle Operator as an OLM operator from the platform OperatorHub. Covers publishing the package with the violet command-line tool, installing the operator, verifying that the CSV reports Succeeded, the controller pod is Running, and the MCPServer CRD is registered, and upgrading the operator. +- [docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx): Installs Alauda Build of MCP Lifecycle Operator as an OLM operator from the platform OperatorHub. Covers uploading the operator package, installing the operator, verifying that the CSV reports Succeeded, the controller pod is Running, and the MCPServer CRD is registered, and upgrading the operator. - [docs/en/ai_applications/components/mcp_lifecycle_operator/intro.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/intro.mdx): Introduction to Alauda Build of MCP Lifecycle Operator, based on the Kubernetes SIG Apps MCP Lifecycle Operator. Describes the `MCPServer` custom resource in `mcp.x-k8s.io/v1alpha1` that declares an MCP server image, port, endpoint path, environment, configuration mounts, resources, and security settings, plus key features, how it works, and use cases. - [docs/en/ai_applications/components/mcp_lifecycle_operator/quickstart.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/quickstart.mdx): Quickstart that creates an `MCPServer` resource and verifies that the operator produces the backing Deployment and Service. Covers prerequisites, creating and verifying the resource, testing the MCP endpoint, updating the port or path, common configuration options, and cleanup. - [docs/en/ai_applications/demo_advanced_secure_profile.mdx](docs/en/ai_applications/demo_advanced_secure_profile.mdx): Advanced variant of the weather agent demo in which AuthBridge is injected on both the agent and the tool, and every agent-to-tool call performs an OAuth 2.0 RFC 8693 token exchange so the tool receives an audience-scoped token. Covers the ConfigMaps involved, the Keycloak client-id convention, deploying the tool with a sidecar, configuring token exchange, preparing outbound routes, and deploying the secured agent. @@ -63,18 +63,15 @@ ## deploy - [docs/en/deploy/components/envoy_ai_gateway/index.mdx](docs/en/deploy/components/envoy_ai_gateway/index.mdx): Component landing page for Alauda Build of Envoy AI Gateway, linking its introduction and installation guide. -- [docs/en/deploy/components/envoy_ai_gateway/install.mdx](docs/en/deploy/components/envoy_ai_gateway/install.mdx): Procedure for deploying the Alauda Build of Envoy AI Gateway cluster plugin: obtain the package from the Customer Portal, upload it to ACP per the CLI tools guide, then install via Administrator -> Marketplace -> Cluster Plugin against the target cluster. Verifies success with kubectl get pods -n envoy-gateway-system | grep ai-gateway and describes upgrades through Clusters -> Functional Components -> Upgrade. -- [docs/en/deploy/components/envoy_ai_gateway/intro.mdx](docs/en/deploy/components/envoy_ai_gateway/intro.mdx): Introduction to Alauda Build of Envoy AI Gateway, a Kubernetes-native AI gateway layer built on Envoy Gateway that provides AI-aware routing by model name and backend availability, an OpenAI-compatible API, and policy enforcement for inference workloads. Links the gateway guides and upstream documentation. +- [docs/en/deploy/components/envoy_ai_gateway/intro.mdx](docs/en/deploy/components/envoy_ai_gateway/intro.mdx): Introduction to Alauda Build of Envoy AI Gateway, a Kubernetes-native AI gateway layer built on Envoy Gateway that provides AI-aware routing by model name and backend availability, an OpenAI-compatible API, and policy enforcement for inference workloads. Links the gateway guides and upstream documentation. Managed by Alauda AI through the `envoyAIGateway` component of the `default` `AmlCluster`. - [docs/en/deploy/components/index.mdx](docs/en/deploy/components/index.mdx): Section index for the components the Deploy chapter relies on, gathering the introduction and installation guides for KServe, Envoy AI Gateway, LeaderWorkerSet, and InferNex Bridge. - [docs/en/deploy/components/infernex_bridge/index.mdx](docs/en/deploy/components/infernex_bridge/index.mdx): Component landing page for Alauda Build of InferNex Bridge, linking its introduction and installation guide. - [docs/en/deploy/components/infernex_bridge/install.mdx](docs/en/deploy/components/infernex_bridge/install.mdx): Installs Alauda Build of InferNex Bridge v26.6.0 from the OLM alpha channel on arm64 clusters. Covers required platform and inference dependencies, the CRDs the operator installs, runtime templates and images, optional dependencies including EagleEye observability, uploading and installing the operator, verification, community examples, and the upgrade and rollback procedures. - [docs/en/deploy/components/infernex_bridge/intro.mdx](docs/en/deploy/components/infernex_bridge/intro.mdx): Introduction to Alauda Build of InferNex Bridge, based on the openFuyao InferNex project. Connects KServe `LLMInferenceService` workloads with the InferNex inference acceleration stack and provides native `InferNexService` APIs for clusters that do not use KServe. Describes both deployment modes, supported inference engine topologies, and capabilities. - [docs/en/deploy/components/kserve/index.mdx](docs/en/deploy/components/kserve/index.mdx): Component landing page for Alauda Build of KServe, linking its introduction and installation guide. -- [docs/en/deploy/components/kserve/install.mdx](docs/en/deploy/components/kserve/install.mdx): End-to-end install of **Alauda Build of KServe** on ACP: lists required dependencies (Envoy Gateway Operator, Envoy AI Gateway, LeaderWorkerSet cluster plugins, and bundled GIE), uploads the `kserve-operator.ALL.xxxx.tgz` via `violet push`, installs through OperatorHub, then creates a `KServe` CR (`components.aml.dev/v1alpha1`) named `default-kserve` configured with `clusterName`, `deployFlavor`, `platformAddress`, registry address, ingress `domain`, and `kserve.controller.deploymentMode` (`Knative` for scale-to-zero or `Standard`). Documents preset configuration tables for `envoy_gateway`, `envoy_ai_gateway`, `kserve_gateway`, and `GIE` (with `gie.builtIn` to disable bundled GIE), plus verification via `kubectl get kserve default-kserve -n kserve-operator` showing `DEPLOYED: True`, and the upgrade path. -- [docs/en/deploy/components/kserve/intro.mdx](docs/en/deploy/components/kserve/intro.mdx): Introduces **Alauda Build of KServe** as Alauda's distribution of upstream KServe, splitting coverage into Generative AI features (llm-d distributed inference with KV-cache-aware scheduling and Leader/Worker multi-node parallelism, vLLM runtime with PagedAttention, OpenAI-compatible `/chat/completions` streaming, autoscaling including scale-to-zero, and Envoy Gateway + GIE integration) and Predictive AI features (the `InferenceService` CRD with canary rollouts, pre-integrated runtimes like TensorFlow Serving, TorchServe, Triton, SKLearn, XGBoost, `ClusterServingRuntime`/`ServingRuntime` for custom runtimes, and `InferenceGraph` for pipeline composition). Includes upstream documentation links for KServe, llm-d, LeaderWorkerSet, Envoy Gateway, Envoy AI Gateway, and Gateway API Inference Extension. +- [docs/en/deploy/components/kserve/intro.mdx](docs/en/deploy/components/kserve/intro.mdx): Introduces **Alauda Build of KServe** as Alauda's distribution of upstream KServe, splitting coverage into Generative AI features (llm-d distributed inference with KV-cache-aware scheduling and Leader/Worker multi-node parallelism, vLLM runtime with PagedAttention, OpenAI-compatible `/chat/completions` streaming, autoscaling including scale-to-zero, and Envoy Gateway + GIE integration) and Predictive AI features (the `InferenceService` CRD with canary rollouts, pre-integrated runtimes like TensorFlow Serving, TorchServe, Triton, SKLearn, XGBoost, `ClusterServingRuntime`/`ServingRuntime` for custom runtimes, and `InferenceGraph` for pipeline composition). Includes upstream documentation links for KServe, llm-d, LeaderWorkerSet, Envoy Gateway, Envoy AI Gateway, and Gateway API Inference Extension. Managed by Alauda AI through the `kserve` component of the `default` `AmlCluster`; see Install Alauda AI for enabling it and the KServe service configuration. - [docs/en/deploy/components/lws/index.mdx](docs/en/deploy/components/lws/index.mdx): Component landing page for Alauda Build of LeaderWorkerSet, linking its introduction and installation guide. -- [docs/en/deploy/components/lws/install.mdx](docs/en/deploy/components/lws/install.mdx): Installation procedure for the Alauda Build of LeaderWorkerSet cluster plugin obtained from the Customer Portal: upload through the ACP cluster plugin tooling, deploy from `Administrator > Marketplace > Cluster Plugin`, verify with `kubectl get pods -n cpaas-system | grep lws`, and upgrade via the cluster's Functional Components page. -- [docs/en/deploy/components/lws/intro.mdx](docs/en/deploy/components/lws/intro.mdx): Introduces Alauda Build of LeaderWorkerSet, a packaging of the upstream Kubernetes SIG `lws` project that provides a `LeaderWorkerSet` CRD for deploying groups of one leader and N worker pods as a unit; highlights co-scheduling with topology spread constraints for NVLink/InfiniBand interconnects, multi-node LLM inference for models like Llama 3.1 405B using tensor/pipeline parallelism (required by Alauda Build of KServe), distributed training with PyTorch DDP/DeepSpeed/Megatron-LM, group-level rolling updates and failure recovery, and leader-first startup sequencing. +- [docs/en/deploy/components/lws/intro.mdx](docs/en/deploy/components/lws/intro.mdx): Introduces Alauda Build of LeaderWorkerSet, a packaging of the upstream Kubernetes SIG `lws` project that provides a `LeaderWorkerSet` CRD for deploying groups of one leader and N worker pods as a unit; highlights co-scheduling with topology spread constraints for NVLink/InfiniBand interconnects, multi-node LLM inference for models like Llama 3.1 405B using tensor/pipeline parallelism (required by Alauda Build of KServe), distributed training with PyTorch DDP/DeepSpeed/Megatron-LM, group-level rolling updates and failure recovery, and leader-first startup sequencing. Managed by Alauda AI through the `lws` component of the `default` `AmlCluster`. - [docs/en/deploy/gateway/cost_management_chargeback.mdx](docs/en/deploy/gateway/cost_management_chargeback.mdx): How-to guide that defines a custom cost model for AI Gateway token usage and turns it into per-namespace bills through Alauda Cost Management. Adds a collection configuration on the agent cluster, a display and storage configuration on the server cluster, and a price in the cost model, consuming the OpenTelemetry GenAI token metric keyed on the `user_namespace` label. - [docs/en/deploy/gateway/external_provider_routing.mdx](docs/en/deploy/gateway/external_provider_routing.mdx): How-to guide for fronting external LLM providers such as OpenAI, AWS Bedrock, Azure OpenAI, GCP Vertex AI, and Anthropic behind one OpenAI-compatible endpoint. Injects upstream credentials with a `BackendSecurityPolicy`, routes by model name, and fails over between providers so consumers never hold provider keys and the gateway becomes the controlled egress point. - [docs/en/deploy/gateway/identity_authentication.mdx](docs/en/deploy/gateway/identity_authentication.mdx): How-to guide for authenticating inference requests at the Envoy AI Gateway edge with an Envoy Gateway `SecurityPolicy` attached to the `HTTPRoute` generated by an `AIGatewayRoute`. Copies selected claims into request headers that token quotas and usage metering consume as the per-tenant key, turning an SSO token or API key into an identity for multi-tenant policy. @@ -100,7 +97,9 @@ - [docs/en/deploy/inference_service/troubleshooting/index.mdx](docs/en/deploy/inference_service/troubleshooting/index.mdx): Section landing page that renders an `` index of inference-service troubleshooting topics (currently MLServer timeout diagnostics). - [docs/en/deploy/inference_service/troubleshooting/infer_timeout.mdx](docs/en/deploy/inference_service/troubleshooting/infer_timeout.mdx): Diagnoses inference-experience `502 Bad Gateway` timeouts on the Seldon MLServer runtime, attributing them to insufficient compute, overly long generated outputs, or MLServer's non-streaming response behavior that blocks until the full inference completes. Recommends upgrading CPU/GPU/memory, capping `max_new_tokens` at request time, and reducing model size/input complexity via quantization, pruning, or input preprocessing to bring response times back under the gateway's timeout window. - [docs/en/deploy/inference_service/troubleshooting/pod_security_admission_violation.mdx](docs/en/deploy/inference_service/troubleshooting/pod_security_admission_violation.mdx): Troubleshoots inference services stuck out of Running state on Alauda Container Platform when the Deployment exists but no Pod is created and FailedCreate events cite Pod Security Admission baseline violations such as hostIPC=true. Walks through inspecting the runtime for privileged requirements like host namespaces, removing them when possible, or otherwise raising the namespace's Enforce/Audit/Warn Pod Security Standard to Privileged via the Project view's Update Pod Security Admission action. -- [docs/en/deploy/maas.mdx](docs/en/deploy/maas.mdx): Describes Model as a Service, the managed model-serving gateway for Alauda AI built on Alauda Build of Envoy Gateway and Envoy AI Gateway with Authorino for authentication. Covers model sources, the administrator experience of publishing models, managing access, configuring token quotas, and monitoring usage, and the developer experience of calling subscribed models with API keys. +- [docs/en/deploy/maas/intro.mdx](docs/en/deploy/maas/intro.mdx): Overview of Model as a Service (MaaS), a managed OpenAI-compatible model-serving gateway built on Alauda Build of Envoy Gateway, Envoy AI Gateway, and Authorino. Introduces the MaaS value proposition, request flow, core concepts, roles, model sources, and links to role-specific guides. +- [docs/en/deploy/maas/administrator.mdx](docs/en/deploy/maas/administrator.mdx): Administrator guide for MaaS. Covers publishing platform `LLMInferenceService` models, importing external model services and credentials, publishing external model names, creating subscriptions and per-model token quotas, configuring guardrails, and reviewing usage reports with CSV export. +- [docs/en/deploy/maas/user.mdx](docs/en/deploy/maas/user.mdx): Developer and end-user guide for MaaS. Covers opening My Subscriptions, finding accessible models and endpoints, creating API keys with one-time plaintext display, making OpenAI-compatible curl and Python requests, understanding per-user quotas, viewing personal usage, interpreting 401/403/429 errors, and securing keys. - [docs/en/deploy/model_compression/compressor_by_workbench.mdx](docs/en/deploy/model_compression/compressor_by_workbench.mdx): Guide to running LLM Compressor model compression workflows on Alauda AI from a workbench. Covers the data-free and calibration-dataset example notebooks, creating a workbench and model repository, uploading models and optional datasets including from S3-compatible object storage, cloning them in JupyterLab, running the compression notebooks, uploading the compressed model, and deploying it for inference. - [docs/en/deploy/model_compression/index.mdx](docs/en/deploy/model_compression/index.mdx): Top-level landing page for the LLM Compressor section, embedding `` to surface the introduction and how-to subsections. - [docs/en/deploy/model_compression/intro.mdx](docs/en/deploy/model_compression/intro.mdx): Introduces LLM Compressor as the vLLM project's open-source model-compression library that supports quantization (int8, W4A16 weight-only, W8A8 weight+activation), sparsity, and file-size compression with native Hugging Face and vLLM integration; lists supported algorithms AWQ, GPTQ, FP8 dynamic per-token, SparseGPT, and SmoothQuant, and explains how computed scales/zero-points (per-tensor, channel, group, or token) produce compressed models suitable for resource-limited deployment. @@ -120,25 +119,25 @@ - [docs/en/develop/components/data_science_pipelines/install.mdx](docs/en/develop/components/data_science_pipelines/install.mdx): Installs the Data Science Pipelines Operator as an OLM operator from the platform OperatorHub. Lists prerequisites including ACP v4.0 or later, amd64 or arm64 clusters, OLM, mutual exclusion with the shared `kfp-operator` install, and the Argo Workflow CRDs; covers uploading and installing the operator, the optional external endpoint provider, and verifying the CSV, controller pod, and CRDs. - [docs/en/develop/components/data_science_pipelines/intro.mdx](docs/en/develop/components/data_science_pipelines/intro.mdx): Introduction to the Data Science Pipelines Operator, a Kubernetes-native operator built on the OpenDataHub project that deploys and manages Kubeflow Pipelines v2 stacks. Each data-science project runs its own isolated pipeline runtime declared through a `DataSciencePipelinesApplication` custom resource. Covers what it deploys, key concepts, mutual exclusion with the shared Kubeflow Pipelines install, and when to use it. - [docs/en/develop/components/feast/index.mdx](docs/en/develop/components/feast/index.mdx): Component landing page for Alauda Build of Feast, linking its introduction, installation guide, and quickstart. -- [docs/en/develop/components/feast/install.mdx](docs/en/develop/components/feast/install.mdx): Walks through installing the Feast Operator on Alauda Container Platform: upload the feast-operator.ALL.xxxx.tgz package with violet push, install Alauda Build of Feast from Marketplace -> Operator Hub against the target cluster, then verify the controller pod in feast-operator-system and the featurestores.feast.dev CRD with kubectl. Notes that installation only provisions the controller and CRDs, and that Feast services come up after a FeatureStore custom resource is applied per the Quickstart. +- [docs/en/develop/components/feast/install.mdx](docs/en/develop/components/feast/install.mdx): Installs Alauda Build of Feast by uploading the Feast Operator package and setting the `feast` component to `Managed` in the `default` `AmlCluster`; the operator registers the `featurestores.feast.dev` CRD. After the component is reconciled, a `FeatureStore` custom resource deploys Feast services per the Quickstart. - [docs/en/develop/components/feast/intro.mdx](docs/en/develop/components/feast/intro.mdx): Introduces Feast as an open-source feature store deployed on Kubernetes through the Feast Operator, where a FeatureStore CR manages the offline store, online store, registry, UI, and a generated client feature_store.yaml ConfigMap. Defines core concepts (Project via spec.feastProject, Entity, Data Source, Feature View, Feature Service, Registry, Materialization, Push Source, Permission), outlines the typical workflow from feast apply through materialization to client reads, and describes Operator responsibilities such as PVC provisioning, Git-based feature repo init, and Kubernetes Role creation for RoleBasedPolicy authorization. - [docs/en/develop/components/feast/quickstart.mdx](docs/en/develop/components/feast/quickstart.mdx): End-to-end guide for configuring and deploying a FeatureStore custom resource after the Feast Operator is installed, covering the main spec fields (feastProject, feastProjectDir, services.offlineStore/onlineStore/registry/ui, authz, replicas) and runnable YAML for persistence patterns: PVC-backed DuckDB plus local file registry, Redis online store with SQL registry on PostgreSQL 16, PostgreSQL-only online plus registry, and registry files in S3/GCS. Also documents Secret layout per backend type (redis, postgres, sql keys), feature repository initialization via feast init/init template/Git, deploying with kubectl apply and watching status.phase, the feast--online/offline/registry/ui Service names and clientConfigMap, using the Feast CLI/Python SDK with feast apply, and Kubernetes-based authorization combining feast-reader/feast-writer roles, Feast Permission objects with RoleBasedPolicy, RoleBindings, ServiceAccount tokens, and SDK token configuration via authz_config.user_token or LOCAL_K8S_TOKEN. - [docs/en/develop/components/index.mdx](docs/en/develop/components/index.mdx): Section index for the components the Develop chapter relies on, gathering the introduction and installation guides for Kubeflow, Data Science Pipelines, MLflow, KubeRay, Spark Operator, Label Studio, and Feast. - [docs/en/develop/components/kubeflow/faq.mdx](docs/en/develop/components/kubeflow/faq.mdx): Troubleshooting FAQ for Kubeflow on Alauda AI covering four scenarios: relabeling namespaces from PSA `restricted` to `baseline` to unblock notebooks and pipeline runs; overriding the `oidcAuthURL` in the `kfbase` `ModuleInfo` resource and restarting `oauth2-proxy` when login must use an alternative platform address; creating a `kfp-launcher` ConfigMap to point Kubeflow Pipelines runs at external S3/MinIO instead of the in-cluster `minio-service.kubeflow:9000`; adding custom GPU vendors (e.g. `your-custom.com/gpu`, Ascend) to `jupyter-web-app-config`; and fixing `CrashLoopBackOff`/probe timeouts on kube-ovn CNI by adding an `allow-kubelet-probes` NetworkPolicy that permits ingress from the kube-ovn join subnet CIDR (typically `100.64.0.0/16`). - [docs/en/develop/components/kubeflow/index.mdx](docs/en/develop/components/kubeflow/index.mdx): Component landing page for Alauda support for Kubeflow, linking its introduction, installation guide, upgrade guide, and FAQ. -- [docs/en/develop/components/kubeflow/install.mdx](docs/en/develop/components/kubeflow/install.mdx): Step-by-step deployment guide for the kfbase, model-registry-operator, kfp, kftraining (deprecated), and kubeflow-trainer cluster plugins on Alauda AI 2.0, including prerequisites like ASM v2, LWS, and Alauda Build of KServe. Covers configuring Dex redirection via Platform Access URLs, wiring oauth2-proxy through ASM extensionProviders (envoyExtAuthzHttp) for both ASM v1 and v2, uploading packages with violet, creating Kubeflow Profile resources to bind users to namespaces such as kubeflow-admin-cpaas-io, and installing the Model Registry operator with MySQL storage settings. +- [docs/en/develop/components/kubeflow/install.mdx](docs/en/develop/components/kubeflow/install.mdx): Deployment guide for the Kubeflow OLM operators on Alauda AI 2.3+: `kfbase-operator`, `kfp-operator`, `kubeflow-trainer-operator` (replacing the deprecated `kftraining` plugin), and `model-registry-operator`. Covers prerequisites including ASM v2, LWS, and Alauda Build of KServe, configuring Dex redirection via Platform Access URLs, wiring oauth2-proxy through ASM extensionProviders (envoyExtAuthzHttp), downloading and uploading the operator bundles, creating Kubeflow Profile resources to bind users to namespaces, and installing the Model Registry operator with MySQL storage settings. - [docs/en/develop/components/kubeflow/intro.mdx](docs/en/develop/components/kubeflow/intro.mdx): Brief introduction positioning Alauda's Kubeflow integration as a Kubernetes-native ML platform combining Kubeflow Pipelines for workflow orchestration, Kubeflow Training for jobs, and Model Registry for versioning, and notes that namespaces must have Pod Security Admission set to privileged for Kubeflow components to function. - [docs/en/develop/components/kubeflow/upgrade.mdx](docs/en/develop/components/kubeflow/upgrade.mdx): Captures manual post-upgrade actions for the kfbase plugin, including the switch from NodePort to gateway-based dashboard access for upgrades from v1.10.13 or earlier (requiring DNS or hosts updates pointing kubeflowDomain to the kubeflow-external-gateway IP), instructions for re-enabling NodePort by editing the kubeflow-istio-ingressgateway service, and the v1.10.10 requirement to set a default StorageClass for the pgStorageClass parameter when upgrading from v1.10.9 or earlier. - [docs/en/develop/components/kuberay/index.mdx](docs/en/develop/components/kuberay/index.mdx): Component landing page for Alauda Build of KubeRay Operator, linking its introduction and installation guide. -- [docs/en/develop/components/kuberay/install.mdx](docs/en/develop/components/kuberay/install.mdx): Installation guide for the Alauda Build of KubeRay Operator cluster plugin on ACP v4.0 or later, covering downloading the package from the Customer Portal, uploading it with the violet CLI tool, deploying it from Administrator > Marketplace > Cluster Plugin to the target cluster, and verifying with kubectl get pods -n cpaas-system | grep kuberay-operator. +- [docs/en/develop/components/kuberay/install.mdx](docs/en/develop/components/kuberay/install.mdx): Installation guide for the Alauda Build of KubeRay Operator cluster plugin on ACP v4.0 or later, covering downloading the package from the Customer Portal and uploading it following the shared Upload Packages guide, deploying it from Administrator > Marketplace > Cluster Plugin to the target cluster, and verifying with kubectl get pods -n cpaas-system | grep kuberay-operator. - [docs/en/develop/components/kuberay/intro.mdx](docs/en/develop/components/kuberay/intro.mdx): Introduces the Alauda Build of KubeRay Operator as a Kubernetes-native operator built on the open-source KubeRay project for running Ray on Kubernetes. Describes the three core CRDs (RayCluster for lifecycle and autoscaling, RayJob for auto-provisioned job execution with cleanup, RayService for zero-downtime Ray Serve deployments), key features like heterogeneous compute, fault tolerance, and ecosystem integration with Kueue/Volcano/Prometheus, and use cases spanning distributed training, batch inference, hyperparameter tuning with Ray Tune, and LLM serving. - [docs/en/develop/components/label_studio/features.mdx](docs/en/develop/components/label_studio/features.mdx): Lists Label Studio's main feature areas: multi-user annotation with user management, collaborative labeling, task assignment, and quality control; multi-type data support spanning image (classification, object detection, semantic segmentation), text, audio, video, time-series, and multi-modal data; XML-based annotation configuration with template library and built-in tools; JSON/CSV data import/export with batch operations; and machine learning integration via ML backend, pre-annotation, REST API, and Python SDK. - [docs/en/develop/components/label_studio/index.mdx](docs/en/develop/components/label_studio/index.mdx): Component landing page for Label Studio, linking its introduction, installation guide, feature reference, and quickstart. -- [docs/en/develop/components/label_studio/install.mdx](docs/en/develop/components/label_studio/install.mdx): Step-by-step guide for deploying Label Studio to a Kubernetes cluster: push the `label-studio.ALL.xxxx.tgz` package with `violet push`, prepare a CSI/`PersistentVolume`-backed StorageClass and a PostgreSQL 13+ cluster (and optional standalone-mode Redis from Data Services), then install the `3rdparty/chart-label-studio` Catalog application. Covers custom values for persistence sizing, `pgConfig`/`redisConfig` with SSL secrets, Service type and Ingress with `LABEL_STUDIO_HOST`, OAuth2 Proxy with ACP Dex as OIDC provider (via an `OAuth2Client` CR), and disabling open registration with `LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK`. +- [docs/en/develop/components/label_studio/install.mdx](docs/en/develop/components/label_studio/install.mdx): Step-by-step guide for deploying Label Studio to a Kubernetes cluster: download and upload the `label-studio.ALL.xxxx.tgz` package, prepare a CSI/`PersistentVolume`-backed StorageClass and a PostgreSQL 13+ cluster (and optional standalone-mode Redis from Data Services), then install the `3rdparty/chart-label-studio` Catalog application. Covers custom values for persistence sizing, `pgConfig`/`redisConfig` with SSL secrets, Service type and Ingress with `LABEL_STUDIO_HOST`, OAuth2 Proxy with ACP Dex as OIDC provider (via an `OAuth2Client` CR), and disabling open registration with `LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK`. - [docs/en/develop/components/label_studio/intro.mdx](docs/en/develop/components/label_studio/intro.mdx): Introduces Label Studio as an open-source multi-type data labeling tool with a Django/Python REST backend, React frontend, PostgreSQL 13+ storage, and optional Redis cache. Explains the core concepts (Project, Labeling Interface, Data Manager, Annotations, Machine Learning Integration), supported data types (image, audio, text, time series, video), cloud storage import from AWS S3 and Google Cloud Storage, and capabilities like pre-annotation, online/active learning, and model comparison. - [docs/en/develop/components/label_studio/quickstart.mdx](docs/en/develop/components/label_studio/quickstart.mdx): Walks users through integrating Label Studio with S3-compatible object storage (Amazon S3, Ceph RGW) for data import and annotation export. Covers configuring Source/Target Cloud Storage under `Settings > Cloud Storage`, fields like Bucket Name, S3 Endpoint, Access Key ID/Secret, Bucket Prefix, File Filter Regex, pre-signed URL expiration, and SSE KMS Key ID; syncing buckets, exporting JSON annotations, and using the Label Studio SDK converter to transform them into COCO, Pascal VOC, YOLO, or CSV for downstream model training pipelines. - [docs/en/develop/components/mlflow/index.mdx](docs/en/develop/components/mlflow/index.mdx): Component landing page for MLflow, linking its introduction and installation guide. -- [docs/en/develop/components/mlflow/install.mdx](docs/en/develop/components/mlflow/install.mdx): Installs MLflow as an OLM operator from the platform OperatorHub and then creates one `MLflow` custom resource that reconciles the tracking-server stack. Lists prerequisites including Alauda AI, OLM, and a PostgreSQL 12 or later database; covers both the kubectl and Alauda Console UI methods, verification of the operand, Deployment, and pod, and high availability and storage options. +- [docs/en/develop/components/mlflow/install.mdx](docs/en/develop/components/mlflow/install.mdx): Installs MLflow in Alauda AI 2.8.x by uploading the MLflow Operator package and setting `spec.components.mlflow.managementState` to `Managed` in the `default` `AmlCluster`; enabling the component installs the MLflow Operator only. Then creates one `MLflow` custom resource that reconciles the tracking-server stack. Lists prerequisites including Alauda AI, OLM, and a PostgreSQL 12 or later database; covers both the kubectl and Alauda Console UI methods, verification of the operand, Deployment, and pod, and high availability and storage options. - [docs/en/develop/components/mlflow/intro.mdx](docs/en/develop/components/mlflow/intro.mdx): Introduction to the MLflow Operator, which deploys and manages a multi-tenant MLflow Tracking Server integrated with platform single sign-on and Kubernetes RBAC so each team sees only its own runs, experiments, and registered models. Covers what it deploys, key concepts, the multi-tenancy model, and when to use it. - [docs/en/develop/components/spark_operator/index.mdx](docs/en/develop/components/spark_operator/index.mdx): Component landing page for Alauda Build of Spark Operator, linking its introduction and installation guide. - [docs/en/develop/components/spark_operator/install.mdx](docs/en/develop/components/spark_operator/install.mdx): Installs Alauda Build of Spark Operator as an OLM operator from the platform OperatorHub. Lists prerequisites including ACP v4.0 or later, amd64 or arm64 clusters, and OLM; covers uploading and installing the operator and verifying that the CSV reports Succeeded, the controller and webhook pods are Running, and the CRDs are registered. @@ -176,7 +175,7 @@ - [docs/en/evaluate_safety/components/index.mdx](docs/en/evaluate_safety/components/index.mdx): Section index for the components the Evaluate and Safety chapter relies on, gathering the introduction and installation guides for TrustyAI. - [docs/en/evaluate_safety/components/trustyai/deploy_tas.mdx](docs/en/evaluate_safety/components/trustyai/deploy_tas.mdx): Guide to deploying the TrustyAI Service, which runs alongside KServe models and collects the inference data that drift detection and bias metrics are computed from. Covers creating the `TrustyAIService` resource in either DATABASE mode with a MySQL 8.x database or PVC mode with a working default StorageClass, and verifying deployment readiness. - [docs/en/evaluate_safety/components/trustyai/index.mdx](docs/en/evaluate_safety/components/trustyai/index.mdx): Component landing page for Alauda Build of TrustyAI, linking its introduction, installation guide, and TrustyAI Service deployment guide. -- [docs/en/evaluate_safety/components/trustyai/install.mdx](docs/en/evaluate_safety/components/trustyai/install.mdx): Installs the TrustyAI operator by uploading the trustyai-service-operator tarball with the violet CLI and then selecting Alauda Build of TrustyAI from the OperatorHub Marketplace. Lists the CRDs (TrustyAIService, LMEvalJob, GuardrailsOrchestrator, NemoGuardrails) made available for inference monitoring, LLM evaluation, and AI guardrails orchestration. +- [docs/en/evaluate_safety/components/trustyai/install.mdx](docs/en/evaluate_safety/components/trustyai/install.mdx): Installs Alauda Build of TrustyAI by uploading the TrustyAI Operator package and setting `spec.components.trustyAI.managementState` to `Managed` in the `default` `AmlCluster`; the operator registers the TrustyAI CRDs. Lists the CRDs (TrustyAIService, LMEvalJob, GuardrailsOrchestrator, NemoGuardrails) made available for inference monitoring, LLM evaluation, and AI guardrails orchestration. - [docs/en/evaluate_safety/components/trustyai/intro.mdx](docs/en/evaluate_safety/components/trustyai/intro.mdx): Introduction to Alauda Build of TrustyAI, based on the TrustyAI Kubernetes operator, which simplifies deploying model explainability, fairness monitoring, LLM evaluation, and AI guardrails. Describes the TrustyAI Service that collects KServe inference data, LM-Eval for model evaluation, and the guardrails components, and links their respective guides. - [docs/en/evaluate_safety/index.mdx](docs/en/evaluate_safety/index.mdx): Chapter landing page for evaluation and safety, covering model and RAG evaluation and guardrails for LLM inputs and outputs, with the chapter's components listed under Components. Notes that bias and drift monitoring with TrustyAI is covered in the Monitor chapter. - [docs/en/evaluate_safety/lm_eval.mdx](docs/en/evaluate_safety/lm_eval.mdx): Runs LLM evaluations against a KServe InferenceService using the LMEvalJob CRD with the lm-evaluation-harness backend and Unitxt. Shows a full example targeting the arc_easy task with local-completions or local-chat-completions model types, modelArgs (base_url, tokenizer, num_concurrent), allowOnline/allowCodeExecution flags, pvcManaged outputs, and status.state lifecycle (New, Scheduled, Running, Complete). Also covers offline mode with offline.storage.pvcName plus HF_HOME, HF_DATASETS_CACHE, and TRANSFORMERS_CACHE env wiring for air-gapped runs. @@ -185,12 +184,13 @@ ## installation -- [docs/en/installation/ai-cluster.mdx](docs/en/installation/ai-cluster.mdx): Installs Alauda AI on a target cluster. Covers downloading the Alauda AI Operator and optional Knative Operator packages, prerequisites including Authorino, PostgreSQL, and Redis, uploading packages with the violet tool through a scripted push, installing the Alauda AI Operator from OperatorHub, enabling optional Knative functionality, configuring the AmlCluster instance, and importing models into a registry. -- [docs/en/installation/ai-essentials.mdx](docs/en/installation/ai-essentials.mdx): Walks an ACP platform administrator through deploying the **Alauda AI Essentials** cluster plugin (`aml-global-xxx.tgz`) to the **global** cluster, which supplies the UI and RBAC resources required by Alauda AI. Covers downloading the `violet` packaging tool from Marketplace, pushing the plugin tarball via `violet push --clusters=global`, and installing it from the Cluster Plugins page until the tile reports `Installed`. +- [docs/en/installation/ai-cluster.mdx](docs/en/installation/ai-cluster.mdx): Installs Alauda AI on a target cluster. Covers downloading the Alauda AI operator package (and optional dependency packages) and uploading them with `violet`, installing the Alauda AI Operator from OperatorHub, configuring the cluster-scoped `default` `AmlCluster` instance, enabling the managed components through the per-component `managementState` settings (`Managed`, `Unmanaged`, `SharedManaged`, `Removed`), and importing models into a registry. +- [docs/en/installation/ai-essentials.mdx](docs/en/installation/ai-essentials.mdx): Configures the Alauda AI entry point and branding for the single-cluster deployment model (Alauda AI 2.6+). Runs the `global-install.sh` script against the global cluster to create the OAuth2Client, the OIDC credential Secret, and the ProductEntry that registers Alauda AI in the console left navigation. Also covers multi-install display-name overrides via ProductEntry patching and in-product branding through the `ui-config` ConfigMap in the `kubeflow` namespace, and warns that the legacy Alauda AI Essentials cluster plugin must be uninstalled after upgrading from an earlier version. - [docs/en/installation/index.mdx](docs/en/installation/index.mdx): Chapter landing page for installation, listing the hardware requirements of at least two nodes with 16 cores and 32 GB memory plus GPU and disk capacity for serving, the software requirements, and the ordered installation steps for the platform. +- [docs/en/installation/package-upload.mdx](docs/en/installation/package-upload.mdx): The shared procedure for downloading Alauda AI component packages from the Customer Portal and uploading them to the target cluster with the `violet` packaging tool. Covers downloading the matching package (OLM operator bundle or Cluster Plugin), downloading and making `violet` executable, the `violet push` command with `--platform-address`/`--platform-username`/`--platform-password`/`--clusters`, uploading to an external registry with `--dst-repo`, uploading multiple packages with a shell script, and installing the uploaded component from OperatorHub or Cluster Plugins. - [docs/en/installation/pre-configuration.mdx](docs/en/installation/pre-configuration.mdx): Pre-install configuration for Alauda AI: provisions the (deprecated but still supported) GitLab service used by Model Management — version 15+ with HTTPS, Git LFS enabled, self-hosted, and access-token expiration disabled — and walks through generating an admin impersonation token with full `api` scope, then creating the `aml-gitlab-admin-token` secret under the `cpaas-system` namespace. Also covers preparing an Alauda Build of Harbor registry for Model Catalog (HTTPS production mode with anonymous pull) and configuring `spec.helmValues.gitlab.webservice.ingress.proxyBodySize: "0"` and `proxyReadTimeout: "3600"` plus related nginx ingress annotations on `GitLabOfficial` CR to fix HTTP 413 errors when pushing large LFS objects to GitLab 18.5+. - [docs/en/installation/tools.mdx](docs/en/installation/tools.mdx): Explains the AML 1.3+ auto-discovery mechanism for the left-navigation **Tools** menu, which scans all `ConfigMap` resources labeled `aml.cpaas.io/centralMenuItem: "true"` and merges them into `centraldashboard-config` (manual edits are reverted on upgrade). Shows an MLFlow example ConfigMap defining `link`, `parentUid: advanced`, `title`/`titleI18N`, `tags`, and `order` fields, and describes the `aml-feature-tags` whitelist with the `aml.cpaas.io/centralTagsFilter` annotation that hides any menu item whose tags fall outside the filter. -- [docs/en/installation/workbench.mdx](docs/en/installation/workbench.mdx): Installs Workbench, delivered as an OLM Helm operator from v0.2.0. Covers prerequisites including Alauda AI, OLM, cert-manager for the workspace-controller webhook, and the aml-skipper routing component; uploading and installing the operator, creating a `Workbench` custom resource that deploys workspace-controller and the built-in JupyterLab and code-server WorkspaceKinds, verification, and feature gate configuration. +- [docs/en/installation/workbench.mdx](docs/en/installation/workbench.mdx): Installs Workbench in Alauda AI 2.8.x, where it is managed by Alauda AI. Covers prerequisites including Alauda AI and OLM, setting `spec.components.workbench.managementState` to `Managed` in the `default` `AmlCluster` (the default is `Unmanaged`), optional Alauda Service Mesh (`istio`) integration, and verification that the AmlCluster reports `Phase=Ready` and the workspace-controller deployment is available. No separate operator upload or `Workbench` custom resource is required. ## monitor @@ -270,6 +270,6 @@ - [docs/en/upgrade/index.mdx](docs/en/upgrade/index.mdx): Chapter landing page for upgrade and uninstall, linking the version upgrade procedure, the Knative Operator migration guide, the Workbench upgrade guide, and the uninstall procedure. - [docs/en/upgrade/migrating-to-knative-operator.mdx](docs/en/upgrade/migrating-to-knative-operator.mdx): Migration path from the legacy Alauda AI Model Serving operator (1.x series) to the upstream Knative Operator (2.x series) for inference-service serverless capability. Instructs operators to set spec.knativeServing.managementState to Removed on the default Alauda AI instance, then install Knative Operator and create a KnativeServing instance via the Marketplace per the Enabling Knative Functionality guide. Notes that AI 1.x deployments must also upgrade Alauda AI to 2.x. -- [docs/en/upgrade/uninstall.mdx](docs/en/upgrade/uninstall.mdx): Step-by-step uninstall procedure for both Alauda AI Model Serving (Serverless) and the core Alauda AI Operator/instance. Requires setting BuiltIn Knative Serving (1.4+) or Knative Serving Mode (1.3) to Unmanaged before deleting the default-knative-serving instance and the kserveless-operator, then deleting the default AmlCluster, the aml-operator, and the residual CRDs amls.components.aml.dev, kserves.components.aml.dev, amlclusters.amlclusters.aml.dev, and knativeservings.components.aml.dev. Warns about the amlnamespaces.manage.aml.dev finalizer. -- [docs/en/upgrade/upgrade-from-previous-version.mdx](docs/en/upgrade/upgrade-from-previous-version.mdx): Upgrades Alauda AI from 1.5 to 2.3 covering operator-bundle uploads, the pre-upgrade requirement to annotate stopped InferenceServices with serving.kserve.io/stop=true for KServe community-native stop semantics, upgrading Alauda AI Essentials via Functional Components and the Alauda AI Operator via OperatorHub, installing the new standalone Alauda Build of KServe operator, and upgrading the Workbench, MLflow, Volcano, and kftraining cluster plugins (with an ACP 4.0.x ModuleInfo/ModulePlugin kubectl patch workaround). Post-upgrade requires editing every existing InferenceService to remove spec.predictor.model.name and verifying the AmlCluster and default-kserve KServe resources reach Ready/DEPLOYED. -- [docs/en/upgrade/workbench.mdx](docs/en/upgrade/workbench.mdx): Guide to upgrading Workbench, which from v0.2.0 is delivered as an OLM Helm operator instead of a Cluster Plugin with no in-place upgrade between the two forms. Covers migrating from the Workbench Cluster Plugin while preserving user PVCs, retained WorkspaceKind resources, and the aml-workbench-config ConfigMap, and migrating from Kubeflow Notebook. +- [docs/en/upgrade/uninstall.mdx](docs/en/upgrade/uninstall.mdx): Step-by-step uninstall procedure for Alauda AI 2.8 and its components. Components managed by the `default` `AmlCluster` are uninstalled by setting `managementState` to `Removed` (for `Managed` components) or to `Unmanaged` then uninstalling the Operator in OperatorHub (for `SharedManaged` components); components not managed by the AmlCluster are uninstalled from OperatorHub or Cluster Plugins. The core Alauda AI is removed by deleting the `default` `AmlCluster` instance, then uninstalling the Alauda AI Operator. Warns that data-related resources remain in the `kubeflow` and `aml-maas` namespaces after uninstallation, that operator instances should be deleted before the Operator, and that uninstalling does not remove CRDs. +- [docs/en/upgrade/upgrade-from-previous-version.mdx](docs/en/upgrade/upgrade-from-previous-version.mdx): Upgrades Alauda AI from 2.3.x to 2.8.x, migrating from the 2.3 deployment model to the single-cluster `AmlCluster` model. Covers preparing and uploading the 2.8 package set, preserving existing resources, removing the old Cluster Plugin forms of Alauda AI Essentials, LeaderWorkerSet, Workbench, and MLflow, running `global-install.sh` and `migrate-roles.sh` on the global cluster, upgrading the Alauda AI Operator (which auto-upgrades the `default` `AmlCluster` and enables PostgreSQL, Redis, Authorino, KServe, and LWS), manually enabling optional components (MLflow, Workbench, Serving Runtime), and post-upgrade verification. +- [docs/en/upgrade/workbench.mdx](docs/en/upgrade/workbench.mdx): Guide to upgrading Workbench in Alauda AI 2.8.x, where Workbench is managed by Alauda AI through the `default` `AmlCluster`. Covers migrating from the Workbench Cluster Plugin while preserving user PVCs, retained WorkspaceKind resources, and the aml-workbench-config ConfigMap, enabling `spec.components.workbench.managementState: Managed` (no separate `Workbench` custom resource), migrating from Kubeflow Notebook, and preserving data.