Deploy a RAG-based IT support chatbot, experiment with prompt and model combinations, and track every result to find the best-performing configuration.
Enterprise IT service desks face mounting pressure as software portfolios grow and support teams struggle to keep pace with the volume and complexity of user inquiries. Inconsistent answers, long resolution times, and reliance on tribal knowledge lead to poor customer satisfaction and expensive escalation paths. An AI-powered support chatbot built on retrieval-augmented generation (RAG) can address these challenges by grounding its responses in up-to-date documentation, but deploying such a chatbot reliably requires rigorous experimentation. Teams must compare prompt strategies, evaluate proprietary versus open-source models, and measure answer quality with metrics like cosine similarity, BERTScore, and LLM-as-a-judge scoring before promoting a model to production. This quickstart demonstrates that full workflow — from chunking and embedding a knowledge base, to running a prompt-engineering sweep across models, to logging every experiment with MLflow — so that IT organizations can build, evaluate, and continuously improve a documentation chatbot with confidence.
AI teams need a centralized platform to track experiments, manage dataset versions, and collaborate on model development. While cloud-based MLOps solutions exist, many organizations require their AI infrastructure to remain within their private environment to meet security, compliance, and data privacy requirements. This AI quickstart helps teams deploy DagsHub® MLOps platform on Red Hat OpenShift AI® to track experiments, manage datasets, and collaborate on AI projects.
DagsHub provides a complete MLOps platform with experiment tracking, model registry, and data versioning capabilities that can be deployed entirely within your organization's infrastructure. Deploying DagsHub with OpenShift AI enables teams to maintain full control over their AI workflows and data while benefiting from enterprise-grade security and scalability.
Important
A license and service account from DagsHub are required for this quickstart. See the prerequisites section below for details.
Your Red Hat OpenShift® cluster must have sufficient resources to run DagsHub and its dependencies:
- CPU: 8 vCPUs (minimum 4 vCPUs)
- Memory: 16 GB RAM (minimum 8 GB)
- Storage: 100 GB persistent storage for data, experiments, and models
- Node capacity: At least 2 worker nodes for high availability
Note: These are minimum requirements for a small team (5-10 users). For production deployments with larger teams or extensive workloads, scale resources accordingly.
- Red Hat OpenShift: Version 4.20 or later
- Red Hat OpenShift AI: Version 3.4 or later
- Command-line tools on your local machine:
ocCLI (OpenShift command-line tool)helmCLI - Version 3.11 or latermake- GNU Make utility
- Cluster access:
- Cluster administrator privileges (ability to create namespaces, deploy workloads, and manage secrets)
- Authenticated session to your OpenShift cluster
- DagsHub credentials (obtained from DagsHub.com):
- Enterprise license key
- GCP service account JSON file for container registry access
- Network access:
- OpenShift cluster must be able to pull container images from DagsHub's GCP registry (
gcr.ioandus-docker.pkg.dev) - Ability to expose services via OpenShift routes (for HTTPS access)
- OpenShift cluster must be able to pull container images from DagsHub's GCP registry (
Before deploying DagsHub on your OpenShift cluster, you need to obtain a license and service account from DagsHub:
- Visit DagsHub.com and contact their team to request an enterprise license and GCP service account credentials
- Download the service account JSON file provided by DagsHub (e.g.,
service-account.json) - Ensure you have access to your OpenShift cluster with the required tools installed
-
Log in to your OpenShift cluster and verify admin access:
# Log in to your OpenShift cluster oc login https://your-openshift-cluster-url:6443 # Verify you have admin privileges oc auth can-i create project
You should see
yesas the output. If not, contact your cluster administrator. -
Clone this repository:
git clone https://github.com/rh-ai-quickstart/dagshub-ai-dev-platform-support cd dagshub-ai-dev-plaform-support -
Place your service account file in the project directory:
# Copy your service account JSON file to the project root cp /path/to/your/service-account.json ./service-account.json -
Run the installation command:
make install-dagshub \ NAMESPACE=dagshub \ SERVICE_ACCOUNT=service-account.json \ URL=https://dagshub.yourcompany.com
Parameters:
NAMESPACE: OpenShift namespace for DagsHub (default:dagshub)SERVICE_ACCOUNT: Path to your GCP service account JSON fileURL: Public HTTPS URL where DagsHub will be accessible (must start withhttps://and end with.com)
-
Wait for the installation to complete. The Makefile will:
- Create the required namespaces
- Configure container registry secrets
- Deploy DagsHub using Helm
- Expose the service via an OpenShift route
-
Access DagsHub at the URL you specified (e.g.,
https://dagshub.yourcompany.com) -
Enter your license key on first access to complete the initial setup
To remove DagsHub from your OpenShift cluster:
make uninstall-dagshub NAMESPACE=dagshubThis command will:
- Uninstall the DagsHub Helm release
- Prompt you to delete the container registry secrets (optional)
- Prompt you to delete the namespace (optional)
Deploy a pre-configured Jupyter workbench with the DagsHub LLM tutorial to get started with AI development on OpenShift AI:
# Deploy the LLM tutorial workbench
make deploy-workbench \
NAMESPACE=dagshub \
URL=https://dagshub.yourcompany.com
# Check workbench status
make workbench-status NAMESPACE=dagshub
# Remove workbench
make uninstall-workbench NAMESPACE=dagshubThe workbench includes:
- hello_world_llm.ipynb: Complete end-to-end LLM tutorial covering RAG, fine-tuning, and evaluation
- Pre-configured environment: Ready-to-use Jupyter environment with DagsHub integration
- Persistent storage: Your work is automatically saved and preserved
After deployment, access the workbench through the OpenShift AI dashboard → Data Science Projects → Your namespace → Workbench.
📖 For detailed workbench documentation, see deploy/helm/workbench/README.md
Red Hat OpenShift AI (RHOAI) 3.4+ ships a built-in MLflow instance that is shared across the cluster. To isolate experiments per namespace, RHOAI requires every API request to carry an X-MLFLOW-WORKSPACE header identifying the caller's workspace. DagsHub's MLflow client does not add this header natively, so requests to the built-in MLflow endpoint would be rejected.
The MLflow workspace proxy solves this by sitting between DagsHub and the RHOAI MLflow service. It is a lightweight Nginx (OpenResty) sidecar deployed in the DagsHub namespace that:
- Injects the workspace header every proxied request gets
X-MLFLOW-WORKSPACEset to the DagsHub namespace, so experiments are scoped correctly. - Handles authentication the proxy attaches a bearer token from a dedicated ServiceAccount that has
mlflow-edit,mlflow-view, andmlflow-integrationRBAC roles. - Strips unsupported fields RHOAI rejects
artifact_locationon experiment creation when workspaces are enabled, so the proxy removes it from those requests.
DagsHub is then configured to use http://mlflow-workspace-proxy.<namespace>.svc:8080 as its MLflow tracking URI instead of calling RHOAI directly.
The proxy is deployed automatically as part of make install-dagshub. To check its status or remove it independently:
make mlflow-status NAMESPACE=dagshub
make uninstall-mlflow NAMESPACE=dagshubCheck that all pods are running:
make status NAMESPACE=dagshubView logs from the main DagsHub pod:
make logs NAMESPACE=dagshubThe Makefile provides the following commands:
# Installation
make install-dagshub NAMESPACE=<namespace> SERVICE_ACCOUNT=<file> URL=<url>
# Management
make status NAMESPACE=<namespace> # Check deployment status
make logs NAMESPACE=<namespace> # View main pod logs
make restart-main-pod NAMESPACE=<namespace> # Restart main pod
# Cleanup
make uninstall-dagshub NAMESPACE=<namespace> # Remove DagsHub
make delete-secrets NAMESPACE=<namespace> # Remove secrets only
# Help
make help # Show all optionsPods not starting:
oc describe pod <pod-name> -n <namespace>
oc logs <pod-name> -n <namespace>
oc get events -n <namespace> --sort-by='.lastTimestamp'License issues:
- Verify you have a valid enterprise license from DagsHub
- Check that the license key was entered correctly during setup
Network connectivity:
- Ensure cluster can reach
gcr.ioandus-docker.pkg.dev - Verify service account credentials are valid
Access issues:
- Check that the route was created:
oc get routes -n <namespace> - Verify DNS resolution for your custom domain
- Title: Build and evaluate an AI-powered IT support chatbot
- Description: Track experiments, manage datasets, and collaborate on AI projects on a unified platform
- Industry: Media and IT services
- Product: OpenShift AI
- Use Case: Experiment tracking, managing datasets, and collaborating on IT services chatbot
- Partner: DagsHub
- Contributing org: Red Hat, DagsHub
