docs: add External LLM Gateways integration page

[rajshah4]

Summary

Add a new Integrations page covering how to chain the OpenHands Enterprise built-in LiteLLM to an external LLM gateway (LiteLLM or Bifrost).

• New page: enterprise/integrations/external-llm-gateways.mdx
• Nav: added enterprise/integrations/external-llm-gateways to the Enterprise → Integrations group in docs.json

Why

Many customers already run an LLM gateway (LiteLLM, Bifrost, or a similar OpenAI-compatible proxy) to route, rate-limit, audit, and track cost across multiple LLM providers. OHE ships with its own built-in LiteLLM instance, and that built-in instance can forward requests to the customer's existing gateway instead of calling LLM providers directly. This guide documents how to set that up and how to attribute traffic per team or per user as it flows through both gateways.

How it works

OHE does not point the OpenHands runtime directly at an external gateway. The built-in LiteLLM forwards requests to the external gateway, which in turn forwards to the actual LLM provider:

OpenHands Runtime → built-in LiteLLM (in cluster) → external gateway → LLM provider

The doc covers:

• The chaining model and why the runtime talks to the built-in LiteLLM, not the external gateway directly.
• What the operator needs from the external gateway owner (base URL, virtual key, model list).
• Two configuration paths for the built-in LiteLLM: the admin API (testing only) and Helm values (production).
• Gateway-specific configuration for LiteLLM and Bifrost, including the provider/model naming Bifrost requires.
• Multi-model patterns: explicit per-model entries (recommended) versus wildcard passthrough (not recommended, causes the model dropdown to fill with hundreds of OpenAI models).
• Model discovery and a verified capability matrix (chat, tool calling, streaming, multi-step agent loops, token tracking).
• Five identity and cost attribution recipes: per-team keys, per-profile extra_headers, static custom_llm_extra_headers, LiteLLM spend-log metadata, and batch reconciliation with conversation tags.
• Tracing calls back to a conversation via the SDK's automatic x-litellm-session-id header.
• Security notes and a troubleshooting accordion for the common failure modes.

Key finding documented in the attribution section: LLM.extra_headers is already a first-class field on every OHE LLM profile, exposed in the UI under Settings > LLM > Advanced Options. The SDK forwards it to LiteLLM on every call. With forward_client_headers_to_llm_api: true on the built-in LiteLLM, any header starting with x- reaches the upstream gateway. This makes per-user attribution workable today (one profile per user with the user ID in extra_headers) without waiting on OSS-4865, which is the long-term fix for scaling without per-user profile duplication.

The doc uses the same Mintlify components (Info, Warning, Note, Tabs/Tab, AccordionGroup/Accordion) and voice as the existing enterprise integration pages (azure-devops, slack, external-postgres).

Verification status

Each link in the chain is verified independently on a self-hosted Replicated cluster:

• extra_headers field exposed on the LLM profile schema in the OHE app and persists through the profile API round-trip.
• SDK forwards llm.extra_headers to LiteLLM on every call (code inspection: openhands-sdk/openhands/sdk/llm/options/common.py).
• Built-in LiteLLM forwards x-* headers to the upstream gateway when forward_client_headers_to_llm_api: true (end-to-end with a capture service standing in for the upstream gateway).

One gap: a single end-to-end capture from a real OHE conversation was blocked by an unrelated LiteLLM virtual-key issue (401 after pod restart wiped the verification token cache). Each link in the chain is verified independently. Filed as a follow-up.

Notes

• The guide cross-links to the existing OSS/Cloud LiteLLM Proxy page and LLM Configuration page so readers landing on the Enterprise integration know when to use the built-in-LiteLLM chaining path versus the direct-proxy path.
• The custom_llm_extra_headers recipe references the setting shipped in OpenHands-Cloud#669. It is available now; provider-agnostic static headers are tracked in draft PR OpenHands-Cloud#674.
• Supersedes docs: add External LLM Gateways integration page #701 (same content, retargeted to a same-repo branch so the Mintlify preview bot can deploy).
• Related Linear ticket: PLTF-2906

Style constraints

No em dashes, no en dashes, no "e.g." or "i.e." (uses "for example"), no HTML tags. Mintlify .mdx format.

---

This PR was created by an AI agent (OpenHands) on behalf of rajshah4.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
all-hands-ai	🟢 Ready	View Preview	Aug 7, 2026, 11:42 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.