diff --git a/docs.json b/docs.json index 1170f317..b350479d 100644 --- a/docs.json +++ b/docs.json @@ -520,6 +520,7 @@ "integrations/guardrails/aporia", "integrations/guardrails/acuvity", "integrations/guardrails/akto", + "integrations/guardrails/alice", "integrations/guardrails/azure-guardrails", "integrations/guardrails/bedrock-guardrails", "integrations/guardrails/cato", diff --git a/integrations/guardrails/alice.mdx b/integrations/guardrails/alice.mdx new file mode 100644 index 00000000..f42d2837 --- /dev/null +++ b/integrations/guardrails/alice.mdx @@ -0,0 +1,182 @@ +--- +title: "Alice" +description: "Evaluate prompts and model responses against the Alice policies configured for your application — block, mask, or record a detection." +--- + +[Alice](https://alice.io/) evaluates prompts and model responses against the policies you configure per application, and returns a verdict Portkey enforces: allow it, block it, mask the flagged spans before the model sees them, or record a detection and let it through. + + + +## Using Alice with Portkey + +### 1. Add Alice Credentials to Portkey + +* Navigate to the `Integrations` page under `Sidebar` +* Click on the edit button for the Alice integration +* Add your Alice API key — create one in WonderSuite under **Account Settings → API Keys** + +`API base URL` is optional. Leave it blank unless you run a private Alice deployment. + +### 2. Add Alice's Guardrail Check + +* Navigate to the `Guardrails` page and click the `Create` button +* Search for **Evaluate** and click `Add` +* Set the **Application** the check applies to (see below), and any `actions` you want on your check +* Create the Guardrail! + + + Guardrail Actions allow you to orchestrate your guardrails logic. You can learn them [here](/product/guardrails#there-are-6-types-of-guardrail-actions) + + +| Check Name | Description | Parameters | Supported Hooks | +|------------|-------------|------------|-----------------| +| Evaluate | Evaluate the prompt or the model response against the Alice policies configured for your application, and enforce the verdict. | `Application`, `Timeout` | `beforeRequestHook`, `afterRequestHook` | + +**Parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `appId` | string | — | **Required.** The application whose policies apply | +| `timeout` | number | `5000` | Timeout in milliseconds | + +## Naming the application + +This is the one setting most likely to trip you up on a first run, so it is worth a section of its own. + +Alice configures policies per **application**, and a project usually holds several — a support bot and a payments agent do not want the same rules. One Alice API key fronts all of them, so the application cannot be part of the credential. It is a parameter on the check instead: + +```json +{ + "id": "alice.evaluate", + "parameters": { "appId": "payments-bot" } +} +``` + +`appId` takes the application's UUID from your Alice **Application Inventory**, or your own id for it if you filled in the optional **Application ID** field when you added the application. A guardrail that names no application fails its check rather than being evaluated against a guess — which would measure your traffic against the wrong policy set. + +If you need different policies for different traffic, create one guardrail per application and attach each to the Config that routes that traffic. + + + The application is deliberately **not** read from request metadata. `x-portkey-metadata` is written by the caller, so an application named there would let a caller point their own traffic at policies laxer than the ones you configured for them. Guardrail configuration is the only place it is read from. + + +## What is sent + +The plugin forwards the hook context as it received it and enforces the verdict that comes back. It selects nothing and renames nothing — which parts of a turn are worth evaluating, and how a verdict is reached, are decided on Alice's side, so those can change without you upgrading your Gateway. + +Credentials are the single exception. `context.request.headers` carries your caller's `Authorization` and `x-portkey-api-key` in the clear, and a provider credential can ride along under a nested field, so any key named `headers`, `credentials`, `api_key`, `apiKey`, `authorization` or `cookie` is dropped **at every nesting depth** before the body is serialised, and never leaves your Gateway. Only the outbound copy is affected — the real context continues down the pipeline untouched. + +Portkey's `x-portkey-trace-id` is sent alongside, so the turns of one conversation group together in Alice rather than scattering. + +## What each verdict does + +| Verdict | Portkey does | +|---|---| +| `ALLOW` | The request proceeds | +| `MASK` | The flagged spans are replaced before the model sees them, and the request proceeds | +| `BLOCK` | The check fails, carrying your policy's own message | +| `DETECT` | The request proceeds; the explanation carries the correlation id of Alice's record | + +A `MASK` verdict is applied **all-or-nothing**: if any replacement cannot be written back, the check fails instead of applying the rest, so content Alice meant to replace can never reach the model alongside content that was replaced. + +## When Alice cannot be reached + +A transport failure, a timeout, and an answer that cannot be read are all reported as a **check error**, never as a pass. So is a `4xx` — a rejected API key is a misconfiguration, not a reason to let traffic through unscreened. + +That makes `failOnError` on the check the single place that decides what happens next: + +* `failOnError: false` (default) — the guardrail is skipped and the request proceeds +* `failOnError: true` — the check fails, and your guardrail actions decide whether to deny + +Set it to `true` if unscreened traffic is not acceptable for this application. + + + Masking applies to the content part currently being handled. On a **streamed** response, blocking still works, but the streamed text is not rewritten. + + +### 3. Add Guardrail ID to a Config and Make Your Request + +* When you save a Guardrail, you'll get an associated Guardrail ID - add this ID to the `input_guardrails` or `output_guardrails` params in your Portkey Config +* Create these Configs in Portkey UI, save them, and get an associated Config ID to attach to your requests. [More here](/product/ai-gateway/configs). + +Here's an example config: + +```json +{ + "input_guardrails": ["guardrails-id-xxx"], + "output_guardrails": ["guardrails-id-xxx"] +} +``` + + + + +```js +const portkey = new Portkey({ + apiKey: "PORTKEY_API_KEY", + config: "pc-***" // Supports a string config id or a config object +}); +``` + + + +```py +portkey = Portkey( + api_key="PORTKEY_API_KEY", + config="pc-***" # Supports a string config id or a config object +) +``` + + + +```js +const openai = new OpenAI({ + apiKey: 'OPENAI_API_KEY', + baseURL: PORTKEY_GATEWAY_URL, + defaultHeaders: createHeaders({ + apiKey: "PORTKEY_API_KEY", + config: "CONFIG_ID" + }) +}); +``` + + + +```py +client = OpenAI( + api_key="OPENAI_API_KEY", + base_url=PORTKEY_GATEWAY_URL, + default_headers=createHeaders( + provider="openai", + api_key="PORTKEY_API_KEY", + config="CONFIG_ID" + ) +) +``` + + + +```sh +curl https://api.portkey.ai/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -H "x-portkey-api-key: $PORTKEY_API_KEY" \ + -H "x-portkey-config: $CONFIG_ID" \ + -d '{ + "model": "gpt-4o-mini", + "messages": [{ + "role": "user", + "content": "Hello!" + }] + }' +``` + + + +For more, refer to the [Config documentation](/product/ai-gateway/configs). + +Your requests are now guarded by Alice and you can see the Verdict and any action you take directly on Portkey logs! + +## Get Support + +If you face any issues with the Alice integration, see the [Alice docs](https://docs.alice.io/) or ping the Portkey team on the [community forum](https://discord.gg/portkey-llms-in-prod-1143393887742861333). diff --git a/product/guardrails/list-of-guardrail-checks.mdx b/product/guardrails/list-of-guardrail-checks.mdx index 4eed52a1..b3f4b60e 100644 --- a/product/guardrails/list-of-guardrail-checks.mdx +++ b/product/guardrails/list-of-guardrail-checks.mdx @@ -21,6 +21,14 @@ Each Guardrail Check has a specific purpose, it's own parameters, supported hook * Scan prompts and responses for security threats * Detect PII, toxicity, and prompt injections * Real-time content analysis and filtering + + * Enforce your own policies, configured per application * Block, mask the + flagged spans, or record a detection and let the request through * Screens + both the prompt and the model response +