Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions crates/builtins/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ oauth = [
"dep:moka",
]
elicitation-ciba = ["_engine-core", "_plugins", "dep:base64"]
# Durable per-consumer token quota against a standalone Limitador. Carries no
# HTTP client of its own: the Limitador calls go through the host transport, so
# it needs no dep beyond core and the shared serde/bytes already in this crate.
quota = ["_engine-core", "_plugins"]
cedar = ["_engine-apl-core", "_pdps", "dep:cedar-policy", "dep:stacker"]
# The feature and the crate share a name, so `dep:cel` is required: without it
# the feature would shadow the implicit optional-dependency feature instead of
Expand All @@ -54,6 +58,11 @@ valkey = [
]
secrets-vault = ["_engine-core", "_secrets"]

# Exposes the quota plugin's per-request hot-path functions for its
# microbenchmark. Off by default, so a normal build keeps them private. Paired
# with `quota` by the `[[bench]]` target below.
bench = []

# Private upstream-edge markers. Each engine dependency is named in exactly one
# place, so changing how an edge is declared is a one-line edit rather than five.
_engine-core = ["dep:praxis-policy-core"]
Expand Down Expand Up @@ -303,6 +312,10 @@ rand = "0.8"
testcontainers-modules = { version = "0.15", features = ["valkey"] }
testcontainers = "0.27"

# Hot-path microbenchmark harness for the quota plugin, behind the `bench`
# feature. Light dep tree, so it stays a dev-dependency.
divan = "0.1"

# One harness per extension, each gated on its own single feature.
# `required-features` is all-or-nothing, so a harness covering several
# extensions would be skipped entirely under a single-feature build and its
Expand Down Expand Up @@ -348,5 +361,16 @@ name = "valkey"
path = "tests/valkey/main.rs"
required-features = ["valkey"]

[[test]]
name = "quota"
path = "tests/quota/main.rs"
required-features = ["quota"]

[[bench]]
name = "quota_hot_path"
path = "benches/quota_hot_path.rs"
harness = false
required-features = ["quota", "bench"]

[lints]
workspace = true
59 changes: 59 additions & 0 deletions crates/builtins/benches/quota_hot_path.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright (c) 2026 Praxis Contributors

//! Microbenchmark for the per-request CPU the quota plugin adds: resolving
//! the principal from the identity extension, and the response-body usage
//! fallback parse. The Limitador round trip dominates real request latency
//! and is out of scope here. This guards the plugin's own cost and the
//! borrow-not-clone identity path.
//!
//! Run with `cargo bench -p praxis-policy-builtins --features quota,bench`.

use std::sync::Arc;

use praxis_policy_builtins::plugins::quota::handlers::bench::{extract_usage, resolve_identity};
use praxis_policy_core::extensions::{Extensions, SecurityExtension, SubjectExtension};

fn main() {
divan::main();
}

fn ext_with_sub(sub: &str) -> Extensions {
Extensions {
security: Some(Arc::new(SecurityExtension {
subject: Some(SubjectExtension {
id: Some(sub.to_owned()),
..Default::default()
}),
..Default::default()
})),
..Default::default()
}
}

/// The default `sub` path: borrows the subject id, no allocation.
#[divan::bench]
fn resolve_sub(bencher: divan::Bencher) {
let ext = ext_with_sub("user-a1b2c3d4-e5f6-7890");
bencher.bench(|| resolve_identity(divan::black_box(&ext), divan::black_box("sub")));
}

/// A missing subject, the common unauthenticated path that skips metering.
#[divan::bench]
fn resolve_absent(bencher: divan::Bencher) {
let ext = Extensions::default();
bencher.bench(|| resolve_identity(divan::black_box(&ext), divan::black_box("sub")));
}

/// The body fallback parse, taken only when the gateway's typed usage is
/// absent. An OpenAI-shaped body with the usage object at the tail.
#[divan::bench]
fn extract_usage_from_body(bencher: divan::Bencher) {
let body = r#"{"choices":[{"index":0,"message":{"role":"assistant","content":"ok"}}],"usage":{"prompt_tokens":57,"completion_tokens":71,"total_tokens":128}}"#;
bencher.bench(|| {
extract_usage(
divan::black_box(body),
divan::black_box("usage.total_tokens"),
)
});
}
1 change: 1 addition & 0 deletions crates/builtins/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
//! | `api-key` | `plugins::identity_api_key` |
//! | `oauth` | `plugins::delegator_oauth` |
//! | `elicitation-ciba` | `plugins::elicitation_ciba` |
//! | `quota` | `plugins::quota` |
//! | `cedar` | `pdps::cedar_direct` |
//! | `cel` | `pdps::cel` |
//! | `opa` | `pdps::opa` |
Expand Down
4 changes: 4 additions & 0 deletions crates/builtins/src/plugins/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
//! - `identity_api_key` (`api-key`) — kind `identity/api-key`
//! - `delegator_oauth` (`oauth`) — kind `delegator/oauth`
//! - `elicitation_ciba` (`elicitation-ciba`) — kind `elicitation/ciba`
//! - `quota` (`quota`) — kind `quota`

#[cfg(feature = "jwt")]
pub mod identity_jwt;
Expand All @@ -22,3 +23,6 @@ pub mod delegator_oauth;

#[cfg(feature = "elicitation-ciba")]
pub mod elicitation_ciba;

#[cfg(feature = "quota")]
pub mod quota;
112 changes: 112 additions & 0 deletions crates/builtins/src/plugins/quota/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Token quota (Limitador)

A per-principal token budget, enforced as a policy against a standalone
[Limitador](https://github.com/Kuadrant/limitador). The plugin registers two
hooks: a pre-invoke check on `cmf.llm_input` that admits or refuses a request,
and a post-invoke debit on `cmf.llm_output` that charges the tokens the
response used. The counter lives in Limitador, so the budget persists across
restarts and across replicas.

The budget is a soft cap. The check probes with a delta of one and charges
nothing. The debit records the real spend after the response. Concurrent
requests for the same principal each pass their own check before those debits
land, so a burst can exceed the budget by roughly the number of requests in
flight. Size budgets with that headroom in mind. Do not treat the quota as a
hard admission limit.

## Config

Written under `plugins[<name>].config`.

| Field | Type | Default | Meaning |
|---|---|---|---|
| `endpoint` | string | required | Limitador base URL, e.g. `http://limitador.grid-system.svc:8080`. The plugin POSTs to `{endpoint}/check` and `{endpoint}/report`. |
| `namespace` | string | required | Limitador limit namespace the counters live under, e.g. `grid-tokens`. The budget value itself lives in Limitador's `limits.yaml`, not here. |
| `identity_claim` | string | `sub` | Which resolved-identity value keys the budget, and the Limitador descriptor key. `sub` reads the authenticated subject id. Must be a verified, always-present claim (see Identity). |
| `on_error` | `deny` \| `allow` | `deny` | What to do when a Limitador call fails for a transient reason (timeout, refused connection, dropped socket, oversize response, or an unexpected Limitador status). `deny` fails closed, `allow` serves. Governs only those transient failures, never an over-budget verdict and never a permanent fault (see Failure behavior). |
| `usage_json_path` | string | `usage.total_tokens` | Fallback path to the token total in the response body, read only when the gateway's typed usage is absent. Segments split on `.` or `/`. |
| `timeout_seconds` | integer | `5` | Per-call HTTP timeout, so a slow Limitador fails fast into the failure path rather than stalling the request. |
| `allow_unauthenticated` | bool | `false` | Whether to serve a request that carries no resolved identity. Default denies (nothing to meter, so fail closed). Set `true` only when authentication is enforced upstream and an unauthenticated request should pass unmetered by design. Every such request then logs a warning. |

`endpoint` and `namespace` must be non-empty or the plugin fails to construct.
An unknown config key is rejected rather than ignored.

## Capabilities

The plugin needs all three. A missing `perform_http` always fails closed:
every metered request is denied with `quota.backend_unavailable`. A missing
`read_subject` or `read_claims` resolves the identity to `None`, which fails
closed by default (`quota.no_identity`) and serves unmetered only when
`allow_unauthenticated` is `true`.

| Capability | Why |
|---|---|
| `perform_http` | The check and debit are outbound calls, made through the host HTTP transport. Without it every metered request is denied with `quota.backend_unavailable`, regardless of `on_error`. |
| `read_subject` | Reads `security.subject.id`, which `identity_claim: sub` keys on. Without it the identity is `None`, which denies by default (see `allow_unauthenticated`). |
| `read_claims` | Reads a non-`sub` `identity_claim` from the subject claims. Required when `identity_claim` names anything other than `sub`. |

## Example

```yaml
plugins:
- name: token-quota
kind: quota
hooks: [cmf.llm_input, cmf.llm_output]
capabilities: [read_subject, read_claims, perform_http]
config:
endpoint: http://limitador.grid-system.svc:8080
namespace: grid-tokens
identity_claim: sub
on_error: deny
usage_json_path: usage.total_tokens
timeout_seconds: 5
allow_unauthenticated: false
```

The plugin registers both hooks itself; the `hooks` list in config has no
effect.

## Identity

`identity_claim` must name a claim the gateway verifies and that is always
present on an authenticated request. `sub` is the only safe default. It reads
the authenticated subject id. A client-supplied claim can be dropped or forged
to change the descriptor the budget is keyed on, so do not key a budget on one.

## Deployment

The check and debit run through the host HTTP transport, which enforces the
host's egress policy. Limitador is normally an in-cluster Service on a private
(RFC 1918) ClusterIP. A transport that blocks private destinations by default
(an SSRF guard) refuses every call to it. The plugin then denies with
`quota.egress_denied`, regardless of `on_error`, because a refused call never
reached Limitador.

Configure the host transport to permit the Limitador address, either by
allowing private destinations or by an egress allowlist entry for the Limitador
ClusterIP. Without that, no request is metered, and under the default
`on_error: deny` none is served. This is a host-transport setting. The plugin
does not control it.

## Failure behavior

The check path is the gate. Its outcomes:

| Outcome | Result | Deny code |
|---|---|---|
| Within budget | allow | |
| Over budget | deny (HTTP 429) | `quota.exhausted` |
| No resolved identity, `allow_unauthenticated: false` | deny | `quota.no_identity` |
| No resolved identity, `allow_unauthenticated: true` | allow (logged) | |
| No transport, `perform_http` withheld, or a malformed request | deny, regardless of `on_error` | `quota.backend_unavailable` |
| Host refused the call (egress policy, SSRF guard, open circuit) | deny, regardless of `on_error` | `quota.egress_denied` |
| Transient Limitador failure (timeout, connect, io, oversize, or unexpected status) | `on_error`: `deny` denies, `allow` serves | `quota.backend_unavailable` (under `deny`) |

`on_error` governs only the last row. Every permanent fault fails closed on its
own, so a misconfiguration cannot silently stop enforcement.

The debit path (`cmf.llm_output`) never denies. The response is already out, so
a failed debit is logged and the balance lags rather than blocking the request.
Neither call retries: `/report` increments unconditionally, so a repeat would
double-charge, and `/check` skips retry to keep tail latency off the admission
path.
Loading