diff --git a/docs.json b/docs.json index 1170f317..1d85a43a 100644 --- a/docs.json +++ b/docs.json @@ -1365,6 +1365,7 @@ "guides/use-cases/comparing-top10-lmsys-models-with-portkey", "guides/use-cases/metadata-use-cases", "guides/use-cases/track-costs-using-metadata", + "guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags", "guides/use-cases/deepseek-r1", "guides/use-cases/openai-computer-use", "guides/use-cases/private-mcp-servers", @@ -2375,6 +2376,7 @@ "guides/use-cases/comparing-top10-lmsys-models-with-portkey", "guides/use-cases/metadata-use-cases", "guides/use-cases/track-costs-using-metadata", + "guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags", "guides/use-cases/enforcing-limits-and-budgets", "guides/use-cases/deepseek-r1", "guides/use-cases/openai-computer-use", diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx new file mode 100644 index 00000000..799f0ad7 --- /dev/null +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -0,0 +1,419 @@ +--- +title: "AWS Bedrock Cost Attribution with STS Session Tags" +description: "Attribute Amazon Bedrock costs per application, team, or environment by forwarding Portkey metadata as AWS STS session tags through a shared assumed role." +--- + +LLM gateways typically share a single IAM role across all tenants. Without per-request tagging, every Bedrock call attributes to the same identity in AWS Cost Explorer — making cost breakdowns impossible. + +Portkey solves this by forwarding the `x-portkey-metadata` header as [AWS STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) during `AssumeRole`. Each unique tag set produces a distinct STS session, so AWS records cost and usage per application, team, or environment — no extra IAM roles required. + + +This feature is available exclusively for **Enterprise self-hosted gateway** customers. It is not available on the Portkey managed cloud. + + +## How It Works + +```mermaid +sequenceDiagram + participant Client + participant Gateway as Portkey Gateway
(self-hosted) + participant STS as AWS STS + participant Bedrock as Amazon Bedrock + + Client->>Gateway: Request + x-portkey-metadata
{"app":"chatbot", "team":"platform"} + Gateway->>Gateway: Parse & sanitise metadata + Gateway->>STS: AssumeRole + Tags.member.N
(Key/Value pairs from metadata) + STS-->>Gateway: Temporary credentials
(tagged session) + Gateway->>Bedrock: InvokeModel
(using tagged credentials) + Bedrock-->>Gateway: Response + Gateway-->>Client: Response + Note over STS,Bedrock: Tags land in CUR 2.0 (iamPrincipal/ prefix),
CloudTrail, and Cost Explorer +``` + +1. Client sends `x-portkey-metadata` with key-value pairs on each request. +2. Gateway parses the metadata, sanitises it (drops reserved `aws:` prefixes, enforces AWS limits), and passes the pairs as `Tags.member.N.Key / Tags.member.N.Value` in the STS `AssumeRole` POST. +3. AWS STS issues temporary credentials tagged with those values. +4. Gateway uses those credentials to call Bedrock `InvokeModel`. +5. Tags appear in **CUR 2.0** (`iamPrincipal/` prefix), **CloudTrail**, and **Cost Explorer**. + +## Prerequisites + +| Requirement | Details | +|---|---| +| **Gateway version** | `≥ 2.19.0` | +| **Auth type** | Bedrock with **AWS Assumed Role** ([setup guide](/product/model-catalog/connect-bedrock-with-amazon-assumed-role)) | +| **Deployment** | Enterprise self-hosted (ECS, EKS, EC2, or Docker) | + +--- + +## Step 1: Configure IAM Permissions on AWS + +Session tags require `sts:TagSession` in **two places**: the caller's IAM policy and the target role's trust policy. Missing either one causes an `AccessDenied` error. Complete this setup **before** enabling the feature on the gateway. + +### 1a. Caller principal (the gateway's execution role) + +The IAM role the gateway runs under (e.g. an ECS task role, an EC2 instance profile, or a Kubernetes service account role) must be allowed to both assume and tag the target Bedrock role. + +Add or update the policy attached to the **caller role**: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "sts:AssumeRole", + "sts:TagSession" + ], + "Resource": "arn:aws:iam:::role/" + } + ] +} +``` + +Replace `` and `` with the actual values for the role Portkey assumes to invoke Bedrock. + + +**Cross-account setups:** When the gateway and the Bedrock role live in different AWS accounts, both the caller permissions (in the gateway account) and the target trust policy (in the Bedrock account) must allow `sts:TagSession`. This is the most common source of `AccessDenied` errors. + + +### 1b. Target role trust policy (the Bedrock invocation role) + +Open the target role in IAM, go to **Trust relationships → Edit trust policy**, and ensure the `Action` includes both `sts:AssumeRole` and `sts:TagSession`: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::role/" + }, + "Action": [ + "sts:AssumeRole", + "sts:TagSession" + ], + "Condition": { + "StringEquals": { + "sts:ExternalId": "" + } + } + } + ] +} +``` + + +The `Condition` block is optional but recommended. If you configured an external ID when setting up the assumed role in Portkey, include it here. + + +### 1c. Bedrock invocation permissions (no change needed) + +The existing permission policy on the target role (`bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream`) does not need modification. Session tags only affect the `AssumeRole` call, not the Bedrock API calls themselves. + +--- + +## Step 2: Enable Session Tags on the Gateway + +Once IAM permissions are in place, enable the feature by setting the environment variable on the gateway container: + +```sh +AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + +```json +{ + "containerDefinitions": [{ + "name": "portkey-gateway", + "environment": [ + {"name": "AWS_BEDROCK_STS_SESSION_TAGS_ENABLED", "value": "true"} + ] + }] +} +``` + + + +```yaml +containers: + - name: portkey-gateway + env: + - name: AWS_BEDROCK_STS_SESSION_TAGS_ENABLED + value: "true" +``` + + + +```yaml +services: + portkey-gateway: + environment: + - AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + + +Enable this flag **only after** completing the IAM setup in Step 1. If `sts:TagSession` is not permitted on both the caller and target roles, all Bedrock requests through assumed roles will fail with `AccessDenied` once the flag is enabled. + + +Restart or redeploy the gateway after adding the variable. + +--- + +## Step 3: Send Metadata with Requests + +Pass key-value pairs in the `x-portkey-metadata` header (or the SDK `metadata` option). Every key-value pair becomes an STS session tag. + + + +```python Python +from portkey_ai import Portkey + +portkey = Portkey( + api_key="PORTKEY_API_KEY", + provider="bedrock" +) + +response = portkey.with_options( + metadata={ + "app": "chatbot", + "team": "ai-platform", + "env": "production", + "cost_center": "CC-1234" + } +).chat.completions.create( + model="anthropic.claude-sonnet-4-20250514-v1:0", + messages=[{"role": "user", "content": "Hello"}] +) + +print(response.choices[0].message) +``` + +```js JavaScript +import Portkey from "portkey-ai" + +const portkey = new Portkey({ + apiKey: "PORTKEY_API_KEY", + provider: "bedrock" +}) + +const response = await portkey.chat.completions.create({ + model: "anthropic.claude-sonnet-4-20250514-v1:0", + messages: [{ role: "user", content: "Hello" }] +}, { + metadata: { + app: "chatbot", + team: "ai-platform", + env: "production", + cost_center: "CC-1234" + } +}) + +console.log(response.choices) +``` + +```sh cURL +curl https://your-gateway.example.com/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "x-portkey-api-key: $PORTKEY_API_KEY" \ + -H "x-portkey-provider: bedrock" \ + -H 'x-portkey-metadata: {"app":"chatbot","team":"ai-platform","env":"production","cost_center":"CC-1234"}' \ + -d '{ + "model": "anthropic.claude-sonnet-4-20250514-v1:0", + "messages": [{"role": "user", "content": "Hello"}] + }' +``` + + + +--- + +## Step 4: Verify Tags in CloudTrail + +After sending a tagged request, confirm the tags appear in the `AssumeRole` CloudTrail event: + +```sh +aws cloudtrail lookup-events \ + --region \ + --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRole \ + --max-results 5 \ + --output json \ + --query "Events[?contains(CloudTrailEvent, '')].CloudTrailEvent" \ + | jq -r '.[] | fromjson | {eventTime, requestParameters}' +``` + +A successful tagged request shows a `tags` array in `requestParameters`: + +```json +{ + "eventTime": "2026-09-02T10:15:30Z", + "requestParameters": { + "roleArn": "arn:aws:iam::123456789012:role/portkey-bedrock-invokemodel", + "roleSessionName": "202692", + "externalId": "...", + "tags": [ + {"key": "app", "value": "chatbot"}, + {"key": "team", "value": "ai-platform"}, + {"key": "env", "value": "production"}, + {"key": "cost_center", "value": "CC-1234"} + ] + } +} +``` + +If the `tags` field is absent, double-check: +- Gateway version is `≥ 2.19.0` +- `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is set +- The gateway was restarted after adding the variable + +--- + +## Step 5: View Costs in AWS Cost Explorer + +Session tags flow into **AWS Cost and Usage Reports (CUR 2.0)** with the `iamPrincipal/` prefix. To view them in Cost Explorer: + +1. Open **Billing and Cost Management → Cost Allocation Tags**. +2. Filter for **User-defined cost allocation tags**. +3. Locate your tag keys (e.g. `app`, `team`, `env`) and click **Activate**. + + +Tags take up to **24 hours** to appear after the first tagged API call, and a further **24 hours** to activate in Cost Explorer. + + +4. In **Cost Explorer**, group or filter by **Tag** → select your activated tag key. + +This breaks down Bedrock spend by application, team, environment, or any other dimension passed in metadata — all through a single shared IAM role. + +--- + +## Tag Constraints and Sanitisation + +The gateway automatically sanitises metadata before passing it to AWS STS. Understanding the constraints helps avoid silent tag drops. + +| Constraint | Limit | Gateway behavior | +|---|---|---| +| Maximum tags per request | 50 | Excess tags silently dropped | +| Key length | 128 characters | Truncated to 128 | +| Value length | 256 characters | Truncated to 256 | +| `aws:` key prefix | Reserved by AWS | Dropped silently | +| Empty keys or values | Not allowed | Dropped silently | +| Non-scalar values (objects, arrays) | Not supported | Dropped during metadata parsing | +| Value types | String only | Numbers and booleans auto-coerced to strings | + +**Source:** [AWS STS session tag limits](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html), [AssumeRole API reference](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) + +--- + +## Credential Caching + +The gateway caches STS temporary credentials to minimize `AssumeRole` API calls. No custom cache configuration is needed — caching works out of the box. + +### Cache key composition + +Each cached credential is keyed by the combination of: + +| Parameter | Source | Effect on cache | +|---|---|---| +| `roleArn` | Virtual key / provider config | Different target roles → separate cache entries | +| `externalId` | Virtual key / provider config | Different external IDs → separate cache entries | +| `region` | Virtual key / provider config / `AWS_ASSUME_ROLE_REGION` env | Different regions → separate cache entries | +| `sourceCredentialHash` | SHA-256 of the gateway's own credentials (access key + secret key + session token) | If the gateway's source credentials rotate, the cache naturally invalidates | +| `tagsHash` | FNV-1a hash of the sanitised metadata key-value pairs | **Different metadata → different cache entries → separate STS sessions** | + +Parameters **not** in the cache key: model name, Portkey API key, virtual key slug, request body. These do not affect the STS call and are irrelevant to credential identity. + +### TTL and reuse + +| Setting | Value | Configurable? | +|---|---|---| +| Credential cache TTL | **5 minutes** (300 seconds) | Not configurable via env — hardcoded in the gateway | +| Cache backend | **Redis** (with 30-second local in-process cache) when Redis is configured; **in-memory** otherwise | Follows the gateway's `CACHE_STORE` setting | +| Local cache layer | When using Redis, a 30-second in-process local cache avoids a Redis round-trip on every request | Automatic when Redis is configured | + +### When does a new AssumeRole call happen? + +A new STS `AssumeRole` call is made **only** when no cached credential exists for the computed cache key. In practice: + +- **Same metadata, same role** — Reuses cached credentials for up to 5 minutes. One STS call per 5-minute window. +- **Different metadata, same role** — Each unique metadata set triggers its own STS call and produces a separate cached entry. For example, 3 distinct `team` values means 3 STS calls (then cached for 5 min each). +- **Gateway restart** — Clears the in-memory cache. Redis cache (if configured) survives restarts. +- **Source credential rotation** — When the gateway's own credentials change (e.g. ECS task role refresh), the `sourceCredentialHash` changes and new STS calls are made. + +### Estimating STS call volume + +``` +STS calls ≈ (unique metadata combinations × unique roles × unique regions) / 5 min +``` + +For a gateway serving 3 teams (`ai-platform`, `mobile`, `backend`) through 1 Bedrock role in 1 region, expect ~3 STS calls every 5 minutes — regardless of request volume. + + +AWS STS has no hard per-second rate limit, but sustained high call rates may trigger throttling. The 5-minute cache keeps call volume well within safe bounds for typical deployments. If you have hundreds of unique metadata combinations, consider limiting the metadata keys sent as session tags to a small set of high-cardinality dimensions (e.g. `team` and `env`, not per-request IDs). + + +--- + +## Troubleshooting + + + +This is the most common error. The full message looks like: + +``` +User: arn:aws:sts:::assumed-role//... +is not authorized to perform: sts:TagSession on resource: +arn:aws:iam:::role/ +``` + +**Fix:** Add `sts:TagSession` to **both** the caller's IAM policy (Step 1a) **and** the target role's trust policy (Step 1b). In cross-account setups, both accounts must allow the action. + + + + +Possible causes: + +1. **Gateway version too old** — Confirm `>= 2.19.0` by calling `GET /health`. +2. **Feature flag not set** — Verify `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is in the container environment (Step 2). +3. **Gateway not restarted** — Environment variables are read at startup. Redeploy after adding the variable. +4. **Metadata not sent** — Confirm the `x-portkey-metadata` header is present in Portkey's request inspector. +5. **Cached credentials** — Previously cached (untagged) credentials may still be in use. Wait for the credential TTL to expire or restart the gateway to clear the cache. + + + + +1. Tags must be **activated** as cost allocation tags in the Billing console (Step 5). +2. Activation takes up to **24 hours** after the first tagged Bedrock call. +3. Cost Explorer data may lag an additional **24 hours**. +4. Ensure you are looking under **User-defined cost allocation tags**, not IAM principal type tags. + + + + +If Bedrock requests fail immediately after enabling `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED`, the most likely cause is missing `sts:TagSession` permissions. As a quick mitigation: + +1. Disable the feature flag: set `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` to `false` (or remove it). +2. Restart the gateway. +3. Complete the IAM setup in Step 1, then re-enable. + + + + +STS session tags do **not** survive role chaining by default. If the gateway performs a two-hop assumption (source role → target role), tags passed on the first hop are dropped on the second unless marked as [transitive](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html#id_session-tags_role-chaining). The gateway does not currently set `TransitiveTagKeys`. + + +--- + +## Reference + +- [AWS: Pass session tags in STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) +- [AWS: AssumeRole API — Tags parameter](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) +- [AWS: IAM principal cost attribution for Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-iam-principal-tracking.html) +- [AWS: Using IAM principal for cost allocation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/iam-principal-cost-allocation.html) +- [Portkey: Connect Bedrock with Assumed Role](/product/model-catalog/connect-bedrock-with-amazon-assumed-role) + +import PrismaAirsCta from "/snippets/prisma-airs-cta.mdx"; + + diff --git a/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx b/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx index 0b7e5854..d1d6b406 100644 --- a/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx +++ b/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx @@ -125,10 +125,10 @@ Once the role is created, copy the role ARN and paste it into the Bedrock integr ## Attribute Cost and Usage with Session Tags -You can attribute Bedrock cost and audit trails to individual applications, teams, or environments while sharing a single assumed role. Any metadata you send on a request is forwarded to AWS as [STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html), so you can segregate spend in AWS Cost Explorer and CloudTrail without provisioning a separate IAM role per application. +Attribute Bedrock cost and audit trails to individual applications, teams, or environments while sharing a single assumed role. Any metadata sent on a request is forwarded to AWS as [STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html), so spend can be segregated in AWS Cost Explorer and CloudTrail without provisioning a separate IAM role per application. -This feature requires the `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` environment variable to be set to `true` on the gateway container. +This feature requires gateway version **≥ 2.19.0** and the `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` environment variable set to `true` on the gateway container. Pass the tags using the `metadata` field (or the `x-portkey-metadata` header): @@ -139,11 +139,11 @@ curl https://api.portkey.ai/v1/chat/completions \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: $PROVIDER_SLUG" \ -H 'x-portkey-metadata: {"app": "chatbot", "team": "platform", "env": "prod"}' \ - -d '{ "model": "anthropic.claude-3-5-sonnet-20240620-v1:0", "messages": [{"role": "user", "content": "Hello"}] }' + -d '{ "model": "anthropic.claude-sonnet-4-20250514-v1:0", "messages": [{"role": "user", "content": "Hello"}] }' ``` -For AWS to accept session tags, add the `sts:TagSession` action to the trust policy of the role Portkey assumes, alongside the existing `sts:AssumeRole` action. +For AWS to accept session tags, add the `sts:TagSession` action to **both** the caller principal's IAM policy **and** the target role's trust policy. See the [full setup cookbook](/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags) for step-by-step IAM configuration, CloudTrail verification, and Cost Explorer activation.