diff --git a/README.md b/README.md index 9038289..d473225 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ Web 端采用 **Astro + TypeScript + Preact Islands**，首期生成静态 HTML - 接入 AI：先读 [AI 成本与安全](./docs/ai-safety-and-cost.md)。 - 实现 AI 调用：遵循 [AI Gateway v1 合同](./docs/ai-gateway-contract-v1.md)。 - Token Forge AI 规划：遵循 [AI 任务拆分](./docs/token-forge-ai-planning.md)。 +- Token Forge AI 成本边界：遵循 [流量与成本策略](./docs/token-forge-ai-traffic-policy.md)。 - Token Forge 本地导出：遵循 [Markdown / GitHub Issue 导出](./docs/token-forge-export.md)。 - Token Forge 正式页面：遵循 [页面与事件合同](./docs/token-forge-page.md)。 - Token Forge 当前顺序与 Beta 门槛：遵循 [优先路线图](./docs/token-forge-roadmap.md)。 @@ -104,5 +105,8 @@ Score-only 的确定性 SVG 下载，Incident Detective MVP 六个任务全部 缺失字段、厂商扩展、SMART 不可用和冲突信号。P3-002 已上线浏览器端只读解析工作台和 Parse Result v1。P3-003 已增加序列号、WWN、主机名、IP 与常见 Secret 的本地脱敏预览， 以及不含自由文本的 Boundary Projection v1；URL、日志和 Analytics 进一步收窄为固定元数据， -AI 与导出尚未启用，现有结果仍不构成厂商保修判断。Token Forge 下一步是 P4-004，为生产 -AI 建立预算、限流和熔断边界；完整顺序以 Token Forge 优先路线图为准。 +AI 与导出尚未启用，现有结果仍不构成厂商保修判断。P4-004 已为 Token Forge 建立按最坏 +两次 Provider 尝试预留的 Token/微美元预算、匿名用户与全局限流、并发和熔断状态机， +快照不含输入正文；它目前仍是 Provider-neutral 的离线核心。Token Forge 下一步是 P1-008， +为该状态机接入原子持久化、真实 Provider 与页面模板降级；完整顺序以 Token Forge 优先 +路线图为准。 diff --git a/TODO.md b/TODO.md index 52d359e..8d01ff3 100644 --- a/TODO.md +++ b/TODO.md @@ -81,7 +81,7 @@ - 验收：只访问公开 GitHub 固定 API Origin；不要求 Token；明确显示读取、忽略、截断和未知计数；超时、限流、私有/无效仓库或疑似 Secret 时保留 P1-002 模板路径。 - [ ] **P1-008 [L] 接通受预算保护的生产 AI 规划** - - 依赖：P4-004。 + - 依赖：P4-004（已完成）。 - 产物：服务端 AI Gateway 端点、Provider Adapter、页面 AI 增强入口、流式或明确等待状态和模板降级。 - 验收：浏览器不含密钥；AI 只接收版本化 allowlist 投影；预算、超时、限流、无效响应和熔断均保留可导出的模板计划；响应重新通过全部确定性合同。 @@ -166,7 +166,12 @@ - Production：`https://lab.margrop.net`。 - 验收：Custom Domain、HTTPS、首页、核心内容和 `robots.txt` 在线检查通过；生产发布保持手动触发。 - [ ] **P4-003 [M] 为 Token Forge 建立不采集输入正文的 Analytics** -- [ ] **P4-004 [M] 建立 Token Forge AI 日/用户预算、限流与熔断** +- [x] **P4-004 [M] 建立 Token Forge AI 日/用户预算、限流与熔断** + - 产物：`token-forge-ai-policy-v1`、匿名用户/Lab/全站三层 Token 与微美元日预算、滑动 + 限流、并发预留、熔断状态机、可验证隐私快照和 Gateway 安全失败映射。 + - 验收：Provider 调用前覆盖最多两次尝试的最坏成本；可信成功结算退回差额，失败或预留 + 超时全额计费；重复请求不重复预留；半开只允许一个探针；快照不包含目标、Prompt、仓库 + URL、路径、正文、原始网络标识或 Provider 错误；无网络、存储和日志副作用。 - [ ] **P4-005 [S] 为博客生成 Token Forge 统一 CTA 片段** - [ ] **P4-006 [S] 在博客导航突出 Token Forge 入口** - [ ] **P4-007 [M] 建立 Labs 首页与 Token Forge 的 SEO、Open Graph 和 sitemap** @@ -174,7 +179,8 @@ ## 推荐下一项任务 -接下来只做 **P1-007**：把已经完成的公开 GitHub 仓库摘要适配器接入 Token Forge 页面， -形成“可选仓库 URL → 受限只读摘要 → 覆盖/未知项 → 模板计划”的可用闭环。仓库不可用、 -限流、超时、疑似 Secret 或输入无效时必须保留无需网络的模板结果；不得要求用户提供 -GitHub Token，不得提前接入真实 AI。每次仍只处理一个任务。 +接下来只做 **P1-008**：为 P4-004 状态机接入服务端原子持久化、不可逆匿名用户键、固定 +Provider/模型/价格表和所有 Provider 尝试的可信成本结算，再把 P1-004 AI 增强接到页面。 +浏览器不得持有密钥、价格或结算字段；预算、限流、熔断、超时、Provider 与输出失败都必须 +保留可导出的 P1-002 模板计划。Preview 验证后仍由人工触发 Production 发布，每次仍只 +处理一个任务。 diff --git a/apps/api/README.md b/apps/api/README.md index 0853943..75f13a0 100644 --- a/apps/api/README.md +++ b/apps/api/README.md @@ -8,4 +8,7 @@ - 公开 GitHub 仓库的受限读取； - 不含正文的调用元数据。 -服务端不得成为确定性解析和脱敏的唯一实现，也不得记录用户输入正文。P0-006 确认合同后再选择运行时。 +服务端不得成为确定性解析和脱敏的唯一实现，也不得记录用户输入正文。P4-004 已提供 +Token Forge 的 Provider-neutral 流量策略与可序列化快照；P1-008 选择运行时时必须为它 +提供原子持久化、服务端匿名用户键、固定价格表和所有 Provider 尝试的可信结算。完整边界见 +[Token Forge AI 流量与成本策略](../../docs/token-forge-ai-traffic-policy.md)。 diff --git a/docs/ai-gateway-contract-v1.md b/docs/ai-gateway-contract-v1.md index 01dd45f..fe5c742 100644 --- a/docs/ai-gateway-contract-v1.md +++ b/docs/ai-gateway-contract-v1.md @@ -109,17 +109,26 @@ P2-005 的 `incident-detective.case-proposal-v1` 操作只发送 ID、主题、 Payload、内部答案、canonical attempt 或评分规则；输出只能进入人工审核，不能自动发布。 详见[受约束案例生成与审核](./incident-detective-case-generation.md)。 +P4-004 为 `token-forge.plan-v1` 增加 Provider-neutral 的准入状态机。它在 Provider 调用前 +按最多两次尝试预留 56,000 可计费 Token 和 $0.10，执行匿名用户、Lab、全站三层日预算、 +滑动限流、并发与熔断；成功后按所有尝试的可信汇总用量退款，失败或预留超时按全额计费。 +固定数值、快照隐私和 P1-008 接入要求见 +[Token Forge AI 流量与成本策略](./token-forge-ai-traffic-policy.md)。 + ## Fixture 与验证 - `ai-gateway-request.valid.json` - `ai-gateway-response.valid.json` +- `token-forge-ai-policy.valid.json` 测试使用内存中的合成 Adapter，覆盖公共 Schema、服务端控制字段、请求/响应/Token 上限、超时、限流、暂时不可用、无效模型结构、秘密回流、Provider 错误隔离和 HTTP 映射。没有真实 Provider、真实仓库内容或真实凭据。 ## 已知限制 -- 没有 HTTP 路由、身份识别、限流存储、日预算存储和部署配置； +- 没有 HTTP 路由、匿名身份派生、策略快照的原子持久化和部署配置； - 没有具体 Provider Adapter、模型选择、SDK 或精确 Tokenizer； - 通用请求 Schema 不能替代每个操作的输入 Schema； - Gateway 自身的最小秘密检测不能替代 P0-007 和各操作的允许字段脱敏； -- `request_id` 目前只是合同字段，幂等缓存需要未来服务端状态实现。 +- 通用 Gateway 的 `request_id` 仍没有响应缓存；Token Forge 策略只提供 24 小时有限去重； +- P4-004 不读取模型价格，也不汇总 Provider 尝试；P1-008 必须在服务端完成这两项后才能 + 提交可信结算。 diff --git a/docs/ai-safety-and-cost.md b/docs/ai-safety-and-cost.md index b3c78bd..42bd495 100644 --- a/docs/ai-safety-and-cost.md +++ b/docs/ai-safety-and-cost.md @@ -21,3 +21,8 @@ 至少限制：单请求输入、单请求输出、IP/匿名会话频率、Lab 日预算、全站日预算、最大并发和 Provider 超时。 预算耗尽时，样例模式和确定性核心继续可用。日志只记录 Lab ID、模型代号、耗时、Token 数、状态和粗粒度错误码。 + +Token Forge 的固定数值、先预留后结算、匿名用户键、熔断和持久化要求见 +[AI 流量与成本策略](./token-forge-ai-traffic-policy.md)。成本预留必须覆盖一次 Gateway +请求内的全部 Provider 尝试；只有服务端汇总的可信实际 Token 与微美元成本可以触发退款， +失败、进程中断或预留过期均按全额计费。 diff --git a/docs/token-forge-ai-planning.md b/docs/token-forge-ai-planning.md index 822cb5a..951d950 100644 --- a/docs/token-forge-ai-planning.md +++ b/docs/token-forge-ai-planning.md @@ -84,6 +84,17 @@ AI Gateway 的 Provider Request 不包含 `provider`、`model`、`system_prompt` 模板输入也会在本地脱敏，所以降级计划不会把导致 AI 拒绝的 Secret 原样写回 Prompt。 +## 生产流量边界 + +P4-004 已提供 `token-forge.plan-v1` 专属的日 Token/微美元预算、匿名用户滑动限流、并发 +预留与熔断状态机。每次生产调用必须在 Provider 前预留最多两次尝试的最坏成本；成功后按 +所有尝试的可信汇总用量结算，任意失败或预留超时则保留全额预留。准入拒绝映射为现有 +Gateway 错误码，P1-004 继续返回完整模板计划。 + +策略核心不包含 Provider、端点、匿名身份派生或持久化。P1-008 必须按 +[AI 流量与成本策略](./token-forge-ai-traffic-policy.md) 接入原子状态存储后，才能把本 +模块连接到正式页面。 + ## 测试范围 测试使用内存中的合成 Adapter 和合成公开仓库摘要，覆盖： @@ -101,6 +112,6 @@ AI Gateway 的 Provider Request 不包含 `provider`、`model`、`system_prompt` - 尚无真实 Provider Adapter、服务端操作注册表或 HTTP API； - Token Forge 正式页面当前只接入确定性模板，AI 路径仍未连接； -- 未实现用户级/每日预算、持久化限流或熔断； +- 用户级/每日预算、限流和熔断已有离线状态机，但尚无生产端点与原子持久化； - 自然语言护栏不能替代最小权限与人工确认； - 导出边界由 P1-005 另行实现。 diff --git a/docs/token-forge-ai-traffic-policy.md b/docs/token-forge-ai-traffic-policy.md new file mode 100644 index 0000000..a671326 --- /dev/null +++ b/docs/token-forge-ai-traffic-policy.md @@ -0,0 +1,133 @@ +# Token Forge AI 流量与成本策略 + +P4-004 为 `token-forge.plan-v1` 定义 Provider-neutral 的预算、滑动限流、并发预留和熔断 +核心。实现位于 `@margrop-labs/ai-gateway/token-forge-policy`，只操作经过验证的内存状态， +不创建 HTTP 端点、Provider Adapter、密钥、数据库或部署配置。 + +策略合同由 +[`token-forge-ai-policy-v1.schema.json`](../schemas/token-forge-ai-policy-v1.schema.json) +约束。P1-008 必须先通过这层准入，再调用 Provider；任一拒绝都沿用 AI Gateway 的安全失败 +信封并保留确定性模板计划。 + +## 固定硬边界 + +| 边界 | 默认硬上限 | +| --- | ---: | +| 单个 Gateway 请求可计费 Token | 56,000 | +| 单个 Gateway 请求成本 | $0.10 | +| 匿名用户每日 Token | 224,000 | +| Token Forge 每日 Token | 2,240,000 | +| 全站每日 Token | 5,600,000 | +| 匿名用户每日成本 | $0.25 | +| Token Forge 每日成本 | $5.00 | +| 全站每日成本 | $10.00 | +| 匿名用户 / Token Forge / 全站每日请求 | 12 / 200 / 400 | +| 60 秒内匿名用户 / Token Forge / 全站请求 | 2 / 20 / 40 | +| 匿名用户 / Token Forge / 全站并发 | 1 / 4 / 6 | +| 预留有效期 | 60 秒 | +| 连续失败后熔断 | 5 次 | +| 熔断打开时间 | 60 秒 | +| 半开探针 | 1 个 | + +微美元是整数成本单位：`1 USD = 1,000,000 microUSD`，避免浮点金额累计误差。56,000 +Token 覆盖 AI Gateway 最多两次 Provider 尝试的最坏情况，而不是只覆盖最终成功的一次。 +P1-008 的服务端价格表必须用固定模型价格和最大尝试次数计算预留 Token 与预留成本；浏览器 +不能提交价格、模型、预留额或结算用量。 + +运行配置只能收紧 Schema 中的预算、频率、并发和失败阈值，不能扩大上述硬边界。若调整真实 +模型、最大尝试次数或价格，必须先验证单次成本上限仍覆盖最坏情况，再发布配置。 + +## 匿名用户边界 + +策略层只接受形如 `anon_<32-64 个安全字符>` 的 `actor_key`。它必须由 P1-008 在服务端 +通过签名匿名会话或带密钥的不可逆摘要派生，不能使用原始 IP、Cookie、User-Agent、邮箱或 +仓库标识。密钥、原始标识与映射关系不得进入策略快照。 + +`actor_key` 是滥用控制标识，不是登录账户，也不能用于跨产品画像。公开响应不会返回它。 + +## 准入、预留与结算 + +每个请求执行以下状态机： + +1. 验证固定 `lab_id`、`operation`、UUID、匿名用户键、服务端时间和预留额； +2. 清理过期预留与限流窗口，拒绝时间回退和重复 `request_id`； +3. 依次检查熔断、滑动限流、并发和 UTC 日预算； +4. 在调用 Provider 前，按最坏情况同时预留 Token 和微美元，并增加请求次数； +5. 成功后用服务端汇总的所有 Provider 尝试实际 Token 与成本结算，退回未使用差额； +6. Gateway 失败、进程中断或 60 秒未结算时保留全额预留，防止通过崩溃、超时或重试绕过 + 成本上限； +7. 实际用量超过预留时失败关闭、保留全额预留，并把该次执行记为熔断失败。 + +重复请求 ID 不会再次预留。已结算 ID 保留 24 小时的有限去重标记；重复结算是幂等空操作。 +日请求数不会因成功结算而退回，避免用大量低 Token 请求绕过频率边界。 + +## 熔断 + +以下调用后失败会增加连续失败数： + +- Provider 限流或预算拒绝； +- Provider 超时或不可用； +- Provider 响应无效、截断或超大小； +- 输出命中安全策略； +- Gateway 内部失败。 + +连续 5 次后，`token-forge.plan-v1` 熔断 60 秒。到期后只允许一个半开探针：成功则关闭并 +清零连续失败；失败或预留超时则重新打开。输入合同错误和请求体超限不代表 Provider +故障，不计入熔断。 + +## 安全失败映射 + +| 内部拒绝 | Gateway 错误码 | 页面行为 | +| --- | --- | --- | +| 日 Token、成本或请求预算耗尽 | `budget_exhausted` | 保留模板计划 | +| 滑动限流或并发已满 | `rate_limited` | 保留模板计划，可稍后重试 | +| 熔断已打开或探针占用 | `provider_unavailable` | 保留模板计划，可稍后重试 | +| 时间回退、无效上下文或重复 ID | `invalid_request` | 保留模板计划 | + +公共失败只包含错误码、是否可稍后重试、最多 3,600 秒的 `retry_after_seconds` 和 +`attempt_count: 0`。内部作用域、匿名用户键、原始错误、请求正文和预算快照不会返回浏览器。 + +## 状态与持久化边界 + +策略快照只允许： + +- UTC 日桶中的 Token、微美元和请求计数； +- 不可逆匿名用户键； +- 请求 UUID、时间戳和预留数值； +- 限流时间戳与熔断状态。 + +快照有 256 KiB 上限、字段白名单、记录数上限和时间单调校验。目标、Prompt、计划、仓库 +URL、Owner、文件路径、正文、IP、Cookie、User-Agent、Provider 原始错误和密钥都不在合同 +内。 + +P4-004 只提供可序列化状态机。P1-008 必须在单个原子持久化边界内完成“读取快照 → 准入或 +结算 → 写回快照”，否则多实例并发会突破预算。刚完成连接、部署或迁移时也不能把空内存 +Ledger 当成生产预算存储。 + +## 验证 + +合成测试覆盖： + +- 策略 Schema、未知字段、硬上限和作用域顺序； +- Token、微美元、请求、滑动窗口和并发边界； +- 成功退款、失败全额计费、超时预留和重复请求； +- 关闭、打开、单探针半开、恢复和重新打开； +- 快照恢复、时间回退、敏感字段缺失和安全 Gateway 失败； +- 无网络、日志或持久化副作用。 + +测试不使用真实用户标识、仓库内容、Provider、价格、密钥或生产数据。 + +## P1-008 接入清单 + +P1-008 仍需完成： + +1. 选择服务端运行时和原子持久化适配器； +2. 通过服务端秘密派生可轮换匿名用户键； +3. 固定 Provider、模型、指令、价格表和最大尝试次数； +4. 在每次 Provider 调用前预留所有尝试的最坏成本； +5. 汇总每次 Provider 尝试的可信可计费用量后结算； +6. 把预算、限流和熔断失败接回 P1-004 的模板降级； +7. 使用合成 Provider 做并发、重启、跨 UTC 日期和故障恢复测试； +8. 经过 Preview 验证后再由人工触发 Production 发布。 + +在这八项完成前，正式页面不得调用真实模型。 diff --git a/docs/token-forge-roadmap.md b/docs/token-forge-roadmap.md index ffd3b57..abe9693 100644 --- a/docs/token-forge-roadmap.md +++ b/docs/token-forge-roadmap.md @@ -21,9 +21,10 @@ Token Forge 已有版本化输入/计划合同、确定性模板、公开 GitHub Provider-neutral AI 规划核心、Markdown / GitHub Issue 导出核心和可用的 Alpha 页面。 当前页面已接通“输入 → 本地模板 → 导出”路径和可选的公开仓库覆盖证据；仓库失败不会 -阻断模板。AI 规划尚未进入正式页面，用户还不能编辑、锁定和重排计划，也没有面向 Coding -Agent 的完整执行包、生产 AI 预算、端到端质量基准或真实转化数据。因此 MVP 核心虽已存在， -产品闭环仍未完成。 +阻断模板。P4-004 已提供 Provider-neutral 的生产 AI 日 Token/微美元预算、匿名用户限流、 +并发预留和熔断状态机，但尚无原子持久化、真实 Provider 和页面入口。用户还不能编辑、锁定 +和重排计划，也没有面向 Coding Agent 的完整执行包、端到端质量基准或真实转化数据。因此 +MVP 核心虽已存在，产品闭环仍未完成。 ## 执行顺序 @@ -50,6 +51,9 @@ Agent 的完整执行包、生产 AI 预算、端到端质量基准或真实转 P1-008 必须依赖 P4-004，不能先上线无预算边界的真实模型调用。P4-003 只能在核心产品闭环稳定 后接入，避免为了收集数据提前扩大隐私面。 +P1-007 与 P4-004 已完成；当前唯一推荐任务是 P1-008。P4-004 的内存状态机不能直接视为 +生产限流存储，P1-008 必须先提供原子持久化与可信的全尝试成本结算。 + ## “特别完善”的完成门槛 只有同时满足以下五类门槛，Token Forge 才视为达到当前战略要求： diff --git a/labs/token-forge/README.md b/labs/token-forge/README.md index 006f64c..4e4cd80 100644 --- a/labs/token-forge/README.md +++ b/labs/token-forge/README.md @@ -57,6 +57,12 @@ Token Forge 现为仓库唯一产品开发主线。公开仓库接入、生产 A 任务去重规则。Secret 输入、超时、不可用或无效输出都会整体丢弃，并返回 P1-002 模板计划。 P1-004 仍不接真实 Provider、HTTP API 或页面。 +[P4-004 AI 流量与成本策略](../../docs/token-forge-ai-traffic-policy.md) 已增加匿名用户、 +Token Forge 与全站三层日 Token/微美元预算、60 秒滑动限流、并发预留和熔断状态机。它 +按最多两次 Provider 尝试预留最坏成本，成功后退款，失败或预留超时全额计费；快照不含 +目标、Prompt、仓库 URL、路径或正文。当前仍没有生产 Provider、端点和原子持久化， +P1-008 完成前页面保持模板模式。 + ## Markdown / GitHub Issue 导出 [P1-005 导出核心](../../docs/token-forge-export.md) 只接收通过 v1 合同的结构化计划，生成 diff --git a/labs/token-forge/fixtures/token-forge-ai-policy.valid.json b/labs/token-forge/fixtures/token-forge-ai-policy.valid.json new file mode 100644 index 0000000..41ffb31 --- /dev/null +++ b/labs/token-forge/fixtures/token-forge-ai-policy.valid.json @@ -0,0 +1,35 @@ +{ + "schema_version": "1.0", + "lab_id": "token-forge", + "operation": "token-forge.plan-v1", + "max_request_billable_tokens": 56000, + "max_request_cost_microusd": 100000, + "daily_budgets": { + "actor_tokens": 224000, + "lab_tokens": 2240000, + "site_tokens": 5600000, + "actor_cost_microusd": 250000, + "lab_cost_microusd": 5000000, + "site_cost_microusd": 10000000, + "actor_requests": 12, + "lab_requests": 200, + "site_requests": 400 + }, + "rate_limit": { + "window_seconds": 60, + "actor_requests": 2, + "lab_requests": 20, + "site_requests": 40 + }, + "concurrency": { + "actor_in_flight": 1, + "lab_in_flight": 4, + "site_in_flight": 6 + }, + "circuit_breaker": { + "consecutive_failures": 5, + "open_seconds": 60, + "half_open_requests": 1 + }, + "reservation_ttl_seconds": 60 +} diff --git a/packages/ai-gateway/README.md b/packages/ai-gateway/README.md index 5d83d3e..490a7a3 100644 --- a/packages/ai-gateway/README.md +++ b/packages/ai-gateway/README.md @@ -1,6 +1,6 @@ # `@margrop-labs/ai-gateway` -Provider 中立的 AI Gateway v1 合同与离线执行核心。 +Provider 中立的 AI Gateway v1 合同、离线执行核心和 Token Forge 流量策略。 包含： @@ -10,6 +10,11 @@ Provider 中立的 AI Gateway v1 合同与离线执行核心。 - Provider Adapter 接口和严格返回值解析； - 粗粒度错误与 HTTP 状态映射； - 操作专属输入/输出验证钩子； -- 无效模型响应、秘密回流与超限输出失败关闭。 +- 无效模型响应、秘密回流与超限输出失败关闭； +- Token Forge 的 Token/微美元日预算、滑动限流、并发预留与熔断状态机； +- 可验证、可序列化且不含输入正文的策略快照。 -不包含网络路由、Provider SDK、模型选择、真实密钥、数据库、限流存储或 AI 业务功能。完整合同见 [`docs/ai-gateway-contract-v1.md`](../../docs/ai-gateway-contract-v1.md)。 +不包含网络路由、Provider SDK、模型选择、真实密钥、数据库、持久化适配器或 AI 业务功能。 +通用合同见 [`docs/ai-gateway-contract-v1.md`](../../docs/ai-gateway-contract-v1.md)，Token +Forge 的成本与流量合同见 +[`docs/token-forge-ai-traffic-policy.md`](../../docs/token-forge-ai-traffic-policy.md)。 diff --git a/packages/ai-gateway/package.json b/packages/ai-gateway/package.json index f4a77fc..acffbbe 100644 --- a/packages/ai-gateway/package.json +++ b/packages/ai-gateway/package.json @@ -4,7 +4,8 @@ "private": true, "type": "module", "exports": { - ".": "./src/index.ts" + ".": "./src/index.ts", + "./token-forge-policy": "./src/token-forge-policy.ts" }, "engines": { "node": ">=22.12.0" diff --git a/packages/ai-gateway/src/token-forge-policy.test.ts b/packages/ai-gateway/src/token-forge-policy.test.ts new file mode 100644 index 0000000..df9c0d0 --- /dev/null +++ b/packages/ai-gateway/src/token-forge-policy.test.ts @@ -0,0 +1,629 @@ +import { readFile } from "node:fs/promises"; + +import { describe, expect, it, vi } from "vitest"; + +import { validateAiGatewayResponse } from "./index"; +import { + type TokenForgeAiAdmissionRequest, + type TokenForgeAiTrafficPolicy, + TokenForgeAiPolicyContractError, + TokenForgeAiPolicyLedger, + admissionDecisionToAiGatewayFailure, + tokenForgeAiTrafficHardPolicy, + validateTokenForgeAiPolicySnapshot, + validateTokenForgeAiTrafficPolicy, +} from "./token-forge-policy"; + +const fixtureUrl = new URL( + "../../../labs/token-forge/fixtures/token-forge-ai-policy.valid.json", + import.meta.url, +); +const startMs = Date.UTC(2026, 0, 2, 0, 0, 0); +const actorA = `anon_${"a".repeat(32)}`; +const actorB = `anon_${"b".repeat(32)}`; +const actorC = `anon_${"c".repeat(32)}`; + +const requestId = (sequence: number): string => + `00000000-0000-4000-8000-${sequence.toString().padStart(12, "0")}`; + +const admission = ( + sequence: number, + overrides: Partial = {}, +): TokenForgeAiAdmissionRequest => ({ + request_id: requestId(sequence), + actor_key: actorA, + lab_id: "token-forge", + operation: "token-forge.plan-v1", + now_ms: startMs, + reserved_tokens: 100, + reserved_cost_microusd: 100, + ...overrides, +}); + +const policyWith = ( + mutate: (policy: TokenForgeAiTrafficPolicy) => void, +): TokenForgeAiTrafficPolicy => { + const policy = JSON.parse( + JSON.stringify(tokenForgeAiTrafficHardPolicy), + ) as TokenForgeAiTrafficPolicy; + mutate(policy); + return policy; +}; + +const expectAllowed = ( + decision: ReturnType, +): Extract => { + expect(decision.status).toBe("allowed"); + if (decision.status !== "allowed") { + throw new Error("Expected the synthetic admission to be allowed."); + } + return decision; +}; + +const settleSuccess = ( + ledger: TokenForgeAiPolicyLedger, + sequence: number, + nowMs: number, + actualTokens = 50, + actualCost = 50, +) => + ledger.settle({ + request_id: requestId(sequence), + now_ms: nowMs, + outcome: { + status: "success", + actual_tokens: actualTokens, + actual_cost_microusd: actualCost, + }, + }); + +const settleFailure = ( + ledger: TokenForgeAiPolicyLedger, + sequence: number, + nowMs: number, +) => + ledger.settle({ + request_id: requestId(sequence), + now_ms: nowMs, + outcome: { + status: "failure", + error_code: "provider_unavailable", + }, + }); + +describe("Token Forge AI traffic policy v1", () => { + it("accepts the published policy fixture and rejects loosened or unknown fields", async () => { + const fixture = JSON.parse(await readFile(fixtureUrl, "utf8")) as unknown; + expect(validateTokenForgeAiTrafficPolicy(fixture)).toEqual( + tokenForgeAiTrafficHardPolicy, + ); + + const loosened = { + ...(fixture as TokenForgeAiTrafficPolicy), + max_request_cost_microusd: 100_001, + }; + expect(() => validateTokenForgeAiTrafficPolicy(loosened)).toThrow( + TokenForgeAiPolicyContractError, + ); + + expect(() => + validateTokenForgeAiTrafficPolicy({ + ...(fixture as TokenForgeAiTrafficPolicy), + provider_api_key: "synthetic-secret-value", + }), + ).toThrow(/additional properties/); + try { + validateTokenForgeAiTrafficPolicy({ + ...(fixture as TokenForgeAiTrafficPolicy), + provider_api_key: "synthetic-secret-value", + }); + } catch (error) { + expect((error as Error).message).not.toContain("synthetic-secret-value"); + } + }); + + it("reserves worst-case tokens and micro-USD before allowing a request", () => { + const ledger = new TokenForgeAiPolicyLedger(); + const allowed = expectAllowed(ledger.admit(admission(1))); + const snapshot = ledger.exportSnapshot(); + const actorUsage = snapshot.days["2026-01-02"]?.actors[actorA]; + + expect(allowed).toEqual({ + status: "allowed", + request_id: requestId(1), + reservation: { + reserved_tokens: 100, + reserved_cost_microusd: 100, + expires_at_ms: startMs + 60_000, + }, + circuit_state: "closed", + }); + expect(actorUsage).toEqual({ + tokens: 100, + cost_microusd: 100, + requests: 1, + }); + }); + + it("enforces actor token, lab cost, and shared site daily budgets", () => { + const actorPolicy = policyWith((policy) => { + policy.max_request_billable_tokens = 100; + policy.daily_budgets.actor_tokens = 100; + }); + const actorLedger = new TokenForgeAiPolicyLedger(actorPolicy); + expectAllowed(actorLedger.admit(admission(1))); + expect( + actorLedger.admit( + admission(2, { + actor_key: actorB, + reserved_tokens: 101, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "invalid_context", + }); + expect( + actorLedger.admit( + admission(2, { + now_ms: startMs + 1, + reserved_tokens: 1, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "actor_concurrency", + }); + settleSuccess(actorLedger, 1, startMs + 2, 100, 50); + expect( + actorLedger.admit( + admission(2, { + now_ms: startMs + 3, + reserved_tokens: 1, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "actor_daily_budget", + gateway_error_code: "budget_exhausted", + }); + + const labPolicy = policyWith((policy) => { + policy.max_request_cost_microusd = 100; + policy.daily_budgets.actor_cost_microusd = 100; + policy.daily_budgets.lab_cost_microusd = 150; + }); + const labLedger = new TokenForgeAiPolicyLedger(labPolicy); + expectAllowed(labLedger.admit(admission(10, { actor_key: actorA }))); + expect( + labLedger.admit( + admission(11, { + actor_key: actorB, + now_ms: startMs + 1, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "lab_daily_budget", + }); + + const sitePolicy = policyWith((policy) => { + policy.max_request_billable_tokens = 100; + policy.daily_budgets.actor_tokens = 100; + policy.daily_budgets.lab_tokens = 200; + policy.daily_budgets.site_tokens = 250; + }); + const seed = new TokenForgeAiPolicyLedger(sitePolicy).exportSnapshot(); + seed.days["2026-01-02"] = { + actors: {}, + lab: { tokens: 0, cost_microusd: 0, requests: 0 }, + site: { tokens: 200, cost_microusd: 0, requests: 0 }, + }; + seed.last_seen_at_ms = startMs; + const siteLedger = new TokenForgeAiPolicyLedger(sitePolicy, seed); + expect( + siteLedger.admit( + admission(20, { + actor_key: actorC, + reserved_tokens: 100, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "site_daily_budget", + }); + }); + + it("refunds unused reservation capacity after trusted success settlement", () => { + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + + expect(settleSuccess(ledger, 1, startMs + 1, 25, 30)).toEqual({ + status: "settled", + charged_tokens: 25, + charged_cost_microusd: 30, + circuit_state: "closed", + }); + + const usage = ledger.exportSnapshot().days["2026-01-02"]?.actors[actorA]; + expect(usage).toEqual({ + tokens: 25, + cost_microusd: 30, + requests: 1, + }); + }); + + it("keeps the full reservation charged on failures and expired calls", () => { + const policy = policyWith((candidate) => { + candidate.reservation_ttl_seconds = 5; + }); + const ledger = new TokenForgeAiPolicyLedger(policy); + expectAllowed(ledger.admit(admission(1))); + expect(settleFailure(ledger, 1, startMs + 1)).toMatchObject({ + status: "settled", + charged_tokens: 100, + charged_cost_microusd: 100, + }); + + expectAllowed( + ledger.admit( + admission(2, { + actor_key: actorB, + now_ms: startMs + 2, + }), + ), + ); + expectAllowed( + ledger.admit( + admission(3, { + actor_key: actorB, + now_ms: startMs + 5_002, + }), + ), + ); + expect(ledger.exportSnapshot().days["2026-01-02"]?.actors[actorB]).toEqual({ + tokens: 200, + cost_microusd: 200, + requests: 2, + }); + }); + + it("enforces sliding rate limits and releases concurrency after settlement", () => { + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + expect(ledger.admit(admission(2, { now_ms: startMs + 1 }))).toMatchObject({ + status: "denied", + reason: "actor_concurrency", + gateway_error_code: "rate_limited", + }); + + settleSuccess(ledger, 1, startMs + 2); + expectAllowed(ledger.admit(admission(2, { now_ms: startMs + 3 }))); + settleSuccess(ledger, 2, startMs + 4); + expect(ledger.admit(admission(3, { now_ms: startMs + 5 }))).toMatchObject({ + status: "denied", + reason: "actor_rate_limit", + retry_after_seconds: 60, + }); + + expectAllowed(ledger.admit(admission(3, { now_ms: startMs + 60_001 }))); + }); + + it("enforces Lab rate, concurrency, and non-refundable daily request caps", () => { + const ratePolicy = policyWith((policy) => { + policy.rate_limit.lab_requests = 2; + policy.rate_limit.site_requests = 3; + }); + const rateLedger = new TokenForgeAiPolicyLedger(ratePolicy); + expectAllowed(rateLedger.admit(admission(1, { actor_key: actorA }))); + settleSuccess(rateLedger, 1, startMs + 1); + expectAllowed( + rateLedger.admit( + admission(2, { + actor_key: actorB, + now_ms: startMs + 2, + }), + ), + ); + settleSuccess(rateLedger, 2, startMs + 3); + expect( + rateLedger.admit( + admission(3, { + actor_key: actorC, + now_ms: startMs + 4, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "lab_rate_limit", + }); + + const concurrencyPolicy = policyWith((policy) => { + policy.concurrency.lab_in_flight = 2; + policy.concurrency.site_in_flight = 3; + }); + const concurrencyLedger = new TokenForgeAiPolicyLedger(concurrencyPolicy); + expectAllowed( + concurrencyLedger.admit(admission(10, { actor_key: actorA })), + ); + expectAllowed( + concurrencyLedger.admit( + admission(11, { + actor_key: actorB, + now_ms: startMs + 1, + }), + ), + ); + expect( + concurrencyLedger.admit( + admission(12, { + actor_key: actorC, + now_ms: startMs + 2, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "lab_concurrency", + }); + + const requestPolicy = policyWith((policy) => { + policy.daily_budgets.actor_requests = 2; + }); + const requestLedger = new TokenForgeAiPolicyLedger(requestPolicy); + expectAllowed(requestLedger.admit(admission(20))); + settleSuccess(requestLedger, 20, startMs + 1, 0, 0); + expectAllowed( + requestLedger.admit( + admission(21, { + now_ms: startMs + 60_001, + }), + ), + ); + settleSuccess(requestLedger, 21, startMs + 60_002, 0, 0); + expect( + requestLedger.admit( + admission(22, { + now_ms: startMs + 120_002, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "actor_daily_budget", + }); + }); + + it("prevents duplicate request IDs from reserving or settling twice", () => { + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + expect(ledger.admit(admission(1, { now_ms: startMs + 1 }))).toMatchObject({ + status: "denied", + reason: "duplicate_request", + }); + + expect(settleSuccess(ledger, 1, startMs + 2)).toMatchObject({ + status: "settled", + }); + expect(settleSuccess(ledger, 1, startMs + 3)).toEqual({ + status: "already_settled", + }); + expect( + ledger.exportSnapshot().days["2026-01-02"]?.actors[actorA], + ).toMatchObject({ requests: 1 }); + }); + + it("opens the circuit, permits one half-open probe, and closes on success", () => { + const policy = policyWith((candidate) => { + candidate.circuit_breaker.consecutive_failures = 2; + }); + const ledger = new TokenForgeAiPolicyLedger(policy); + + expectAllowed(ledger.admit(admission(1))); + settleFailure(ledger, 1, startMs + 1); + expectAllowed( + ledger.admit( + admission(2, { + actor_key: actorB, + now_ms: startMs + 2, + }), + ), + ); + settleFailure(ledger, 2, startMs + 3); + + expect( + ledger.admit( + admission(3, { + actor_key: actorC, + now_ms: startMs + 4, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "circuit_open", + gateway_error_code: "provider_unavailable", + retry_after_seconds: 60, + }); + + const probeTime = startMs + 60_004; + expect( + expectAllowed( + ledger.admit( + admission(3, { + actor_key: actorC, + now_ms: probeTime, + }), + ), + ).circuit_state, + ).toBe("half_open"); + expect( + ledger.admit( + admission(4, { + actor_key: actorA, + now_ms: probeTime + 1, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "circuit_open", + }); + + expect(settleSuccess(ledger, 3, probeTime + 2)).toMatchObject({ + status: "settled", + circuit_state: "closed", + }); + expectAllowed( + ledger.admit( + admission(4, { + actor_key: actorA, + now_ms: probeTime + 3, + }), + ), + ); + }); + + it("reopens the circuit when the half-open probe fails", () => { + const policy = policyWith((candidate) => { + candidate.circuit_breaker.consecutive_failures = 1; + }); + const ledger = new TokenForgeAiPolicyLedger(policy); + + expectAllowed(ledger.admit(admission(1))); + settleFailure(ledger, 1, startMs + 1); + const probeTime = startMs + 60_001; + expectAllowed( + ledger.admit( + admission(2, { + actor_key: actorB, + now_ms: probeTime, + }), + ), + ); + settleFailure(ledger, 2, probeTime + 1); + expect( + ledger.admit( + admission(3, { + actor_key: actorC, + now_ms: probeTime + 2, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "circuit_open", + retry_after_seconds: 60, + }); + }); + + it("restores budgets and circuit state from a bounded privacy-safe snapshot", () => { + const policy = policyWith((candidate) => { + candidate.circuit_breaker.consecutive_failures = 1; + }); + const ledger = new TokenForgeAiPolicyLedger(policy); + expectAllowed(ledger.admit(admission(1))); + settleFailure(ledger, 1, startMs + 1); + + const serialized = JSON.stringify(ledger.exportSnapshot()); + expect(serialized).not.toMatch( + /goal|repository|url|prompt|cookie|ip_address|user_agent|provider_error/i, + ); + + const restored = new TokenForgeAiPolicyLedger( + policy, + JSON.parse(serialized), + ); + expect( + restored.admit( + admission(2, { + actor_key: actorB, + now_ms: startMs + 2, + }), + ), + ).toMatchObject({ + status: "denied", + reason: "circuit_open", + }); + }); + + it("rejects malformed snapshots and time rollback without echoing values", () => { + const snapshot = new TokenForgeAiPolicyLedger().exportSnapshot(); + expect(() => + validateTokenForgeAiPolicySnapshot({ + ...snapshot, + raw_prompt: "synthetic-sensitive-value", + }), + ).toThrow("snapshot validation failed"); + + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1, { now_ms: startMs + 10 }))); + expect( + ledger.admit( + admission(2, { + now_ms: startMs + 9, + }), + ), + ).toEqual({ + status: "denied", + reason: "invalid_context", + gateway_error_code: "invalid_request", + }); + }); + + it("fails closed when trusted usage exceeds the reservation", () => { + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + expect(settleSuccess(ledger, 1, startMs + 1, 101, 100)).toEqual({ + status: "rejected", + reason: "usage_exceeded_reservation", + }); + expect(ledger.exportSnapshot().days["2026-01-02"]?.actors[actorA]).toEqual({ + tokens: 100, + cost_microusd: 100, + requests: 1, + }); + }); + + it("maps internal denials to the existing safe Gateway failure envelope", () => { + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + const denial = ledger.admit(admission(2, { now_ms: startMs + 1 })); + expect(denial.status).toBe("denied"); + if (denial.status !== "denied") { + throw new Error("Expected a synthetic admission denial."); + } + + const failure = admissionDecisionToAiGatewayFailure(denial, requestId(2)); + expect(validateAiGatewayResponse(failure)).toEqual(failure); + expect(failure).toEqual({ + schema_version: "1.0", + request_id: requestId(2), + status: "error", + error: { + code: "rate_limited", + retryable: true, + retry_after_seconds: 60, + }, + meta: { + attempt_count: 0, + }, + }); + }); + + it("has no network, persistence, or logging side effects", () => { + const fetchSpy = vi.spyOn(globalThis, "fetch"); + const logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined); + const warnSpy = vi + .spyOn(console, "warn") + .mockImplementation(() => undefined); + const errorSpy = vi + .spyOn(console, "error") + .mockImplementation(() => undefined); + + const ledger = new TokenForgeAiPolicyLedger(); + expectAllowed(ledger.admit(admission(1))); + settleSuccess(ledger, 1, startMs + 1); + ledger.exportSnapshot(); + + expect(fetchSpy).not.toHaveBeenCalled(); + expect(logSpy).not.toHaveBeenCalled(); + expect(warnSpy).not.toHaveBeenCalled(); + expect(errorSpy).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/ai-gateway/src/token-forge-policy.ts b/packages/ai-gateway/src/token-forge-policy.ts new file mode 100644 index 0000000..ee16ba7 --- /dev/null +++ b/packages/ai-gateway/src/token-forge-policy.ts @@ -0,0 +1,1139 @@ +import Ajv2020, { type ErrorObject } from "ajv/dist/2020.js"; +import type { AnySchema } from "ajv"; +import addFormats from "ajv-formats"; + +import policySchema from "../../../schemas/token-forge-ai-policy-v1.schema.json"; +import type { AiGatewayErrorCode, AiGatewayFailureResponse } from "./index"; + +export type TokenForgeAiTrafficPolicy = { + schema_version: "1.0"; + lab_id: "token-forge"; + operation: "token-forge.plan-v1"; + max_request_billable_tokens: number; + max_request_cost_microusd: number; + daily_budgets: { + actor_tokens: number; + lab_tokens: number; + site_tokens: number; + actor_cost_microusd: number; + lab_cost_microusd: number; + site_cost_microusd: number; + actor_requests: number; + lab_requests: number; + site_requests: number; + }; + rate_limit: { + window_seconds: number; + actor_requests: number; + lab_requests: number; + site_requests: number; + }; + concurrency: { + actor_in_flight: number; + lab_in_flight: number; + site_in_flight: number; + }; + circuit_breaker: { + consecutive_failures: number; + open_seconds: number; + half_open_requests: 1; + }; + reservation_ttl_seconds: number; +}; + +type UsageCounter = { + tokens: number; + cost_microusd: number; + requests: number; +}; + +type DayBucket = { + actors: Record; + lab: UsageCounter; + site: UsageCounter; +}; + +type Reservation = { + actor_key: string; + day_key: string; + reserved_tokens: number; + reserved_cost_microusd: number; + expires_at_ms: number; + circuit_probe: boolean; +}; + +type CircuitSnapshot = { + state: "closed" | "open" | "half_open"; + consecutive_failures: number; + opened_at_ms?: number; + half_open_request_id?: string; +}; + +export type TokenForgeAiPolicySnapshot = { + schema_version: "1.0"; + last_seen_at_ms: number; + days: Record; + rate_events: { + actors: Record; + lab: number[]; + site: number[]; + }; + reservations: Record; + completed_request_ids: Record; + circuit: CircuitSnapshot; +}; + +export type TokenForgeAiAdmissionRequest = { + request_id: string; + actor_key: string; + lab_id: "token-forge"; + operation: "token-forge.plan-v1"; + now_ms: number; + reserved_tokens: number; + reserved_cost_microusd: number; +}; + +export type TokenForgeAiAdmissionDenialReason = + | "invalid_context" + | "duplicate_request" + | "actor_daily_budget" + | "lab_daily_budget" + | "site_daily_budget" + | "actor_rate_limit" + | "lab_rate_limit" + | "site_rate_limit" + | "actor_concurrency" + | "lab_concurrency" + | "site_concurrency" + | "circuit_open"; + +export type TokenForgeAiAdmissionDecision = + | { + status: "allowed"; + request_id: string; + reservation: { + reserved_tokens: number; + reserved_cost_microusd: number; + expires_at_ms: number; + }; + circuit_state: "closed" | "half_open"; + } + | { + status: "denied"; + reason: TokenForgeAiAdmissionDenialReason; + gateway_error_code: + | "invalid_request" + | "rate_limited" + | "budget_exhausted" + | "provider_unavailable"; + retry_after_seconds?: number; + }; + +export type TokenForgeAiSettlementRequest = + | { + request_id: string; + now_ms: number; + outcome: { + status: "success"; + actual_tokens: number; + actual_cost_microusd: number; + }; + } + | { + request_id: string; + now_ms: number; + outcome: { + status: "failure"; + error_code: AiGatewayErrorCode; + }; + }; + +export type TokenForgeAiSettlementResult = + | { + status: "settled"; + charged_tokens: number; + charged_cost_microusd: number; + circuit_state: CircuitSnapshot["state"]; + } + | { + status: "already_settled"; + } + | { + status: "rejected"; + reason: + | "invalid_context" + | "unknown_reservation" + | "usage_exceeded_reservation"; + }; + +export class TokenForgeAiPolicyContractError extends Error { + override name = "TokenForgeAiPolicyContractError"; +} + +const emptyUsage = (): UsageCounter => ({ + tokens: 0, + cost_microusd: 0, + requests: 0, +}); + +const emptySnapshot = (): TokenForgeAiPolicySnapshot => ({ + schema_version: "1.0", + last_seen_at_ms: 0, + days: {}, + rate_events: { + actors: {}, + lab: [], + site: [], + }, + reservations: {}, + completed_request_ids: {}, + circuit: { + state: "closed", + consecutive_failures: 0, + }, +}); + +const hardPolicyCandidate: TokenForgeAiTrafficPolicy = { + schema_version: "1.0", + lab_id: "token-forge", + operation: "token-forge.plan-v1", + max_request_billable_tokens: 56_000, + max_request_cost_microusd: 100_000, + daily_budgets: { + actor_tokens: 224_000, + lab_tokens: 2_240_000, + site_tokens: 5_600_000, + actor_cost_microusd: 250_000, + lab_cost_microusd: 5_000_000, + site_cost_microusd: 10_000_000, + actor_requests: 12, + lab_requests: 200, + site_requests: 400, + }, + rate_limit: { + window_seconds: 60, + actor_requests: 2, + lab_requests: 20, + site_requests: 40, + }, + concurrency: { + actor_in_flight: 1, + lab_in_flight: 4, + site_in_flight: 6, + }, + circuit_breaker: { + consecutive_failures: 5, + open_seconds: 60, + half_open_requests: 1, + }, + reservation_ttl_seconds: 60, +}; + +const ajv = new Ajv2020({ + allErrors: true, + strict: true, +}); +addFormats(ajv); + +const policyValidator = ajv.compile( + policySchema as AnySchema, +); +const textEncoder = new TextEncoder(); +const maxSnapshotBytes = 256 * 1024; +const completedRequestTtlMs = 24 * 60 * 60 * 1_000; +const maxTimestampMs = 8_640_000_000_000_000; +const maxOperationalTimestampMs = maxTimestampMs - completedRequestTtlMs; +const actorKeyPattern = /^anon_[A-Za-z0-9_-]{32,64}$/; +const requestIdPattern = + /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; +const dayKeyPattern = /^\d{4}-\d{2}-\d{2}$/; + +const isRecord = (candidate: unknown): candidate is Record => + typeof candidate === "object" && + candidate !== null && + !Array.isArray(candidate); + +const hasOnlyKeys = ( + candidate: Record, + allowed: readonly string[], +): boolean => Object.keys(candidate).every((key) => allowed.includes(key)); + +const hasExactKeys = ( + candidate: Record, + required: readonly string[], +): boolean => + Object.keys(candidate).length === required.length && + required.every((key) => Object.hasOwn(candidate, key)); + +const isSafeInteger = ( + candidate: unknown, + minimum = 0, + maximum = Number.MAX_SAFE_INTEGER, +): candidate is number => + typeof candidate === "number" && + Number.isSafeInteger(candidate) && + candidate >= minimum && + candidate <= maximum; + +const cloneJson = (candidate: T, contractName: string): T => { + let serialized: string | undefined; + try { + serialized = JSON.stringify(candidate); + } catch { + throw new TokenForgeAiPolicyContractError( + `${contractName} must be serializable JSON.`, + ); + } + + if ( + serialized === undefined || + textEncoder.encode(serialized).byteLength > maxSnapshotBytes + ) { + throw new TokenForgeAiPolicyContractError( + `${contractName} exceeded the serialization limit.`, + ); + } + + return JSON.parse(serialized) as T; +}; + +const deepFreeze = (candidate: T): Readonly => { + if (typeof candidate === "object" && candidate !== null) { + for (const child of Object.values(candidate)) { + deepFreeze(child); + } + Object.freeze(candidate); + } + return candidate; +}; + +const formatValidationErrors = ( + errors: ErrorObject[] | null | undefined, +): string => + (errors ?? []) + .map((error) => { + const location = error.instancePath || "/"; + return `${location} ${error.message ?? "is invalid"}`; + }) + .join("; "); + +const assertScopeOrder = ( + actor: number, + lab: number, + site: number, + field: string, +): void => { + if (actor > lab || lab > site) { + throw new TokenForgeAiPolicyContractError( + `${field} must increase from actor to lab to site.`, + ); + } +}; + +export const validateTokenForgeAiTrafficPolicy = ( + candidate: unknown, +): TokenForgeAiTrafficPolicy => { + const cloned = cloneJson(candidate, "Token Forge AI traffic policy"); + if (!policyValidator(cloned)) { + throw new TokenForgeAiPolicyContractError( + `token-forge-ai-policy-v1 validation failed: ${formatValidationErrors(policyValidator.errors)}`, + ); + } + + const policy = cloned as TokenForgeAiTrafficPolicy; + assertScopeOrder( + policy.daily_budgets.actor_tokens, + policy.daily_budgets.lab_tokens, + policy.daily_budgets.site_tokens, + "Daily token budgets", + ); + assertScopeOrder( + policy.daily_budgets.actor_cost_microusd, + policy.daily_budgets.lab_cost_microusd, + policy.daily_budgets.site_cost_microusd, + "Daily cost budgets", + ); + assertScopeOrder( + policy.daily_budgets.actor_requests, + policy.daily_budgets.lab_requests, + policy.daily_budgets.site_requests, + "Daily request budgets", + ); + assertScopeOrder( + policy.rate_limit.actor_requests, + policy.rate_limit.lab_requests, + policy.rate_limit.site_requests, + "Rate limits", + ); + assertScopeOrder( + policy.concurrency.actor_in_flight, + policy.concurrency.lab_in_flight, + policy.concurrency.site_in_flight, + "Concurrency limits", + ); + + if ( + policy.max_request_billable_tokens > policy.daily_budgets.actor_tokens || + policy.max_request_cost_microusd > policy.daily_budgets.actor_cost_microusd + ) { + throw new TokenForgeAiPolicyContractError( + "Per-request reservations must fit inside the actor daily budget.", + ); + } + + return policy; +}; + +export const tokenForgeAiTrafficHardPolicy: Readonly = + deepFreeze(validateTokenForgeAiTrafficPolicy(hardPolicyCandidate)); + +const validateUsage = (candidate: unknown): candidate is UsageCounter => + isRecord(candidate) && + hasExactKeys(candidate, ["tokens", "cost_microusd", "requests"]) && + isSafeInteger(candidate.tokens) && + isSafeInteger(candidate.cost_microusd) && + isSafeInteger(candidate.requests); + +const validateTimestampArray = (candidate: unknown): candidate is number[] => + Array.isArray(candidate) && + candidate.length <= 400 && + candidate.every((value) => isSafeInteger(value, 0, maxTimestampMs)); + +const validateSnapshotShape = ( + candidate: unknown, +): candidate is TokenForgeAiPolicySnapshot => { + if ( + !isRecord(candidate) || + !hasExactKeys(candidate, [ + "schema_version", + "last_seen_at_ms", + "days", + "rate_events", + "reservations", + "completed_request_ids", + "circuit", + ]) || + candidate.schema_version !== "1.0" || + !isSafeInteger(candidate.last_seen_at_ms, 0, maxTimestampMs) || + !isRecord(candidate.days) || + Object.keys(candidate.days).length > 2 || + !isRecord(candidate.rate_events) || + !isRecord(candidate.reservations) || + Object.keys(candidate.reservations).length > 6 || + !isRecord(candidate.completed_request_ids) || + Object.keys(candidate.completed_request_ids).length > 1_000 || + !isRecord(candidate.circuit) + ) { + return false; + } + + for (const [dayKey, rawDay] of Object.entries(candidate.days)) { + if ( + !dayKeyPattern.test(dayKey) || + !isRecord(rawDay) || + !hasExactKeys(rawDay, ["actors", "lab", "site"]) || + !isRecord(rawDay.actors) || + Object.keys(rawDay.actors).length > 1_000 || + !validateUsage(rawDay.lab) || + !validateUsage(rawDay.site) + ) { + return false; + } + + for (const [actorKey, usage] of Object.entries(rawDay.actors)) { + if (!actorKeyPattern.test(actorKey) || !validateUsage(usage)) { + return false; + } + } + } + + if ( + !hasExactKeys(candidate.rate_events, ["actors", "lab", "site"]) || + !isRecord(candidate.rate_events.actors) || + Object.keys(candidate.rate_events.actors).length > 1_000 || + !validateTimestampArray(candidate.rate_events.lab) || + !validateTimestampArray(candidate.rate_events.site) + ) { + return false; + } + + for (const [actorKey, events] of Object.entries( + candidate.rate_events.actors, + )) { + if (!actorKeyPattern.test(actorKey) || !validateTimestampArray(events)) { + return false; + } + } + + for (const [requestId, rawReservation] of Object.entries( + candidate.reservations, + )) { + if ( + !requestIdPattern.test(requestId) || + !isRecord(rawReservation) || + !hasExactKeys(rawReservation, [ + "actor_key", + "day_key", + "reserved_tokens", + "reserved_cost_microusd", + "expires_at_ms", + "circuit_probe", + ]) || + typeof rawReservation.actor_key !== "string" || + !actorKeyPattern.test(rawReservation.actor_key) || + typeof rawReservation.day_key !== "string" || + !dayKeyPattern.test(rawReservation.day_key) || + !isSafeInteger(rawReservation.reserved_tokens, 1, 56_000) || + !isSafeInteger(rawReservation.reserved_cost_microusd, 1, 100_000) || + !isSafeInteger(rawReservation.expires_at_ms, 0, maxTimestampMs) || + typeof rawReservation.circuit_probe !== "boolean" + ) { + return false; + } + } + + for (const [requestId, expiresAt] of Object.entries( + candidate.completed_request_ids, + )) { + if ( + !requestIdPattern.test(requestId) || + !isSafeInteger(expiresAt, 0, maxTimestampMs) + ) { + return false; + } + } + + if ( + !hasOnlyKeys(candidate.circuit, [ + "state", + "consecutive_failures", + "opened_at_ms", + "half_open_request_id", + ]) || + !Object.hasOwn(candidate.circuit, "state") || + !Object.hasOwn(candidate.circuit, "consecutive_failures") || + !["closed", "open", "half_open"].includes( + candidate.circuit.state as string, + ) || + !isSafeInteger(candidate.circuit.consecutive_failures, 0, 1_000) + ) { + return false; + } + + if ( + candidate.circuit.opened_at_ms !== undefined && + !isSafeInteger(candidate.circuit.opened_at_ms, 0, maxTimestampMs) + ) { + return false; + } + + if ( + candidate.circuit.half_open_request_id !== undefined && + (typeof candidate.circuit.half_open_request_id !== "string" || + !requestIdPattern.test(candidate.circuit.half_open_request_id)) + ) { + return false; + } + + return true; +}; + +export const validateTokenForgeAiPolicySnapshot = ( + candidate: unknown, +): TokenForgeAiPolicySnapshot => { + const cloned = cloneJson(candidate, "Token Forge AI policy snapshot"); + if (!validateSnapshotShape(cloned)) { + throw new TokenForgeAiPolicyContractError( + "Token Forge AI policy snapshot validation failed.", + ); + } + return cloned; +}; + +const admissionDenial = ( + reason: TokenForgeAiAdmissionDenialReason, + gatewayErrorCode: Extract< + AiGatewayErrorCode, + | "invalid_request" + | "rate_limited" + | "budget_exhausted" + | "provider_unavailable" + >, + retryAfterSeconds?: number, +): TokenForgeAiAdmissionDecision => ({ + status: "denied", + reason, + gateway_error_code: gatewayErrorCode, + ...(retryAfterSeconds === undefined + ? {} + : { + retry_after_seconds: Math.max( + 1, + Math.min(Math.ceil(retryAfterSeconds), 3_600), + ), + }), +}); + +const validateAdmissionRequest = ( + candidate: unknown, + policy: TokenForgeAiTrafficPolicy, +): candidate is TokenForgeAiAdmissionRequest => + isRecord(candidate) && + hasExactKeys(candidate, [ + "request_id", + "actor_key", + "lab_id", + "operation", + "now_ms", + "reserved_tokens", + "reserved_cost_microusd", + ]) && + typeof candidate.request_id === "string" && + requestIdPattern.test(candidate.request_id) && + typeof candidate.actor_key === "string" && + actorKeyPattern.test(candidate.actor_key) && + candidate.lab_id === policy.lab_id && + candidate.operation === policy.operation && + isSafeInteger(candidate.now_ms, 0, maxTimestampMs) && + candidate.now_ms <= maxOperationalTimestampMs && + isSafeInteger( + candidate.reserved_tokens, + 1, + policy.max_request_billable_tokens, + ) && + isSafeInteger( + candidate.reserved_cost_microusd, + 1, + policy.max_request_cost_microusd, + ); + +const validateSettlementRequest = ( + candidate: unknown, +): candidate is TokenForgeAiSettlementRequest => { + if ( + !isRecord(candidate) || + !hasExactKeys(candidate, ["request_id", "now_ms", "outcome"]) || + typeof candidate.request_id !== "string" || + !requestIdPattern.test(candidate.request_id) || + !isSafeInteger(candidate.now_ms, 0, maxTimestampMs) || + candidate.now_ms > maxOperationalTimestampMs || + !isRecord(candidate.outcome) || + typeof candidate.outcome.status !== "string" + ) { + return false; + } + + if (candidate.outcome.status === "success") { + return ( + hasExactKeys(candidate.outcome, [ + "status", + "actual_tokens", + "actual_cost_microusd", + ]) && + isSafeInteger(candidate.outcome.actual_tokens) && + isSafeInteger(candidate.outcome.actual_cost_microusd) + ); + } + + const errorCodes: readonly AiGatewayErrorCode[] = [ + "invalid_request", + "request_too_large", + "input_token_limit_exceeded", + "rate_limited", + "budget_exhausted", + "provider_timeout", + "provider_unavailable", + "invalid_provider_response", + "output_token_limit_exceeded", + "response_too_large", + "policy_blocked", + "internal_error", + ]; + + return ( + candidate.outcome.status === "failure" && + hasExactKeys(candidate.outcome, ["status", "error_code"]) && + typeof candidate.outcome.error_code === "string" && + errorCodes.includes(candidate.outcome.error_code as AiGatewayErrorCode) + ); +}; + +const dayKeyFor = (nowMs: number): string => + new Date(nowMs).toISOString().slice(0, 10); + +const secondsUntil = (targetMs: number, nowMs: number): number => + Math.max(1, Math.min(Math.ceil((targetMs - nowMs) / 1_000), 3_600)); + +const secondsUntilNextUtcDay = (nowMs: number): number => { + const now = new Date(nowMs); + const nextDay = Date.UTC( + now.getUTCFullYear(), + now.getUTCMonth(), + now.getUTCDate() + 1, + ); + return secondsUntil(nextDay, nowMs); +}; + +const circuitFailureCodes = new Set([ + "rate_limited", + "budget_exhausted", + "provider_timeout", + "provider_unavailable", + "invalid_provider_response", + "output_token_limit_exceeded", + "response_too_large", + "policy_blocked", + "internal_error", +]); + +const subtractUsage = ( + usage: UsageCounter, + tokenRefund: number, + costRefund: number, +): void => { + usage.tokens = Math.max(0, usage.tokens - tokenRefund); + usage.cost_microusd = Math.max(0, usage.cost_microusd - costRefund); +}; + +export class TokenForgeAiPolicyLedger { + readonly policy: Readonly; + private state: TokenForgeAiPolicySnapshot; + + constructor( + policy: unknown = tokenForgeAiTrafficHardPolicy, + snapshot?: unknown, + ) { + this.policy = deepFreeze(validateTokenForgeAiTrafficPolicy(policy)); + this.state = + snapshot === undefined + ? emptySnapshot() + : validateTokenForgeAiPolicySnapshot(snapshot); + + if ( + Object.values(this.state.reservations).some( + (reservation) => + reservation.reserved_tokens > + this.policy.max_request_billable_tokens || + reservation.reserved_cost_microusd > + this.policy.max_request_cost_microusd, + ) + ) { + throw new TokenForgeAiPolicyContractError( + "Token Forge AI policy snapshot exceeds the active policy.", + ); + } + } + + admit(candidate: unknown): TokenForgeAiAdmissionDecision { + if (!validateAdmissionRequest(candidate, this.policy)) { + return admissionDenial("invalid_context", "invalid_request"); + } + + if (candidate.now_ms < this.state.last_seen_at_ms) { + return admissionDenial("invalid_context", "invalid_request"); + } + + this.prepare(candidate.now_ms); + + if ( + Object.hasOwn(this.state.reservations, candidate.request_id) || + Object.hasOwn(this.state.completed_request_ids, candidate.request_id) + ) { + return admissionDenial("duplicate_request", "invalid_request"); + } + + const circuitDecision = this.prepareCircuit(candidate.now_ms); + if (circuitDecision) { + return circuitDecision; + } + + const windowMs = this.policy.rate_limit.window_seconds * 1_000; + const actorEvents = + this.state.rate_events.actors[candidate.actor_key] ?? []; + const rateChecks = [ + { + events: actorEvents, + limit: this.policy.rate_limit.actor_requests, + reason: "actor_rate_limit" as const, + }, + { + events: this.state.rate_events.lab, + limit: this.policy.rate_limit.lab_requests, + reason: "lab_rate_limit" as const, + }, + { + events: this.state.rate_events.site, + limit: this.policy.rate_limit.site_requests, + reason: "site_rate_limit" as const, + }, + ]; + + for (const check of rateChecks) { + if (check.events.length >= check.limit) { + const oldest = check.events[0] ?? candidate.now_ms; + return admissionDenial( + check.reason, + "rate_limited", + secondsUntil(oldest + windowMs, candidate.now_ms), + ); + } + } + + const activeReservations = Object.values(this.state.reservations); + const actorActive = activeReservations.filter( + (reservation) => reservation.actor_key === candidate.actor_key, + ); + const concurrencyChecks = [ + { + reservations: actorActive, + limit: this.policy.concurrency.actor_in_flight, + reason: "actor_concurrency" as const, + }, + { + reservations: activeReservations, + limit: this.policy.concurrency.lab_in_flight, + reason: "lab_concurrency" as const, + }, + { + reservations: activeReservations, + limit: this.policy.concurrency.site_in_flight, + reason: "site_concurrency" as const, + }, + ]; + + for (const check of concurrencyChecks) { + if (check.reservations.length >= check.limit) { + const earliestExpiry = Math.min( + ...check.reservations.map((reservation) => reservation.expires_at_ms), + ); + return admissionDenial( + check.reason, + "rate_limited", + secondsUntil(earliestExpiry, candidate.now_ms), + ); + } + } + + const dayKey = dayKeyFor(candidate.now_ms); + const day = this.getDay(dayKey); + const actorUsage = day.actors[candidate.actor_key] ?? emptyUsage(); + const dailyChecks = [ + { + usage: actorUsage, + tokenLimit: this.policy.daily_budgets.actor_tokens, + costLimit: this.policy.daily_budgets.actor_cost_microusd, + requestLimit: this.policy.daily_budgets.actor_requests, + reason: "actor_daily_budget" as const, + }, + { + usage: day.lab, + tokenLimit: this.policy.daily_budgets.lab_tokens, + costLimit: this.policy.daily_budgets.lab_cost_microusd, + requestLimit: this.policy.daily_budgets.lab_requests, + reason: "lab_daily_budget" as const, + }, + { + usage: day.site, + tokenLimit: this.policy.daily_budgets.site_tokens, + costLimit: this.policy.daily_budgets.site_cost_microusd, + requestLimit: this.policy.daily_budgets.site_requests, + reason: "site_daily_budget" as const, + }, + ]; + + for (const check of dailyChecks) { + if ( + check.usage.tokens + candidate.reserved_tokens > check.tokenLimit || + check.usage.cost_microusd + candidate.reserved_cost_microusd > + check.costLimit || + check.usage.requests + 1 > check.requestLimit + ) { + return admissionDenial( + check.reason, + "budget_exhausted", + secondsUntilNextUtcDay(candidate.now_ms), + ); + } + } + + day.actors[candidate.actor_key] = actorUsage; + for (const usage of [actorUsage, day.lab, day.site]) { + usage.tokens += candidate.reserved_tokens; + usage.cost_microusd += candidate.reserved_cost_microusd; + usage.requests += 1; + } + + actorEvents.push(candidate.now_ms); + this.state.rate_events.actors[candidate.actor_key] = actorEvents; + this.state.rate_events.lab.push(candidate.now_ms); + this.state.rate_events.site.push(candidate.now_ms); + + const circuitProbe = this.state.circuit.state === "half_open"; + const expiresAt = + candidate.now_ms + this.policy.reservation_ttl_seconds * 1_000; + this.state.reservations[candidate.request_id] = { + actor_key: candidate.actor_key, + day_key: dayKey, + reserved_tokens: candidate.reserved_tokens, + reserved_cost_microusd: candidate.reserved_cost_microusd, + expires_at_ms: expiresAt, + circuit_probe: circuitProbe, + }; + + if (circuitProbe) { + this.state.circuit.half_open_request_id = candidate.request_id; + } + + return { + status: "allowed", + request_id: candidate.request_id, + reservation: { + reserved_tokens: candidate.reserved_tokens, + reserved_cost_microusd: candidate.reserved_cost_microusd, + expires_at_ms: expiresAt, + }, + circuit_state: circuitProbe ? "half_open" : "closed", + }; + } + + settle(candidate: unknown): TokenForgeAiSettlementResult { + if ( + !validateSettlementRequest(candidate) || + candidate.now_ms < this.state.last_seen_at_ms + ) { + return { status: "rejected", reason: "invalid_context" }; + } + + this.prepare(candidate.now_ms); + const reservation = this.state.reservations[candidate.request_id]; + + if (!reservation) { + return Object.hasOwn( + this.state.completed_request_ids, + candidate.request_id, + ) + ? { status: "already_settled" } + : { status: "rejected", reason: "unknown_reservation" }; + } + + let chargedTokens = reservation.reserved_tokens; + let chargedCost = reservation.reserved_cost_microusd; + + if (candidate.outcome.status === "success") { + if ( + candidate.outcome.actual_tokens > reservation.reserved_tokens || + candidate.outcome.actual_cost_microusd > + reservation.reserved_cost_microusd + ) { + this.finishReservation(candidate.request_id, candidate.now_ms); + this.recordCircuitFailure(candidate.now_ms); + return { + status: "rejected", + reason: "usage_exceeded_reservation", + }; + } + + chargedTokens = candidate.outcome.actual_tokens; + chargedCost = candidate.outcome.actual_cost_microusd; + const day = this.state.days[reservation.day_key]; + const actorUsage = day?.actors[reservation.actor_key]; + if (!day || !actorUsage) { + this.finishReservation(candidate.request_id, candidate.now_ms); + this.recordCircuitFailure(candidate.now_ms); + return { status: "rejected", reason: "invalid_context" }; + } + + const tokenRefund = reservation.reserved_tokens - chargedTokens; + const costRefund = reservation.reserved_cost_microusd - chargedCost; + for (const usage of [actorUsage, day.lab, day.site]) { + subtractUsage(usage, tokenRefund, costRefund); + } + this.finishReservation(candidate.request_id, candidate.now_ms); + this.recordCircuitSuccess(); + } else { + this.finishReservation(candidate.request_id, candidate.now_ms); + if (circuitFailureCodes.has(candidate.outcome.error_code)) { + this.recordCircuitFailure(candidate.now_ms); + } else if (reservation.circuit_probe) { + delete this.state.circuit.half_open_request_id; + } + } + + this.pruneDays(candidate.now_ms); + return { + status: "settled", + charged_tokens: chargedTokens, + charged_cost_microusd: chargedCost, + circuit_state: this.state.circuit.state, + }; + } + + exportSnapshot(): TokenForgeAiPolicySnapshot { + return validateTokenForgeAiPolicySnapshot(this.state); + } + + private prepare(nowMs: number): void { + this.state.last_seen_at_ms = nowMs; + const windowStart = nowMs - this.policy.rate_limit.window_seconds * 1_000; + const keepRecent = (timestamp: number): boolean => timestamp > windowStart; + + this.state.rate_events.lab = this.state.rate_events.lab + .filter(keepRecent) + .sort((a, b) => a - b); + this.state.rate_events.site = this.state.rate_events.site + .filter(keepRecent) + .sort((a, b) => a - b); + for (const [actorKey, events] of Object.entries( + this.state.rate_events.actors, + )) { + const recent = events.filter(keepRecent).sort((a, b) => a - b); + if (recent.length === 0) { + delete this.state.rate_events.actors[actorKey]; + } else { + this.state.rate_events.actors[actorKey] = recent; + } + } + + for (const [requestId, expiresAt] of Object.entries( + this.state.completed_request_ids, + )) { + if (expiresAt <= nowMs) { + delete this.state.completed_request_ids[requestId]; + } + } + + for (const [requestId, reservation] of Object.entries( + this.state.reservations, + )) { + if (reservation.expires_at_ms <= nowMs) { + this.finishReservation(requestId, nowMs); + this.recordCircuitFailure(nowMs); + } + } + + this.pruneDays(nowMs); + } + + private prepareCircuit( + nowMs: number, + ): TokenForgeAiAdmissionDecision | undefined { + const circuit = this.state.circuit; + if (circuit.state === "open") { + const openedAt = circuit.opened_at_ms ?? nowMs; + const retryAt = + openedAt + this.policy.circuit_breaker.open_seconds * 1_000; + if (nowMs < retryAt) { + return admissionDenial( + "circuit_open", + "provider_unavailable", + secondsUntil(retryAt, nowMs), + ); + } + + circuit.state = "half_open"; + delete circuit.half_open_request_id; + } + + if ( + circuit.state === "half_open" && + circuit.half_open_request_id !== undefined + ) { + const probe = this.state.reservations[circuit.half_open_request_id]; + return admissionDenial( + "circuit_open", + "provider_unavailable", + probe + ? secondsUntil(probe.expires_at_ms, nowMs) + : this.policy.reservation_ttl_seconds, + ); + } + + return undefined; + } + + private getDay(dayKey: string): DayBucket { + const existing = this.state.days[dayKey]; + if (existing) { + return existing; + } + + const created: DayBucket = { + actors: {}, + lab: emptyUsage(), + site: emptyUsage(), + }; + this.state.days[dayKey] = created; + return created; + } + + private finishReservation(requestId: string, nowMs: number): void { + const reservation = this.state.reservations[requestId]; + if (reservation?.circuit_probe) { + delete this.state.circuit.half_open_request_id; + } + delete this.state.reservations[requestId]; + this.state.completed_request_ids[requestId] = nowMs + completedRequestTtlMs; + } + + private recordCircuitFailure(nowMs: number): void { + const circuit = this.state.circuit; + circuit.consecutive_failures += 1; + if ( + circuit.state === "half_open" || + circuit.consecutive_failures >= + this.policy.circuit_breaker.consecutive_failures + ) { + circuit.state = "open"; + circuit.opened_at_ms = nowMs; + delete circuit.half_open_request_id; + } + } + + private recordCircuitSuccess(): void { + this.state.circuit = { + state: "closed", + consecutive_failures: 0, + }; + } + + private pruneDays(nowMs: number): void { + const currentDay = dayKeyFor(nowMs); + const activeDays = new Set( + Object.values(this.state.reservations).map( + (reservation) => reservation.day_key, + ), + ); + for (const dayKey of Object.keys(this.state.days)) { + if (dayKey !== currentDay && !activeDays.has(dayKey)) { + delete this.state.days[dayKey]; + } + } + } +} + +export const admissionDecisionToAiGatewayFailure = ( + decision: Extract, + requestId?: string, +): AiGatewayFailureResponse => { + const retryable = + decision.gateway_error_code === "rate_limited" || + decision.gateway_error_code === "provider_unavailable"; + + return { + schema_version: "1.0", + ...(requestId !== undefined && requestIdPattern.test(requestId) + ? { request_id: requestId } + : {}), + status: "error", + error: { + code: decision.gateway_error_code, + retryable, + ...(decision.retry_after_seconds === undefined + ? {} + : { retry_after_seconds: decision.retry_after_seconds }), + }, + meta: { + attempt_count: 0, + }, + }; +}; diff --git a/schemas/README.md b/schemas/README.md index 6e86fda..1c847a9 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -19,5 +19,6 @@ | [`smart-rma-boundary-projection-v1.schema.json`](./smart-rma-boundary-projection-v1.schema.json) | Allowlisted SMART data for AI and export boundaries | | [`ai-gateway-request-v1.schema.json`](./ai-gateway-request-v1.schema.json) | Web-to-Gateway request envelope | | [`ai-gateway-response-v1.schema.json`](./ai-gateway-response-v1.schema.json) | Validated Gateway result or safe error | +| [`token-forge-ai-policy-v1.schema.json`](./token-forge-ai-policy-v1.schema.json) | Token Forge AI budgets, rate limits and circuit bounds | Published Schema files are immutable contracts. Breaking changes require a new version, fixtures, consumer tests and migration notes. diff --git a/schemas/token-forge-ai-policy-v1.schema.json b/schemas/token-forge-ai-policy-v1.schema.json new file mode 100644 index 0000000..5407fff --- /dev/null +++ b/schemas/token-forge-ai-policy-v1.schema.json @@ -0,0 +1,185 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://lab.margrop.net/schemas/token-forge-ai-policy-v1.schema.json", + "title": "Token Forge AI Traffic Policy v1", + "type": "object", + "additionalProperties": false, + "required": [ + "schema_version", + "lab_id", + "operation", + "max_request_billable_tokens", + "max_request_cost_microusd", + "daily_budgets", + "rate_limit", + "concurrency", + "circuit_breaker", + "reservation_ttl_seconds" + ], + "properties": { + "schema_version": { + "const": "1.0" + }, + "lab_id": { + "const": "token-forge" + }, + "operation": { + "const": "token-forge.plan-v1" + }, + "max_request_billable_tokens": { + "type": "integer", + "minimum": 1, + "maximum": 56000 + }, + "max_request_cost_microusd": { + "type": "integer", + "minimum": 1, + "maximum": 100000 + }, + "daily_budgets": { + "type": "object", + "additionalProperties": false, + "required": [ + "actor_tokens", + "lab_tokens", + "site_tokens", + "actor_cost_microusd", + "lab_cost_microusd", + "site_cost_microusd", + "actor_requests", + "lab_requests", + "site_requests" + ], + "properties": { + "actor_tokens": { + "type": "integer", + "minimum": 1, + "maximum": 224000 + }, + "lab_tokens": { + "type": "integer", + "minimum": 1, + "maximum": 2240000 + }, + "site_tokens": { + "type": "integer", + "minimum": 1, + "maximum": 5600000 + }, + "actor_cost_microusd": { + "type": "integer", + "minimum": 1, + "maximum": 250000 + }, + "lab_cost_microusd": { + "type": "integer", + "minimum": 1, + "maximum": 5000000 + }, + "site_cost_microusd": { + "type": "integer", + "minimum": 1, + "maximum": 10000000 + }, + "actor_requests": { + "type": "integer", + "minimum": 1, + "maximum": 12 + }, + "lab_requests": { + "type": "integer", + "minimum": 1, + "maximum": 200 + }, + "site_requests": { + "type": "integer", + "minimum": 1, + "maximum": 400 + } + } + }, + "rate_limit": { + "type": "object", + "additionalProperties": false, + "required": [ + "window_seconds", + "actor_requests", + "lab_requests", + "site_requests" + ], + "properties": { + "window_seconds": { + "type": "integer", + "minimum": 60, + "maximum": 3600 + }, + "actor_requests": { + "type": "integer", + "minimum": 1, + "maximum": 2 + }, + "lab_requests": { + "type": "integer", + "minimum": 1, + "maximum": 20 + }, + "site_requests": { + "type": "integer", + "minimum": 1, + "maximum": 40 + } + } + }, + "concurrency": { + "type": "object", + "additionalProperties": false, + "required": ["actor_in_flight", "lab_in_flight", "site_in_flight"], + "properties": { + "actor_in_flight": { + "type": "integer", + "minimum": 1, + "maximum": 1 + }, + "lab_in_flight": { + "type": "integer", + "minimum": 1, + "maximum": 4 + }, + "site_in_flight": { + "type": "integer", + "minimum": 1, + "maximum": 6 + } + } + }, + "circuit_breaker": { + "type": "object", + "additionalProperties": false, + "required": [ + "consecutive_failures", + "open_seconds", + "half_open_requests" + ], + "properties": { + "consecutive_failures": { + "type": "integer", + "minimum": 1, + "maximum": 5 + }, + "open_seconds": { + "type": "integer", + "minimum": 60, + "maximum": 3600 + }, + "half_open_requests": { + "const": 1 + } + } + }, + "reservation_ttl_seconds": { + "type": "integer", + "minimum": 5, + "maximum": 60 + } + } +}