chore: cache campaign_assistant system prefix via cache_control

[grsmv]

Summary

Adds an ephemeral Anthropic prompt-caching (cache_control) breakpoint on the
system field for the campaign_assistant's planning-model requests. A multi-turn
conversation now reuses the tools + system tokens across turns instead of paying
full input price on every turn (Haiku cached-read is ~0.1× input).

This is token-level prompt caching — distinct from the strict-tool grammar cache
that the existing tool-order stabilizer already keeps warm. Before this change the
campaign_assistant set no cache_control at all, despite a code comment implying
caching "still applied".

How

The breakpoint is injected at the wire, inside the existing tool-order
RoundTripper (server.anthropicToolOrderTransport) that already rewrites the
outgoing /v1/messages body to sort tools. This sidesteps the Genkit abstraction
entirely — no dependency on whether the Genkit Anthropic plugin forwards
cache_control. Anthropic assembles the prefix as tools → system, so one
breakpoint on the last system block caches the tool schemas and the system
prompt together.

addAnthropicSystemCacheControl handles both wire forms of system (a bare JSON
string → promoted to a cached text block; an array → cache_control on the last
block), and is a no-op when there's no system, it's empty, or a breakpoint already
exists.

Scope & safety

• Model-scoped. Caching applies only to requests whose model matches
  cfg.PlanningModelID (the Haiku model behind campaign_assistant). Other flows
  (generation/quality) — whose system prompts vary per request — are left uncached
  so they don't pay the ~1.25× cache-write premium for reads that never come.
• Opt-out preserved. Gated behind the existing AnthropicStableToolOrder flag;
  cachePrefixModel == "" disables the caching while keeping the tool-order sort.
• Tool ordering unchanged for every Anthropic request.

⚠️ Verify before relying on it

The cached prefix (tools + system) is borderline against Haiku 4.5's ~4096-token
minimum (estimated ~4.2–5.0K). Prompt caching is a **silent no-op below
— no error, just no cache. Two checks:

1. Authoritative token count: run messages/count_tokens on a representative
   tools + system payload and confirm it clears ~4096.
2. Post-deploy: on turn 2 of a conversation, confirm usage.cache_read_input_tokens > 0
   (via KindCacheRead metering or ANTHROPIC_DEBUG_HTTP). Zero means th
   under the floor or the date/brief rotated the cache between turns.

Note the cached prefix includes today's date + the live brief (concatenated after the
static prompt), so it rotates daily and on brief edits — fine for the real
pattern (consecutive turns within the 5-min TTL share it).

Test plan

• go build ./... ✅
• go test ./src/server/ ./src/genkit/flows/campaign_assistant/ ✅
• New unit tests cover the transform (string / array / no-op cases) and model-scoped
  transport behavior (matching model caches; other model and empty config

Follow-up (not in this PR)

A call-site variant in run.go could send system as two blocks
([{static, cache_control}, {dynamic}]) so the cached prefix survives across days and
brief edits — but that split can't be done reliably at the wire and depend
Genkit plugin forwarding multi-block cache_control. Deferred.

Summary by CodeRabbit

• Performance

  • Improved Anthropic prompt caching for planning requests, helping reduce repeated prompt processing.
  • Stabilized tool ordering for more consistent request handling.

• Reliability

  • Prompt caching is applied only to the configured planning model, leaving other flows unchanged.
  • Added safeguards to preserve request content when applying caching metadata.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 067f94f8-1c21-4be1-ac21-113d2036200f

📥 Commits

Reviewing files that changed from the base of the PR and between d1c6ab6 and 8d9d124.

📒 Files selected for processing (4)

• src/genkit/flows/campaign_assistant/context.go
• src/server/anthropic_tool_order.go
• src/server/anthropic_tool_order_test.go
• src/server/genkit_runtime.go

---

Walkthrough

The Anthropic transport now adds an ephemeral system cache breakpoint only for requests matching the configured planning model, while retaining tool-order sorting. Runtime wiring supplies that model, and campaign context comments document prompt-prefix determinism.

Changes

Anthropic prompt caching

Layer / File(s)	Summary
Request transformation and coverage src/server/anthropic_tool_order.go, src/server/anthropic_tool_order_test.go	Requests retain deterministic tool ordering; matching models receive an ephemeral system cache breakpoint for string and block-array forms, with no-op and model-scope cases covered by tests.
Runtime model wiring and documentation src/server/genkit_runtime.go, src/genkit/flows/campaign_assistant/context.go	Runtime setup passes PlanningModelID to the stabilizer, and comments describe deterministic static prompt prefixes and dynamic context content.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant newGenkitRuntime
  participant anthropicToolOrderTransport
  participant AnthropicMessagesAPI
  newGenkitRuntime->>anthropicToolOrderTransport: Configure planning model
  anthropicToolOrderTransport->>anthropicToolOrderTransport: Sort tools and match model
  anthropicToolOrderTransport->>AnthropicMessagesAPI: Send request with conditional ephemeral cache_control

Loading

Possibly related PRs

• ogen-app/ogen#68: Uses Anthropic stable tool ordering and planning-model routing affected by this transport integration.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the new Anthropic cache_control prefix caching for campaign assistant system prompts.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/campaign-assistant-prompt-cache

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}