Guardrails: migrate guardrails to the v1/checks endpoint

[liavweiss]

Summary

• Migrate the ai_guardrails NeMo provider from /v1/guardrail/checks to /v1/checks
• Parse the new response shape (passed / blocked / modified + content + rail)
• Evaluate each user/assistant message with cumulative context (one NeMo call per message)
• Map provider HTTP errors to fail-closed behavior (replaces in-body "error" status)

Breaking change

NeMo endpoint URLs in proxy config must use /v1/checks instead of /v1/guardrail/checks.

Out of scope

• Applying modified/content redaction to request/response bodies (follow-up: ai_guardrails: NeMo mask / redact action #49)

Test plan

• cargo test -p praxis-ai-filters guardrails (70 tests)
• cargo test -p praxis-tests-integration guardrails (35 tests)
• A real nemo test with a real v1/chat/completion passing forward the ai_guardrails filter.

close: #1044

[praxis-bot]

PR Review

Migrates the NeMo guardrails provider from /v1/guardrail/checks to /v1/checks, parsing the new response shape and issuing cumulative per-message evaluations.

Assessment

Clean migration with solid test coverage for the new response mapping and cumulative evaluation logic. Two findings.

Severity	Count
Large	1
Medium	1

[leseb]

P1 – modified short-circuits evaluation. A modified earlier message causes later blocked messages to go unchecked and pass through.

P1 – wrong NeMo rail direction. Mixed user/assistant history auto-runs both rail types; request-phase modifications can be reported as passed. Send phase-specific rail_types.

P1 – unbounded callout amplification. One request can trigger arbitrarily many sequential NeMo calls, each with its own timeout. Add a call limit or overall deadline.

[liavweiss]

P1 – modified short-circuits evaluation. A modified earlier message causes later blocked messages to go unchecked and pass through.

Addressed - modified no longer short-circuits the loop; only blocked fails fast. See apply_slice_result.

P1 – wrong NeMo rail direction. Mixed user/assistant history auto-runs both rail types; request-phase modifications can be reported as passed. Send phase-specific rail_types.

Addressed - we now send explicit rail_types (["input"] for request phase, ["output"] for response phase) to prevent NeMo from auto-running both rail families on mixed-role slices.

P1 – unbounded callout amplification. One request can trigger arbitrarily many sequential NeMo calls, each with its own timeout. Add a call limit or overall deadline.

Addressed - same max_message_checks limit as item 1.

[leseb]

@liavweiss

P1: timeout_ms restarts for each of up to 32 sequential calls, permitting roughly 320 seconds total by default. Use one overall evaluation deadline.
P2: Valid NeMo modified responses with empty content are rejected. Empty-string redaction must be accepted; only a missing field should fail.
P2 docs: Document the NeMo v0.24+ requirement and the new max_message_checks: 32 fail-closed limit.
P3: Example and PR description still reference #579; the correct redaction issue is #49.

[liavweiss]

@liavweiss

P1: timeout_ms restarts for each of up to 32 sequential calls, permitting roughly 320 seconds total by default. Use one overall evaluation deadline.
P2: Valid NeMo modified responses with empty content are rejected. Empty-string redaction must be accepted; only a missing field should fail.
P2 docs: Document the NeMo v0.24+ requirement and the new max_message_checks: 32 fail-closed limit.
P3: Example and PR description still reference #579; the correct redaction issue is #49.

Done.
Regarding the "only a missing field should fail." - nemo will always return the content field.

[leseb]

@liavweiss lint is failing