refactor(library): support canonical conversation history in topic safety by Pouyanpi · Pull Request #2237 · NVIDIA-NeMo/Guardrails

Pouyanpi · 2026-07-29T10:56:35Z

Description

Accept canonical messages directly in the topic-safety action.
Preserve event conversion for existing LLMRails execution.
Prevent current-message duplication while preserving multi-turn ordering and metadata.
Characterize explicit and event-derived self-check variants.

Related Issue(s)

Verification

AI Assistance

No AI tools were used.
AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

I've read the CONTRIBUTING guidelines.
This PR links to a triaged issue assigned to me.
My PR title follows the project commit convention.
I've updated the documentation if applicable.
I've added tests if applicable.
I've noted any verification beyond CI and any checks I couldn't run.
I did not update generated changelog files manually.
I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
@mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

Improvements
- Topic safety checks now support canonical conversation history directly, preserving message metadata and turn order.
- Legacy event-based conversations remain supported.
- Explicit self-check variants are consistently honored over inferred legacy context.
Documentation
- Added clarifying documentation for legacy recovery flows and engine-neutral variant handling.
Tests
- Expanded coverage for topic-safety history handling, metadata preservation, and multi-turn conversations.
- Added validation for self-check variant precedence.

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

greptile-apps · 2026-07-29T11:36:33Z


    if cache:
-        cache_key = create_normalized_cache_key(messages)
+        cache_key = create_normalized_cache_key(prompt_messages)


Exclude metadata from cache identity

Canonical messages now pass volatile fields such as provider request IDs into create_normalized_cache_key, even though those fields are removed before model inference. Model-equivalent conversations therefore receive different cache keys, causing unnecessary topic-safety model calls and reducing cache effectiveness.

Knowledge Base Used: Library Rails

Prompt To Fix With AI

This is a comment left during a code review. Path: nemoguardrails/library/topic_safety/actions.py Line: 127 Comment: **Exclude metadata from cache identity** Canonical messages now pass volatile fields such as provider request IDs into `create_normalized_cache_key`, even though those fields are removed before model inference. Model-equivalent conversations therefore receive different cache keys, causing unnecessary topic-safety model calls and reducing cache effectiveness. **Knowledge Base Used:** [Library Rails](https://app.greptile.com/nvidia-public-github/-/custom-context/knowledge-base/nvidia-nemo/guardrails/-/docs/library-rails.md) --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-07-29T11:36:36Z

Greptile Summary

The PR teaches the topic-safety action to consume canonical conversation histories while preserving event-based compatibility.

Gives canonical messages precedence over legacy event conversion and avoids re-appending the current turn.
Preserves message ordering and metadata when constructing the topic-safety prompt.
Adds focused unit and end-to-end multi-turn coverage.
Clarifies why self-check and topic-safety actions retain legacy event parameters.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking cache-efficiency issue for canonical messages carrying volatile provider metadata.

Canonical conversation handling is covered across direct and LLMRails execution, but cache keys are computed from metadata-bearing dictionaries even though provider metadata is removed before model inference.

Files Needing Attention: nemoguardrails/library/topic_safety/actions.py

Important Files Changed

Filename	Overview
nemoguardrails/library/topic_safety/actions.py	Adds canonical-message handling and preserves the legacy event fallback, but volatile provider metadata now participates in cache identity.
nemoguardrails/library/self_check/input_check/actions.py	Adds explanatory comments distinguishing legacy event injection from manifest-bound variants without changing behavior.
nemoguardrails/library/self_check/output_check/actions.py	Adds explanatory comments distinguishing legacy event injection from manifest-bound variants without changing behavior.
tests/test_topic_safety_actions.py	Adds focused coverage for canonical metadata, event-derived history, absent history, and empty canonical history.
tests/test_topic_safety_railoutcome_flow.py	Adds integration coverage proving multi-turn topic-safety prompt ordering without duplication.
tests/test_multiple_self_check_rails.py	Extends task-resolution coverage to prove explicit variants override both legacy context and event-derived variants.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Topic-safety action] --> B{Canonical messages supplied?}
    B -->|Yes| C[Copy canonical conversation]
    B -->|No| D[Convert legacy Colang events]
    D --> E[Append current user input]
    C --> F[Prepend system prompt]
    E --> F
    F --> G[Topic-safety LLM call]
    G --> H[Allow or block RailOutcome]

Prompt To Fix All With AI

### Issue 1
nemoguardrails/library/topic_safety/actions.py:127
**Exclude metadata from cache identity**

Canonical messages now pass volatile fields such as provider request IDs into `create_normalized_cache_key`, even though those fields are removed before model inference. Model-equivalent conversations therefore receive different cache keys, causing unnecessary topic-safety model calls and reducing cache effectiveness.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "docs(library): explain topic safety even..." | Re-trigger Greptile}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nemoguardrails/library/self_check/input_check/actions.py`:
- Around line 46-48: Remove the added inline comment blocks from both self-check
actions: nemoguardrails/library/self_check/input_check/actions.py lines 46-48
and nemoguardrails/library/self_check/output_check/actions.py lines 46-48. No
code behavior changes are required.

In `@tests/test_topic_safety_railoutcome_flow.py`:
- Around line 75-93: Update
test_multiturn_history_reaches_topic_safety_once_and_in_order to assert
topic_model.generate_async was awaited exactly once before accessing await_args.
Keep the existing prompt ordering assertions unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1d9b25a5-71c4-453f-b30f-43c4f0769aaa

📥 Commits

Reviewing files that changed from the base of the PR and between ccc8255 and 3a8faa1.

📒 Files selected for processing (6)

nemoguardrails/library/self_check/input_check/actions.py
nemoguardrails/library/self_check/output_check/actions.py
nemoguardrails/library/topic_safety/actions.py
tests/test_multiple_self_check_rails.py
tests/test_topic_safety_actions.py
tests/test_topic_safety_railoutcome_flow.py

coderabbitai · 2026-07-29T11:43:32Z

+    # LLMRails injects Colang events only to recover variants for legacy 1.0
+    # flows, including parallel rails. Manifests bind `variant` instead so
+    # engine-neutral callers do not provide or synthesize events.


📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove the unrequested inline comments in both self-check actions.

The repository guideline prohibits adding comments in Python files unless explicitly requested.

nemoguardrails/library/self_check/input_check/actions.py#L46-L48: remove the added comment block or confirm explicit maintainer approval.

nemoguardrails/library/self_check/output_check/actions.py#L46-L48: remove the added comment block or confirm explicit maintainer approval.

📍 Affects 2 files

nemoguardrails/library/self_check/input_check/actions.py#L46-L48 (this comment)

nemoguardrails/library/self_check/output_check/actions.py#L46-L48

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nemoguardrails/library/self_check/input_check/actions.py` around lines 46 - 48, Remove the added inline comment blocks from both self-check actions: nemoguardrails/library/self_check/input_check/actions.py lines 46-48 and nemoguardrails/library/self_check/output_check/actions.py lines 46-48. No code behavior changes are required.

Source: Coding guidelines

coderabbitai · 2026-07-29T11:43:32Z

+def test_multiturn_history_reaches_topic_safety_once_and_in_order():
+    chat = _chat_with_verdict("on-topic")
+    topic_model = chat.app.runtime.registered_action_params["llms"]["topic_control"]
+    topic_model.generate_async = AsyncMock(wraps=topic_model.generate_async)
+
+    chat.app.generate(
+        messages=[
+            {"role": "user", "content": "earlier question"},
+            {"role": "assistant", "content": "earlier answer"},
+            {"role": "user", "content": "current question"},
+        ]
+    )
+
+    prompt = topic_model.generate_async.await_args.args[0]
+    assert [(message.role, message.content) for message in prompt[1:]] == [
+        ("user", "earlier question"),
+        ("assistant", "earlier answer"),
+        ("user", "current question"),
+    ]


🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash set -euo pipefail uv run --locked python - <<'PY' from unittest.mock import AsyncMock assert hasattr(AsyncMock(), "assert_awaited_once") PY

Repository: NVIDIA-NeMo/Guardrails

Length of output: 200

🏁 Script executed:

#!/usr/bin/env bash set -euo pipefail python3 - <<'PY' from unittest.mock import AsyncMock print("assert_awaited_once", hasattr(AsyncMock(), "assert_awaited_once")) PY printf '\n--- file outline ---\n' ast-grep outline tests/test_topic_safety_railoutcome_flow.py --view expanded || true printf '\n--- relevant lines ---\n' cat -n tests/test_topic_safety_railoutcome_flow.py | sed -n '1,180p' PY

Repository: NVIDIA-NeMo/Guardrails

Length of output: 4622

🏁 Script executed:

#!/usr/bin/env bash set -euo pipefail python3 - <<'PY' from unittest.mock import AsyncMock print("assert_awaited_once", hasattr(AsyncMock(), "assert_awaited_once")) PY printf '\n--- file outline ---\n' ast-grep outline tests/test_topic_safety_railoutcome_flow.py --view expanded || true printf '\n--- relevant lines ---\n' cat -n tests/test_topic_safety_railoutcome_flow.py | sed -n '1,180p'

Repository: NVIDIA-NeMo/Guardrails

Length of output: 4581

🏁 Script executed:

#!/usr/bin/env bash set -euo pipefail python3 - <<'PY' from unittest.mock import AsyncMock print("assert_awaited_once", hasattr(AsyncMock(), "assert_awaited_once")) PY printf '\n--- file outline ---\n' ast-grep outline tests/test_topic_safety_railoutcome_flow.py --view expanded || true printf '\n--- relevant lines ---\n' sed -n '1,180p' tests/test_topic_safety_railoutcome_flow.py | cat -n

Repository: NVIDIA-NeMo/Guardrails

Length of output: 4581

🏁 Script executed:

#!/usr/bin/env bash set -euo pipefail python3 - <<'PY' from unittest.mock import AsyncMock print("assert_awaited_once", hasattr(AsyncMock(), "assert_awaited_once")) PY echo '--- relevant lines ---' nl -ba tests/test_topic_safety_railoutcome_flow.py | sed -n '70,100p'

Repository: NVIDIA-NeMo/Guardrails

Length of output: 249

Assert the topic model is awaited exactly once.

await_args only reflects the last await, so this test can still pass if generate_async runs multiple times. Add assert_awaited_once() before reading await_args.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_topic_safety_railoutcome_flow.py` around lines 75 - 93, Update test_multiturn_history_reaches_topic_safety_once_and_in_order to assert topic_model.generate_async was awaited exactly once before accessing await_args. Keep the existing prompt ordering assertions unchanged.

Pouyanpi added 3 commits July 29, 2026 11:36

test(actions): characterize canonical topic and self-check inputs

1e4e141

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

refactor(actions): accept canonical topic safety history

d74395e

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

test(actions): verify topic safety history through LLMRails

69ef27f

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

Pouyanpi added this to the v0.24.0 milestone Jul 29, 2026

Pouyanpi self-assigned this Jul 29, 2026

github-actions Bot added status: needs triage New issues that have not yet been reviewed or categorized. size: M labels Jul 29, 2026

Pouyanpi added 2 commits July 29, 2026 13:06

docs(library): explain legacy self-check events

d1448af

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

docs(library): explain topic safety event fallback

3a8faa1

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>

Pouyanpi marked this pull request as ready for review July 29, 2026 11:34

Pouyanpi added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Jul 29, 2026

greptile-apps Bot reviewed Jul 29, 2026

View reviewed changes

coderabbitai Bot reviewed Jul 29, 2026

View reviewed changes

immu4989 mentioned this pull request Aug 2, 2026

feature: multi-turn jailbreak detection rail (Crescendo, role-play, progressive specificity) #2028

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(library): support canonical conversation history in topic safety - #2237

refactor(library): support canonical conversation history in topic safety#2237
Pouyanpi wants to merge 5 commits into
developfrom
pouyanpi/actions-canonical-conversation-inputs

Pouyanpi commented Jul 29, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

greptile-apps Bot Jul 29, 2026

Uh oh!

greptile-apps Bot commented Jul 29, 2026

Confidence Score: 4/5

Flowchart

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Jul 29, 2026

Uh oh!

coderabbitai Bot Jul 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Pouyanpi commented Jul 29, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Verification

AI Assistance

Checklist

Summary by CodeRabbit

Uh oh!

greptile-apps Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Jul 29, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Flowchart

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Pouyanpi commented Jul 29, 2026 •

edited by coderabbitai Bot

Loading