feat(server): allow tool calls outside passthrough mode without dialog flows

docs/configure-rails/guardrail-catalog/tool-calling.mdx

@@ -348,7 +348,7 @@ IORails streams the text response as it arrives, accumulates the tool-call fragm
 
 The NVIDIA NeMo Guardrails library includes other tool-related capabilities that are distinct from the IORails tool-calling rails described here:
 
-- [Tools Integration](/integration-with-third-party-libraries/tools-integration) covers LangChain tool passthrough and output-rail validation on the `LLMRails` engine.
+- [Tools Integration](/integration-with-third-party-libraries/tools-integration) covers LangChain tool calling and output-rail validation on the `LLMRails` engine.
 - [Rail types](/about-nemo-guardrails-library/rail-types) describes Colang execution rails, which run actions before and after execution within the `LLMRails` event-driven pipeline.
 
 The rails on this page operate at the request and response boundary of the IORails engine and do not execute tools.

docs/integration/langchain/langgraph-integration.mdx

@@ -401,7 +401,7 @@ result = graph.invoke({
 
 ### 1. Passthrough Mode Configuration
 
-For tool calling and complex flows, use `passthrough=True` to maintain the original prompt structure:
+Use `passthrough=True` to maintain the original prompt structure for complex flows.
 
 ```python
 guardrails = RunnableRails(config=config, passthrough=True)
@@ -427,7 +427,7 @@ guardrails = RunnableRails(config=config, passthrough=True, verbose=True)
 
 ### Common Issues
 
-1. **Empty Content with Tool Calls**: When using tools, ensure `passthrough=True` is set.
+1. **Empty Content with Tool Calls**: Check that the guardrails configuration does not define dialog flows.
 2. **Authorization Errors**: Verify API keys for both main model and safety models.
 3. **Configuration Not Found**: Ensure guardrail config paths are correct.
 

docs/integration/langchain/runnable-rails.mdx

@@ -199,14 +199,14 @@ The `passthrough` parameter controls this behavior.
 - **`passthrough=False`**: Allows guardrails to modify prompts for enhanced protection.
 
 ```python
-# Minimal intervention (required for tool calling)
+# Minimal intervention
 guardrails = RunnableRails(config, passthrough=True)
 
 # Enhanced guardrails (modifies prompts as needed)
 guardrails = RunnableRails(config, passthrough=False)
 ```
 
-**Tool Calling Requirement**: Set `passthrough=True` for proper tool call handling.
+**Tool Calling**: Tool calls work with either `passthrough` setting, as long as the guardrails configuration does not define dialog flows (`user_messages` or `rails.dialog.single_call.enabled`).
 
 ### Custom Input/Output Keys
 
@@ -238,13 +238,13 @@ When a guardrail is triggered and predefined messages must be returned instead o
 
 ## Tool Calling
 
-`RunnableRails` supports LangChain tool calling with full metadata preservation and streaming. Tool calling requires `passthrough=True` to work properly.
+`RunnableRails` supports LangChain tool calling with full metadata preservation and streaming. Tool calling works with or without `passthrough`, as long as the guardrails configuration does not define dialog flows (`user_messages` or `rails.dialog.single_call.enabled`). Configurations that define dialog flows drop tool calls and return only the generated text.
 
-The following steps are required to use tool calling with `RunnableRails`:
+To use tool calling with `RunnableRails`:
 
-- Set `passthrough=True` when creating `RunnableRails` instance.
 - Use `bind_tools()` to attach tools to your model.
 - Handle tool execution in your application logic.
+- Avoid guardrail configurations that define dialog flows if you need tool calls to surface in the response.
 
 ### Basic Tool Setup
 

docs/integration/tools-integration.mdx

@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: Apache-2.0
 title: "Tools Integration with the NeMo Guardrails Library"
 sidebar-title: "Tools Integration"
-description: "Integrate LangChain tools with guardrails using passthrough mode and output rail validation."
+description: "Integrate LangChain tools with guardrails and output rail validation."
 content:
   type: "how_to"
 ---
@@ -52,9 +52,11 @@ For detailed information on creating custom tools, refer to the [LangChain Tools
 
 ## Configuration Settings
 
-### Passthrough Mode
+### Passthrough Mode and Dialog Flows
 
-When using tools with the NeMo Guardrails library, it's recommended to use **passthrough mode**. This mode is essential because:
+Tool calls are supported in any guardrails configuration, with or without `passthrough`, as long as the configuration does not define dialog flows. A configuration defines dialog flows when it declares `user_messages` (canonical form) or when `rails.dialog.single_call.enabled` is `true`. Those dialog-flow paths drop tool calls and return only the generated text.
+
+If your configuration does not use dialog flows, using **passthrough mode** is still recommended because:
 
 - Internal NeMo Guardrails library tasks do not require tool use and might provide erroneous results if tools are enabled
 - It ensures that the LLM can properly handle tool calls and responses

docs/reference/engine-feature-support.mdx

@@ -135,7 +135,7 @@ A topic-control or jailbreak flow on the output rail is a fallback condition.
 | Tool-result validation rail | ✓ | ✓ | IORails flow: `tool result validation` |
 
 Both engines support passing model tool calls through to the caller and validating tool calls and tool results.
-`LLMRails` handles these through the Colang runtime and tool rails.
+`LLMRails` handles these through the Colang runtime and tool rails, and surfaces tool calls regardless of the `passthrough` setting, as long as the configuration does not define dialog flows (`user_messages` or `rails.dialog.single_call.enabled`); those flows drop tool calls and return only the generated text.
 
 `IORails` validates tool calls and tool results through directional flows: `tool call validation` on the tool-output rail and `tool result validation` on the tool-input rail.
 Tool calls are returned in the OpenAI-style `tool_calls` field of the response message.

nemoguardrails/actions/llm/generation.py

@@ -32,6 +32,7 @@
 from nemoguardrails.actions.llm.utils import (
     flow_to_colang,
     get_and_clear_reasoning_trace_contextvar,
+    get_and_clear_tool_calls_contextvar,
     get_first_nonempty_line,
     get_last_bot_intent_event,
     get_last_user_intent_event,
@@ -653,17 +654,10 @@ async def _emit_general_bot_turn(
             context_updates["bot_thinking"] = reasoning_trace
             output_events.append(new_event_dict("BotThinking", content=reasoning_trace))
 
-        if self.config.passthrough:
-            from nemoguardrails.actions.llm.utils import (
-                get_and_clear_tool_calls_contextvar,
-            )
+        tool_calls = get_and_clear_tool_calls_contextvar()
 
-            tool_calls = get_and_clear_tool_calls_contextvar()
-
-            if tool_calls:
-                output_events.append(new_event_dict("BotToolCalls", tool_calls=tool_calls))
-            else:
-                output_events.append(new_event_dict("BotMessage", text=text))
+        if tool_calls:
+            output_events.append(new_event_dict("BotToolCalls", tool_calls=tool_calls))
         else:
             output_events.append(new_event_dict("BotMessage", text=text))
 

nemoguardrails/server/api.py

@@ -542,17 +542,29 @@ async def chat_completion(body: GuardrailsChatCompletionRequest, request: Reques
             detail="thread_id message-history replay is not supported for Colang 2.0.",
         )
 
-    if (body.tools or body.tool_choice is not None or body.parallel_tool_calls is not None) and (
-        llm_rails.config.passthrough is not True or body.stream
-    ):
-        raise HTTPException(
-            status_code=422,
-            detail=(
-                "The 'tools', 'tool_choice', and 'parallel_tool_calls' parameters are only "
-                "supported for non-streaming requests when the guardrails configuration has 'passthrough: true'."
-            ),
-        )
+    tools_requested = body.tools or body.tool_choice is not None or body.parallel_tool_calls is not None
+    if tools_requested:
+        if body.stream:
+            raise HTTPException(
+                status_code=422,
+                detail=(
+                    "The 'tools', 'tool_choice', and 'parallel_tool_calls' parameters are only "
+                    "supported for non-streaming requests."
+                ),
+            )
 
+        # Dialog-flow configs (canonical-form user_messages or single_call dialog
+        # rails) never surface tool calls to the request, so reject explicitly
+        has_dialog_flows = bool(llm_rails.config.user_messages) or llm_rails.config.rails.dialog.single_call.enabled
+        if has_dialog_flows:
+            raise HTTPException(
+                status_code=422,
+                detail=(
+                    "The 'tools', 'tool_choice', and 'parallel_tool_calls' parameters are not "
+                    "supported when the guardrails configuration defines dialog flows "
+                    "(user_messages or single_call dialog rails)."
+                ),
+            )
     try:
         messages = body.messages or []
         if body.guardrails.context:

tests/integrations/langchain/test_tool_calling_passthrough_only.py → tests/integrations/langchain/test_tool_calling_passthrough.py

@@ -13,7 +13,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Test that tool calling ONLY works in passthrough mode."""
+"""Test tool-calling behavior across passthrough and non-dialog-flow general modes."""
 
 from unittest.mock import MagicMock
 
@@ -94,16 +94,12 @@ def config_no_passthrough():
 
 
 class TestToolCallingPassthroughOnly:
-    """Test that tool calling only works in passthrough mode."""
+    """Test that tool calling works in passthrough mode."""
 
     def test_config_passthrough_true(self, config_passthrough):
         """Test that passthrough config is correctly set."""
         assert config_passthrough.passthrough is True
 
-    def test_config_passthrough_false(self, config_no_passthrough):
-        """Test that non-passthrough config is correctly set."""
-        assert config_no_passthrough.passthrough is False
-
     @pytest.mark.asyncio
     async def test_tool_calls_work_in_passthrough_mode(self, config_passthrough, mock_llm_with_tool_calls):
         """Test that tool calls create BotToolCalls events in passthrough mode."""
@@ -136,13 +132,54 @@ async def test_tool_calls_work_in_passthrough_mode(self, config_passthrough, moc
         assert len(result.events) == 1
         assert result.events[0]["type"] == "BotToolCalls"
         stored = result.events[0]["tool_calls"]
+
         assert len(stored) == 1
         assert stored[0]["function"]["name"] == "test_tool"
         assert stored[0]["function"]["arguments"] == {"param": "value"}
         assert stored[0]["id"] == "call_123"
 
     @pytest.mark.asyncio
-    async def test_tool_calls_ignored_in_non_passthrough_mode(self, config_no_passthrough, mock_llm_with_tool_calls):
+    async def test_no_tool_calls_creates_bot_message_in_passthrough(self, config_passthrough, mock_llm_with_tool_calls):
+        """Test that no tool calls creates BotMessage event even in passthrough mode."""
+        tool_calls_var.set(None)
+
+        mock_response_no_tools = AIMessage(content="Regular text response")
+        mock_llm_with_tool_calls.ainvoke.return_value = mock_response_no_tools
+        mock_llm_with_tool_calls.invoke.return_value = mock_response_no_tools
+
+        generation_actions = LLMGenerationActions(
+            config=config_passthrough,
+            llm=LangChainLLMAdapter(mock_llm_with_tool_calls),
+            llm_task_manager=MagicMock(),
+            get_embedding_search_provider_instance=MagicMock(return_value=None),
+        )
+
+        events = [{"type": "UserMessage", "text": "test"}]
+        context = {}
+
+        result = await generation_actions.generate_user_intent(
+            events=events, context=context, config=config_passthrough
+        )
+
+        assert len(result.events) == 1
+        assert result.events[0]["type"] == "BotMessage"
+
+    def test_llm_rails_integration_passthrough_mode(self, config_passthrough, mock_llm_with_tool_calls):
+        """Test LLMRails with passthrough mode allows tool calls."""
+        rails = LLMRails(config=config_passthrough, llm=LangChainLLMAdapter(mock_llm_with_tool_calls))
+
+        assert rails.config.passthrough is True
+
+
+class TestToolCallingNonPassthrough:
+    """Test that tool calling works in non-passthrough mode."""
+
+    def test_config_no_passthrough_true(self, config_no_passthrough):
+        """Test that non-passthrough config is correctly set."""
+        assert config_no_passthrough.passthrough is False
+
+    @pytest.mark.asyncio
+    async def test_tool_calls_non_passthrough_mode(self, config_no_passthrough, mock_llm_with_tool_calls):
         """Test that tool calls are ignored when not in passthrough mode."""
         tool_calls = [
             {
@@ -169,20 +206,27 @@ async def test_tool_calls_ignored_in_non_passthrough_mode(self, config_no_passth
         )
 
         assert len(result.events) == 1
-        assert result.events[0]["type"] == "BotMessage"
-        assert "tool_calls" not in result.events[0]
+        assert result.events[0]["type"] == "BotToolCalls"
+        stored = result.events[0]["tool_calls"]
+
+        assert len(stored) == 1
+        assert stored[0]["function"]["name"] == "test_tool"
+        assert stored[0]["function"]["arguments"] == {"param": "value"}
+        assert stored[0]["id"] == "call_123"
 
     @pytest.mark.asyncio
-    async def test_no_tool_calls_creates_bot_message_in_passthrough(self, config_passthrough, mock_llm_with_tool_calls):
-        """Test that no tool calls creates BotMessage event even in passthrough mode."""
+    async def test_no_tool_calls_creates_bot_message_in_non_passthrough(
+        self, config_no_passthrough, mock_llm_with_tool_calls
+    ):
+        """Test that no tool calls creates BotMessage event even in non-passthrough mode."""
         tool_calls_var.set(None)
 
         mock_response_no_tools = AIMessage(content="Regular text response")
         mock_llm_with_tool_calls.ainvoke.return_value = mock_response_no_tools
         mock_llm_with_tool_calls.invoke.return_value = mock_response_no_tools
 
         generation_actions = LLMGenerationActions(
-            config=config_passthrough,
+            config=config_no_passthrough,
             llm=LangChainLLMAdapter(mock_llm_with_tool_calls),
             llm_task_manager=MagicMock(),
             get_embedding_search_provider_instance=MagicMock(return_value=None),
@@ -192,20 +236,106 @@ async def test_no_tool_calls_creates_bot_message_in_passthrough(self, config_pas
         context = {}
 
         result = await generation_actions.generate_user_intent(
-            events=events, context=context, config=config_passthrough
+            events=events, context=context, config=config_no_passthrough
         )
 
         assert len(result.events) == 1
-        assert result.events[0]["type"] == "BotMessage"
+        assert result.events[0]["type"] == "BotToolCalls"
+        stored = result.events[0]["tool_calls"]
+        assert len(stored) == 1
+        assert stored[0]["function"]["name"] == "test_tool"
+        assert stored[0]["function"]["arguments"] == {"param": "value"}
+        assert stored[0]["id"] == "call_123"
 
-    def test_llm_rails_integration_passthrough_mode(self, config_passthrough, mock_llm_with_tool_calls):
-        """Test LLMRails with passthrough mode allows tool calls."""
-        rails = LLMRails(config=config_passthrough, llm=LangChainLLMAdapter(mock_llm_with_tool_calls))
+    @pytest.mark.asyncio
+    async def test_tool_calls_with_prompt_mode_non_passthrough(self, config_no_passthrough, mock_llm_with_tool_calls):
+        """Test that tool calls are ignored when not in passthrough mode."""
+        tool_calls = [
+            {
+                "id": "call_123",
+                "type": "tool_call",
+                "name": "test_tool",
+                "args": {"param": "value"},
+            }
+        ]
+        tool_calls_var.set(tool_calls)
 
-        assert rails.config.passthrough is True
+        generation_actions = LLMGenerationActions(
+            config=config_no_passthrough,
+            llm=LangChainLLMAdapter(mock_llm_with_tool_calls),
+            llm_task_manager=MagicMock(),
+            get_embedding_search_provider_instance=MagicMock(return_value=None),
+        )
+
+        events = [{"type": "UserMessage", "text": "test"}]
+        context = {}
+
+        result = await generation_actions.generate_user_intent(
+            events=events, context=context, config=config_no_passthrough
+        )
+        assert len(result.events) == 1
+        assert result.events[0]["type"] == "BotToolCalls"
+        stored = result.events[0]["tool_calls"]
+
+        assert len(stored) == 1
+        assert stored[0]["function"]["name"] == "test_tool"
+        assert stored[0]["function"]["arguments"] == {"param": "value"}
+        assert stored[0]["id"] == "call_123"
+
+    @pytest.mark.asyncio
+    async def test_complex_tool_calls_non_passthrough(self, config_no_passthrough, mock_llm_with_tool_calls):
+        """Test that complex tool calls are ignored when not in passthrough mode."""
+        tool_calls = [
+            {
+                "id": "call_123",
+                "type": "tool_call",
+                "name": "test_tool",
+                "args": {"param": "value"},
+            }
+        ]
+        tool_calls_var.set(tool_calls)
+        generation_actions = LLMGenerationActions(
+            config=config_no_passthrough,
+            llm=LangChainLLMAdapter(mock_llm_with_tool_calls),
+            llm_task_manager=MagicMock(),
+            get_embedding_search_provider_instance=MagicMock(return_value=None),
+        )
 
-    def test_llm_rails_integration_non_passthrough_mode(self, config_no_passthrough, mock_llm_with_tool_calls):
-        """Test LLMRails without passthrough mode."""
-        rails = LLMRails(config=config_no_passthrough, llm=LangChainLLMAdapter(mock_llm_with_tool_calls))
+        events = [{"type": "UserMessage", "text": "test"}]
+        context = {}
 
-        assert rails.config.passthrough is False
+        result = await generation_actions.generate_user_intent(
+            events=events, context=context, config=config_no_passthrough
+        )
+        assert len(result.events) == 1
+        assert result.events[0]["type"] == "BotToolCalls"
+        stored = result.events[0]["tool_calls"]
+        assert len(stored) == 1
+        assert stored[0]["function"]["name"] == "test_tool"
+        assert stored[0]["function"]["arguments"] == {"param": "value"}
+        assert stored[0]["id"] == "call_123"
+
+    @pytest.mark.asyncio
+    async def test_get_and_clear_tool_calls_called_correctly_non_passthrough(
+        self, config_no_passthrough, mock_llm_with_tool_calls
+    ):
+        generation_actions = LLMGenerationActions(
+            config=config_no_passthrough,
+            llm=LangChainLLMAdapter(mock_llm_with_tool_calls),
+            llm_task_manager=MagicMock(),
+            get_embedding_search_provider_instance=MagicMock(return_value=None),
+        )
+
+        events = [{"type": "UserMessage", "text": "test"}]
+        context = {}
+
+        result = await generation_actions.generate_user_intent(
+            events=events, context=context, config=config_no_passthrough
+        )
+        assert len(result.events) == 1
+        assert result.events[0]["type"] == "BotToolCalls"
+        stored = result.events[0]["tool_calls"]
+        assert len(stored) == 1
+        assert stored[0]["function"]["name"] == "test_tool"
+        assert stored[0]["function"]["arguments"] == {"param": "value"}
+        assert stored[0]["id"] == "call_123"