feat(iorails): Inference-time http headers - #2227
Draft
tgasser-nv wants to merge 6 commits into
Draft
Conversation
github-actions
Bot
added
status: needs triage
tgasser-nv
added
status: triaged
tgasser-nv
marked this pull request as ready for review
July 27, 2026 22:00
Contributor
Greptile SummaryAdds request-scoped HTTP headers to the IORails inference pipeline.
|
| Filename | Overview |
|---|---|
| nemoguardrails/guardrails/guardrails.py | Extends the public generation and streaming facade with explicit IORails header forwarding and LLMRails rejection. |
| nemoguardrails/guardrails/iorails.py | Binds inference headers to non-streaming and streaming request lifecycles before rail and model execution. |
| nemoguardrails/guardrails/guardrails_types.py | Introduces the request-scoped header ContextVar, coercion, nesting, and cross-context teardown handling. |
| nemoguardrails/guardrails/engine_registry.py | Reads request headers at the outbound-call choke points and keeps them separate from provider request bodies. |
| nemoguardrails/guardrails/model_engine.py | Implements configured and inference-time header precedence while reserving transport-only parameter names. |
| nemoguardrails/guardrails/api_engine.py | Applies inference-time headers to arbitrary API rail requests without adding them to JSON bodies. |
| nemoguardrails/guardrails/_http.py | Adds a non-mutating, case-insensitive header merge helper. |
| docs/configure-rails/yaml-schema/model-configuration.mdx | Documents configured and per-request headers, precedence, coercion, and IORails-only runtime support. |
Sequence Diagram
sequenceDiagram
participant Client
participant Facade as Guardrails
participant IO as IORails
participant Context as Request Header Context
participant Registry as EngineRegistry
participant Engine as ModelEngine/APIEngine
participant Provider
Client->>Facade: generate/stream(http_headers)
Facade->>IO: Forward request headers
IO->>Context: Bind headers for request scope
IO->>Registry: Run main and rail calls
Registry->>Context: Read current headers
Registry->>Engine: Call with http_headers
Engine->>Engine: Merge base, default, request headers
Engine->>Provider: HTTP request
Provider-->>Engine: Response
Engine-->>Client: Guarded response or stream
IO->>Context: Restore prior header context
Reviews (1): Last reviewed commit: "Initial checkin of inference-time http h..." | Re-trigger Greptile
Contributor
📝 WalkthroughWalkthroughAdds ChangesHTTP header composition
Request propagation
End-to-end validation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant IORails
participant EngineRegistry
participant ModelEngine
participant Provider
Caller->>IORails: generate or stream with http_headers
IORails->>IORails: Bind request-scoped headers
IORails->>EngineRegistry: Execute provider calls
EngineRegistry->>ModelEngine: Pass request headers
ModelEngine->>Provider: Send merged HTTP headers
IORails->>IORails: Reset request-scoped headers
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nemoguardrails/guardrails/model_engine.py`:
- Around line 91-100: Update the parameter-merging flow in the model engine to
remove or reject the runtime GenerationOptions.llm_params entry named
http_headers before forwarding arguments to engine.chat_completion(). Ensure it
cannot collide with the explicit http_headers keyword, while preserving the
existing filtering of config-time parameters.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5e0ed380-df0b-4352-a4e9-51e0a3000e26
📒 Files selected for processing (16)
docs/configure-rails/yaml-schema/model-configuration.mdxnemoguardrails/guardrails/_http.pynemoguardrails/guardrails/api_engine.pynemoguardrails/guardrails/engine_registry.pynemoguardrails/guardrails/guardrails.pynemoguardrails/guardrails/guardrails_types.pynemoguardrails/guardrails/iorails.pynemoguardrails/guardrails/model_engine.pytests/guardrails/test__http.pytests/guardrails/test_api_engine.pytests/guardrails/test_engine_registry.pytests/guardrails/test_guardrails.pytests/guardrails/test_guardrails_types.pytests/guardrails/test_iorails.pytests/guardrails/test_iorails_streaming.pytests/guardrails/test_model_engine.py
Comment on lines
+91
to
+100
| # client-only options — these configure transport, not the | ||
| # chat-completion request body, so they are never forwarded as body | ||
| # fields. | ||
| "default_headers", | ||
| "default_query", | ||
| # inference-time header argument name — reserved so a model configured | ||
| # with `parameters.http_headers` (the per-request argument mistaken for | ||
| # the config key `default_headers`) can't collide with the explicit | ||
| # http_headers keyword the engine registry passes into chat_completion(). | ||
| "http_headers", |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check whether llm_params / GenerationOptions filters reserved keys like "http_headers" before merging.
rg -nP -C5 'llm_params' nemoguardrails/guardrails/engine_registry.py nemoguardrails/rails/llm/options.py 2>/dev/nullRepository: NVIDIA-NeMo/Guardrails
Length of output: 5098
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '190,230p' nemoguardrails/guardrails/engine_registry.py
printf '\n---\n'
sed -n '1,220p' nemoguardrails/guardrails/model_engine.pyRepository: NVIDIA-NeMo/Guardrails
Length of output: 11304
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "_RESERVED_LLM_PARAMETERS|http_headers|get_http_headers\(|merged_params = \{\*\*engine.body_param_defaults, \*\*kwargs\}" nemoguardrails/guardrails nemoguardrails/rails/llmRepository: NVIDIA-NeMo/Guardrails
Length of output: 7151
Prevent llm_params.http_headers from colliding with the explicit header kwarg. The reserved set only filters config-time parameters; a runtime GenerationOptions.llm_params value named http_headers can still reach engine.chat_completion(...) and trigger a duplicate-key TypeError. Reject or strip that key before merging.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@nemoguardrails/guardrails/model_engine.py` around lines 91 - 100, Update the
parameter-merging flow in the model engine to remove or reject the runtime
GenerationOptions.llm_params entry named http_headers before forwarding
arguments to engine.chat_completion(). Ensure it cannot collide with the
explicit http_headers keyword, while preserving the existing filtering of
config-time parameters.
tgasser-nv
marked this pull request as draft
July 28, 2026 13:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds inference-time headers to IORails with connection through Guardrails facade. This allows clients to set per-inference headers (for tracking or governance)
Related Issue(s)
Verification
AI Assistance
Checklist
Summary by CodeRabbit
New Features
Documentation