feat(http): add canonical outbound HTTP foundation

[Pouyanpi]

Description

Define the transport-neutral outbound HTTP foundation used by built-in library
rails.

The neutral response owns decoded bytes rather than exposing a live transport
response. Callers can therefore inspect a response after the underlying client
context closes. The HTTPX adapter provides pooled connections while keeping
transport-specific types out of rail actions.

Retry behavior is bounded and conservative by default. Safe methods are
retryable, POST retries require an explicit rail-owned policy, and vendor
override headers are ignored unless a policy opts in. Retry-After remains
bounded by policy, total request timeouts cover the complete operation, and
redirect following is disabled unless explicitly enabled.

Design

There is one real transport and a set of thin, single-purpose decorators, not a
family of client implementations. HTTPClient is a runtime-checkable protocol,
so HttpxHTTPClient, a recording fake, and any future aiohttp adapter satisfy
the same contract without sharing a base class. Rail actions and tests depend on
the protocol and never import a concrete transport.

Behavior is added by composition. The default stack is
InstrumentedHTTPClient(RetryingHTTPClient(HttpxHTTPClient(...))), where each
layer satisfies HTTPClient and wraps the next:

• HttpxHTTPClient is the only layer that touches the network. It owns the
  connection pool, the total-operation timeout, and TLS.
• RetryingHTTPClient re-issues a request under a bounded policy and knows
  nothing about HTTPX or telemetry.
• InstrumentedHTTPClient opens one span per logical call and knows nothing
  about HTTPX or retries. It is a transparent pass-through when no tracer is
  supplied, so importing the module never requires the OpenTelemetry SDK.

This design keeps each concern independently testable and configurable: retry
logic is unit-tested against a fake transport with no HTTPX and no telemetry,
and a layer is only present when it is needed (retries only when a policy is
passed, instrumentation only when tracing is enabled). The composition order is
load-bearing. Instrumenting outside retry yields one span per logical operation
that records the resend count. The reverse order would emit one sibling span per
attempt and make a single operation look like several unrelated calls.

Impact

• Library integrations gain one transport-neutral HTTP contract.
• Response parsing and status errors no longer depend on a live HTTPX response.
• POST requests are not retried unless the integration explicitly declares
  that resending is safe.
• Redirect, timeout, retry-header, and backoff behavior is deterministic and
  covered by focused tests.
• No library rail is migrated in this PR; call-site migrations remain isolated
  in later stack layers.

Stack

Layer	Branch	Base	PR
HTTP foundation	pouyanpi/rail-library-stack-9-http-contract	develop	#2209
HTTP instrumentation	pouyanpi/rail-library-stack-10-http-instrumentation	Stack 9	#2219
Request lifecycle	pouyanpi/rail-library-stack-11-http-observability	Stack 9	#2210
Request migrations	pouyanpi/rail-library-stack-12-http-request-migrations	Stack 11	#2211
Action and specialized migrations	pouyanpi/rail-library-stack-13-http-action-migrations	Stack 12	#2212

Summary by CodeRabbit

• New Features
  • Added an asynchronous HTTP client with request and response handling.
  • Added response text/JSON parsing and clear status, timeout, connection, and decoding errors.
  • Added configurable retries with exponential backoff, retry limits, and Retry-After support.
  • Added optional redirect handling, timeout enforcement, and safe resource cleanup.
• Tests
  • Added comprehensive coverage for HTTP requests, errors, retries, timeouts, redirects, and response parsing.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR introduces the transport-neutral outbound HTTP foundation for NeMo Guardrails library rails: a runtime_checkable HTTPClient protocol, an HTTPX adapter (HttpxHTTPClient), a policy-driven retry decorator (RetryingHTTPClient), transport-neutral request/response value objects, a credential-stripping URL sanitizer, and a RecordingHTTPClient test double.

• Layered composition — each layer satisfies the same protocol and is independently testable against the fake transport without touching the network or telemetry.
• Retry policy — bounded exponential backoff with full-jitter, Retry-After parsing (delta-seconds and HTTP-date), opt-in X-Should-Retry vendor override, and POST excluded from the default retryable set; RetryingHTTPClient now includes __aenter__/__aexit__.
• Response ownership — HTTPResponse materializes body bytes before returning; credential sanitization in error messages is covered by a dedicated URL helper and parametrized test suite.

Confidence Score: 5/5

Safe to merge — no functional defects found across the protocol, transport, retry, and testing layers.

Retry accounting, method gating, backoff math, Retry-After parsing, credential sanitization, and response materialization are all logically correct and thoroughly covered by focused unit tests with injectable fakes. The two observations are documentation clarity and a connection-pool efficiency trade-off, neither of which affects correctness.

Files Needing Attention: No files require special attention; the minor observations are in transport.py (timeout mechanism) and retry.py (docstring terminology).

Important Files Changed

Filename	Overview
nemoguardrails/http/transport.py	New HTTPX adapter translating transport errors to neutral types; uses asyncio.wait_for for total-operation deadline with HTTPX internal timeouts disabled; owned vs injected client distinction enforced at construction time.
nemoguardrails/http/retry.py	New RetryPolicy dataclass and RetryingHTTPClient decorator; bounded exponential backoff with jitter; Retry-After parsing; X-Should-Retry opt-in; context manager and idempotent close confirmed correct.
nemoguardrails/http/types.py	Frozen HTTPRequest and HTTPResponse dataclasses; charset-aware text decoding with LookupError fallback; json() raises HTTPResponseDecodeError without leaking body.
nemoguardrails/http/_url.py	sanitize_url strips credentials, query params, and fragment; handles IPv6, invalid ports, missing hostname, and relative URLs correctly.
nemoguardrails/http/errors.py	Clean exception hierarchy under HTTPClientError; retry_count on base class; URL credential redaction in HTTPStatusError.
nemoguardrails/http/client.py	runtime_checkable HTTPClient and ClosableHTTPClient protocols; no concrete inheritance required from implementors.
nemoguardrails/testing/http.py	RecordingHTTPClient queues responses or exceptions, records requests, tracks close_calls; satisfies both protocols structurally.
tests/http/test_retry.py	Comprehensive retry tests covering method gating, status triggers, Retry-After, X-Should-Retry, transport errors, context manager, clamped backoff, and boundary validation.
tests/http/test_transport.py	Tests cover forwarding, error translation, close idempotency, deadline enforcement, timeout validation, redirect flag, and injected-client option rejection.
tests/http/test_contract.py	Contract tests verify text/JSON parsing, charset fallback, credential redaction, protocol satisfaction, and exception hierarchy.
tests/http/test_url.py	Parametrized sanitize_url tests cover standard, IPv6, missing hostname, scheme-relative, invalid port, relative, query-only, and empty inputs.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Retrying as RetryingHTTPClient
    participant Transport as HttpxHTTPClient
    participant HTTPX as httpx.AsyncClient

    Caller->>Retrying: request(method, url, ...)
    Note over Retrying: can_retry_method = policy.can_retry_method(method)

    loop Up to max_attempts
        Retrying->>Transport: request(method, url, ...)
        Transport->>HTTPX: "asyncio.wait_for(client.request(...), timeout=deadline)"
        alt Success
            HTTPX-->>Transport: httpx.Response (buffered)
            Transport-->>Retrying: HTTPResponse(status_code, headers, content, extensions)
            alt should_retry(response) AND attempts remain
                Note over Retrying: sleep(retry_after or backoff)
                Note over Retrying: retries += 1
            else Done
                Note over Retrying: inject retry_count into extensions
                Retrying-->>Caller: HTTPResponse with retry_count
            end
        else asyncio.TimeoutError
            Transport-->>Retrying: HTTPTimeoutError
            alt retry_transport_errors AND attempts remain
                Note over Retrying: sleep(backoff)
            else
                Retrying-->>Caller: "HTTPTimeoutError(retry_count=N)"
            end
        else httpx.RequestError
            Transport-->>Retrying: HTTPConnectionError
            alt retry_transport_errors AND attempts remain
                Note over Retrying: sleep(backoff)
            else
                Retrying-->>Caller: "HTTPConnectionError(retry_count=N)"
            end
        end
    end

Loading

_{Reviews (5): Last reviewed commit: "test(http): cover outbound client branch..." | Re-trigger Greptile}

[coderabbitai]

📝 Walkthrough

Walkthrough

Introduces a transport-neutral asynchronous HTTP client package with request/response types, standardized errors, an HTTPX adapter, configurable retries, lifecycle management, public exports, and deterministic test coverage.

Changes

Async HTTP Client

Layer / File(s)	Summary
HTTP contracts and value objects nemoguardrails/http/types.py, nemoguardrails/http/client.py, nemoguardrails/http/errors.py, nemoguardrails/http/testing.py, nemoguardrails/http/__init__.py, tests/http/test_contract.py	Defines neutral request/response models, client protocols, error types, sanitized status messages, public exports, and a recording test client with contract tests.
HTTPX transport adapter nemoguardrails/http/transport.py, tests/http/test_transport.py	Translates HTTPX requests and responses, maps transport failures, enforces total timeouts, supports redirect configuration, and manages owned client closure.
Retry policy and wrapper nemoguardrails/http/retry.py, tests/http/test_retry.py	Adds retry configuration, retryable status and transport-error handling, Retry-After parsing, exponential backoff, retry metadata, and idempotent closure behavior.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant RetryingHTTPClient
  participant HttpxHTTPClient
  participant httpx.AsyncClient
  Caller->>RetryingHTTPClient: request(method, url, request data)
  RetryingHTTPClient->>HttpxHTTPClient: send attempt
  HttpxHTTPClient->>httpx.AsyncClient: execute HTTP request
  httpx.AsyncClient-->>HttpxHTTPClient: response or transport exception
  HttpxHTTPClient-->>RetryingHTTPClient: HTTPResponse or mapped error
  RetryingHTTPClient-->>Caller: retry or return final result

Loading

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 19.15% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes	✅ Passed	PR description includes testing info: it says the HTTP foundation is covered by focused tests and unit-tested against a fake transport.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: a new canonical outbound HTTP foundation for the http package.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch pouyanpi/rail-library-stack-9-http-contract

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nemoguardrails/http/errors.py`:
- Around line 25-36: Update the hostname-None fallback in _safe_url so it
removes any userinfo from parts.netloc before rebuilding the URL, including
empty-host values such as user:pass@. Preserve the existing scheme, path, and
query/fragment redaction behavior, and add coverage for this credential-bearing
fallback case.

In `@nemoguardrails/http/transport.py`:
- Around line 35-64: Update the HTTP request exception handling in the transport
adapter to catch all relevant httpx.RequestError subclasses, including
DecodingError and TooManyRedirects, rather than only TransportError. Preserve
the existing transport-neutral wrapping and retry behavior for these errors.
- Around line 106-116: Update the exception mapping in the HTTP request flow
around the client request and existing httpx exception handlers to catch
request-level httpx failures, including httpx.DecodingError and
httpx.TooManyRedirects, and translate them into the established neutral
HTTPConnectionError path. Preserve the existing timeout mappings and exception
chaining.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4aad114e-00e6-4f57-bbce-0bf8a33a34cd

📥 Commits

Reviewing files that changed from the base of the PR and between 286bdd3 and 84e9e61.

📒 Files selected for processing (10)

• nemoguardrails/http/__init__.py
• nemoguardrails/http/client.py
• nemoguardrails/http/errors.py
• nemoguardrails/http/retry.py
• nemoguardrails/http/testing.py
• nemoguardrails/http/transport.py
• nemoguardrails/http/types.py
• tests/http/test_contract.py
• tests/http/test_retry.py
• tests/http/test_transport.py

[tgasser-nv]

Looks good! Can you check the code coverage and add tests to improve this? Looks like missing lines in the owned-client request-path (where client is None) and a couple of other places.

[github-actions]

Staged Fern docs preview: https://nvidia-preview-pr-2209.docs.buildwithfern.com/nemo/guardrails

[Pouyanpi]

Thanks @tgasser-nv for the review. I've added coverage and addressed your review comments.