perf(manifests): load built-in catalog without discovery

[Pouyanpi]

Description

Goal

Materialize the built-in rail catalog from a generated, packaged manifest artifact instead of recursively scanning nemoguardrails/library and importing every built-in rail.py module.

Why

The built-in catalog is immutable for a given package build, but the current runtime reconstructs it from source on every fresh process. That requires a recursive filesystem scan and imports 33 manifest modules while RailsConfig builds its typed schema. This work is unnecessary at runtime, adds cold-start latency to every application, and couples catalog availability to a source-tree layout.

This change:

• generates nemoguardrails/manifests/builtin_rails.json deterministically from source manifests;
• loads and validates the packaged artifact with importlib.resources;
• retains RailCatalog.discover_built_ins() for generation and source-conformance tests;
• provides Makefile targets to regenerate the artifact or check it without modifying the worktree;
• runs the freshness check during pre-commit for catalog-relevant changes and during every CI lint validation;
• keeps source generation independent of package initialization so missing or malformed artifacts remain recoverable;
• preserves process caching, locking, concurrent access, and re-entrancy protection;
• fails clearly for missing, malformed, or unsupported artifacts instead of silently scanning the filesystem;
• adds no public API or configuration flag.

The generated artifact is never edited manually. A stale, malformed, or missing artifact fails with the exact regeneration command. Tests require it to match source discovery exactly and verify that runtime loading performs no recursive scan or built-in rail.py import.

Performance

Fresh subprocesses were alternated between the parent commit and this PR for each sample: 2 warmups and 20 measured samples per metric on the current machine.

Benchmark	Parent median / p95	PR 1 median / p95	Median change
Package import	476.76 / 510.69 ms	458.28 / 505.93 ms	3.9% faster
Catalog loading	12.94 / 15.65 ms	1.54 / 1.76 ms	88.1% faster (8.4×)
Single-rail Colang 1 construction	69.86 / 73.63 ms	74.32 / 94.01 ms	No improvement in this PR

The catalog target of 2 ms or less is met. Single-rail construction still performs the library-wide Colang scan; that independent bottleneck is addressed by the follow-up PR.

Areas for careful review

• The generated artifact format and source-to-artifact drift gate.
• Strict failure behavior for missing or invalid packaged data.
• Preservation of catalog collision validation and cache concurrency semantics.

No user-facing documentation change is required because runtime behavior, public APIs, and configuration syntax are unchanged.

Related Issue(s)

• Fixes #<issue_number>
• Issue assignee: @

Replace both placeholders with the shared triaged, assigned issue before submission.

Verification

• make test TEST='tests/manifests tests/rails/llm/test_builtin_rail_manifests.py' WORKERS=1 — 110 passed.
• make check-builtin-rail-catalog passed.
• Fresh, missing, malformed, and stale catalog check paths are covered.
• Fresh subprocesses regenerated valid catalogs from missing and malformed artifact paths.
• Generated artifact matches source discovery exactly.
• Runtime catalog loading was verified not to scan recursively or import built-in rail.py modules.
• Missing, malformed, unsupported-version, and invalid-record artifacts fail clearly.
• uv build produced the source distribution and wheel.
• The wheel contains nemoguardrails/manifests/builtin_rails.json.
• An extracted wheel installation successfully loaded all 33 catalog records.
• make pre-commit passed on the completed stack containing this commit.
• make test passed on the completed stack: 6,160 passed, 178 skipped.
• No live or credentialed provider calls were made.

AI Assistance

• No AI tools were used.
• AI tools were used; a human reviewed and can explain every change (tool: Codex).

Codex assisted with implementation, tests, benchmarking, and this draft. The PR author must review the changes and check the second box before submission.

Checklist

• I've read the CONTRIBUTING guidelines.
• This PR links to a triaged issue assigned to me.
• My PR title follows the project commit convention.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• I've noted any verification beyond CI and any checks I couldn't run.
• I did not update generated changelog files manually.
• I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
• @mentions of the person or team responsible for reviewing proposed changes.

[github-actions]

Staged Fern docs preview: https://nvidia-preview-pr-2232.docs.buildwithfern.com/nemo/guardrails

[codecov]

Codecov Report

❌ Patch coverage is 97.67442% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
nemoguardrails/manifests/registry.py	97.67%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!