chore: bump renderers to 0.1.11, verifiers and prime-envs to main - #3441
Merged
Conversation
renderers 0.1.11 makes transformers optional, splitting it into the [transformers] and [multimodal] (transformers + pillow) extras. Depend on renderers[multimodal] in prime-rl, which loads HF tokenizers and renders image content, and bump the floor in prime-rl-configs, which only imports config types and stays extra-free. Update the renderers submodule to the 0.1.11 release and refresh the lock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
samsja
previously approved these changes
Aug 31, 2026
verifiers moves to current main (renderers 0.1.11 bump, legacy RendererClient removal, MCP 2026-07-28 adoption, aiohttp>=3.14). Its mcp==2.0.0 pin conflicts with automationbench's and browsecomp_plus's mcp 1.x pins, so prime-envs moves to main plus PrimeIntellect-ai/prime-envs#789, which drops those pins; re-point to prime-envs main once that PR merges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hallerite
changed the title
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PrimeIntellect-ai/prime-envs#789 now removes automationbench_env instead of migrating it; the env and its automation-bench git dep leave the workspace lock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
enterprise_ops_gym and mcp_atlas now run their dynamic-schema tool servers on MCPServer overrides and declare mcp>=2,<3 directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…erged mcp2 branch verifiers main now includes the legacy (v0) stack removal (#2480); prime-rl only consumes verifiers.v1. prime-envs moves to PrimeIntellect-ai/prime-envs#789 merged with its main (judge standardization, Prime VM defaults). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
xeophon
pushed a commit
to PrimeIntellect-ai/prime-envs
that referenced
this pull request
Aug 31, 2026
## Summary Makes prime-envs compatible with verifiers main, which pins `mcp==2.0.0` (the MCP 2026-07-28 adoption, PrimeIntellect-ai/verifiers#2474) and removes `mcp.server.fastmcp`. Unblocks the verifiers submodule bump in prime-rl (PrimeIntellect-ai/prime-rl#3441). - **automationbench_env**: removed entirely rather than migrated (team decision — superseded by the Verified fork; its `mcp<2` pin was the hard workspace-lock conflict). Also drops its row from the tool_use README index. Nothing else in prime-envs or prime-rl references it. - **browsecomp_plus**: upstream #777 already dropped its `mcp>=1,<2` pin; on the merged tree this PR removes the stale FastMCP comment it left behind (changelog entry added). It imports nothing from mcp itself. - **enterprise_ops_gym / mcp_atlas**: both tool servers advertised dynamic, service-defined tool schemas by installing fastmcp `Tool` objects with a passthrough `FuncMetadata` into the private tool manager — runtime-broken under mcp 2. Migrated to the pattern verifiers' nemo_gym toolset established in #2474: `register` overrides `MCPServer.list_tools` / `call_tool` (the public methods its protocol handlers dispatch through at request time), serving `mcp.types.Tool` entries with the live schemas verbatim and wrapping forwarded results in `CallToolResult`. Argument validation stays with the backing services, as before. Both now declare `mcp>=2,<3` directly and are version-bumped (enterprise-ops-gym 0.2.0, mcp-atlas 0.4.0) for the breaking change. ## E2E results (hosted model via Prime Inference) Same tasks, same model (`openai/gpt-5.6-luna` via api.pinference.ai), same flags (`-n 2 -r 1`, max-turns 8, max-tokens 1024), prime VM runtimes. **main stack** = env from prime-envs main + released verifiers 0.3.1 (mcp 1.x, FastMCP path); **PR stack** = env from this branch + verifiers main (mcp 2, MCPServer path). | Run | main stack | PR stack | |---|---|---| | enterprise_ops_gym task 0 (calendar kickoff) | reward **1.0**, verifier pass rate 1.0 | reward **1.0**, verifier pass rate 1.0 | | enterprise_ops_gym task 1 (Helios roadmap) | reward **0.0**, pass rate 0.4 | reward **0.0**, pass rate 0.4 | The model received the dynamic tool catalog over the wire, called the gym MCP tools, mutated the seeded databases through them, and the SQL end-state verifiers scored **identically on both stacks** — including matching per-verifier pass rates on the failed task (a model/task failure, identical on both sides). This exercises the migrated `register`/`list_tools`/`call_tool` path end-to-end against live services. mcp_atlas (with the documented source installs for env + verifiers) completed on both stacks — healthy multi-turn tool loops (5–8 tool-call rounds per rollout, no tool errors), rollouts `ok`, judge-scored: | Run | main stack | PR stack | |---|---|---| | mcp_atlas task 0 (local-files QA) | coverage **1.0** (6 calls) | coverage **0.0** (8 calls) | | mcp_atlas task 1 (Santiago communes QA) | coverage **0.3** (7 calls) | coverage **0.5** (5 calls) | Unlike EOG's deterministic SQL verifiers, atlas scores are single-sample LLM-judge coverage under temperature sampling — the per-task deltas go both directions and there is no mechanical divergence in the traces (both stacks listed and called the dynamic Atlas tools over MCP without errors). Earlier atlas attempts failed **identically on both stacks** at infra layers before any MCP code (GHCR image rejection — since fixed upstream and merged in; then the source-install requirements). ## Verification - Functional smoke test on mcp 2.0.0: both migrated toolsets driven through a real `MCPServer`'s public `list_tools`/`call_tool` — dynamic schemas served verbatim (`input_schema` intact), forwarded results wrapped in `CallToolResult`, unknown tools raise. - mcp 2.0.0 source check (re Bugbot): the SDK's protocol handlers dispatch late through `self.list_tools()` / `self.call_tool(...)` — there is no `on_list_tools`/`on_call_tool` init binding — so the instance-attribute overrides are the supported interception point. - ruff check clean; root `uv lock --check` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Removes an entire taskset and changes how two stateful MCP proxies advertise and dispatch tools; behavior is intended to be unchanged, but this touches multi-turn grading paths. > > **Overview** > Updates prime-envs for **verifiers main**, which pins **MCP 2** and drops `mcp.server.fastmcp`. > > **`automationbench_env` is removed** (package, docs, and tool-use index row) because it could not be reconciled with the MCP 2 lock and is superseded elsewhere. > > **`enterprise_ops_gym`** and **`mcp_atlas`** no longer register dynamic tools via FastMCP internals. Their tool servers override **`MCPServer.list_tools` / `call_tool`** to expose live service schemas as `mcp.types.Tool` and return **`CallToolResult`**, matching the verifiers pattern for dynamic catalogs. Both declare **`mcp>=2,<3`** and are version-bumped (**0.2.0** / **0.4.0**). > > **`browsecomp_plus`** only drops a stale FastMCP-era dependency comment and notes MCP 2 in the changelog; it does not import MCP directly. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 4cb5269. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Migrate `enterprise_ops_gym` and `mcp_atlas` toolsets to MCP 2 and remove `automationbench_env` > - Replaces FastMCP internals (`FastMCP`, `Tool`, `ArgModelBase`, `_PassthroughMetadata`) in `enterprise_ops_gym` and `mcp_atlas` with MCP 2 `MCPServer` registration that overrides `list_tools` and `call_tool` directly on the server instance. > - `register` now assigns instance coroutines for dynamic tool listing and dispatch; `call_tool` returns `CallToolResult` with `TextContent` and raises `ValueError` for unknown tool names. > - Deletes the `automationbench_env` environment package and removes its taskset row from the `tool_use` README. > - Both updated packages bump versions and add a direct `mcp>=2,<3` dependency constraint in their `pyproject.toml`. > - Risk: installing these packages now requires MCP 2.x; consumers still on MCP 1.x / FastMCP will fail to resolve dependencies. The removed `_PassthroughMetadata` and `_EMPTY_ARG_MODEL` shims in [toolset.py](https://github.com/PrimeIntellect-ai/prime-envs/pull/789/files#diff-801ef4ed1395fe376f6de7d8884a88b72fdbbbddb93d8dff3e9e8c2588d2c5a9) and [toolset.py](https://github.com/PrimeIntellect-ai/prime-envs/pull/789/files#diff-e51faf9f76fd939f40f8e0f3256c2e72307821c3a3f306a03fb06e0bee62f8b4) mean any external code relying on those private symbols breaks. > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 4cb5269.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…merged Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikasenghaas
approved these changes
Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
renderers[multimodal]>=0.1.11in prime-rl: renderers 0.1.11 makestransformersoptional behind the[transformers]/[multimodal]extras, and prime-rl loads HF tokenizers (renderers.base.load_tokenizer/create_renderer) and renders image content — the multimodal extra carriestransformers+pillow.renderers>=0.1.11in prime-rl-configs, which only imports config types andMODEL_RENDERER_MAPand needs no extra.66915aaa): renderers 0.1.11 bump + legacyRendererClientremoval (chore: bump renderers to 0.1.11; drop legacy RendererClient verifiers#2478), MCP 2026-07-28 adoption (mcp==2.0.0),aiohttp>=3.14.39906ac2, which includes feat: make prime-envs compatible with verifiers main / mcp 2 prime-envs#789): verifiers'mcp==2.0.0pin was unresolvable against automationbench'smcp<2and browsecomp_plus'smcp>=1,<2pins in the shared workspace lock; [feat] max vio in sft #789 removed automationbench_env entirely (itsautomation-benchgit dep leaves the lock), dropped browsecomp_plus's stale pin, and migrated the enterprise_ops_gym / mcp_atlas tool servers to mcp 2 (validated e2e via Prime Inference — see [feat] max vio in sft #789's description for results).prime-rl doesn't use the renderer pool APIs removed in renderers 0.1.11, and nothing in prime-rl imports
verifiers.legacy.enterprise_ops_gymandmcp_atlas, which importedmcp.server.fastmcpinternals at runtime, are migrated to the mcp 2MCPServeroverride pattern in PrimeIntellect-ai/prime-envs#789 (included via the submodule pointer).Submodule commits pulled in by this bump
renderers
cb824391(v0.1.10) →9b47901d(v0.1.11), 5 commits:9b47901Unify renderer template parity coverage (Unify renderer template parity coverage renderers#127)5d50274feat: add DeepSeek V4 Flash 0731 renderer (feat: add DeepSeek V4 Flash 0731 renderer renderers#140)c4772acfeat: support offsetless BYO tokenizers (feat: support offsetless BYO tokenizers renderers#142)ef49eb9feat: make transformers optional (Is transformers necessary or tokenizers is enough? renderers#31) (feat: make transformers optional (#31) renderers#70)32e6622feat: add Qwen3.8 Flash Next renderer (feat: add Qwen3.8 Flash Next renderer renderers#141)verifiers
b2e4e815→e2103d6f(main), 24 commitse2103d6ffix(v1): reconcile concurrent graph commits (fix(v1): reconcile concurrent graph commits verifiers#2477)66a60649feat!: remove the legacy (v0) stack (feat!: remove the legacy (v0) stack verifiers#2480)66915aaafix: require aiohttp 3.14 or newer (fix: require aiohttp 3.14 or newer verifiers#2479)a205eb3aConsolidate v1 MCP client utilities (Consolidate v1 MCP client utilities verifiers#2467)ab569b10chore: bump renderers to 0.1.11; drop legacy RendererClient (chore: bump renderers to 0.1.11; drop legacy RendererClient verifiers#2478)df618d99Clean up empty NeMo Gym extra (Clean up empty NeMo Gym extra verifiers#2476)9fe6c140Adopt MCP 2026-07-28 (Adopt MCP 2026-07-28 verifiers#2474)c51c094afix(v1): prefer final judge choices over draft boxes (fix(v1): prefer final judge choices over draft boxes verifiers#2465)8a993eceFix cache salt forwarding in train client (Fix cache salt forwarding in train client verifiers#2463)651484c2Report unchecked gold validation (Report unchecked gold validation verifiers#2466)4ba184d1[fix] [v1]: reject dupe artifact sources ([fix] [v1]: reject dupe artifact sources verifiers#2225)573bfa16fix: batch Prime background job polling (fix: batch Prime background job polling verifiers#2462)5f24fb84fix(v1): clear handled model call errors (fix(v1): clear handled model call errors verifiers#2457)79d13da2fix: end the null harness cleanly on context overflow (fix: end the null harness cleanly on context overflow verifiers#2456)2e8d1867feat: relay overlong prompt errors to the harness (feat: relay overlong prompt errors to the harness verifiers#2453)1e8e1c87fix(v1): retain Terminus reasoning history (fix(v1): retain Terminus reasoning history verifiers#2455)eba9a7f6fix(v1): pin Prime Agent v0.8.1 (fix(v1): pin Prime Agent v0.8.1 verifiers#2445)aeeae7c6fix(v1): allow concurrent borrowed runtime environments (fix(v1): allow concurrent borrowed runtime environments verifiers#2418)baa6a86bfix(v1): use typed ACP turn in Prime Agent harness (fix(v1): use typed ACP turn in Prime Agent harness verifiers#2443)c5fe6462feat(v1): add Prime Agent ACP harness (feat(v1): add Prime Agent ACP harness verifiers#2440)b082e3c5fix(v1): use typed ACP turn in RLM harness (fix(v1): use typed ACP turn in RLM harness verifiers#2442)ca941d37feat(v1): integrate nano-RLM training over ACP (feat(v1): integrate nano-RLM training over ACP verifiers#2439)f4e6864ffeat(v1): type ACP prompt results (feat(v1): type ACP prompt results verifiers#2438)c521e414feat: add verifiers release skill (feat: add verifiers release skill verifiers#2435)prime-envs
26dafdc9→1f1e050a(main), 7 commits1f1e050aDefault-deny environment runtime networking (Default-deny environment runtime networking prime-envs#780)39906ac2feat: make prime-envs compatible with verifiers main / mcp 2 (feat: make prime-envs compatible with verifiers main / mcp 2 prime-envs#789)ec44a3f0Yield environment tasks lazily (Yield environment tasks lazily prime-envs#788)084b9fb5Standardize native Verifiers judges (Standardize native Verifiers judges prime-envs#779)46e93fccUpdate environments for Verifiers 0.3.1 and Prime VM support (Update environments for Verifiers 0.3.1 and Prime VM support prime-envs#777)279995a6Simplify high-LOC environments (Simplify high-LOC environments prime-envs#787)3b2af76eFix environments for Verifiers 0.3.1 (Fix environments for Verifiers 0.3.1 prime-envs#772)Verification
🤖 Generated with Claude Code
Note
Medium Risk
Wide dependency and MCP 2.0 upgrades affect many evaluation environments and HTTP/MCP tool servers; regressions are most likely in multimodal rendering and MCP-based envs rather than core RL entrypoints.
Overview
This PR raises dependency floors and refreshes the workspace lock to match updated verifiers and prime-envs submodules.
Renderers:
prime-rland the lock now requirerenderers[multimodal]>=0.1.11so multimodal rendering (HF tokenizers / images) still pullstransformersandpillowafter renderers made those optional.prime-rl-configsonly bumps torenderers>=0.1.11(no extra).Verifiers ecosystem: Lock metadata moves many prime-envs packages to
verifiers>=0.3.1and drops redundant direct pins ondatasets,openai,httpx, andprime-sandboxeswhere verifiers now supplies them. Verifiers itself pinsmcp==2.0.0,aiohttp>=3.14,renderers[multimodal]>=0.1.11, and dropsopenai-agents(and related packages) from the resolved graph.MCP / tool envs:
automationbench-envis removed from the workspace manifest and lock.enterprise-ops-gym(0.2.0) andmcp-atlas(0.4.0) addmcp>=2,<3and shed older HTTP/sandbox pins aligned with MCP 2.Transitive updates in
uv.lockinclude mcp 2.0.0 (via httpx2), aiohttp 3.14.3, openai 2.54.0, and tau2-bench 0.3.1, plus renderers optional-deps layout (multimodal/transformersextras).Reviewed by Cursor Bugbot for commit 1386784. Bugbot is set up for automated code reviews on this repo. Configure here.