Repository navigation
Bump claude-agent-acp pin to 0.81.0 for claude-opus-5-5 - #1184
Open
shubham2345 wants to merge 1 commit into
Open
shubham2345 wants to merge 1 commit into
shubham2345 wants to merge 1 commit into
Conversation
shubham2345
force-pushed
the
feature/acp_versioning
branch
2 times, most recently
from
October 15, 2026 07:11
20209fe to
5e2f4ac
Compare
0.73.0 bundles Claude Code 2.1.257, which claude-opus-5-5 rejects (needs >= 2.1.280). 0.81.0 bundles sdk 0.3.280 / Claude Code 2.1.280.
shubham2345
force-pushed
the
feature/acp_versioning
branch
from
October 15, 2026 07:18
5e2f4ac to
713106a
Compare
1 of 5 tasks
3 tasks
Benjamin-eecs
left a comment
Contributor
There was a problem hiding this comment.
Non-binding review from the #1137 reporter: this matches what FrontierPhysics ran.
- FrontierPhysics has run its claude-opus-5-5 cells on claude-agent-acp 0.81.0 since 2026-09-23 through a local re-pin of this constant (FrontierPhysics-Infra 3e20674,
FPINFRA_CLAUDE_AGENT_ACP_VERSION=0.81.0), through the fullbench eval runpath on Daytona and on docker.agent/runtime.jsonrecordssandbox.claude_code_version = 2.1.280, and themodelandeffortconfig options took effect. That covers the "Not run" item in the description. - 0.81.0 does not change #1138: a large image read still produces multi-MB ACP lines, as on 0.73.0.
- release/0.8 solves #1137 differently (6bd05f2: adapter 0.81.2 plus
@anthropic-ai/claude-code@2.1.280throughCLAUDE_CODE_EXECUTABLE). For main, this one-line bump is enough; PR #1139 is the same bump and could be closed in favour of this one. - Optional follow-up, not needed here: an env override for the adapter (or CLI) version, so the next model that needs a newer Claude Code does not wait for a release.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1137
Problem
bench eval run --agent claude-agent-acp --model claude-opus-5-5fails on every turn withACP error -32603 ... API Error: 400 Claude Code 2.1.257 does not support this model; version 2.1.280 or newer is required.The
_CLAUDE_AGENT_ACP_PACKAGEpin (0.73.0) bundlesclaude-agent-sdk0.3.257, i.e. Claude Code 2.1.257, which is too old forclaude-opus-5-5.Change
src/benchflow/agents/registry.py: raise the pin to@agentclientprotocol/claude-agent-acp@0.81.0(sdk 0.3.280, Claude Code 2.1.280) and update the explanatory comment.tests/test_agent_registry.py: update the pin assertion.CHANGELOG.md: add a Fixed entry.Verification
tests/test_agent_registry.py: 31 passed.claude-opus-5-5andclaude-fable-5-1succeeds and a one-word prompt returnsend_turn. On 0.73.0, selectingclaude-opus-5-5fails withInvalid value for config option model.bench eval runthrough the runner and a sandbox.Notes
tests/test_acp_pinned_protocol_guard.py,RUN_ACP_DEP_GUARD=1) is unchanged. On 0.81.0 its Fable check needsANTHROPIC_API_KEY, because the adapter validates a concrete model against the API when it is selected.BENCHFLOW_CLAUDE_AGENT_ACP_VERSION) mentioned in the issue; this PR only raises the pin.