Skip to content

chi-bench: Actava · claude-code · anthropic/claude-opus-5 - #20

Merged
haolin-chen-actava merged 1 commit into
mainfrom
sub/chi-bench/2026-07-24-opus-5-claude-code
Aug 12, 2026
Merged

chi-bench: Actava · claude-code · anthropic/claude-opus-5#20
haolin-chen-actava merged 1 commit into
mainfrom
sub/chi-bench/2026-07-24-opus-5-claude-code

Conversation

@haolin-chen-actava

Copy link
Copy Markdown
Contributor

Claude Opus 5 on the stock claude-code harness (MCP tools), single attempt per task.

pass@1
Overall 37.3% (28/75)
Provider PA 20.0% (5/25)
Payer UM 32.0% (8/25)
Care management 60.0% (15/25)

Full-run cost $359.25. The 60.0% care-management slice is the strongest CM result on the board apart from MedGuard's specialist entry.

Why this is filed late. The run completed 2026-07-24T23:22:36Z on Modal and a results infographic was produced, but the leaderboard submission was never filed — the packet sat in logs/submissions/opus-5-claude-code/ without ever being prepared. This is that original run manifest, unmodified.

Provenance note. chi_bench_git_sha pins 2395e9e (current main, via actava-ai/chi-bench#42) so the packet is reproducible from a clean clone. provenance.original_run_sha records c4c1188, the SHA the run actually executed against.

Full cost coverage — all 75 trials report usage, no trajectory repair needed.

Validator: 0 errors, 0 warnings.

🤖 Generated with Claude Code

Filed 2026-08-12 from the original 2026-07-22/24 run; aggregation unmodified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Submission validation — PR diff

All checks passed.

@haolin-chen-actava
haolin-chen-actava merged commit 823b9da into main Aug 12, 2026
4 checks passed
@haolin-chen-actava
haolin-chen-actava deleted the sub/chi-bench/2026-07-24-opus-5-claude-code branch August 12, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants