Skip to content

docs: add evaluation reproduction notes - #253

Open
Rohithmatham12 wants to merge 1 commit into
bytedance:mainfrom
Rohithmatham12:codex/evaluation-reproduction-faq
Open

docs: add evaluation reproduction notes#253
Rohithmatham12 wants to merge 1 commit into
bytedance:mainfrom
Rohithmatham12:codex/evaluation-reproduction-faq

Conversation

@Rohithmatham12

Copy link
Copy Markdown

Summary

  • add a top-level evaluation/reproduction FAQ covering requirements, prompt templates, deterministic inference settings, history formatting, coordinate handling, action mapping, mobile scroll vs drag, and deployment endpoint checks
  • link the new guide from the main README header
  • document the public action-space mappings previously clarified in issue discussions
  • clarify that browser benchmarks should be run as multimodal GUI tasks and that exact reproduction depends on matching preprocessing, prompt, parser, history window, endpoint, and benchmark state

Issues

Addresses #246.
Addresses #247.
Addresses #221.
Addresses #169.
Addresses #96.
Addresses #86.
Helps #225.
Helps #121.
Helps #143.
Helps #122.
Helps #127.
Helps #150.
Helps #215.

Testing

git diff --check
python3 doc sanity check for README_evaluation.md and README link

This is docs-only; I did not run model benchmark jobs locally.

@CLAassistant

CLAassistant commented Aug 16, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants