Repository navigation

All pull requests

Labels (8)Milestones (0)New pull request

fix(simpleqa): correct height example typo in GRADER_TEMPLATE
#126reginaldalfret opened Oct 14, 2026
Fix MGSM crash when the sampler raises
#124rioyu123 opened Sep 27, 2026
Fix MATH grading to use last Answer and harden equality check
#119shoemoney opened Aug 24, 2026
Make the grader parser robust
#118anjali411 opened Aug 24, 2026
2
feat: add batching support to GPQAEval via sampler.batch()
#111dcaayushd opened May 24, 2026
1
Add ArxivRollBench eval
#110liangzid opened May 24, 2026
fix: correct regex group extraction in browsecomp_eval.py
#93Neph0s opened Jul 4, 2025
Anca/b
#92anca-loo opened Jun 23, 2025
Anca br
#91anca-loo opened Jun 23, 2025
Enhance simple-evals for beginner to run
#87ECNU3D opened Jun 5, 2025
Jules version of beginner friendly README
#81rarhs opened May 20, 2025
fix(queue): prevent duplicate message send (race condition)
#72Yuu6798 opened May 8, 2025
feat: add len_var scorer (B-0)
#71Yuu6798 opened May 8, 2025
fix: add semantic_match scorer to detect drift in math_eval
#69Yuu6798 opened May 4, 2025
fix regex bug in browsecomp
#67tengyaolong2000 opened Apr 22, 2025
1
fix: import collision for types
#66Ithanil opened Apr 21, 2025
Small typo in grader
#57chiruu12 opened Mar 25, 2025
2
add aime task
#55jason9693 opened Mar 12, 2025
Initial commit
#45osmanjamalfarag opened Feb 1, 2025
Grok Sampler
#40rolandgvc opened Jan 9, 2025
correct string spelling error
#37owos opened Dec 27, 2024
Fix MMLU answer extraction regex for repeated "Answer: LETTER" pattern
#34lucasresck opened Dec 10, 2024
Update SimpleQAEval to use direct CSV URL instead of blobfile
#26ZhangYiqun018 opened Oct 31, 2024
Use correct _pack_message function name
#12andrewmbenton opened May 20, 2024
1
fix typo
#10dongZheX opened May 20, 2024
1