You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Eval progress helper crashes on an empty sample set #288
gpt_oss.evals.report.map_with_progress() creates its worker pool with ThreadPool(min(num_threads, len(xs))). When xs is empty, that becomes ThreadPool(0), which raises ValueError: Number of processes must be at least 1.
Empty eval inputs are otherwise representable. For example, HealthBenchEval(num_examples=0) samples zero examples and then routes the empty list through map_with_progress().
Impact
An empty evaluation fails in the threading helper before the eval can return an empty aggregate/result. The failure is unrelated to model sampling or grading and makes zero-example/debug configurations brittle.
Proposed resolution
Return an empty list immediately when xs is empty, before constructing a thread pool. Add regression coverage for both progress-enabled and progress-disabled calls so empty inputs never invoke the mapped function or create workers.
Activity