Issues

• Aggregation Extension's default (all, null graders excluded) does not describe how most producers in this repo derive Result.passed

  spec
  adhabnr-ux/evalport#129 · Filter by author adhabnr-ux opened on Oct 7, 2026

• Unknown fields: SPEC says "ignore", JSON Schemas say "reject", SDKs say "accept" — pick one

  adhabnr-ux/evalport#107 · Filter by author adhabnr-ux opened on Sep 29, 2026
  1

• Conformance README overstates raw-schema agreement: several rejected fixtures are hand-rolled-only by design

  documentationgood first issue
  adhabnr-ux/evalport#96 · Filter by author adhabnr-ux opened on Sep 27, 2026

• evalport.org does not resolve: $schema URLs in the spec are dead

  bugdocumentation
  adhabnr-ux/evalport#87 · Filter by author adhabnr-ux opened on Sep 27, 2026

• eval-ai-library adapter: support multi-turn ConversationalEvalTestCase / ConversationalTestCaseResult

  enhancementhelp wanted
  adhabnr-ux/evalport#77 · Filter by author adhabnr-ux opened on Sep 27, 2026

• TypeScript SDK: port from_deepeval and from_inspect, and expose them through evalport convert

  enhancementhelp wanted
  adhabnr-ux/evalport#76 · Filter by author adhabnr-ux opened on Sep 27, 2026

• Migration guides: DeepEval/Inspect snippets import from the wrong module, and the OpenAI Evals guide is missing

  documentationgood first issue
  adhabnr-ux/evalport#72 · Filter by author adhabnr-ux opened on Sep 27, 2026

• Nested validation error paths contain repeated $. segments (e.g. $.test_cases[0].$.graders[0].$.params.pattern)

  buggood first issue
  adhabnr-ux/evalport#69 · Filter by author adhabnr-ux opened on Sep 27, 2026

• Should GraderResult/Result carry a required denominator/floor for rate-based scores? (raised via NVIDIA/garak#2110)

  adhabnr-ux/evalport#58 · Filter by author adhabnr-ux opened on Sep 22, 2026
  1

• Adapters wanted: how to add EvalPort support for your favorite framework

  adhabnr-ux/evalport#6 · Filter by author adhabnr-ux opened on Aug 9, 2026
  26