Skip to content

Evidence capture rejects an absolute in-workspace symlink with '..' parts ("Unsafe evidence path"): a verified cell never gets a review #1157

Closed

Description

@Benjamin-eecs

Status 2026-10-26


Problem. review.evidence._link_target does not normalise an absolute symlink target before its safety check. It strips the workspace prefix and passes the rest to _relative_path, which rejects any .. part. A link the agent made as /root/analysis/scripts/../figures resolves inside the workspace, yet the capture fails with EvidenceError: Unsafe evidence path: 'analysis/scripts/../figures'. Relative targets go through posixpath.normpath first, so only absolute ones are hit.

Evidence

  • 2026-10-09 21:16 UTC run, Claude Haiku 5.5, glass-nonequilibrium-landscape-evolution trial 3 on Daytona. The verifier passed (reward 1.0), then the in-run review failed with the error above. The six review attempts and a post-hoc bench eval score failed the same way within four minutes. The sandbox was gone by then, so the stored export_error is re-raised on every later score, and the verified cell never gets a review.
  • Traceback: review/evidence.py line 212 _relative_path(str(target)) in _link_target, line 198 raise EvidenceError(f"Unsafe evidence path: {value!r}").

Where (main e965eee, same as 0.7.8)

  • src/benchflow/review/evidence.py:202-213 _link_target: the absolute branch checks target.relative_to(workspace) unnormalised. The relative branch (214-219) normalises.

Proposed fix

  • Normalise the absolute target after relative_to(workspace) (posixpath.normpath). Raise "escapes workspace" when the result is .. or starts with ../. Otherwise check and return posixpath.relpath(normalized, parent).
  • A test with an absolute in-workspace link containing .. (archived), and one that leaves after normalising (rejected).

Workaround in use: FrontierPhysics-Infra 3111e65 (bench patch 43) wraps _link_target this way for 0.7.8. The affected cell is re-run, because its workspace was never captured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions