fix(v1): collect isolated judge artifacts in owned rollouts

[xeophon]

Overview

Collect isolated solver artifacts through the normal Agent.run owned-runtime lifecycle, after task finalization and before scoring and teardown.

Details

• Add an opt-in collect_artifacts argument to Agent.run and Rollout.
• Preserve artifacts produced by a task finalizer; otherwise use the existing artifact collection helper within the same finalize boundary.
• Let the isolated agentic judge request collection without provisioning or borrowing the solver runtime itself.
• Exercise the generic path by removing the echo fixture custom collection finalizer.

Related work

This is an alternative implementation of the artifact-lifecycle problem reported in #2342 by @crazywriter1. It preserves Agent.run ownership and whole-rollout retry semantics while reusing the same artifact representation consumed by JudgeTask.

It complements #2464: that PR hardens artifact transfer and restoration boundaries, while this PR controls when the existing collection path runs before an owned solver runtime is torn down.

---

Note

Medium Risk
Changes rollout teardown timing and artifact population semantics; callers that relied on task finalize for artifacts must opt in with collect_artifacts=True, and isolated judging now depends on container solver runtimes.

Overview
Adds an opt-in collect_artifacts flag on Agent.run (and Rollout) so declared task artifacts are gathered after finalize while the owned container runtime is still up, without tasks implementing their own collection.

IsolatedAgenticJudgeEnv enables this on the solver rollout so JudgeTask.from_trace(..., share_runtime=False) receives solution.state.artifacts before the solver box is torn down. collect_artifacts=True is rejected for subprocess runtimes (host-only temp cwd).

Rollout finalize is now bounded with asyncio.timeout instead of wait_for; collection runs only when artifacts are still empty, so a custom finalize that already populated trace.state.artifacts is left alone. The echo agentic fixture drops its task-level finalize in favor of this path.

^{Reviewed by Cursor Bugbot for commit 0df3f33. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add collect_artifacts flag to Rollout and Agent.run for isolated judge rollouts

• Adds a collect_artifacts boolean parameter (default False) to Rollout.__init__, Agent.run, Agent._run_once, and _EpisodeAgent.run, propagating the flag through to the rollout.
• When collect_artifacts=True, Rollout.close collects artifacts via collect(runtime, self.task.data.artifacts) after task.finalize if trace.state.artifacts is still empty. IsolatedAgenticJudgeEnv.run now passes collect_artifacts=True to the solver agent so judge can access solver-produced artifacts.
• Replaces asyncio.wait_for with an asyncio.timeout context manager in Rollout.close for the finalize phase; timeout behavior is unchanged.
• Removes the EchoAgenticTask.finalize method in the test fixture since artifact collection is now handled by the rollout.
• Risk: Agent._run_once raises TypeError when collect_artifacts=True and the resolved runtime is a SubprocessConfig; callers passing the flag with a subprocess runtime will fail at runtime.

^{Macroscope summarized 0df3f33.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-29T08:27:37.855809Z	0df3f33	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6ca8baa71

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This change modifies the production rollout lifecycle so isolated judging can transfer artifacts before the solver runtime is destroyed. Unresolved concrete findings still raise a possible pre-solver failure in the episode-agent forwarding path, so that execution path requires verification.

Not approved because:

• 1 blocking correctness issue found at or above your repo's Minimum Blocking Severity

^{Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7c99e06f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".