Skip to content

fix(v1): serialize restricted borrowed runtimes #2446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xeophon wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(v1): serialize restricted borrowed runtimes #2446

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 22 additions & 10 deletions verifiers/v1/rollout.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ def __init__(
self._interception = interception
self.runtime = runtime
self._borrowed_runtime = runtime
self._borrow_lock: asyncio.Lock | None = None
self.trace: Trace = Trace(
task=TraceTask(
type=type(task).__name__,
Expand Down Expand Up @@ -204,6 +205,9 @@ async def open(self) -> bool:
if self._borrowed_runtime is None:
runtime.env = runtime_env
else:
if runtime.network_restricted:
await runtime.borrow_lock.acquire()

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid waiting for the borrow lock while holding the agent gate

When an episode starts a second interaction or run on the same restricted borrowed runtime while the first interaction is open between turns, the second operation holds _EpisodeAgent._gate and blocks here. The first interaction already owns borrow_lock, but its next turn() or close() must reacquire that gate (agent.py:173 and agent.py:227); with the default max_concurrent_agents=1, neither can progress. Acquire the serialization lock before the agent gate, or release the gate while waiting for it.

Useful? React with 👍 / 👎.

self._borrow_lock = runtime.borrow_lock

@cursor cursor Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open can leak borrow lock

Medium Severity

After borrow_lock is acquired, open's except Exception path calls fail, which can re-raise when a borrowed runtime was stopped, instead of returning false or calling abort. That exception leaves open with the lock still held. interaction awaits open outside its cleanup try/finally, so neither close nor abort runs and the shared lock is leaked.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit ff7b988. Configure here.

runtime = runtime.with_env(runtime_env)
self.runtime = runtime
if self.task.data.prompt is None and not self._has_user:
Expand Down Expand Up @@ -423,17 +427,22 @@ async def abort(self) -> None:
(a cancellation mid-setup, a lifetime bug raised to the caller) means the
driver will never reach `close()`. Safe after a partial `close()`."""
self._closed = True
if self._harness_session is not None:
with contextlib.suppress(Exception):
await self._harness_session.close()
with contextlib.suppress(Exception):
await self._stack.aclose()
if self.runtime is not None:
with contextlib.suppress(Exception):
await self.harness.cleanup(self.trace, self.runtime)
if self._borrowed_runtime is None and self.runtime is not None:
try:
if self._harness_session is not None:
with contextlib.suppress(Exception):
await self._harness_session.close()
with contextlib.suppress(Exception):
await self.runtime.stop()
await self._stack.aclose()
if self.runtime is not None:
with contextlib.suppress(Exception):
await self.harness.cleanup(self.trace, self.runtime)
if self._borrowed_runtime is None and self.runtime is not None:
with contextlib.suppress(Exception):
await self.runtime.stop()
finally:
if self._borrow_lock is not None:
self._borrow_lock.release()
self._borrow_lock = None

async def close(self) -> Trace:
"""Finish the rollout: tool servers and interception down, task `finalize`
Expand Down Expand Up @@ -522,6 +531,9 @@ async def close(self) -> Trace:
logger.warning(
"runtime teardown failed (rollout %s)", trace.id, exc_info=True
)
if self._borrow_lock is not None:
self._borrow_lock.release()
self._borrow_lock = None

@cursor cursor Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Close can skip lock release

High Severity

close releases _borrow_lock only after several cancellable awaits in the same finally, and suppress(Exception) does not swallow CancelledError. Cancellation mid-cleanup skips the release. abort correctly nests release in its own finally, but callers such as interaction that tear down via close alone can leave borrow_lock held forever and stall every later restricted borrower.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit ff7b988. Configure here.

logger.info(
"rollout done: id=%s task=%s reward=%.3f turns=%d stop=%s",
trace.id,
Expand Down
3 changes: 3 additions & 0 deletions verifiers/v1/runtimes/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,9 @@ def __init__(self, name: str | None = None) -> None:
# Per-run task values live on the runtime rather than its serializable config/info.
# Explicit process values (model credentials, proxy settings, etc.) override these.
self.env: dict[str, str] = {}
# Restricted runtimes have one sandbox-wide network policy. Borrowers hold
# this lock for their full rollout so setup cannot widen another agent's policy.
self.borrow_lock = asyncio.Lock()
self._uv_interpreters: dict[str, str] = {}
self._uv_script_locks: dict[str, asyncio.Lock] = {}
self._setup_claimed = False
Expand Down
Loading