fix(sessions): surface working_directory and agent_profile on list_sessions - #497
fix(sessions): surface working_directory and agent_profile on list_sessions#497tedswinyar wants to merge 12 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #497 +/- ##
=======================================
Coverage ? 91.14%
=======================================
Files ? 179
Lines ? 23321
Branches ? 0
=======================================
Hits ? 21256
Misses ? 2065
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
fanhongy
left a comment
There was a problem hiding this comment.
Summary
The additive response shape is compatible and the focused tests pass, but two lifecycle cases make the new ownership metadata unreliable. Both can cause an orchestrator to associate a live session with the wrong project.
Findings
P2: Persist the effective launch directory, not the raw optional argument
/tmp/cao-pr-review-awslabs-cli-agent-orchestrator-pr-497/checkout/src/cli_agent_orchestrator/services/terminal_service.py:308
The new database write stores working_directory exactly as received even though the backend resolves a different effective value. POST /sessions and ops MCP launch_session both allow the argument to be omitted; tmux/herdr then launch in os.getcwd(), but this line stores NULL. After the pane changes directory, list_sessions falls back to that drifting live path, which is the behavior persistence was intended to prevent. A relative input such as . is also persisted literally while tmux launches in a canonical absolute path; enrichment prefers the persisted . and never consults the pane. This makes sessions launched with the default or a relative path fail as stable, comparable ownership signals.
Resolve the effective canonical directory once before backend creation and pass that same value to both the backend and db_create_terminal, or read the newly created pane's directory immediately and persist it. Add coverage for omitted and relative launch paths, including a later pane-CWD change.
P2: Stale terminal rows misattribute a reused session name
/tmp/cao-pr-review-awslabs-cli-agent-orchestrator-pr-497/checkout/src/cli_agent_orchestrator/services/session_service.py:102
Enrichment accepts the first persisted profile/directory for a session without checking that the terminal row belongs to the current backend session incarnation. If a tmux session is killed outside CAO, its DB rows remain until retention cleanup. Relaunching the same custom session name creates another row but does not remove the old rows; the current SQLite query returns the older row first, and this loop immediately reports its directory/profile. A focused reproduction with /old/project followed by /new/project returned /old/project and the old profile for the live cao-reused session.
When creating a new backend session after session_exists is false, purge stale terminal rows for that session name before recording the new terminal, or otherwise validate terminal rows against live windows/session incarnation before using them. Add a regression test for external backend deletion followed by same-name relaunch.
P3: Remove the new changed-source mypy error
/tmp/cao-pr-review-awslabs-cli-agent-orchestrator-pr-497/checkout/src/cli_agent_orchestrator/services/session_service.py:114
next((...), None) introduces Argument 2 to "next" has incompatible type "None"; expected "dict[str, Any]" [arg-type]. Repository-wide mypy is non-blocking, but changed-source checking isolates this as the only error in the four modified source files. Use an empty-dict sentinel, an explicitly typed helper, or a small loop so the added code remains type-clean.
Validation
uv run pytest test/services/test_session_service.py test/clients/test_database.py test/ops_mcp_server/test_server.py test/services/test_terminal_service_full.py -q- 144 passeduv run black --check src/ test/- passeduv run isort --check-only src/ test/- passedgit diff --check 77befe84e69c0785c25ec024bb7e6391202cf305...HEAD- passed- Changed-source mypy - failed only at
session_service.py:114 - Focused reproductions confirmed raw
.resolves to an absolute launch path and stale-first rows produce the old directory/profile
gutosantos82
left a comment
There was a problem hiding this comment.
PR Review: #497 — fix(sessions): surface working_directory and agent_profile on list_sessions
Summary
This PR addresses a real multi-orchestrator hazard (an external orchestrator nearly shut down another caller's live sessions as "orphans") by enriching list_sessions entries with working_directory and agent_profile. The design is additive and careful: a nullable DB column following the caller_id migration pattern, best-effort per-session enrichment that never blanks the list, a pane-cwd fallback, and a SessionListEntry model with extra="allow" so existing fields survive. All 144 change-selected tests pass in the PR worktree and CI is fully green. However, two lifecycle gaps — independently verified here — mean the persisted "launch-time directory" (the PR's own headline design point) is absent or wrong in exactly the common cases: default launches persist NULL (degrading to the drifting pane path the PR says persistence exists to prevent), and stale terminal rows from an externally killed session misattribute a relaunched same-name session to the old project. Recommend one round of changes.
Blocking (must fix before merge)
- [correctness] src/cli_agent_orchestrator/services/terminal_service.py:308 — the DB write persists the raw optional argument, not the effective launch directory. Verified structurally:
TmuxClient.create_sessioncalls_resolve_and_validate_working_directory()internally (tmux.py:135), which resolvesNoneto a default and canonicalizes relative paths — butdb_create_terminalreceives the pre-resolution value. Consequences: (a) any launch that omitsworking_directory(the default forcao launchwithout--working-directory, and for handoff/assign unlessCAO_ENABLE_WORKING_DIRECTORY=true) persistsNULL, solist_sessionsfalls back to the live pane path — which drifts when the agentcds, which is precisely the unreliability the PR body says persistence was designed to prevent; (b) a relative input like.is persisted literally and, being truthy, is preferred over the pane path, so it never self-corrects into a comparable absolute path. Fix: resolve the effective canonical directory once and pass the same value to both the backend anddb_create_terminal(or read the newly created pane's cwd immediately after creation and persist that). Add coverage for omitted and relative launch paths. - [correctness] src/cli_agent_orchestrator/services/session_service.py:102 (
_enrich_session_ownership) — stale terminal rows misattribute a reused session name.list_terminals_by_session(database.py:597) has noORDER BYand no liveness/incarnation check; if a tmux session is killed outside CAO, its rows persist until retention cleanup, and relaunching the same custom session name adds a new row without purging the old ones. SQLite returns oldest-first, and the enrichment loop takes the first truthy profile/directory — so the live session reports the previous project's directory and profile. For a feature whose entire purpose is letting an orchestrator decide "is this session mine?" before acting (including shutdown), silently reporting another incarnation's identity is the failure mode that caused the motivating incident. Fix: purge stale rows for a session name when creating a new backend session, or filter rows against live windows before use. Add a regression test for external kill + same-name relaunch. (The maintainer-side review reports a focused reproduction returning/old/projectfor the live relaunched session; the code structure confirms the mechanism.)
Important (should fix)
- [types] src/cli_agent_orchestrator/services/session_service.py:114 —
next((t for t in terminals if t.get("tmux_window")), None)introduces the only changed-source mypy error:Argument 2 to "next" has incompatible type "None"; expected "dict[str, Any]". Verified: mypy on the worktree flags exactly this line (the other errors are pre-existing in unrelated files). Repo-wide mypy is non-blocking, but don't add new errors — a small loop or a typed helper keeps it clean.
Nits (optional)
- [correctness] src/cli_agent_orchestrator/services/session_service.py:131 — the new comment claims the
.get("id", "")filter hardens against a backend dict that "must not blank the entire list", but it only covers a missing key. Verified dynamically: a backend returning{"id": None}raisesAttributeErrorin the comprehension, is swallowed by the outerexcept, and returns[]— blanking the list, the exact failure class the comment says it prevents.(s.get("id") or "")closes it for one character of code. - [performance] session_service.py
list_sessions— now N+1: one DB query per session, plus one tmux subprocess (get_pane_working_directory) per session whose persisted cwd is NULL. Given the Blocking finding above, that is every default-launched and every pre-migration session, on every poll of/sessions(the web UI polls it). Fine at local scale, but worth a note — fixing Blocking #1 also mostly eliminates the subprocess churn. - [design]
_enrich_session_ownershipmerge semantics —agent_profileandworking_directorycan come from different terminals (verified dynamically: terminal-1's directory + terminal-2's profile merge into one entry). For supervisor+worker sessions this is usually harmless (first row is the supervisor), but the field description "the session's first known terminal" is not quite what the code does. Either take both fields from the first terminal that has either, or adjust the description. - [conventions] CHANGELOG.md — no
[Unreleased]entry; sibling fixes in this area carry hand-curated entries. Convention-by-precedent.
Tests
Strong for the paths it covers: persisted-value preference (with an AssertionError-armed fake proving the pane is not consulted), pane fallback, per-field failure isolation, orphan sessions, per-session enrichment failure isolation, migration idempotency against fresh and legacy databases, and full round-trip through a real SQLite engine. The ops MCP test pins extra="allow" field preservation (session_name, terminal_count). What's missing maps exactly to the blocking findings: no test launches with an omitted or relative working_directory and asserts what gets persisted, and no test covers external kill + same-name relaunch. The suite tests the read path thoroughly but assumes the write path stored the right thing.
Verification
Dynamic verification ran in a fresh worktree at the PR head (683d78b), PYTHONPATH=<worktree>/src + main-checkout venv (Python 3.12).
- ✓ VERIFIED — change-selected suites (
test_session_service.py,test_database.py,test_server.py(ops),test_terminal_service_full.py): 144 passed in 6.6s. Matches the maintainer-side review's count. CI green on 3.10/3.11/3.12 + security scans. - ✓ VERIFIED —
get_pane_working_directoryexists onTerminalBackendbase, tmux backend, and herdr backend — the fallback works on both backends. - ✓ VERIFIED —
{"id": None}from the backend blanks the entire session list (log:Failed to list sessions: 'NoneType' object has no attribute 'startswith') — see nit. - ✓ VERIFIED — enrichment does not mutate the backend's session dict (copies first); mixed-terminal merge takes profile and directory from different rows — see nit.
- ✓ VERIFIED — mypy flags
session_service.py:114as the only new changed-source error. - ✓ VERIFIED (structurally) —
TmuxClient.create_sessionresolves/canonicalizesworking_directoryafter the raw value is already persisted to the DB; nothing writes the resolved value back. Confirms Blocking #1. - ✓ VERIFIED (structurally) —
list_terminals_by_sessionhas no ORDER BY and no liveness filter; no purge of same-name rows on session creation. Confirms the mechanism of Blocking #2 (maintainer-side review reports a live reproduction).
Verdict
Request changes — the direction, API shape, and test discipline are all good, but the persisted launch directory must be the effective resolved directory (not NULL/relative passthrough), and stale same-name rows must not misattribute a live session. Both are cheap, well-localized fixes with clear regression tests, and both sit on the exact code path the feature exists to make trustworthy.
There was a problem hiding this comment.
Pull request overview
This PR adds best-effort ownership metadata to session listings so callers can distinguish “their” sessions from other orchestrators sharing the same cao-server, by surfacing each session’s working_directory and agent_profile.
Changes:
- Persist
working_directoryin the terminals DB table and plumb it through terminal creation and terminal metadata reads. - Enrich
services.session_service.list_sessions()results withworking_directory(persisted preferred, tmux-pane cwd fallback) andagent_profile. - Update ops MCP server models and expand test coverage across session service, database, terminal service, and ops MCP server.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/services/test_terminal_service_full.py | Updates terminal creation test to pass the new working_directory parameter. |
| test/services/test_session_service.py | Adds coverage for session list enrichment (persisted cwd, pane fallback, orphaned sessions, partial failures). |
| test/ops_mcp_server/test_server.py | Extends ops MCP list-sessions test to assert the new field is present in dumped session output. |
| test/clients/test_database.py | Extends DB tests for working_directory read/write and schema migration idempotency. |
| src/cli_agent_orchestrator/services/terminal_service.py | Persists working_directory into the terminals metadata record during terminal creation. |
| src/cli_agent_orchestrator/services/session_service.py | Enriches list_sessions() entries with working_directory/agent_profile via DB metadata and tmux fallback. |
| src/cli_agent_orchestrator/ops_mcp_server/models.py | Introduces SessionListEntry model (extra fields allowed) and updates SessionListResult sessions typing. |
| src/cli_agent_orchestrator/clients/database.py | Adds working_directory column, migration, and includes it in terminal CRUD/list APIs. |
Comments suppressed due to low confidence (1)
src/cli_agent_orchestrator/services/session_service.py:122
- Same as above: this exception is intentionally swallowed, but the log message omits the traceback (and uses eager f-string formatting). Including exc_info=True makes pane-cwd resolution failures actionable in production logs.
try:
enriched["working_directory"] = backend.get_pane_working_directory(
session_name, terminal["tmux_window"]
)
except Exception as e:
logger.warning(f"Failed to resolve working directory for {session_name}: {e}")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| try: | ||
| terminals = list_terminals_by_session(session_name) | ||
| except Exception as e: | ||
| logger.warning(f"Failed to load terminal metadata for {session_name}: {e}") | ||
| terminals = [] |
| try: | ||
| terminals = list_terminals_by_session(session_name) | ||
| except Exception as e: |
683d78b to
876a9c8
Compare
|
Thanks @fanhongy and @gutosantos82 — both blocking findings were real and are fixed, along with the mypy and Blocking 1 — persist the effective launch directory ( Blocking 2 — stale rows misattribute a reused session name ( Important — mypy: the Nit — Nit — merge semantics: VerificationBecause the rebase brought in #513 — which adds a second launch path (
Never Change-selected suites: 157 passed (including #513's new tests). black/isort clean; mypy clean on changed files. Two items deliberately not changed here
One thing the review surfaced that is not introduced by this PR: |
fanhongy
left a comment
There was a problem hiding this comment.
lgtm, merge if @gutosantos82 agree.
sujoydc
left a comment
There was a problem hiding this comment.
Verified the round-2 fixes at 876a9c8 against @gutosantos82's original review rather than
the summary. All seven items are accounted for: both blocking findings, the mypy next()
error, the {"id": None} list-blanking nit, the two-terminal merge nit, and the missing
[Unreleased] CHANGELOG entry are closed; the N+1 is openly deferred. Reproduced the test
claim: 157 passed on session service + database + terminal service + ops MCP server, up
from the 144 at the previous head.
Spot-checking the two blocking fixes rather than taking them on faith:
- Blocking 1 is properly closed.
_resolve_working_directorypassesallow_create=False, allow_file=False, description="Working directory"with the sameNone -> os.getcwd()
default asclients/tmux.py::_resolve_and_validate_working_directory, and the single
resolved value goes to bothbackend.create_sessionanddb_create_terminal. The
resulting double resolution is idempotent:/tmp/xcanonicalizes to/private/tmp/xon
both passes, so the tmux client's second pass can't re-transform what was persisted. - Blocking 2 is properly closed and correctly scoped.
delete_terminals_by_sessionis an
exacttmux_session ==filter, and thesession_existsguard raises at line 244 before
create_session, so the purge at line 260 can only ever touch rows for a name the
backend reported absent. It can't delete a live incarnation's rows.
One new finding and one scope note.
The change is additive for tmux but not for herdr. herdr_backend does no path
validation of its own, so hoisting resolution into terminal_service.create_terminal adds
a gate on that path rather than relocating one. Details inline on terminal_service.py:152.
I think hoisting is right, and centralizing beats one backend enforcing it — the ask is
just a caveat in the body and CHANGELOG, since "The change is additive" is currently stated
without qualification and a herdr operator will see new ValueErrors from a change that
doesn't look related.
Scope note worth a sentence in the body. Blocking 1 fixed NULL-versus-drift, but when
working_directory is omitted the resolved value is the server's os.getcwd() — the same
string for every caller sharing that server. working_directory is an explicit optional
param on every launch path (api/main.py POST /sessions and /sessions/{name}/terminals,
ops MCP launch_session), and nothing forwards the caller's own cwd automatically. So a
populated working_directory is only an ownership signal when the caller passed one
explicitly. The body frames the directory as "the strongest signal," which holds for
explicit launches but not defaults, and a downstream orchestrator reading a populated field
as proof of ownership is the exact failure this change exists to prevent. Worth stating so
consumers treat the field as a hint rather than an assertion.
876a9c8 to
b8f2bd4
Compare
|
Rebased onto current The conflicts were in All five commits are unchanged in content. Verification after the rebase:
@gutosantos82 — your review is pinned to @fanhongy — thanks for the approval; flagging that the head moved for the rebase in case you want another look. |
Graft two valuable test cases from the rival sonnet implementation: 1. Orphaned tmux session (session exists but no DB terminals) — should list with null metadata, not fail 2. Per-session exception isolation — one session's DB failure should not blank the entire list Both tests pass against the current codex implementation, proving the DB-persistence approach already handles these edge cases correctly via the per-session try/except in _enrich_session_ownership. The rival's pane-only approach validated these same cases; grafting the tests ensures the stronger DB-persistence approach doesn't regress on graceful degradation.
Adversarial review (caom-5oa) found the SESSION_PREFIX filter used direct
s["id"] indexing, so a backend returning a session without an id would
raise KeyError and blank the whole list. Use s.get("id", ""). Not reachable
with shipped backends; hardens against a future one.
Use `enriched.get("id") or ""` so an explicit id=None collapses to the
empty string like the sibling list_sessions guard, instead of the truthy
string "None". Review nit (kiro-sonnet5 + reviewer-opus).
b8f2bd4 to
eed7803
Compare
gutosantos82
left a comment
There was a problem hiding this comment.
PR Review: #497 — fix(sessions): surface working_directory and agent_profile on list_sessions
Summary
Delta review against the previously reviewed head b8f2bd48 (verdict there: Request changes).
The author's claim that this push is "purely a rebase" is verified: git range-diff shows
the same five commits replayed onto current main (752be53f, the v2.4.1 release); four are
byte-identical and the fifth differs only in CHANGELOG conflict placement. The rebase resolves
the previous round's CONFLICTING gate — the branch is MERGEABLE again and CI is fully green.
Substance is unchanged: this remains good, well-motivated work (ownership metadata on
list_sessions to prevent cross-orchestrator session shutdown), with all of the first round's
blocking findings still fixed. However, the one live blocker from the prior round — the
test-isolation regression introduced by the unmocked delete_terminals_by_session call —
reproduces identically at this head (18 failed / 34 passed on a clean environment, all
sqlite3.OperationalError: no such table: terminals, while upstream main passes 52/52 under
the same conditions). The maintainer's CHANGES_REQUESTED also still stands. One small,
well-scoped fix away from approval.
Blocking (carried forward — re-verified at this head)
- [tests] test/services/test_terminal_service_full.py (
TestCreateTerminal,TestCreateTerminalEnvVars) —
re-verified ateed78037in a fresh worktree with a clean tempHOME: 18 failed, 34
passed, every failuresqlite3.OperationalError: no such table: terminals. Baseline
control: the same file on upstreammain(752be53f) under the same cleanHOMEpasses
52/52, so this is unambiguously a PR regression, not environmental. Mechanism unchanged:
the newdelete_terminals_by_session(session_name)call increate_terminal
(terminal_service.py, after tmux session creation) is not in the@patchstacks of these
pre-existing DB-free tests, so it falls through to the real SQLite engine at
$HOME/.aws/cli-agent-orchestrator/db/. Green CI at this head remains an ordering artifact
(an earlier test in the full-suite run initializes the DB), and local runs on a developer
machine silently mutate the developer's real CAO database. Fix remains small: add
delete_terminals_by_sessionto the affected@patchstacks, or add an autouse per-test DB
fixture. Repro:H=$(mktemp -d); HOME=$H python -m pytest test/services/test_terminal_service_full.py -q.
Resolved since last review
- [merge] CONFLICTING gate cleared — the branch is rebased onto current
mainand reports
mergeable: MERGEABLE. Verifiedmainis an ancestor of the PR head and the PR diff vs main
matches the advertised 9 files / +561 −25.
Important (carried forward — unchanged at this head)
- [consistency] terminal_service.py (
_resolve_working_directory) — resolution still runs
for every backend before creation (confirmed at this head: called unconditionally at
create_terminal step 2), changing the herdr backend's contract (must-exist +
blocked-system-dir checks it previously didn't apply). Arguably desirable hardening, but
still undocumented and untested on the herdr path. A one-line statement of intent in the PR
body/CHANGELOG would close this. - [security]
GET /sessionsdisclosure — confirmed at this head the endpoint still carries
no scope dependency (@app.get("/sessions")with noDepends), while this PR adds absolute
filesystem paths (embedding OS usernames/project names) and agent profiles to its response.
Missing auth is pre-existing; the disclosure is new. Cheap mitigation: gate with the same
require_any_scopedependency as/events/history, here or immediately after.
Process note
The maintainer (gutosantos82) review state is still CHANGES_REQUESTED, so
reviewDecision=CHANGES_REQUESTED gates merge regardless of this review. fanhongy has
approved conditional on the maintainer's agreement. Once the test-mocking fix lands, this PR
is in good shape: the first round's substantive blockers (persist effective launch directory,
stale-row purge, mypy, id=None handling) all remain fixed at this head.
…reporting mandatory Two fixes for the self-learning loop, which recorded zero outcomes across its first scheduled runs. 1. Retrospector launch failed for 60s and was misreported as a timeout. `retrospector` is a BUILT-IN profile, so it lists in `cao profile list`, but `cao launch` does not materialize the provider-side agent JSON for built-ins. With no ~/.kiro/agents/retrospector.json, kiro-cli logs 'no agent with name retrospector found. Falling back to user specified default' and renders a prompt with no [profile] prefix. CAO derives its kiro idle pattern from the profile name (kiro_cli.py:190), so it never matches, the terminal never reaches IDLE, and create_terminal fails after provider_init_timeout. The driver now installs the built-in profile when its kiro JSON is absent (idempotent) and captures launch output to pr-review-data/retro-launch.log instead of /dev/null — swallowing stderr is what hid the real cause. Verified: with the JSON removed, the block self-heals and launches in ~10s. Note this is environment setup, not a CAO defect: kiro-cli 2.15.2 still renders the '[profile] N% >' prompt CAO expects, confirmed by capturing raw pane bytes with a profile that does load. 2. Supervisors ignored the outcome-reporting step on clean runs. PR awslabs#497 was reviewed successfully with Step 7 present in the prompt and still reported nothing. Step 7 is now explicitly mandatory and unconditional, states that a clean run must still report (success=true, empty friction_notes), drops the 'load the cao-learning skill first' indirection (the tool is already granted via @cao-mcp-server), and narrows the skip condition to a literal disabled: true response. New Step 8 requires confirming the recorded outcome in the final message so a miss is visible in the transcript. Also defines SCRIPT_DIR/REPO_ROOT in run_reviews.sh, which it never had — the new install path needs REPO_ROOT and set -u would have aborted the run.
…ests stop hitting the real DB Addresses the review finding on awslabs#497: this PR added delete_terminals_by_session(session_name) to create_terminal, but 15 @patch stacks in TestCreateTerminal/TestCreateTerminalEnvVars did not mock it, so the call fell through to the real SQLite engine at $HOME/.aws/cli-agent-orchestrator/db/. Two consequences. On a clean HOME the tests fail outright with "sqlite3.OperationalError: no such table: terminals" (18 failed / 34 passed); green CI was an ordering artifact, since an earlier test in a full-suite run initializes the DB first. On a developer machine they silently mutate the developer's real CAO database. Mocking at the terminal_service import site matches how test/services/test_flow_service.py already isolates the identical call in flow_service.py. The purge behavior itself remains covered: test/clients/test_database.py tests delete_terminals_by_session directly, and test_session_service.py::TestSessionOwnershipIntegration::test_same_name_relaunch_purges_stale_terminal_metadata fails if the create_terminal call is removed — verified by mutation, so this isolates the unit tests without silencing the feature. Verified: test/services/test_terminal_service_full.py => 52 passed on a clean HOME (was 18 failed / 34 passed); upstream main passes 52/52 under the same conditions, confirming the regression was introduced by this PR.
…inal_service_full.py Follow-on to the previous commit. That commit fixes the regression this PR introduced; these three gaps are older, present identically on upstream main, and unrelated to delete_terminals_by_session — but they keep this file touching a real SQLite database, so the previous commit alone still creates $HOME/.aws/cli-agent-orchestrator/db/cli-agent-orchestrator.db on a clean HOME. With this commit the file creates no DB file at all. Found by instrumenting sqlite3.dbapi2.connect (patching sqlite3.connect is not enough — the pysqlite dialect binds `from sqlite3 import dbapi2 as sqlite`). 1. test_create_terminal_session_not_found / test_create_terminal_session_already_exists: both raise with terminal_id already generated, so create_terminal's exception cleanup calls the real db_delete_terminal(). A bare except swallows the error, so the test never fails — it just issues a DELETE against a nonexistent table on a real SQLite file. Now patched. 2. test_send_input_success did not mock status_monitor, so the real singleton's get_status() cascaded into the real provider_manager.get_provider() -> get_terminal_metadata() -> real DB query. Now patched. 3. Three send_input tests reach inject_memory_context(), which constructs a real MemoryService() and queries the DB. The module-level _memory_injected_terminals cache means only the first test to run pays it, so which test connects is order-dependent — all three need the patch. get_curated_memory_context is stubbed to "" to keep injected context falsy, preserving the exact message content two of these tests assert on. None of these three are asserted by name anywhere in the suite, so patching them does not defeat any test's stated purpose. Verified: 52 passed on two different clean HOMEs, and no file is created under .aws/cli-agent-orchestrator/db/ (only the empty directory, which clients/database._ensure_db_dir() mkdirs unconditionally at import time on main as well).
…vice_coverage.py Same awslabs#497 regression as the two prior commits, in the fourth file: four TestCreateTerminalCleanup/TestCreateTerminalSessionCleanupGuard tests drive create_terminal through the new_session=True path far enough to reach delete_terminals_by_session(session_name), which fell through to the real SQLite engine on a clean HOME. Added the patch as the topmost decorator with its mock appended last, mirroring this file's existing db_delete_terminal/ db_create_terminal stacking convention. Verified: 12 passed on a clean HOME (was 8 passed / 4 failed).
…phase0.py Same awslabs#497 regression, third file. test_omitted_engine_launches_as_explicitly_pinned_v2 and test_explicit_model_override_is_probed_even_when_profile_has_none both drive create_terminal through the new_session=True success path far enough to reach delete_terminals_by_session(session_name), hitting the real SQLite engine on a clean HOME. Added patch(f"{_MODULE}.delete_terminals_by_session") to each test's existing `with (patch(...) as x, ...):` block, alongside db_create_terminal, matching this file's context-manager convention (not decorator stacks). Verified: 11 passed on a clean HOME (was 9 passed / 2 failed).
…_emission.py Same awslabs#497 regression, fourth and final file. Both TestTerminalPluginEvents tests that exercise create_terminal's new_session=True success/failure paths reached delete_terminals_by_session(session_name) unmocked, hitting the real SQLite engine on a clean HOME. Checked test/services/conftest.py first -- its only fixture is an unrelated autouse Kiro-probe stub, so decorator stacking (this file's existing convention) is the right fix, not a shared fixture. Added the patch as the topmost decorator with its mock appended last, matching the same positional convention applied in the other three files. Verified: 13 passed on a clean HOME (was 11 passed / 2 failed).
…ile check The 3-file command passed 36/36 after the previous three commits, but the task's explicit acceptance check (no .db FILE under a clean HOME's db/ dir) turned up three more pre-existing gaps -- same class of bug as 040039e, unrelated to delete_terminals_by_session, that happened to be silently swallowed rather than failing outright: 1. test_terminal_service_coverage.py::test_no_kill_session_when_session_already_exists and ::test_kill_session_when_we_created_it_and_later_step_fails: terminal_id is already generated when these raise, so create_terminal's exception cleanup calls the real db_delete_terminal(). Now patched (mirrors the file's existing db_delete_terminal stacking). 2. test_plugin_event_emission.py::test_create_terminal_does_not_dispatch_on_failure: same db_delete_terminal cleanup-path gap as above. 3. test_plugin_event_emission.py::test_send_input_dispatches_post_send_message_event_for_each_orchestration_mode and ::test_send_input_does_not_dispatch_on_failure: neither mocks status_monitor, so the real singleton's get_status() cascades into the real provider_manager.get_provider() -> get_terminal_metadata() -> real DB query (status_monitor.py holds its own provider_manager reference, independent of the one patched on the terminal_service module). The first of the two also reaches inject_memory_context's real MemoryService() on whichever parametrized case runs first, per the module-level _memory_injected_terminals cache -- same fix as 040039e's item 3, get_curated_memory_context stubbed to "" to preserve message content assertions. None of these are asserted by name anywhere in the suite (checked via grep for purge/stale-row assertions before patching), so this isolates the units without silencing coverage of the real behavior. Verified: the 3-file command still passes 36/36, and two separate fresh HOMEs now create only the empty db/ directory -- no .db file.
…tion stops depending on test order
Round-2 review finding. The previous commits mocked each DB call individually,
which left one hole: TestMessagePluginEvents::test_send_input_does_not_dispatch_on_failure
satisfied the "no real DB" bar only because a parametrized sibling ran first and
poisoned the module-global _memory_injected_terminals cache
(terminal_service.py:89,114). Selecting that test alone reached the real
MemoryService:
HOME=$(mktemp -d) pytest test/services/test_plugin_event_emission.py \
-k does_not_dispatch_on_failure # created a real .db file
With no HOME override that reads the developer's live CAO database. It also hits
under pytest --lf, any single-nodeid selection, and pytest-xdist --dist load
(pyproject.toml declares xdist). Sequential CI runs were unaffected, which is
why it stayed hidden.
Fixes it two ways, per the reviewer's recommendation:
1. test_send_input_does_not_dispatch_on_failure gets the MemoryService patch its
siblings already had — the specific gap.
2. All four touched modules now carry
pytestmark = pytest.mark.usefixtures("isolated_memory_db"), routing default
memory sessions to a per-test SQLite DB. This is already the repo idiom
(test/conftest.py:174, used by six modules) and makes these files immune to
the next unmocked DB call rather than only to the ones enumerated today.
The per-call mocks are kept — several tests assert on them
(mock_db_delete.assert_called_once_with(...), call-order assertions), so the
fixture complements them rather than replacing them.
Verified: 88 passed across the four files with no .db file; every one of the 25
collected nodeids run individually on a fresh HOME creates no .db file (was 1);
test/services/ excluding agui = 1649 passed, 6 skipped. The single
test_fifo_reader.py failure is a pre-existing environmental flake — it
reproduces intermittently on this branch (1 of 3 runs) and touches no code in
this change.
|
Thanks @gutosantos82 — the test-isolation regression was real and is fixed, and your review surfaced a second instance of it that I'd missed. Seven commits, all test-only ( Reproduced your finding first. On The fix ( Verifying it isolates rather than silences. A fix that works by adding mocks can convert a real failure into a false green, so I mutation-tested it: removing the The regression was wider than either of us spotted. An adversarial review pass found the same unmocked-call regression in three sibling files that neither of us had checked:
Measured: 8 failed / 28 passed on the PR head vs 36 passed / 0 failed on Two commits are pre-existing cleanup, not this PR's regression — flagged separately so you can weigh them independently: A second review round found one more hole, fixed in Fixed both narrowly (the missing patch) and generically: all four modules now carry Verification
Note this branch is not rebased onto the latest Your two carried "Important" items are still open and I'd like your read on them: the herdr |
gutosantos82
left a comment
There was a problem hiding this comment.
PR Review: #497 — fix(sessions): surface working_directory and agent_profile on list_sessions
Summary
Delta review against previously reviewed head eed78037 (verdict there: Request changes, with
exactly one live blocker — the test-isolation regression from the unmocked
delete_terminals_by_session call). The author's claim that this push is seven test-only
commits is verified: git diff eed78037..b0495f16 --name-only touches only
test/services/test_terminal_service_full.py, test_terminal_service_coverage.py,
test_kiro_engine_phase0.py, and test_plugin_event_emission.py (+118 −6); zero src/ files
change. The blocker is fixed and re-verified empirically (details below). With that
cleared, the substance of the PR — ownership metadata (working_directory, agent_profile)
on list_sessions entries so a caller on a shared cao-server can tell its own sessions
apart, persisted launch-time directory with live-pane fallback, best-effort per-entry
enrichment, idempotent migration — stands as previously assessed: well-motivated, additive,
and carefully scoped, with all of the first round's substantive blockers (persist effective
launch directory, stale-row purge, mypy, id=None handling) still fixed at this head.
Blocker resolution — verified at this head
- [tests] test-isolation regression: FIXED. Re-ran the exact prior repro in a fresh
worktree atb0495f16with a clean tempHOME:
test/services/test_terminal_service_full.pynow passes 52/52 (prior head: 18 failed
withsqlite3.OperationalError: no such table: terminals), and no.dbfile is created
under the cleanHOME. All six suites the PR touches pass together under the same clean-HOME
conditions: 141/141 (test_terminal_service_full,test_terminal_service_coverage,
test_kiro_engine_phase0,test_plugin_event_emission,test_session_service,
test_database,test_server). The fix is the right shape, not a bandage: a file-wide
pytestmark = pytest.mark.usefixtures("isolated_memory_db")routes any residual DB touch to
a per-test SQLite engine (sharedtest/conftest.pyfixture, monkeypatchedSessionLocal,
disposed on teardown), anddelete_terminals_by_sessionis added explicitly to every
affected@patchstack — so the DB-free tests are DB-free again by construction, not by
test ordering. The author also closed pre-existing real-DB touches surfaced by the same
check (e.g.,db_delete_terminalin the session-not-found path) and a second instance the
maintainer's review surfaced in other files. - One
.dbfile does still appear when runningtest/clients/test_database.pyunder a clean
HOME — verified this is pre-existing on upstreammain(same behavior at2a6f20cb),
by design for the DB-client suite, and not a PR regression. - CI: 19/19 checks pass at this head.
Non-blocking (carried forward — src/ unchanged since last review, both re-confirmed)
- [consistency] terminal_service.py (
_resolve_working_directory) — resolution still runs
unconditionally for every backend before creation, tightening the herdr backend's contract
(must-exist + blocked-system-dir checks it previously didn't apply). Reasonable hardening,
still undocumented/untested on the herdr path. A one-line statement of intent in the PR body
or CHANGELOG closes this. - [security]
GET /sessionsdisclosure — re-confirmed at this head:@app.get("/sessions")
(api/main.py:2012) carries no auth/scope dependency, and this PR adds absolute filesystem
paths (embedding OS usernames/project layouts) and agent profiles to its response. Missing
auth is pre-existing; the added disclosure is new. Cheap mitigation now or as fast-follow:
gate with the same scope dependency used by/events/history.
Process note
The maintainer (gutosantos82) submitted CHANGES_REQUESTED on Aug 5, i.e. before the
Aug 6 test-only fix push, so reviewDecision=CHANGES_REQUESTED still gates merge but is
addressed-pending-re-review, not outstanding. fanhongy has an APPROVED review on record.
The finding the maintainer raised (the test-isolation regression) is exactly what this push
fixes, and the author credits the review for surfacing a second instance. The ball is in the
maintainer's court to re-review; nothing further is owed by the author beyond the optional
nits above.
(b) adds a new information-disclosure surface to an unauthenticated endpoint — an operator
should read the /sessions note before acking, and publishing an approve while the
maintainer's CHANGES_REQUESTED is unresolved should be a deliberate choice.
Verification performed
- Fetched PR head
b0495f16and upstreammain(2a6f20cb); worktrees at both. git diff eed78037..b0495f16 --name-only: 4 test files only, zero non-test files.- Clean-HOME repro (
HOME=$(mktemp -d)) of the prior blocker: 52/52 pass at this head
(18F/34P at prior head); no stray.dbcreated by the terminal-service suite. - All 6 touched suites under clean HOME: 141 passed.
- Per-suite DB-touch bisection: only
test_database.pywrites a DB file; same onmain. gh pr checks: 19 pass, 0 fail.mergeable: MERGEABLE.- Inspected fixture implementation (
test/conftest.py:174) and patch-stack additions.
|
@tedswinyar can you please help to resolve the conflicts. |
A single
cao-serverserves every session from one flat namespace filtered only by thecao-prefix, solist_sessionsreturns the sessions of every orchestrator using that server, with no way to tell them apart. When more than one orchestration runs against the same server (for example, two people driving CAO from different project directories, or one person running separate sessions per repo), a caller cannot distinguish its own sessions from another's. This surfaced in practice: an external orchestrator listed the sessions, saw sessions it did not recognize, and nearly shut them down as orphans while they were doing live work.This change adds ownership metadata to each
list_sessionsentry so a caller can identify which sessions are its own. Each entry now carries aworking_directoryand anagent_profile. The working directory is the strongest signal, since a given orchestrator normally drives one project directory.How the working directory is resolved. A new nullable
working_directorycolumn is added to the terminals table and populated with the launch-time directory when a terminal is created.list_sessionsprefers that persisted value, falls back to the live tmux pane's current path when it is absent, and returns null when neither can be resolved. Persisting the launch-time value matters because the pane's current path drifts if the agent changes directory during its run, so the pane path alone is an unreliable ownership marker.Scope and safety. The change is additive. The
cao-prefix filter is unchanged, the new fields are optional, and enrichment is best-effort per session: if one session's metadata cannot be resolved, that entry gets null fields and the rest of the list is returned normally rather than the whole call failing. The database migration follows the existingcaller_idcolumn pattern and is idempotent. Existing callers oflist_sessions(the web UI,cao session list, the HTTP/sessionsendpoint) keep working, and the newSessionListEntrymodel allows extra fields so nothing the backend already returned is dropped.This is a tactical fix, not the whole story. Surfacing the working directory lets a caller identify ownership by convention, but it does not enforce anything: any caller can still see and shut down any session. The deeper question is whether the shared server should scope or namespace sessions per orchestrator at all, and how that should interact with the web UI and CLI, which also list every session. That belongs in a design discussion with the maintainers, and I will open a separate issue and PR for it rather than fold a larger behavioral change into this one. This PR is the low-risk, immediately useful piece.
Tests. Drives
list_sessionsagainst a backend with a known working directory and asserts the persisted value is returned; asserts the pane-path fallback fires when the persisted value is null; asserts an orphan session (tmux present, no database row) lists with null fields rather than crashing; asserts one unresolvable session does not blank the whole list; and covers migration idempotency against both a fresh database and one that predates the column. Relevant suites pass (session service, database, ops MCP server).