feat: side-panel launcher to start a coding agent in a terminal - #17
Closed
whysosaket wants to merge 38 commits into
Closed
whysosaket wants to merge 38 commits into
whysosaket wants to merge 38 commits into
Conversation
Foundation for the extension-triggered Claude launch: a private launch/ subdir under the 0700 state dir for transient launcher artifacts, and a safe_mode reader that mirrors the CLI launcher's getSafeMode so the daemon and `browserops --claude` agree on the danger flag. Path is injectable for hermetic tests.
The detached daemon has no TTY, so it writes a launcher script + a context file under the 0700 launch/ dir and asks the OS to open a terminal that runs it. Per-platform: macOS bash .command opened via 'open -a iTerm/Terminal'; Windows PowerShell .ps1 opened via wt.exe or 'cmd /c start powershell'. Page metadata is seeded into Claude via --append-system-prompt but only ever reaches the shell as the content of a file read into a variable, never interpolated into the script source — so arbitrary page titles can't inject shell commands. claude is invoked by bare name inside the new terminal so it resolves against the user's interactive PATH. Pure builders + an injectable orchestrator are covered by 20 unit tests, incl. an injection-safety case.
Adds a token-authed, origin-pinned POST /launch HTTP endpoint next to /pair and /status. Two independent gates: the Origin must match the pinned extension (a web page's real Origin won't) AND the body must carry the loopback token (only the pinned extension gets it via /pair). On success it calls the injected onLaunch (daemon-runtime wires it to launchClaude) and returns the outcome as JSON so the popup can render real success/error. 13 endpoint tests cover the happy path + field mapping, 401 (bad/missing token), 403 (non-pinned and web origins), empty-origin host_permissions case, 502 handler-failure, 503 unavailable, 400 malformed body, 405, and the OPTIONS preflight.
The button captures the active tab's metadata (url + title always; meta description / og:site_name via a self-contained executeScript that gracefully degrades on restricted pages), reads the loopback token, and POSTs to the bridge's /launch endpoint. The POST is independent of the WS connection, so it works from a fresh popup as long as the daemon is up; a connection refusal renders as "Bridge not running". The button disables while pending to avoid a double-spawn. Pure helpers (body builder, status mapper, text clamp) live in launch.ts with 7 unit tests; index.ts is the DOM/chrome glue.
Executes the actual generated .command against a fake claude on PATH and asserts it execs with exactly [--dangerously-skip-permissions, --append-system-prompt, <context>] — proving the quoted "$CTX" is not word-split, the malicious $(touch PWNED) title is passed as inert data (no command injection at runtime), and the launcher self-deletes its temp files. A second case proves the claude-not-found path exits 1. Gated to darwin; skipped elsewhere.
A single AGENTS catalogue both sides consume: the extension renders it as
the launcher's agent picker, the bridge validates /launch requests against
it. Claude is enabled; Codex/Cursor ship disabled ("coming soon") so the
launcher is agent-agnostic and adding them later is a registry entry plus a
per-agent launch spec.
Renames launch-claude -> launch-agent and generalizes it around the shared
agent registry: launchAgent({agent, instruction, page}) validates the agent
via isLaunchableAgent (codex/cursor and unknown ids -> agent_disabled) and
dispatches through a per-agent AGENT_LAUNCHERS spec (only claude today).
Adds an optional instruction (typed in the composer): written to its own
.instr file, read into $INSTR/$instr, and passed as a separate quoted
positional after the context — same file-not-argv injection invariant as the
page metadata. The /launch endpoint + daemon-runtime now thread agent +
instruction. Tests updated; shell E2E proves the instruction is a distinct
quoted positional and $(...) values stay inert.
Replaces the popup's single launch button with a roomier side-panel
launcher (chrome.sidePanel). The panel renders the shared agent registry as
a picker (Claude selectable; Codex/Cursor shown disabled 'coming soon'), a
current-tab context line, and an instruction composer; Cmd/Ctrl+Enter or the
button POSTs {agent, instruction, page metadata, token} to /launch and shows
the result. The composer is gated on connection (reads storage.local.connected,
live-updates) and the panel refreshes its context as you switch tabs.
The popup now shows an 'Open launcher' entry only when connected, which calls
chrome.sidePanel.open() for the window (window id resolved at load so the call
stays inside the click gesture). Launch helpers moved popup/launch.ts ->
lib/launch-client.ts and gained agent + instruction fields. manifest: adds the
sidePanel permission + side_panel.default_path.
whysosaket
changed the title
The panel now opens as a chat window — agent picker on top, a conversation area, and a message input at the bottom. Sending the first message launches the chosen agent in a terminal (the message becomes its first task; page context still rides along as the system prompt) and drops a confirmation bubble; the conversation then continues in the terminal. For now there's no in-panel streaming, so the composer parks after launch with a 'New' button to start over. Enter sends, Shift+Enter newlines; still connection-gated and context refreshes per active tab. Backend unchanged.
The page context was passed only via --append-system-prompt, which the agent keeps in its (invisible) system prompt — and the TUI clears any pre-launch banner — so the user couldn't tell the context took effect. Split it: the operational guidance (browser_* tools, /browserops skill) stays in the system prompt, while the page facts now lead the VISIBLE first prompt, followed by the user's instruction. The agent shows that as its opening turn, so the page context is plainly visible in the terminal. Launcher files renamed ctx/instr -> sys/prompt; same file-not-argv injection invariant. Tests + shell E2E updated.
Add chat.ts: the agent-agnostic message vocabulary (ChatRole/ChatBlock/
ChatMessage/SessionRef/ChatEvent/ChatPageContext) plus TerminalMode, shared
by the bridge's chat engine and the side-panel client so neither depends on a
specific agent's wire format.
Extend the agent registry with caps {launch, chat} so terminal-launch and
in-panel chat can ship per agent independently; add canChatAgent().
The agent-specific half of the chat engine: map Claude's stream-json events (live) and on-disk JSONL records (history) onto the shared ChatMessage shapes. - normalize.ts: one content/block normalizer shared by both feeds; live event mapper (assistant/user→message, result→idle/error status, init handled by the runtime); drops empty thinking blocks and the thinking signature. - transcript.ts: resolve a session by scanning ~/.claude/projects/*/<id>.jsonl (slug-agnostic, since dotted cwds break the slug rule), whitelist user/assistant turns, drop sidechains + skill/reminder injections, and re-join the per-block assistant records Claude splits a turn into. Fixtures captured from a real claude v2.1.153 stream-json run.
The custom read/send transport behind the AgentDriver seam. Drives Claude as a per-turn stream-json subprocess (validated: first turn --session-id, later --resume continue the same session id + transcript), parses stdout into ChatEvents, fans them to live subscribers, mirrors a human-readable log for the "show in terminal" view, and resolves history by re-reading the transcript. - driver.ts (AgentDriver interface): the pluggable per-agent seam. - claude/bin.ts: login-shell PATH resolution (the daemon PATH is minimal). - claude/runtime.ts: runClaudeTurn — spawn (no shell → argv is injection-inert, stdin ignored to skip claude's 3s stall), NDJSON line-buffer → events. - claude/mirror.ts: pretty mirror-log lines for the read-only terminal tail. - claude/driver.ts: per-session turn queue, replay buffer for the start race, --session-id/--resume selection, stop()=eject (abort + refuse further sends). - paths.ts: chatDir/sessionsDir/workspaceDir under the 0700 state dir. Tests incl. a real-process bash-fake E2E proving a $(...) message stays inert.
ChatEngine is the agent-agnostic orchestrator the HTTP/SSE surface will call. It owns the driver registry (one per agent), the session index, and an in-memory ref cache, and routes every call by session id — resolving live refs from cache or cold from the index after a daemon restart (so a session survives a restart and resumes via --resume). Adding an agent is just registering its driver here. - sessions.ts: SessionIndex — our small manifest of started sessions (history itself stays in the agents' transcripts); atomic, capped, corruption-tolerant. - manager.ts: start/send/subscribe/stop/history/list, EngineError codes. - errors.ts: typed EngineError so the HTTP layer maps codes to status.
…he daemon Expose the in-panel chat surface over the bridge port, gated by the same two gates as /launch (pinned-extension origin + loopback token): - POST /chat/new → engine.start, returns the SessionRef - POST /chat/send → engine.send - POST /chat/terminal → mirror | takeover (stubbed; wired next step) - GET /chat/sessions, /chat/history → list / replay (token in query) - GET /chat/stream → SSE; queues the engine's replayed buffer until headers are written so an unknown session still maps to 404; registered so stop() ends open streams. EngineError codes map to HTTP status (unknown_session->404, ejected->409, ...). daemon-runtime constructs the ChatEngine and adapts it to the ChatService the bridge consumes. ChatService stays an interface so ws-server is engine-agnostic and unit-testable with a fake.
Implement POST /chat/terminal's two modes, reusing the launchAgent opener
machinery (Terminal/iTerm/wt/cmd):
- mirror: open a read-only terminal that tails the session's mirror log
(tail -n +1 -F / Get-Content -Wait); the engine keeps driving.
- takeover: engine.stop() lets go, then open an interactive
claude --resume <id> in the session's cwd — one writer at a time.
Only repo/engine-controlled values (temp script path, session id, workspace
cwd, mirror path) are interpolated, all single-quoted, so the injection-safe
invariant holds. safe_mode is honored on takeover.
Rewire the side panel from fire-and-forget launch into a real bidirectional chat over /chat/*: - first message → POST /chat/new, then open an EventSource on /chat/stream and render assistant/tool/thinking blocks as they arrive; the composer stays live for follow-up turns (POST /chat/send). - header buttons: 'Terminal' (read-only live mirror) and 'Take over' (eject into an interactive terminal; the panel then goes read-only). - a recent-sessions strip lists past sessions (/chat/sessions); clicking one re-reads its transcript (/chat/history) and reconnects the live stream. lib/chat-client.ts holds the pure pieces (reducer, body builder, status copy) so they unit-test in Node; reuses grabPageMeta/clampText from launch-client. The agent picker now keys off caps.chat.
AGENTS.md (loaded as CLAUDE.md): architecture note on the bidirectional chat engine + a Where-to-look row for packages/bridge/src/engine. README: a user-facing 'Chat from the side panel' section (in-panel chat, Terminal mirror, Take over, recent-sessions strip).
Add a chat_transport preference ("terminal" | "headless") to the bridge
config, resolved from BROWSEROPS_CHAT_TRANSPORT then the config file, defaulting
to "terminal". It selects how the in-panel chat engine drives Claude; driver
selection (a later step) degrades terminal->headless when node-pty is missing.
Add the pty_unavailable EngineError code and map it to HTTP 503 in the chat
surface.
Add engine/claude/tail.ts: a polling, byte-offset transcript tailer that emits normalized ChatEvents for the interactive transport's clean chat feed. It advances only on newline boundaries (no partial line / split UTF-8) and tolerates the transcript file being created after the session starts. Turn-end (the transcript feed has no result event) is decided by detectTurnState: a terminal stop_reason (end_turn/stop_sequence) is primary, a quiescence timeout is the universal fallback that also covers interrupts and turns typed directly in the terminal. Add normalize.stopReasonOf and refactor parseTranscript's record loop into a reusable createTranscriptReducer the tailer drives incrementally.
Add engine/claude/pty.ts: a lazily-loaded node-pty wrapper that (1) imports the native module only on demand, so a missing/incompatible binary surfaces as PtyUnavailableError (the engine falls back to headless) rather than crashing the daemon at boot; (2) re-chmods node-pty's spawn-helper +x before the first spawn — pnpm's onlyBuiltDependencies skips the postinstall that would set it, so spawn would otherwise fail with posix_spawnp; and (3) submits turns with a two-step discrete Enter (a single text+CR write is swallowed as a paste by the Ink TUI), bracketed-pasting multi-line input. PtySession adds raw-output fan-out with bounded scrollback replay (so the embedded xterm pane renders the screen when it attaches late) and a quiescence readiness gate. Add node-pty to the bridge deps + the workspace onlyBuiltDependencies list.
Add engine/claude/terminal-driver.ts: a second AgentDriver that spawns ONE long-lived INTERACTIVE claude on a pty (no --print, no skip-permissions; --permission-mode acceptEdits + --allowedTools mcp__browserops in non-safe mode, default-prompting in safe mode). Turns are injected at the tty; the clean chat feed comes from the transcript tailer; the raw TUI is exposed via a new TerminalAttachable capability (attachTerminal) for the embedded xterm pane. Pre-seeds workspace folder-trust in ~/.claude.json (atomic, one-time) so the session doesn't open on a trust prompt that would swallow the seed turn. A node-pty load failure surfaces as pty_unavailable so the engine can fall back to headless. Add TranscriptTailer.markThinking() so turn injection drives the tailer's thinking->idle lifecycle (the tailer can't observe the injection), and make quiescence a long interrupt-only fallback.
Wire driver selection by chat_transport in daemon-runtime: terminal (default) constructs ClaudePtyDriver with the headless ClaudeDriver as an automatic fallback; headless constructs ClaudeDriver alone. ChatEngine now catches pty_unavailable from a primary driver's start() and retries with the registered fallback, remembering the owning driver per session so send/subscribe/history/stop route to whichever driver actually owns it.
Add a token+origin-gated WebSocket route (/chat/pty?token&session_id) that binds
the panel's xterm pane to the live pty: raw pty output is sent to the socket
(scrollback replayed on attach), and JSON {type:input|resize} frames from the pane
are forwarded to the pty. Reuses the existing upgrade path with a custom callback;
sockets are tracked and closed on stop().
ChatEngine.attachTerminal resolves the session's owning driver and returns its
TerminalHandle — unknown_session for cold sessions, pty_unavailable for the
non-terminal headless fallback. ChatService gains attachTerminal, wired in
daemon-runtime.
The Terminal button now toggles a live xterm.js pane inside the side panel, wired to the bridge's /chat/pty WebSocket: it shows the raw interactive Claude TUI and you can type into it — the same pty the chat drives, so a turn typed in the terminal also lands in the chat bubbles. xterm (~330KB) is lazy-imported only when the terminal is opened; Take over still ejects to a real external terminal. Add pure ptyUrl + input/resize frame builders to chat-client (+tests) and a thin pty-terminal module for the xterm<->WS wiring (fit on open/resize).
A live E2E against real claude surfaced two issues in the interactive transport: - Readiness resolved before the TUI mounted, so the seed turn was injected into an unmounted prompt and lost. waitUntilReady now starts its quiescence timer on the FIRST byte of output (with a hard-timeout floor), waiting for the TUI to actually render and settle. - A multi-line turn went through bracketed paste, which Claude's Ink TUI collapses into a 'pasted text' attachment instead of submitting. submitText now flattens to a single line (the proven submit path) and drops bracketed paste. Verified end-to-end: interactive session -> grounded seed reply -> follow-up -> history with both turns.
AGENTS.md: the in-panel chat engine now has two Claude transports (terminal default via ClaudePtyDriver, headless --print fallback), the /chat/pty embedded-xterm WebSocket, and the transcript-tail clean feed. README: the Terminal button is now an embedded in-panel terminal; note the interactive-session safety framing and the chat_transport flag.
…y mirror The takeover eject called chatEngine.stop(), which for the pty transport hard-killed the pty AND the transcript tailer, freezing the panel chat. Add an optional AgentDriver.relinquish() distinct from stop(): the pty driver kills its own pty (one writer at a time) but keeps the tailer running, emitting an 'ejected' status instead of 'end'. The killed pty's onExit is silenced (the death is expected) and send/attachTerminal are refused with session_ejected. ChatEngine.eject() prefers relinquish and falls back to stop() for the headless driver; the takeover path now calls eject(). Adds ChatEngine.terminalCapable()/driver.terminalLive() so the panel can tell whether to offer the embedded-terminal toggle.
…ly-skip-permissions The takeover resume scripts injected --dangerously-skip-permissions in non-safe mode — the exact ban-risk flag the interactive-terminal transport exists to avoid. An ejected session is driven by a human in a terminal, so prompting is fine. Parameterize buildMac/WindowsResumeScript with explicit permissionArgs (via resumePermissionArgs) instead of deriving a skip-permissions flag: acceptEdits + --allowedTools mcp__browserops when not in safe mode, --permission-mode default in safe mode — matching the pty driver's args. NOTE: this also changes the headless transport's takeover (it no longer skip-permissions on eject); unified deliberately so the codebase never ejects with the ban-risk flag. Flagged for review.
…sessions The panel needs to know whether a session has a live pty before offering the embedded-terminal toggle (a headless-fallback, cold post-restart, or already- ejected session has none). Add ChatService.terminalCapable() wired to the engine, include the flag on the /chat/new response and each /chat/sessions item, and thread it into the client ChatState (cleared on the ejected status).
The embedded terminal was a fixed 280px pane shown alongside the chat. Make it a real view toggle: the header button swaps the whole panel between the chat and a full-window live terminal — mutually exclusive, not a cramped pane. The pty session keeps running across the toggle (the chat tailer stays live underneath), so flipping back shows the conversation intact and turns typed in the terminal appear in the chat. The toggle is offered only when the session is terminalCapable (live pty); render() falls back to chat if the session loses its pty (eject, disconnect, cold/new session). CSS drives the swap via a .view-terminal class; xterm stays a lazy chunk.
After '⤢ Take over', keep the SSE stream open instead of closing it: the daemon now keeps tailing the transcript (relinquish), so the chat becomes a LIVE read-only mirror of the turns typed in the external terminal. Only input is locked — the composer is disabled (ejected) and the embedded-terminal view is closed/hidden (terminalCapable cleared). Also reconcile the optimistic user echo with the pty tailer's transcript user record so a composer turn shows once (was a latent double-bubble): a real user record adopts the oldest pending optimistic id but keeps the user's original text; a user record with no pending echo (terminal-typed, or streamed in after eject) just appends.
Document the embedded terminal as a full-window view toggle (chat xor terminal) and the external eject as a live read-only mirror that resumes interactively (never --dangerously-skip-permissions), in AGENTS.md and the README side-panel section.
feat: interactive-terminal chat transport (Claude on a pty) + embedded xterm, headless fallback
feat: bidirectional in-panel chat engine (Claude), pluggable + terminal-mirrored
Step: shared chat contract Substeps: - add per-session chat capabilities and migration metadata - mark Codex and Cursor as chat-capable but not launch-capable - export the new shared capability type
Step: bridge chat engine and backend Substeps: - route driver construction through a factory with Claude fallback support - add generic headless process driver helpers for Codex and Cursor - persist BrowserOps normalized transcripts for migration and fallback history - expose per-session capabilities and a migration endpoint
Step: side-panel integration Substeps: - read per-session capabilities from chat responses - gate terminal and takeover controls independently - add a client helper for future chat migration UI
Contributor
|
Closing for now; branch retained. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Launch a coding agent in a terminal from the extension — now via a side-panel launcher (chrome.sidePanel), not a single button.
The side panel shows an agent picker (Claude selectable; Codex/Cursor visible but disabled, "coming soon"), the current tab as context, and an instruction composer. Hitting Launch (or ⌘/Ctrl+Enter) opens the chosen agent in a new terminal — seeded with the page metadata as
--append-system-promptcontext and your typed instruction as the first task. The composer is gated on connection (only usable when the bridge is connected), and the popup only offers "Open launcher" when connected.Why it's shaped this way
The daemon runs detached with
stdio: "ignore"and has no TTY, so it can'tstdio: "inherit"likebrowserops --claude. It writes a launcher script + a context file + an instruction file under the0700state dir and asks the OS to open a terminal that runs the script.Transport:
POST /launch(next to/pair//status), token-authed + origin-pinned. The side panel already holds the loopback token and the repo already does token'dfetchto the bridge, so this is idiomatic — and HTTP gives the panel a real success/error to render. No changes to the WS protocol or background worker.Agent-agnostic by construction
packages/shared/src/agents.ts— oneAGENTSregistry both sides consume (UI picker + bridge validation). Claudeenabled; Codex/Cursorenabled: false.launch-agent.ts(waslaunch-claude.ts) —launchAgent({agent, instruction, page})validates viaisLaunchableAgent(disabled/unknown →agent_disabled) and dispatches through a per-agentAGENT_LAUNCHERSspec. Adding Codex/Cursor later is a registry flag + one spec entry.Platforms
launch-agent.tsswitches onprocess.platform: macOS bash.commandviaopen -a iTerm/Terminal; Windows PowerShell.ps1viawt.exeorcmd /c start powershell. Linux returnsunsupported_platform(the remaining seam).Security
Two independent gates on
/launch: the Origin must match the pinned extension (a web page's real Origin won't) and the body must carry the loopback token (only the pinned extension receives it via/pair). The launcher gesture is the user-consent event — deliberately not routed through the risky-action approval layer.No shell injection: page metadata and the typed instruction only ever reach the shell as the content of a file read into a variable, never interpolated into the script. A
<title>or instruction of$(rm -rf ~)is inert.Testing
agent_disabled, instruction file, injection-safety), 14/launchendpoint tests (incl. agent + instruction forwarding, auth/origin gates), and a real-shell E2E that runs the generated.commandagainst a fakeclaudeand asserts it execs with[--dangerously-skip-permissions, --append-system-prompt, <ctx>, <instruction>](no word-split), that$(...)values stay inert, and that the script self-deletes. Extension: 133 tests (launch-client helpers). Fullpnpm build+typecheck+testgreen.codex/ghost→502 agent_disabled(no terminal opens), wrong token →401, non-pinned origin →403.Manual E2E left for a human (spawns a real interactive session)
pnpm build && node packages/cli/dist/index.js restart; load-unpackpackages/extension/dist.Follow-ups
open -a iTerm <file>.commandruns vs opens on an iTerm box.