Chat terminal: detect sensitive prompts and surface focus-terminal UI - #314826
Merged
Conversation
…#314796) When a command run via the chat terminal tool produces output that looks like a password / passphrase / secret prompt (e.g. 'Password:', '[sudo] password for user:', 'Passphrase for /home/me/.ssh/id_rsa:', 'Enter PIN:', 'Verification code:'), we never want the model to answer — we route the secret entry to the user directly. The agent system prompt instructs the model not to call ask_questions for secrets, but until now there was no explicit detection or UI affordance, and screen-reader users got no announcement that input was required. Adds: - detectsSensitiveInputPrompt regex covering the common patterns above. - A new onDidDetectSensitiveInputNeeded signal on IOutputMonitor, fired from the same idle-detection path as onDidDetectInputNeeded but routed separately so the regular 'looks like a prompt — answer it' UX is bypassed for secrets. - runInTerminalTool listens for the sensitive signal and shows a chat elicitation that announces the prompt for ARIA, offers a primary 'Focus terminal' action, and a 'Cancel' secondary action. The secret itself is never round-tripped through the model. Tests cover the new regex (positive + negative cases) and verify that a sensitive prompt fires onDidDetectSensitiveInputNeeded but NOT onDidDetectInputNeeded.
meganrogge
added
accessibility
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the run_in_terminal chat tool’s terminal monitoring so that prompts likely requesting secrets (passwords, tokens, OTPs, etc.) are handled via a dedicated “sensitive input needed” signal and UI flow, rather than being routed back through the agent/model.
Changes:
- Added
onDidDetectSensitiveInputNeededtoOutputMonitorand wired detection so sensitive prompts fire this event instead ofonDidDetectInputNeeded. - Added a sensitive-input elicitation flow in
RunInTerminalToolthat surfaces UI to focus the terminal or cancel the command (foreground + background monitoring). - Updated/added unit tests and adjusted test stubs to account for the new event.
Show a summary per file
| File | Description |
|---|---|
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/electron-browser/runInTerminalTool.test.ts | Updates test stubs to include the new sensitive-input event. |
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/outputMonitor.test.ts | Adds coverage for sensitive prompt detection and event routing behavior. |
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/runInTerminalTool.ts | Adds UI elicitation that focuses terminal / cancels execution when sensitive prompts are detected. |
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/monitoring/outputMonitor.ts | Introduces sensitive prompt detection + new event, and ensures it replaces the generic input-needed signal for secrets. |
Copilot's findings
- Files reviewed: 4/4 changed files
- Comments generated: 3
added 4 commits
May 6, 2026 16:15
Previously when the OutputMonitor detected a password/passphrase/secret prompt in foreground mode, it fired onDidDetectSensitiveInputNeeded and showed an elicitation dialog. The dialog's lifetime was tied to the race cleanup, which auto-hid it when the race resolved, AND the race itself never resolved on the sensitive event. The result: the agent's tool call sat indefinitely waiting for the user to focus the terminal and type their secret, with no way to surrender its turn. Changes: - Add a 'sensitiveInputNeeded' race candidate in the foreground path so the tool returns immediately when a sensitive prompt is detected. - Track didSensitiveInputNeeded separately and emit a dedicated steering note that tells the model the user has been shown a focus-terminal dialog and to stop / end its turn (no send_to_terminal, no ask_questions, no guessing the secret). - Stop registering a part.hide() disposable in the elicitation store so the dialog persists after the tool call returns and the user can still interact with it.
…e prompts Previously, when the OutputMonitor detected a sensitive prompt the tool would immediately resolve the race as 'sensitiveInputNeeded' and hand control back to the agent. That worked but gave the user no opportunity to actually engage with the terminal — the tool always returned to the model even when the user was about to focus the terminal and type their secret. Instead, treat the sensitive prompt as a hint, not a verdict: - On detection, start a 45s grace timer and listen for terminal data. Every onData event (the user typing, shell echo) resets the timer. - If executionPromise resolves during the grace window — i.e. the user successfully entered the secret and the command finished — the outer Promise.race takes the 'completed' branch and we never resolve the sensitive promise. - Only if the grace expires with no engagement do we resolve as 'sensitiveInputNeeded' so the agent gets control back rather than hanging forever. The Cancel button in the elicitation still cancels execution immediately, which causes executionPromise to reject and the outer race takes the 'completed' (cancelled) branch.
Drop the 45s grace + sensitiveInputNeeded race branch added in the prior commit. The dialog already gives the user two unambiguous options: - Focus Terminal: focuses the terminal so the user types the secret; when the command finishes, executionPromise wins the race and we report 'completed' as normal. - Cancel Command: cancels execution, which causes executionPromise to resolve and the race takes the 'completed' (cancelled) branch. Either path resolves the race naturally — no need for a separate timer or fallback. This matches the user's expectation: while a sensitive prompt is up, the tool should wait for the user to answer, not start its own clock and surrender to the agent.
…arker outputMonitor is created lazily inside execution.strategy.onDidCreateStartMarker, which fires after the foreground race is set up. The previous synchronous 'if (outputMonitor)' check ran before the start marker had fired, so outputMonitor was undefined and the elicitation listener was never attached — meaning sudo / password / passphrase prompts produced no dialog. Wrap the registration in startMarkerPromise.then(...) the same way the existing onDidDetectInputNeeded listener does.
added 6 commits
May 6, 2026 16:33
In auto-approve / autopilot mode there is no human in the loop to type a secret into the focused terminal, so the prior 'wait for the user' behavior would just hang the command forever. Detect the auto-approve session level inside _registerSensitiveInputElicitation and: - Cancel the running command immediately. - Surface a one-shot informational chat part explaining what happened. - Set didSensitiveAutoCancelled so the tool result text includes a steering note telling the model NOT to retry the command, NOT to call send_to_terminal, and NOT to ask_questions for the secret \u2014 instead to ask the user to run the command interactively. Non-autopilot sessions still get the original Focus Terminal / Cancel Command dialog. Then update the PR description to remove the open question.
…uto-cancelling Previously in auto-approve / autopilot we replaced the dialog with a dismiss-only info part. That removed the Focus Terminal button, so a user who happened to be watching couldn't grab the terminal and finish the command manually. Show the same dialog as the default branch in autopilot (Focus Terminal / Cancel Command), but with a message that mentions the command was auto-cancelled. Set autoCancelled / onAutoCancelled and cancelExecution() right after attaching the part so the agent's tool call still returns immediately with the steering note. The dialog persists so the user can click Focus Terminal to take over after the fact (the running command itself is gone, but they can still type into the terminal as usual).
…t user is unavailable After the autopilot tool call returns, the terminal can't reliably be focused (it may be cleaned up), so a Focus Terminal button there is misleading — clicking it does nothing visible. Replace the Focus/Cancel dialog with a one-shot info part (Dismiss only) and update the steering note in the tool result to explicitly tell the agent the user is unavailable, the command has been cancelled, and to ask the user to run the command interactively when they are available. Default / Ask sessions are unchanged: they still get the Focus Terminal / Cancel Command dialog, since in those sessions the terminal stays alive after the tool call returns and Focus Terminal actually works.
…aces cancel In autopilot there is no human to act on a chat elicitation, and the agent's own reply (driven by the steering note in the tool result) already tells the user the command was cancelled and to run it interactively. Skip the redundant info part and just cancel + steer.
Reverts part of the previous commit. Without any UI affordance, when autopilot cancels a sensitive prompt the user just sees the command stop with no explanation — looks like a bug. Bring back the dismiss-only info chat part so the user always sees what happened, even if the agent doesn't follow up with a message.
…test stub types - Wrap the Focus Terminal accept handler's reveal/focus sequence in a try/catch so a rejection from revealTerminal can't leave the elicitation in a bad state. Still returns Accepted. - Update the runInTerminalTool.test.ts outputMonitor stub type casts to include onDidDetectSensitiveInputNeeded so future refactors that drop the field fail at compile time, not runtime.
Collaborator
Author
|
/requires-eval-assessment terminalbench2 gpt-5.4,claude-opus-4.6,claude-opus-4.7 |
meganrogge
added
the
~requires-eval-assessment
meganrogge
enabled auto-merge (squash)
May 6, 2026 20:53
Contributor
|
⏳ Queued vscode build for
|
dmitrivMS
approved these changes
May 6, 2026
Contributor
|
🚀 Queued eval-assessment publish build for
|
Contributor
|
✅ Eval-assessment build published.
|
vs-code-engineering
Bot
removed
the
~requires-eval-assessment
Contributor
|
🔬 Queued eval-assessment benchmark for
Results will be posted back here when the run completes. |
Contributor
|
📊 Eval-assessment benchmark complete.
Analysis ResultsResolution Rate
|
Contributor
|
📊 Eval-assessment benchmark complete.
Analysis ResultsResolution Rate
|
Contributor
|
📊 Eval-assessment benchmark complete.
Analysis ResultsResolution Rate
|
Contributor
|
📊 Eval-assessment benchmark complete.
Analysis ResultsResolution Rate
|
Contributor
|
📊 Eval-assessment benchmark complete.
Analysis ResultsResolution Rate
|
meganrogge
added
feature-request
Closed
|
this is hot garbage, why would you implement this with no option to the user to override it? Now I have to run every sudo command manually even on my local only raspberry pi? for real? |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Default:
default.mov
Autopilot:
autopilot.mov
Fixes #314796.
When a command run via the chat terminal tool produces output that looks like a password / passphrase / secret prompt (e.g.
Password:,[sudo] password for user:,Passphrase for /home/me/.ssh/id_rsa:,Enter PIN:,Verification code:), we never want the model to answer — the secret has to come from the user directly. The agent system prompt already instructs the model not to callask_questionsfor secrets, but until now there was no explicit detection or UI affordance, and screen-reader users got no announcement that input was required.Changes
detectsSensitiveInputPromptregex covering the common patterns above.onDidDetectSensitiveInputNeededsignal onIOutputMonitor, fired from the same idle-detection path asonDidDetectInputNeededbut routed separately so the regular "looks like a prompt — answer it" UX is bypassed for secrets.runInTerminalTool(foreground + background paths) listens for the sensitive signal. Behavior depends on the session permission level:Default / Ask sessions (human in the loop)
A chat elicitation appears with:
The sensitive signal does not resolve the foreground race candidate. While the dialog is up the tool keeps waiting:
executionPromisewins → result is reported ascompletedwith normal output.completed(cancelled) path.The secret never round-trips through the model.
Auto-approve / Autopilot sessions (no human)
There is no human in the loop and the terminal can be cleaned up after the tool returns, so a Focus Terminal button would be misleading. Instead the tool simply:
send_to_terminal, do not callvscode_askQuestionsfor the secret.The agent