feat(browser-use): add Camoufox noVNC session viewer#921
Open
viper151 wants to merge 78 commits into
Open
Conversation
…wser-use-independent
# Conflicts: # src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx
# Conflicts: # server/index.js
- Rename Browser Use surfaces to Browser - Register Browser MCP under the new server name - Mark CloudCLI-managed MCP servers read-only - Adjust MCP stdio framing and sidebar footer sizing
# Conflicts: # src/i18n/locales/en/settings.json
# Conflicts: # package-lock.json # package.json # server/index.js # src/components/main-content/types/types.ts # src/components/main-content/view/MainContent.tsx # src/components/main-content/view/subcomponents/MainContentHeader.tsx # src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx # src/components/main-content/view/subcomponents/MainContentTitle.tsx # src/components/settings/hooks/useSettingsController.ts # src/components/settings/types/types.ts # src/components/settings/view/Settings.tsx # src/components/settings/view/SettingsSidebar.tsx # src/hooks/useProjectsState.ts # src/i18n/locales/de/common.json # src/i18n/locales/en/common.json # src/i18n/locales/en/settings.json # src/i18n/locales/it/common.json # src/i18n/locales/ja/common.json # src/i18n/locales/ko/common.json # src/i18n/locales/ru/common.json # src/i18n/locales/tr/common.json # src/i18n/locales/zh-CN/common.json # src/i18n/locales/zh-TW/common.json # src/types/app.ts
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds Camoufox VNC browser-use runtime support, viewer proxy/auth wiring, backend and profile settings, and frontend updates for viewer access, engine display, and loading states. ChangesCamoufox VNC Backend and Viewer
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@server/browser-use-mcp.ts`:
- Line 72: The Browser session tool description in the MCP schema is inaccurate
because it implies the configured persistent profile is used by default in all
cases, while resolveSessionProfileName() only falls back to defaultProfileName
when persistSessions is enabled. Update the description on the Browser session
tool definition to match the actual behavior: mention that profileName can
override the profile, and that the persistent profile is only used by default
when session persistence is enabled; otherwise a temporary session is created.
In `@server/index.js`:
- Around line 231-243: The authenticateBrowserUse middleware currently falls
back to authenticateToken(), which lets any logged-in user access viewer routes
without proving ownership of the :sessionId. Update authenticateBrowserUse and
the browserUseService validation path so requests under
/sessions/:sessionId/viewer require a valid viewer token or an ownership-aware
authorization check tied to that session. Make sure the logic in
authenticateBrowserUse and browserUseService.validateViewerToken enforces
per-session access before calling next(), rather than relying on generic
authentication.
In `@server/modules/browser-use/browser-use.service.ts`:
- Around line 258-314: In startVisibleRuntime, readiness is being inferred from
fixed delays after spawnRuntimeProcess, so the visible runtime can be marked
ready even if Xvfb, x11vnc, or websockify never started or exited early. Replace
the sleep-only flow with explicit health checks in the startup path: verify each
spawned process is still alive and confirm the VNC/websockify ports are actually
bound before returning the viewer handle. Use the existing startVisibleRuntime
and spawnRuntimeProcess flow to gate the ready state only after those checks
pass.
In `@server/modules/browser-use/browser-use.settings.ts`:
- Around line 41-55: The profile-name handling in resolveSessionProfileName and
getProfilePath can alias different logical names to the same disk directory
because of case folding and character rewriting. Update the
BrowserUseSettings/profile resolution flow so profileName is canonicalized
before it is stored or echoed back, and add validation to reject any requested
name that would map to an existing normalized on-disk name. Keep the fix
centered around resolveSessionProfileName,
normalizeProfileName/normalizeDefaultProfileName, and getProfilePath so distinct
UI profiles cannot share the same saved session state.
In `@server/modules/browser-use/browser-use.viewer.ts`:
- Around line 8-11: The VIEWER_TOKEN_TTL_MS constant can become NaN or a
non-positive value from the CLOUDCLI_BROWSER_USE_VIEWER_TOKEN_TTL_MS env var, so
clamp it to a finite positive number when initializing browser-use.viewer.ts.
Update the VIEWER_TOKEN_TTL_MS parsing logic to validate the parsed value and
fall back to the default 30 * 60 * 1000 whenever the env value is invalid, zero,
negative, or non-finite, so the token expiry path used by expiresAt and cookie
maxAge remains safe.
In `@server/modules/websocket/services/websocket-server.service.ts`:
- Around line 44-56: The websocket auth path in websocket-server.service.ts is
incorrectly falling back to verifyWebSocketClient() for viewer sockets, which
lets generic app auth accept connections without proving session ownership.
Update the websocket upgrade handling around the viewer route check in the
websocket server service so that
/api/browser-use/sessions/:sessionId/viewer/websockify only succeeds when
getBrowserUseViewerToken() is valid and authenticateBrowserUseViewer() passes,
or otherwise perform an explicit per-session ownership check before returning
true. Keep the generic verifyWebSocketClient() fallback only for non-viewer
websocket paths.
In `@src/components/browser-use/view/BrowserUsePanel.tsx`:
- Around line 274-281: The cache update in BrowserUsePanel’s selection effect
should not bump freshness metadata when only selectedSessionId changes. Update
the useEffect that reads and writes browserUsePanelCache so it only stores the
new selectedSessionId on the existing entry and leaves updatedAt unchanged, and
verify getFreshCacheEntry still relies on the original fetch time rather than
selection-only updates.
In
`@src/components/settings/view/tabs/browser-use-settings/BrowserUseSettingsTab.tsx`:
- Around line 125-129: The settings load failure is being treated as if the
feature were disabled because BrowserUseSettingsTab derives browserEnabled
directly from settings being null; keep a separate “settings loaded” flag in
BrowserUseSettingsTab and use it to distinguish unknown state from an actual
disabled setting. Update the rendering around the browserEnabled-dependent UI so
the error banner from loadSettings() is shown independently of browserEnabled,
and make sure the disabled-state UI is only based on loaded settings, not a
failed load, in the affected browser toggle and related sections.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: e8a72ac2-f54b-4413-85c5-c57a91b49ac5
📒 Files selected for processing (13)
server/browser-use-mcp.tsserver/index.jsserver/modules/browser-use/browser-use.routes.tsserver/modules/browser-use/browser-use.service.tsserver/modules/browser-use/browser-use.settings.tsserver/modules/browser-use/browser-use.types.tsserver/modules/browser-use/browser-use.viewer.tsserver/modules/browser-use/index.tsserver/modules/websocket/services/websocket-server.service.tssrc/components/browser-use/view/BrowserUsePanel.tsxsrc/components/main-content/view/MainContent.tsxsrc/components/main-content/view/subcomponents/MainContentTitle.tsxsrc/components/settings/view/tabs/browser-use-settings/BrowserUseSettingsTab.tsx
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
server/modules/browser-use/tests/browser-use.settings.test.ts (1)
31-57: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoffTests mutate the real user home directory.
getProfilePath/PROFILE_ROOTresolve underos.homedir()/.cloudcli/browser-use/profiles, so this test creates (and thefinallyonly partially removes) directories in the developer's actual home directory. This couples the test to machine state and risks clobbering a real profile if a name ever collides. Consider redirectingPROFILE_ROOTto an isolated temp dir (e.g. viafs.mkdtempSync(os.tmpdir())and an env/override) for the duration of the test.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@server/modules/browser-use/tests/browser-use.settings.test.ts` around lines 31 - 57, The browser profile alias test is writing into the real home-directory-backed profile store, so isolate it from machine state. Update the test setup around resolveSessionProfileName/getProfilePath to point PROFILE_ROOT at a temporary directory for the duration of the test, rather than using os.homedir()-based paths. Ensure the temporary override is restored and the temp directory is cleaned up in teardown so the test no longer creates or removes real user profiles.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@server/modules/browser-use/browser-use.settings.ts`:
- Around line 31-37: The profile name normalization in browser-use.settings.ts
runs the dash-stripping regexes on an unbounded user-controlled string, which
can trigger the CodeQL ReDoS warning. Update the normalization flow in the
profile name handling logic to apply the length cap before the `/^-+|-+$/g`
replacements, so the regex work is limited to MAX_PROFILE_NAME_LENGTH; keep the
existing normalization behavior in the same sequence around the normalized
variable.
---
Nitpick comments:
In `@server/modules/browser-use/tests/browser-use.settings.test.ts`:
- Around line 31-57: The browser profile alias test is writing into the real
home-directory-backed profile store, so isolate it from machine state. Update
the test setup around resolveSessionProfileName/getProfilePath to point
PROFILE_ROOT at a temporary directory for the duration of the test, rather than
using os.homedir()-based paths. Ensure the temporary override is restored and
the temp directory is cleaned up in teardown so the test no longer creates or
removes real user profiles.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 7c91751c-0ebe-4ebc-8823-208bbd4cffa3
📒 Files selected for processing (9)
server/browser-use-mcp.tsserver/index.jsserver/modules/browser-use/browser-use.service.tsserver/modules/browser-use/browser-use.settings.tsserver/modules/browser-use/browser-use.viewer.tsserver/modules/browser-use/tests/browser-use.settings.test.tsserver/modules/websocket/services/websocket-server.service.tssrc/components/browser-use/view/BrowserUsePanel.tsxsrc/components/settings/view/tabs/browser-use-settings/BrowserUseSettingsTab.tsx
💤 Files with no reviewable changes (1)
- src/components/browser-use/view/BrowserUsePanel.tsx
✅ Files skipped from review due to trivial changes (1)
- server/browser-use-mcp.ts
🚧 Files skipped from review as they are similar to previous changes (5)
- server/modules/browser-use/browser-use.viewer.ts
- server/modules/websocket/services/websocket-server.service.ts
- server/index.js
- src/components/settings/view/tabs/browser-use-settings/BrowserUseSettingsTab.tsx
- server/modules/browser-use/browser-use.service.ts
…ariable' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
# Conflicts: # server/index.js
…r-use # Conflicts: # .github/workflows/release.yml # electron/desktopWindow.js # electron/launcher/launcher.js # electron/main.js # electron/preload.cjs # package.json # scripts/release/prepare-desktop-app.js # server/modules/websocket/services/websocket-server.service.ts # src/components/main-content/view/MainContent.tsx # src/components/main-content/view/subcomponents/MainContentTitle.tsx
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
|
||
| const shouldShowTasksTab = Boolean(tasksEnabled && isTaskMasterInstalled); | ||
| const shouldShowBrowserTab = browserUseEnabled; | ||
| const shouldShowComputerTab = COMPUTER_USE_MENUS_ENABLED && computerUseEnabled === true; |
| const normalizeMainTab = (tab: string): SettingsMainTab => { | ||
| // Keep backwards compatibility with older callers that still pass "tools". | ||
| if (tab === 'tools') { | ||
| if (tab === 'tools' || (tab === 'computer' && !COMPUTER_USE_MENUS_ENABLED)) { |
| ]; | ||
|
|
||
| const VISIBLE_NAV_ITEMS = NAV_ITEMS.filter((item) => ( | ||
| COMPUTER_USE_MENUS_ENABLED || item.id !== 'computer' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a visible Browser session mode for agent-driven browsing. The existing Playwright backend works well for fast automated page interaction, screenshots, and frontend checks, but it is limited when a flow requires human involvement, such as logging in, completing MFA, approving a consent screen, or watching exactly what the agent is doing in real time.
To support those workflows, Browser can now run sessions with Camoufox and expose them through noVNC. Users can choose between the existing Playwright backend and the new visible Camoufox + noVNC backend from Browser settings. Playwright remains the lightweight option for quick automated checks, while Camoufox + noVNC is the better fit when a person may need to observe or take control of the live browser session.
The backend manages the visible runtime, proxies authenticated noVNC traffic through the app, and attaches viewer metadata to Browser sessions. The UI now surfaces the selected engine, profile information, runtime readiness, live screenshots, and take-control links from the Browser panel.
Changes
playwrightandcamoufox-vncTesting Steps
Docs
The Browser Use docs are already published and cover the new visible-session flow:
https://cloudcli.ai/docs/features/browser-use
The page includes Browser engine selection, Playwright, Camoufox + noVNC, saved logins/profiles, runtime requirements, and troubleshooting, so this PR links to the existing docs rather than adding separate setup instructions in the PR body.
Summary by CodeRabbit