macOS: cross-process window capture always fails (NSApp windowWithWindowNumber: only sees own-process windows -> 0x0 SCStreamConfiguration -> start_capture panic)

[benjamin-small]

Summary

Capturing any cross-process window on macOS fails at start_capture with "Failed due to an invalid parameter" (and panics, since the failure is .expect()ed). Only display capture and same-process windows work. Root cause diagnosed below; verified on 0.0.8 and present verbatim on main.

Symptom

thread 'main' panicked at scap-0.0.8/src/capturer/engine/mod.rs:93:38:
Failed to start capture: "Failed due to an invalid parameter"

Reproduces with any Target::Window belonging to another application (Chrome, etc.); target enumeration succeeds, the failure is at stream start.

Root cause

Both get_target_dimensions and get_scale_factor in src/targets/mac/mod.rs resolve windows via AppKit:

let ns_app: id = NSApp();
let ns_window: id = msg_send![ns_app, windowWithWindowNumber: cg_win_id as NSUInteger];

NSApp().windowWithWindowNumber: can only return windows owned by the calling process. For any other app's window it returns nil; messaging nil yields zeroed scalars, so:

• get_target_dimensions → (0, 0)
• get_scale_factor → 0.0

get_output_frame_size then computes 0 * 0, the SCStreamConfiguration gets width: 0, height: 0, and SCStream::start_capture rejects it with "invalid parameter" — which Engine::start converts into a panic via .expect(). Cross-process window capture can never have worked through this path.

Suggested fix

ScreenCaptureKit already knows every shareable window's size — the same SCShareableContent listing used by get_all_targets:

• get_target_dimensions: look up the window by window_id in SCShareableContent::current().windows and use its width/height (cross-process by design, same permission gate as capture itself).
• get_scale_factor: the AppKit lookup has the same nil problem; the containing display's backing scale (or main-display scale as an approximation) works cross-process.
• Consider returning Result from start_capture instead of .expect() so embedding applications can handle a failed start.

Working implementation

We hit this in production and published a patched fork while this is open: https://github.com/benjamin-small/scap-vc (patch in the initial commits; verified by capturing a live Chrome window frame, 3232x1988 BGRA). Happy to turn it into a PR against main if useful.

[tembo]

Great analysis - your diagnosis is spot-on. This is a known issue that has been reported before:

• #116 - same "Failed due to an invalid parameter" error when capturing windows
• #173 - null pointer reference in get_target_dimensions on macOS window capture

There's already an open PR that fixes this: #174. It takes essentially the same approach you've outlined - replacing the NSApp().windowWithWindowNumber: calls with ScreenCaptureKit lookups via sc::ShareableContent::current().windows(). The PR also adds fallback values (scale = 1.0, dimensions = 800x600) to prevent panics when window resolution fails.

The fix in PR #174 modifies src/targets/mac/mod.rs:87-117 for get_scale_factor and lines 133-156 for get_target_dimensions, switching both from the AppKit path to SCK queries.

Since you have a working fork at benjamin-small/scap-vc, it might be worth comparing your implementation with PR #174 - if your approach handles any edge cases differently (multi-display setups, display scale computation, etc.), that context could help move #174 forward or inform a combined solution.

[benjamin-small]

What's the hold up? that PR looks like it's just waiting to get merged?