Skip to content

feat: let the user choose a screen or window in the macOS system picker - #592

Merged
devopvoid merged 3 commits into
mainfrom
feat/macos-system-picker
Nov 13, 2026
Merged

devopvoid merged 3 commits into
mainfrom
feat/macos-system-picker

Conversation

@devopvoid

@devopvoid devopvoid commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Follows up on #344, which suggested allow_sck_system_picker so that macOS's own picker chooses the screen or window, as the Wayland portal does.

Builds on the ScreenCaptureKit window and screen capture of #591.

The system picker, opt-in

VideoDesktopSource source = new VideoDesktopSource();

if (VideoDesktopSource.isSystemPickerAvailable()) {
    source.setUseSystemPicker(true);       // instead of setSourceId(...)
}

VideoTrack track = factory.createVideoTrack("screen", source);
track.addTrackEndedListener(t -> { /* cancelled, or sharing stopped */ });

source.start();                            // shows the picker, returns at once
  • macOS 14 and newer: ScreenCaptureKit's SCContentSharingPicker, offering a screen or one or more windows. WebRTC fixes these modes, SingleDisplay | MultipleWindows. VideoTrackDesktopSource::capture() sets allow_sck_system_picker and takes the capturer from DesktopCapturer::CreateGenericCapturer, as the Wayland path already does. ScreenCaptureKit draws the cursor (prefer_cursor_embedded), since DesktopAndCursorComposer would misplace it on a picked window.
  • Wayland: the portal picker, with or without the setting, as before.
  • Elsewhere (Windows, X11, macOS before 14): isSystemPickerAvailable() is false, and a source set to use the picker ends right away. The fallback is not the source ID: its default, -1, would share the whole desktop.

It is opt-in, and the default is unchanged. Apps with their own source list would otherwise see a second chooser and have setSourceId ignored, and the picker needs a person to answer it.

Limits from WebRTC: show the picker for one source at a time, since every source waiting on it receives the user's choice. The app cannot reopen the picker on a running source; stop() and start() show a new one.

The picker on ScreenCapturer and WindowCapturer

The same option is available on DesktopCapturer, for apps that take frames themselves and don't use a track:

ScreenCapturer capturer = new ScreenCapturer();

if (DesktopCapturer.isSystemPickerAvailable()) {
    capturer.setUseSystemPicker(true);     // before start(), instead of selectSource(...)
}

capturer.start(callback);                  // shows the picker, returns at once
// then call captureFrame() periodically
  • Until the user has chosen, captureFrame() reports ERROR_TEMPORARY. A cancelled picker or stopped sharing reports ERROR_PERMANENT. Where isSystemPickerAvailable() is false, every frame reports ERROR_PERMANENT instead of capturing something the user did not choose.
  • Calling setUseSystemPicker() after start() throws IllegalStateException, as a second start() does.
  • Changing the setting rebuilds the native capturer in DesktopCapturer::createCapturer(). The maximum frame rate is kept, but a selected source is forgotten.
  • The picker works the same for a ScreenCapturer and a WindowCapturer, since it offers both screens and windows. A picked window's frames have no window border to crop, so isWindowCapturer() is false with the picker.
  • VideoTrackDesktopSource::isSystemPickerAvailable() now asks jni::DesktopCapturer, so both APIs give the same answer.
  • The default is unchanged, and the existing selectSource() path works as before.

DesktopCapturerPickerExample is a small Swing app that shows the picked content live, and the screen and window capturer docs describe the option.

The end of capture

Cancelling the picker, stopping sharing from the menu bar (macOS 14.4+), and a picker that fails to open all reach VideoTrackDesktopSource as ERROR_PERMANENT. So far that only stopped the capture loop, and the track stayed LIVE. Now:

  • A permanent error ends the source, and its tracks call their ended listeners. This also applies to a captured window being closed, so it is a behaviour change beyond the picker.
  • The listeners run on the capture thread, as they already did for a missing backend. Since stopping or disposing the source is the natural reaction there:
    • The capture thread holds a reference to the source until it ends.
    • stop() called on that thread no longer joins the thread from itself, which would hit RTC_CHECK on POSIX (Native Crash related to Window/Video Capture #186). The next stop(), or the destructor, joins it.
    • If the last reference is dropped on the capture thread, a short-lived thread joins it.
  • dispose() ends the capture without waiting for the thread, consistent with the non-blocking dispose in JNI_VideoDesktopSource. Before, a track holding a disposed source kept receiving frames.

Testing

macOS 14.5, Apple silicon:

  • mvn -pl webrtc test: 1455 tests, 0 failures, 49 skipped (platform and hardware assumptions, plus endsWithoutPicker on a Mac with a picker).
  • -Pjni-check on VideoDesktopSourceTests and DesktopCapturerTests: no FATAL ERROR.

New tests:

  • systemPickerAvailability: true on macOS 14+ and on Wayland, false otherwise.
  • endsWithoutPicker: the source ends at once where no picker exists. It runs on Windows and X11 and skips on macOS 14+.
  • closedWindowEndsTrack (was closedWindowStopsCapture): the track ends, and the frames stop.
  • endedListenerMayStopAndDispose: the ended listener calls stop() and dispose() on the capture thread, without a crash.
  • disposeEndsCapture: no frames reach a track once its source is disposed.
  • DesktopCapturerTests.systemPickerAvailability: matches VideoDesktopSource.isSystemPickerAvailable().
  • pickerAfterStartIsRejected: setUseSystemPicker() after start() throws.
  • noPickerReportsPermanentError: where no picker exists, the first frame reports ERROR_PERMANENT. It skips on macOS 14+.
  • pickerCanBeTurnedOff: turning the picker on and off again gives a capturer that lists the same screens.

After the DesktopCapturer change, DesktopCapturerTests and VideoDesktopSourceTests ran again: 22 tests, 0 failures, 4 skipped (systems without a picker or without a backend).

Verified by hand: the picker appears. It was presented from the capture thread in a plain JVM without AWT, and a screenshot showed macOS's "share this screen" overlay and the menu bar sharing indicator. stop() while the picker is open destroys the capturer cleanly, and the track stays LIVE.

Verified by hand with DesktopCapturerPickerExample (Swing, ScreenCapturer, start() and captureFrame() on one executor thread): choosing a screen or windows delivers frames, and cancelling the picker and stopping sharing report ERROR_PERMANENT.

Still to check by hand, with DesktopPickerExample. This is why the PR is a draft. The picker needs a click, so it stays out of the automated tests.

  • Choosing a screen delivers frames of its size.
  • Choosing one window, then two, delivers frames.
  • Changing the choice from the menu bar continues with the new content.
  • Cancelling the picker ends the track.
  • Stopping sharing from the menu bar ends the track.
  • The same with -XstartOnFirstThread, where the main thread runs no event loop: the picker appears, a screen (3840x2160) and a window (1920x997) deliver frames at 15 fps, and cancelling the picker or stopping sharing ends the track. Changing the choice from the menu bar was not repeated with this flag.
  • The same in a Swing app.

Not tested: macOS 15 and newer, and whether the picker spares an app the Screen Recording permission.

VideoDesktopSource.setUseSystemPicker(true) makes start() show the
operating system's own picker, instead of capturing the source set
with setSourceId(). On macOS 14 and newer that is ScreenCaptureKit's
SCContentSharingPicker, offering a screen or one or more windows; under
Wayland it is the desktop portal, as before. Both come through
DesktopCapturer::CreateGenericCapturer. ScreenCaptureKit draws the
cursor, which DesktopAndCursorComposer would misplace on a picked
window. Where isSystemPickerAvailable() is false, the source ends
instead of capturing something the user did not choose. The default
is unchanged.

A source now ends when its capturer gives up for good: the user
cancelled the picker or stopped sharing, or the captured window was
closed. Before, capture stopped but the track stayed live. The ended
listeners run on the capture thread, as for a missing backend, so
that thread now keeps the source alive until it ends, and stop() called
on it does not join it (#186). Should the last reference go there, a
short-lived thread joins it. dispose() ends the capture without
waiting, even while a track still holds the source.

DesktopPickerExample shows the picker, and the desktop capture guide
describes it.
DesktopCapturer.setUseSystemPicker(true), set before start(), makes
start() show the operating system's own picker instead of capturing the
source chosen with selectSource(): ScreenCaptureKit's
SCContentSharingPicker on macOS 14 and newer, the desktop portal under
Wayland, as VideoDesktopSource already does. Until the user has chosen,
captureFrame() reports ERROR_TEMPORARY; a cancelled picker or stopped
sharing reports ERROR_PERMANENT, as does every frame where
isSystemPickerAvailable() is false. Setting it after start() throws
IllegalStateException.

Changing the setting replaces the native capturer, keeping the maximum
frame rate. VideoDesktopSource now asks DesktopCapturer whether a
picker is available, so both answer alike.
DesktopCapturerPickerExample starts a ScreenCapturer set to use the
system picker, calls captureFrame() at the frame rate on the one thread
that uses the capturer, and shows the frames in a VideoView. Its status
line tells a pending choice from a cancelled picker or stopped sharing.
@devopvoid
devopvoid marked this pull request as ready for review November 13, 2026 19:21
@devopvoid
devopvoid merged commit 1ecb89b into main Nov 13, 2026
27 checks passed
@devopvoid
devopvoid deleted the feat/macos-system-picker branch November 13, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant