Skip to content

feat: capture macOS windows with ScreenCaptureKit, and close the screen capture gaps - #591

Merged
devopvoid merged 3 commits into
mainfrom
feat/macos-sck-window-capture
Nov 12, 2026
Merged

devopvoid merged 3 commits into
mainfrom
feat/macos-sck-window-capture

Conversation

@devopvoid

@devopvoid devopvoid commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Follows up on #344, where sharing a single window didn't work at all for a macOS user. That cause isn't known yet: the reporter hasn't given their macOS version or permission state. This PR moves window capture off the API that #354 had to leave in place.

Windows are captured with ScreenCaptureKit on macOS 14 and newer

On macOS, VideoDesktopSource and WindowCapturer captured windows with WebRTC's WindowCapturerMac, which takes a CGWindowListCreateImage snapshot for every frame. Apple deprecated that call in macOS 14, and it is unavailable to apps built for macOS 15 and newer. WebRTC at branch-heads/8059 still has no ScreenCaptureKit window capturer outside its system picker, so this adds one of our own: ScreenCaptureKitWindowCapturer, a webrtc::DesktopCapturer in src/media/video/desktop/macos/.

It looks up the selected CGWindowID in SCShareableContent and streams that window with initWithDesktopIndependentWindow:. Frames arrive as IOSurfaces and are wrapped and cropped to the window with WebRTC's DesktopFrameIOSurface.

  • Size: the stream follows the window. With scalesToFit off, a window that outgrew the output is scaled down until the next CaptureFrame() reconfigures the output to the window's on-screen size (SCStreamFrameInfoScreenRect). One reconfiguration per resize, e.g. 800x600 → 1600x1000 → 600x500 pixels.
  • Cursor: ScreenCaptureKit draws it, and the frames say so, so that DesktopAndCursorComposer doesn't draw a second one. The composer works in screen coordinates and would misplace the cursor on a window's frame.
  • Shadow: left out (ignoreShadowsSingleWindow), like kCGWindowImageBoundsIgnoreFraming before.
  • Minimized window: ScreenCaptureKit reports it as suspended, and the last image repeats.
  • Closed window: ScreenCaptureKit stops the stream with error -3815, CaptureFrame() reports ERROR_PERMANENT, and capture stops as before.

VideoDesktopSource and jni::DesktopCapturer use it for windows on macOS 14 and newer. Window lists and source IDs are unchanged: GetSourceList() is WebRTC's GetWindowList(), as in WindowCapturerMac.

What still goes to WindowCapturerMac

  • macOS before 14.
  • Windows ScreenCaptureKit cannot capture: for example, no screen recording permission, a window it doesn't list, or a request that takes more than 10 s. The reason is logged.
  • Windows of the capturing process itself. A stream keeps a closed window of its own process alive: still listed, off screen, and "suspended" for ScreenCaptureKit, so the capture never ended. This held even after the stream was stopped and every object from it released. Core Graphics needs no permission for an app's own windows anyway. Windows of other processes don't behave like this.

Windows were captured with WebRTC's WindowCapturerMac, which takes a
CGWindowListCreateImage snapshot for every frame. Apple deprecated that
call in macOS 14 and made it unavailable to apps built for macOS 15, and
WebRTC, also at branch-heads/8059, has no ScreenCaptureKit window
capturer outside its system picker.

On macOS 14 and newer, VideoDesktopSource and WindowCapturer now capture
a window, chosen by the same CGWindowID as before, with a ScreenCaptureKit
stream of our own. Frames arrive as IOSurfaces cropped to the window. The
stream follows the window's size, ScreenCaptureKit draws the cursor, and
a minimized window keeps its last image. For a 2560x1600 pixel window at
30 fps, the capturing process used about 23% of a core instead of 35%
(Apple silicon, macOS 14.5).

Windows of the capturing process itself, and windows ScreenCaptureKit
cannot capture, for example without screen recording permission, still
go to WindowCapturerMac. A stream kept a closed window of its own process
alive, so that capture never ended, and Core Graphics needs no permission
for such windows. For other processes, ScreenCaptureKit stops the stream
when the window closes, and capture stops as before.

Window lists and source IDs are unchanged. Issue #344 reports window
sharing failing on macOS; its cause there is not known yet.
…creenCaptureKit

newer, by setting allow_sck_capturer for VideoDesktopSource. That left
three gaps, all closed by a screen capturer that picks its backend when
started (MacOSScreenCapturer):

- The whole desktop (source ID -1, all screens in one image):
  ScreenCapturerSck cannot capture it, and silently delivered only the
  first screen. The CGDisplayStream capturer captures it again.
- ScreenCapturer.start()/captureFrame() still captured with the
  CGDisplayStream capturer, which can freeze on its first frame
  (#344), because ScreenCaptureKit lists no screens without its system
  picker. Screens are now listed by the CGDisplayStream capturer, as
  before, and a single screen is captured with ScreenCaptureKit, as in
  VideoDesktopSource. Both name a screen by its CGDirectDisplayID.
- ScreenCapturerSck's frames do not say where the screen lies on the
  desktop, while DesktopAndCursorComposer places its cursor in desktop
  coordinates, so on any screen but the first the cursor was missing or
  misplaced. ScreenCaptureKit now draws the cursor itself.

Before macOS 14, every screen is still captured with the CGDisplayStream
capturer.
ownWindowFramesFollowWindow failed once in a full run: after the window
turned white, its frames stayed black. Its capture was running, but the
window had not been redrawn. A window of a test process that is not in
front may not be redrawn in time, and it cannot be put on top, since
WindowCapturer lists only windows of the normal level. The test now
checks that the window is captured at its size, and that the capture
ends when the window closes. Content is checked on another process's
window, which is now ordered to the front.
@devopvoid devopvoid changed the title feat: capture macOS windows with ScreenCaptureKit feat: capture macOS windows with ScreenCaptureKit, and close the screen capture gaps Nov 12, 2026
@devopvoid
devopvoid merged commit 88f6bab into main Nov 12, 2026
27 checks passed
@devopvoid
devopvoid deleted the feat/macos-sck-window-capture branch November 12, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant