Skip to content

Keys reach a SwiftUI List and a WebKit page in the background; web terminals are typed into, never pasted - #240

Merged
iamnbutler merged 6 commits into
mainfrom
fix-keys
Nov 3, 2026
Merged

iamnbutler merged 6 commits into
mainfrom
fix-keys

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

Fixes #226
Fixes #229
Refs #5

#226: keys into a list in a window that isn't key

Cause: there were two layers.

  • A SwiftUI List / onDeleteCommand drops a background key unless its window is key.
  • A background (Accessibility) click on a row selects it but doesn't make the list first responder. The window's focused element is the window itself, so AppKit drops Delete and Return.

Fix:

  • desktop_key makes an inactive app's window key first. It uses the same SkyLight activation and title-bar click that background paste uses, so the app isn't brought forward. The result reports key_window: made_key | already_key | not_key.
  • When the window itself holds focus, desktop_key focuses the window's one list, table or outline with a selected row (list_focused). Focus never leaves the window.
  • The result is now unknown, with KEY_WINDOW_UNAVAILABLE or KEY_NO_FOCUSED_VIEW, when neither works.
  • A second preparation soon after the first no longer clicks the title bar again; macOS read that as a title-bar double-click and zoomed the window.

Live check (native2/fixture/keysfix.swift, app inactive, user's frontmost app unchanged):

  • SwiftUI List Backspace: completed, made_key, list_focused, and the fixture logged list deleted ["two"]. Before the fix it was completed with nothing deleted.
  • AppKit table: Backspace deleted the row; down then Return renamed (already_key).
  • A list window that isn't the app's focused window is still refused, with nothing sent.

#229: Cloud Shell xterm paste

Cause:

  • xterm's helper textarea clears itself after every input, so a paste into it can never be read back. It was reported unknown, and the text stayed on the clipboard.
  • Typing reaches xterm at once, including in a cross-origin iframe in inactive Chrome. A test page with xterm 5.5.0 logged each character from onData.
  • So the late run in Cloud Shell was most likely its stalled backend replaying buffered input after "Terminal tabs have been recovered". This is inferred; I didn't reproduce it against Cloud Shell itself.

Fix:

  • desktop_insert into a web terminal's input (xterm class, or "Terminal input" on a web field) types instead of pasting.
  • route: "paste" there is refused with PASTE_UNVERIFIABLE_RECEIVER; nothing is written to the clipboard.
  • Typed text is unknown with web_terminal: true, and a hint to check with ocr and press Return with desktop_key, never to retype.

Live check (webfix terminal input): paste refused; typing gave web_terminal and the page received e-c-h-o-space-h-i; a newline was refused. In Chrome, xterm in a cross-origin iframe received " xt3". Chrome doesn't name xterm's textarea as focused there, so that result is the generic window_focus unknown.

#5: WebKit page fields and background Command-V (partly)

Findings:

  • The fields were missing on the first inspect because WebKit builds a WKWebView's Accessibility tree only when first asked. The first read finds only an AXGroup; the AXWebArea appears under it milliseconds later. It wasn't the element cap.
  • A WKWebView app names no AXFocusedUIElement, so background keys were refused. The page's field reports AXFocused itself.
  • Edit › Paste is disabled (AXEnabled false) in a background WebKit app before preparation.
  • After the window is made key it is enabled, and an Accessibility press of it pastes, but only with a page field focused. With no field focused it stays disabled even in the key window.

Fix:

  • Inspect re-reads for up to 2 s until the web area appears. Each group is waited on once.
  • Focus checks fall back to the deepest focused element under the web area.
  • Inspect names that field as the focused element when the web area and scroll area also report focus.

Live check (native2/fixture/webfix.swift):

  • The first inspect lists all 4 fields.
  • Background Cmd+V pasted "BG-CMD-V" into the textarea; the page's paste event had text/plain.

Not done:

Fixtures

native2/fixture/keysfix.swift and native2/fixture/webfix.swift; build commands are in their headers.

Tests

  • swift test --package-path native2: passes, including the new KeyboardRouteTests.
  • npm run check: passes.
  • npm test: passes.

…d; web terminals are typed into, never pasted

- desktop_key makes an inactive app's window key first (key_window), focuses the window's one selected list when
  only the window has focus (list_focused), and says when neither is possible instead of reporting completed (#226).
- desktop_insert types into a web terminal's input (xterm.js) and refuses route paste there, since xterm clears its
  input and a paste could never be confirmed (#229).
- desktop_inspect waits for a WKWebView's lazily built tree, and keys reach a WebKit page's focused field (#5).
- Fixtures: native2/fixture/keysfix.swift (AppKit table and SwiftUI List) and webfix.swift (WKWebView page).

Fixes #226
Fixes #229
Refs #5

Why: native2/Sources/DesktopMac/Keyboard.swift#focusSelectedList | an Accessibility row press selects a SwiftUI List row without making the list first responder; a key into a window whose focused element is the window itself is dropped (checked live with native2/fixture/keysfix.swift)
Why: native2/Sources/DesktopMac/Keyboard.swift#focusedWebElement | a WKWebView app reports no AXFocusedUIElement; the page's field reports AXFocused itself and can sit under a label group that doesn't
Why: native2/Sources/DesktopMac/BackgroundPreparation.swift | two title-bar preparation clicks within the double-click interval zoom the window even with click state 1; skip once the app reports AXFrontmost with this window focused
Why: native2/Sources/DesktopHost/Text.swift | xterm's helper textarea clears itself after every input, so a paste into it can never be read back and would stay on the clipboard
Why: native2/Sources/DesktopMac/WebAccessibility.swift#awaitsWebKitTree | WebKit builds a WKWebView's tree on the first Accessibility request; that read finds only an AXGroup, and the AXWebArea appears under it milliseconds later
iamnbutler and others added 4 commits November 2, 2026 20:33
…ays nothing was sent

The key result reused the paste path's refusal text, so it said both that the
keys were sent and that nothing was sent. It now gives only the cause.

Co-Authored-By: Claude <noreply@anthropic.com>
…lank tab strip

Chromium has no standard title bar: its tab strip fills the title row, inside groups that span the whole window,
so preparation found no blank space and every background key into Chrome was unknown (KEY_WINDOW_UNAVAILABLE).
In a Chromium app only leaf elements in the title row count as drawn, and an empty unlabeled group without a press
action is blank strip, where the click goes. Checked live in a temp-profile Chrome that was never activated: keys
into a page field arrive with key_window made_key, then already_key, and the window doesn't move or zoom.

Why: native2/Sources/DesktopMac/BackgroundPreparation.swift#isBlankChrome | Chrome's title row holds a labeled group spanning the whole window ("<title> - Google Chrome") and an empty unlabeled AXGroup right of New Tab; AX hit-testing the blank strip returns that group, not the window
# Conflicts:
#	CHANGELOG.md
#	native2/Sources/DesktopMac/WebAccessibility.swift
@iamnbutler
iamnbutler merged commit af0f626 into main Nov 3, 2026
2 checks passed
@iamnbutler iamnbutler mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant