Repository navigation
WebKit: background Command-V, and Safari page fields missing from Accessibility #5
Description
Focus, clicking, desktop_type and desktop_insert now work in WebKit composers, including Ace's (#82, #68, #88, and ace2 #120, #131 and #141). Two things are still unknown.
1. Background Command-V into a web app that isn't frontmost. A background desktop_key Command-V into a focused WKWebView composer didn't land. Text has a working route through desktop_insert. An image or file paste into the composer doesn't. Before any patch, answer these with Safari in the background and a textarea focused:
- Is Edit › Paste
AXEnabled, both with and without the host's background key-window preparation? - Does an Accessibility press of Edit › Paste paste into the textarea?
- Must the textarea have
AXFocusedfirst?
The answers decide between a menu-press route, a foreground fallback, and a clear refusal.
2. Safari page fields missing from the Accessibility tree. In the Oct 15 study (D6), a local Safari page's <input>, <textarea> and contenteditable weren't in desktop_inspect, so the agent had to click coordinates. Chrome and Electron are handled by #107. For Safari, the cause isn't known: WebKit might load its accessibility lazily, or the 200-element inspect cap might cut the tree short. Reproduce it, then fix it or document the workaround.
Activity
iamnbutler commentedon Oct 5, 2026
iamnbutler commentedon Oct 5, 2026
iamnbutler commentedon Oct 5, 2026
iamnbutler commentedon Oct 5, 2026
iamnbutler commentedon Oct 5, 2026
iamnbutler commentedon Oct 6, 2026
iamnbutler commentedon Oct 6, 2026
iamnbutler commentedon Oct 7, 2026
iamnbutler commentedon Oct 13, 2026
iamnbutler commentedon Oct 15, 2026
iamnbutler commentedon Oct 17, 2026
feat: turn on Chromium and Electron web accessibility before inspect …
feat: turn on Chromium and Electron web accessibility before inspect …
[-]Focus and paste into WebKit composers with native desktop tools[/-][+]WebKit: background Command-V, and Safari page fields missing from Accessibility[/+]iamnbutler commentedon Nov 3, 2026
Answers from a WKWebView fixture (
native2/fixture/webfix.swift, in #240), driven in the background through Desktop Tools 0.19.0 plus the #240 changes. The fixture is a regular app that isn't frontmost, with a standard Edit menu, and the page logs every focus, paste and input event.Is Edit › Paste
AXEnabled?desktop_menurefuses it as disabled.Does an Accessibility press of Edit › Paste paste? Yes, once it's enabled. The page logged
paste notes types=["text/plain"] text="MENU-PASTE".Must the textarea be
AXFocused? Yes, in practice. The paste goes to the page's focused field, and with none focused the menu item is disabled.Why page fields were missing from Accessibility: WebKit builds a WKWebView's tree only when an assistive client first asks. The first read finds only an AXGroup where the page is, and the AXWebArea appears under it milliseconds later; it isn't the element cap. #240 makes
desktop_inspectre-read, for up to 2 s, until the web area appears, so the first inspect lists the fields.Why background Command-V didn't reach the page:
AXFocusedUIElement, since the field lives in the web content process, so background keys were refused.AXFocuseditself.desktop_keyCommand-V then pasted "BG-CMD-V" into the focused textarea, with the app still not frontmost.Still open:
desktop_insertroute: "paste"into a web<input>is still refused before sending, because the field's text can't be read back to confirm the paste. Typing remains the route there.Keys reach a SwiftUI List and a WebKit page in the background; web te…
Release 0.20.0: fix train for #224–#227, #229–#232, #178, #5 (#241)
Docs: catch up with 0.20.0 and the native2-only build (#242)