Repository navigation
perf(trace): capture page state only for commands that change it - #442
Conversation
[Medium impact] The PR appears safe to merge; no blocking issues remain. SummaryThis PR separates timeline rows from page captures.
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Mapped command] --> B{Changes the page?}
B -->|Yes| C[Capture page state]
B -->|No| D{Initial capture needed?}
D -->|Yes| C
D -->|No| E[Reuse earlier capture]
C --> F[Render timeline row]
E --> F
Reviews (2) · Last reviewed commit: "fix(trace): retry the initial frame only..." · Reviewed by Greptile |
| if (snap) { | ||
| upsertRichestSnapshot(ctx.actionSnapshots, snap) | ||
| } else if (initialFrame && !changesPage(cmd.command)) { | ||
| capturer.awaitingInitialFrame = true |
There was a problem hiding this comment.
Failed screenshot stops later attempts
If the first screenshot fails, captureActionSnapshot can still return a truthy object with screenshot: undefined. This branch leaves awaitingInitialFrame false, so later reads never retry. FrameSnapshotIndex skips that capture, and Selenium has no final capture to recover it. A read-only test can therefore finish without frames for its rows.
Clear the initial-frame flag only after a usable screenshot arrives. Apply the same check in Nightwatch and the service.
ReviewThe direction is right and the diff is small. Splitting "does this get a timeline row" from "does this get a capture" via I'd ask for one fix before merging and have one design question. Must fix1. The initial-frame retry never fires. Greptile's finding is correct, and it's slightly worse than it describes.
Suggested fix: gate on Should address2. The service's action rows now show a later moment (not mentioned in the PR). The service captures before a command and stamps the capture with the timestamp of the last page change. Take
The capture count is the same (one per change); only when it's taken moves. Showing the settled state may well be what we want, but it is a user-visible change. The "0 rows differing" table compares page text on suites that probably have no async transitions, so it wouldn't catch this. Please either state it in the changeset or check it on a suite with a loading state. Selenium and Nightwatch capture right after the command, so the adapters now differ here as well. 3. A late result from one test can flip the next test's flag (Selenium/Nightwatch). The capture promise isn't awaited. If test N's first capture resolves after test N+1 has already taken its initial frame, the retry branch sets Nits / questions
Generated by Claude Code |
b012d0c to
14e1b41
Compare
Trace mode took a per-action capture (page source + screenshot, plus DOM on web) for reads and waits as well as actions, because the capture gate was mapCommandToAction, the timeline-rendering table. 5c54ae4 added the reads to that table so they would render as rows, which also made every read, and every poll of a waiting matcher, a capture. On iOS a capture is expensive; a reported suite made 187 page-source calls for 12 interactions and went from 3:10 to 7:10. ACTION_MAP's methods are now classified in a typed METHOD_EFFECT table (an unclassified method fails to compile) and changesPage() gates the capture in all three JS adapters. Reads keep their rows and inherit the latest earlier capture through claimAfter's fallback. The first traced command of each test is still captured so a test opening with reads has a frame. The WDIO service stamps its end-of-test capture at the test's last page change, otherwise that action's row resolves to the capture from before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
captureActionSnapshot reports a failed screenshot probe as a snapshot without a screenshot, not as null, so the retry keyed on a falsy result never fired and a test that opened with reads could end up with no frame. The flag now stays armed until a capture carries a screenshot, under one rule in all three adapters, and is retried at most once per test so a driver that cannot take screenshots does not capture every read again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in the capture-gate tests takeScreenshot is optional on SeleniumDriverLike, so the tests that mock and call it failed the typecheck CI runs. Also documents in CLAUDE.md when each adapter takes its per-action capture now that only page changes take one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tream added Wait commands now open a running row through SessionCapturer.openWait, so the mocked capturer threw on the wait in the gate test once rebased onto main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hat change the page The Python adapter ran its two element-tree reads after every command, which the JS adapters stopped doing. The page-changing and mapped command sets are evaluated from shared's ACTION_MAP/changesPage by the wire-schema generator and emitted into _contract.py, so Python follows the same table without a copy. Each test's first mapped command still captures an initial frame, retried once when it has no screenshot. WebElement.submit() reaches execute as a marked script, so it is gated as shared's submit rather than an unmapped script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…capture off the next test browser.url() with no string argument only reads the url (assert.urlContains reads through it), so gating it as a navigation took a capture per read. And a slow initial-frame capture that resolved after the next test had started could spend that test's one retry; a per-test generation now ignores it. CLAUDE.md scopes the beforeCommand description to the WDIO service and names the initial-frame exception. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
14e1b41 to
3665ae4
Compare
|
@greptileai review |
…te after trailing reads The page-changing allowlist missed commands that change the page without being in ACTION_MAP (an executeScript or actions() click, an alert accept, Nightwatch submitForm), so the reads after them showed the old page in the Selenium, Nightwatch and Python adapters. The gate is now a deny-list: shared skipsCapture() names the reads, waits, lookups and asserts that skip a capture, and everything else captures. Python reads the list from wire-schema.json, and treats selenium's isDisplayed/getAttribute script atoms as the reads they are. Those adapters also capture right after the command, so a click that returns before its navigation commits left a test ending on reads with the old page. Each now takes one end-of-test capture when rows follow its last capture, stamped at that capture (core finalFrameStamp) and replacing it whole (trace replaceSnapshotAt), the way the WDIO service's finalize already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ep what a partial end capture lacks The deny-list missed Nightwatch's own reads (isVisible, isPresent, title, source, getCssProperty, element/elements and more), selenium-webdriver's getDomAttribute/getId/getCapabilities/getSession, and selenium-python's w3cGet*/getElementAria* names, so polling them took a full capture again. An end-of-test capture whose screenshot timed out replaced the last good one whole; it now supersedes field-wise. Python's end capture is taken through the driver that took the test's last capture, so a second driver neither lends its page nor, by quitting, uses up the owner's end capture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… end capture supersede selenium-webdriver sends actions().perform() through driver.execute, which the patcher deliberately leaves unwrapped, so a navigating Actions click took no capture and the reads after it showed the old page. perform() is now wrapped to take a capture on completion without adding a row, since main never listed one. A test that ends its own session leaves the end-of-test capture with no url and no screenshot; its empty elements and placeholder text still superseded the last capture's. core capturedPageState() gates the supersede. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts after it returns Nightwatch's click resolves before its navigation commits, so its per-action capture records the page it is leaving. main hid that by capturing again on every read; with reads skipped, the click and the reads after it showed the old page (iOS Safari, 2 of 2 runs). When anchorAfterNavigation sees the document replaced, the command that triggered it is captured once more and the result supersedes its record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t caused it The re-capture was bound to the command whose poll saw the document replaced. A fill's poll outlasts it, so when the next command (the submit click) navigated during that poll, the fill took the destination page and its row showed /secure before the login had happened. The target is now the latest captured command at the moment the replacement is seen, and overlapping polls re-capture it once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead the page it left The re-capture fired for any command whose poll saw the document replaced, including url(), which resolves after load and had already captured its destination. Its late re-read then landed after the next command navigated again and gave the url() row that later page. A command is now re-captured only when its capture was queued before the destination document's performance.timeOrigin, and the re-capture is dropped if a newer page capture was queued while it ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… perform() still captures A url-only end or post-navigation capture (screenshot probe timed out) was merged over the earlier record, leaving the new page's url on the old page's image. capturedPageState now requires a screenshot, and Python's end capture follows the same rule and keeps a screenshot-only capture as the JS path does. An actions() sequence that moved the page and then rejected now still takes its capture. Nightwatch awaits its post-navigation re-capture even when the drain beside it fails, and logs that failure. Docs and changesets now describe the capture as keyed on the last command that took one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vishnuv688
left a comment
There was a problem hiding this comment.
Thanks for this. Skipping captures for reads, waits and wait polls is the right fix for the iOS slowdown, and generating Python's read list from shared (through wire-schema.json into _contract.py) avoids a hand-copied table. The first-frame retry fix, the per-test generation counter and the url() getter fix all look good, and CI including examples-smoke is green.
I reviewed the current head (1eed261). It has moved past the description, so a few things need to line up before merge.
Before merge
-
Docs and description contradict the code.
- The description says
changesPageis the capture gate in every adapter and that Python is a follow-up. Selenium, Nightwatch and Python now use theskipsCapturedeny-list, and Python is changed in this PR. - In CLAUDE.md, the new Known debt entry says "Only a command that changes the page is captured (
changesPage...)", while the paragraph added under "One DOM snapshot per action" says Selenium, Nightwatch and Python capture every command except known reads. Please make both describe the current behaviour.
- The description says
-
The same logic lands in two adapters. Selenium's and Nightwatch's
SessionCapturerboth gainawaitingInitialFrame,initialFrameRetried,testCaptureGeneration,lastCapturedTimestamp,testStartWallTimeandresetTestCapture(), plus the same first-frame retry block inqueueActionSnapshotand a near-identicalcaptureFinalFrame. Per CLAUDE.md that belongs in core;SessionCapturerBaselooks like the natural home, with each adapter passing only its owncaptureActionSnapshot. -
One question, two predicates. The service gates on
changesPageand the other adapters onskipsCapture. The service only considersACTION_MAPcommands (#isActionCommand), and for thosechangesPage(c) === !skipsCapture(c), so the service can useskipsCaptureandchangesPagecan go. If the difference is meant to stay, please say why in the code. -
Python.
_final_frame_stampsays it mirrors core'sfinalFrameStamp, but there is no section for it inpackages/shared/mirror-vectors.json. It is pure and deterministic, so it should get one, with cases for no capture yet, a capture from the previous test, no rows after the capture, and rows after it.instrumentation.pyadds seven_statekeys. The Known debt entry for that file asks the next change touching_stateto split it per concern first; a smallTestCaptureState(or a module of its own) would do it for these keys.- Nit: the
# Frames are screenshot-backed ...comment is duplicated in_capture_final_frame.
Suggestions
-
Selenium JS reads missing from the deny-list. The JS adapter wraps every
WebDrivermethod exceptINTERNAL_DRIVER_METHODS. Checked against selenium-webdriver 4.50, these reads would now each take a capture:getAllWindowHandles,printPage,getCredentials,getDownloadableFiles.getAllWindowHandlesis common in multi-window tests. Worth adding toREAD_ONLY_COMMANDS. -
afterActionsfires on failure too. Its comment intypes.tssays it fires onceperform()succeeds, butpatchedPerformcalls it fromfinally, so a rejectedperform()also queues a capture, unlike every other command, where an error skips it. Either move the call to the success path or update the comment and say why a failed sequence should capture.
Questions and nits
- iOS is still unmeasured, and it is the case this PR exists for. A run of the reporting suite on this build before merge would confirm the gain. On Python a screenshot is still taken on every command, so there only the element-tree reads are saved; worth saying in the fragment.
wire-schema.jsonnow carriesactionCommands, which is not part of the wire format. ADAPTERS.md documents it, so this is fine if intended; just checking it was a deliberate choice over a separate generated file.- Nightwatch
session.ts: the doc comment oncaptureTraceAtTestEndwas replaced by the one for the newcaptureFinalFrame; please keep both.
Addresses the review on webdriverio#442. - The service gates on skipsCapture like every other adapter; changesPage is gone, since for ACTION_MAP commands the two always agreed. - Selenium and Nightwatch shared the per-test frame state, first-frame retry and end-of-test frame verbatim; that now lives in core's TestCapture. Selenium's end frame reads ctx.driver after the pending captures settle, as before the move. - Python keeps the same state in PerTestCapture rather than seven _state keys, and finalFrameStamp has a mirror-vectors section run by both sides. - getAllWindowHandles, printPage, getCredentials and getDownloadableFiles join READ_ONLY_COMMANDS; wire-schema and the Python contract regenerated. - CLAUDE.md describes the single gate; the afterActions comment says why a failed perform() still captures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@vishnuv688 I have measured the changes on BrowserStack against iOS iOS measurement (BrowserStack App Automate, iPhone 17 / iOS 26)Base is
Smoke runs on head, one each, all passed with a trace written and no unresolved rows: Selenium JS on iOS web and on native, Python on iOS web, and Nightwatch on iOS web. Nightwatch native ran twice on head and wrote a full trace both times. The earlier The reported suite itself (187 page-source reads for 12 interactions) was not rerun. These numbers come from the sample app. |
Upstream's capture timing (09ddb75) is threaded through the per-test capture state this branch moved into core's TestCapture and Python's PerTestCapture: a capture is tagged 'action' or 'wait-poll', and every adapter's end-of-test frame is tagged 'final', as the service's was. Python times every command's screenshot, since it still takes one per command, while the element and accessibility reads stay gated to commands that capture. mirror-vectors keeps both finalFrameStamp and captureTimingSummary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Thanks, this addresses the review, and the iOS numbers settle the open question (GET /source 26 to 8, Mocha 27.9 s to 17.0 s, every head run faster than every main run).
Three small text fixes left, then this is good to merge from my side:
- The PR description is still the first version. It names
changesPageas the gate in every adapter, says iOS is not measured, and lists the Python adapter as a follow-up. Please update it to the deny-list gate, link the iOS measurement above, and drop the Python follow-up. - Changeset: the last sentence, "The WDIO service keeps its page-changing gate.", is out of date now that the service uses
skipsCapturetoo. Something like "The WDIO service applies the same gate to its action rows and keeps capturing before the next command." would match CLAUDE.md. - CLAUDE.md, "Logic implemented in both TypeScript and Python": the "Covered today" list should name the new section (end-of-test frame stamp).
What & why
In trace mode every adapter took a per-action capture (page source and screenshot, plus DOM/a11y on web) for read and wait commands as well as for actions. On iOS a capture is expensive, because WDA computes
visiblefor every node. One reported suite made 187getPageSourcecalls for 12 interactions and slowed from 3:10 to 7:10.The capture gate has always been
mapCommandToAction, the table that decides which commands render as timeline rows. 5c54ae4 (#247) added the reads (getText,is*,waitFor*,getTitle, …) toACTION_MAPso they would render as rows. From then on each of them was also captured, including every poll of a waiting matcher. One table was answering two questions.METHOD_EFFECTclassifies every trace method aschangesorreads.ACTION_MAPis typed against it, so an unclassified entry fails to compile.changesPage(command)is the new capture gate.changesPage. The first traced command of each test is also captured, so a test that opens with reads still has a frame. Read and wait rows keep rendering and inherit the latest earlier capture throughclaimAfter's existinglatestAtOrBeforefallback.captureActionSnapshotreports a failed screenshot as a snapshot with no screenshot rather than asnull. The retry is capped at one per test, so a driver that cannot take screenshots doesn't capture every read again. The rule is the same in all three adapters.Type of change
Packages touched
shared(types and contracts)core(framework-agnostic capture/reporting)elements(published element/snapshot API —@wdio/elements)service(WebdriverIO adapter)nightwatch-devtools(Nightwatch adapter)selenium-devtools(Selenium adapter)selenium-devtools-py(Selenium Python adapter)backend(server)app(UI)script(page-injected runtime)trace(Trace mode) (tests only)Notes for reviewers
Why isn't this in
core? The gate is one call per adapter, but each adapter captures at a different point: the service before the command, Selenium and Nightwatch after it. The shared part is the classification, and that lives inshared.Behavior change in the service. The service captures before a command, so an action's row now shows the page just before the next page-changing command, after any waits and reads in between. Before, it showed the page just before the first read that followed. Take
click(A) → waitForDisplayed → getText → click(C): A's row used to show the moment after the click, possibly mid-transition, and now shows the settled state. The number of captures per page change is unchanged; only their timing moves. Selenium and Nightwatch capture right after each command, so they are unaffected.Real runtime,
upstream/mainvs this branch, 3 runs each,DEVTOOLS_MODE=trace. "Rows differing" compares each row's resolved snapshot page text against the base run. These runs were measured before the retry fix, which only changes the screenshot-failure path.The Nightwatch wait after the submit click still resolves to
/secure: it inherits the click's own capture, which is taken after the click. Wall time on Chrome doesn't move because captures are cheap there.Checks: after the rebase onto
upstream/main,pnpm buildpasses,pnpm testreports 2859 passed and 3 skipped, andpnpm lintreports 0 errors. New tests cover:null;Not measured: iOS, the case that motivated this. A rerun of the reporting suite on this build would confirm the gain.
Known limits:
describe/it: the plugin's per-test hook fires once per module, so the first-command capture happens once per module.browser.url()with no argument: this getter still counts as a page change.executecounts as a page change. That includes Appiummobile:reads, which only read.Screenshots / recordings
n/a (no UI change)
🤖 Generated with Claude Code