[Winify]

Why

Test history lives inside the devtools dashboard, one run at a time. This PR pushes each run's results to any OTLP backend, so the history becomes searchable and aggregatable with tools a team already runs. OTLP is the open standard that log and trace stores already accept, so no vendor lock-in is added. It also lays groundwork for comparing failing and passing attempts of the same test across runs.

What

Four commits, in order:

1. refactor(trace): move the trace zip reader next to the writer — the zip reader (trace-reader*.ts and its tests) moves from packages/backend to packages/trace, so any consumer can parse a trace zip without running the server. Backend keeps readTraceZip as the thin disk-read wrapper.
2. feat(service): record each test's timing and final error in the artifacts manifest — the WDIO service stamps startTime, endTime, durationMs and the final attempt's error onto each test's metadata, and core copies them into devtools-artifacts-<sessionId>.json. The manifest contract moves to packages/shared, so producers and readers share one definition.
3. feat(otel): push devtools test results and traces to OpenTelemetry — new package @wdio/devtools-otel with the devtools-otel CLI. It reads a manifest and its trace zips, then sends one log record per test and, where a zip was retained, one trace per test with the test as the root span and its actions as child spans.
4. docs(examples): add a Grafana LGTM stack with one-click trace replay — examples/o11y is a compose stack (OTel collector, Loki, Tempo, Grafana, and an S3-compatible bucket) with a provisioned test-results dashboard. Each row's Replay link opens the retained trace in a player. The player host is a dashboard variable, player_base, that defaults to https://player.vibium.dev.

Data model: a test's record in Loki is its history entry (one record per test, about 0.5 KB). Its trace in Tempo has one span per action (about 3 KB per trace). The zip itself goes to the user's own bucket, and the record links to it.

Scope

• WebdriverIO only. The package is experimental.
• The service and core/shared change only for the manifest contract (commit 2). This PR does not touch any other adapter. The "why isn't this in core?" question does not apply to the otel package itself, which is a new standalone package.

NOT SUPPORTED

• retain-on-failure and retain-on-first-failure trace policies. Those policies drop the zips of passing tests, so the CLI skips a manifest that has any retained: false artifact and warns. Record with tracePolicy: 'on'.

Known limits

• No dedupe on re-push. Pushing the same manifest twice writes its records twice.
• Loki silently drops a record that is about 1 hour older than the newest record already in its stream.
• specFileRetries is not counted as flaky. Each retry spawns a fresh worker, so the retry is outside the attempt ledger.
• Only the WDIO service writes the manifest timing fields. Other adapters' manifests still lack them, and the CLI tolerates that.
• The example's network-private bucket is POC-grade. It has had no security review. A trace zip holds everything the run typed, passwords included, so the bucket must stay on a private network.

Measurements

From a 150-test mocha run and a 150-test cucumber run, filmstrip off:

• Trace zip size: average 826 KB (mocha) and 751 KB (cucumber).
• Tempo: about 3 KB per trace.
• Loki: about 0.5 KB per record.
• Spans: 19.5 per test (mocha) and 24.5 per test (cucumber).
• Storage projection: 10k tests per day is about 237 GB in S3 at 30-day retention.

Verification

Checks run in this branch's worktree (feat/otel-export, based on upstream/main 1375b2f):

• pnpm build: exit 0.
• pnpm test: 232 test files passed, 3085 tests passed, 3 skipped, 0 failed.
• pnpm lint: 0 errors. Warnings exist in packages other than this PR's. Eslint on the touched TypeScript files: 0 errors, 2 warnings. Both warnings are in packages/service/src/index.ts (async before at 64 lines, file at 635 lines). Upstream's version of that file produces the same two warnings at 633 lines, so this PR adds two lines to an already over-cap file.
• tsc --noEmit -p packages/otel/tsconfig.json: exit 0.
• Dist leak grep (the bare from/require( check from CLAUDE.md) on packages/otel/dist, packages/backend/dist and packages/service/dist: no matches.
• pnpm-lock.yaml was regenerated with pnpm 10.34.9 (CI pins 10.26.1, which drops the libc platform fields on native packages in this lockfile; 10.34.9 keeps them). The diff is 197 insertions and 5 deletions, and adds the packages/otel importer.

End to end, on the same changes before they were rebased onto this base (not re-run here): a real WDIO mocha run (3 tests, one failing by design) and a 150 mocha + 150 cucumber benchmark were pushed through the CLI into the examples/o11y stack. Loki held one record per test with the manifest timing, Tempo held one trace per test with its action spans, and the Replay link played the zip in a real browser on player.vibium.dev. A fail-then-pass retry was checked to report the retried attempt's data.

Not in this PR

• Flipping the filmstrip default (separate branch).
• Backend traceparent correlation.
• A compare tool for failing versus passing attempts.
• Mirroring the docs to the WebdriverIO website.

🤖 Generated with Claude Code