Repository navigation
Mutation gate: forbid Stryker suppression comments in source #2050
Description
Problem
// Stryker disable next-line ... suppression annotations keep spreading through production source files. main currently carries 38 directives across 9 files (Task.ts, pathUtils.ts, lm-studio.ts, qwen-code.ts, openai.ts, openai-codex.ts, abort-signal.ts, webviewMessageHandler.ts, NativeToolCallParser.ts), and every contributor who runs Stryker locally adds more.
The root cause is a policy gap, not a missing tool:
- The changed-code mutation gate (
scripts/stryker-diff.mjs, run by themutation-diffCI job on every PR) already exists. - Its directive policy allows suppressions and validates only their shape (named mutator + reason, no
All). - Malformed directives are reported as advisories — the job still exits 0, so nothing blocks them.
The result is the worst of both worlds: annotations accumulate, each one permanently silences a mutant class on that line, and the codebase trends toward an annotation soup where every contributor's favorite static-analysis tool leaves its fingerprints in comments.
Decision
Forbid Stryker suppression comments in source. The mutation gate is the tool of record in this repository's CI; its exclusions belong in gate configuration (maintainer-owned), not scattered through source as contributor-owned comments.
Plan (two PRs)
- Enforcement PR (ci(mutation-gate): forbid Stryker suppression comments in source #2051):
scripts/stryker-diff.mjsreplaces the advisory hygiene check with a hard failure — any// Stryker disable/// Stryker enablecomment in a touched mutation-tested file fails themutation-diffjob. CI summary guidance, workflow step name,AGENTS.mdandCONTRIBUTING.mdupdated. This PR does NOT touch the existing directives. - Cleanup issue/PR (Mutation gate: remove the 38 legacy Stryker directives on main with evidence-backed central exclusions #2060 / chore(mutation-gate): remove all Stryker directives with evidence-backed exclusions #2054): removes the 38 directives already on main, backed by a per-line mutation repro — the 38 protected lines are re-exposed to Stryker with no directives, and each survivor is resolved either by a kill test (where killable) or by a narrow gate-config exclusion with the equivalence argument recorded (where genuinely equivalent).
Contributors whose mutant is genuinely equivalent explain the equivalence in the PR body; maintainers decide on narrow gate-config exclusions (e.g. excludedPaths in PACKAGE_CONFIGS).
Activity
easonLiangWorldedtech commentedon Oct 22, 2026
#2051 closed by the author: a direction change, not a technical failure
Recording the closure of step 1 of this plan, so the plan of record below matches reality.
Closure: #2051 (
ci(mutation-gate): forbid Stryker suppression comments in source) was closed unmerged by the author at 2026-10-22T09:57:41Z, head44ca22216893a7ab544852cd892243bfa0fb68e9(+140/−55 across 5 files).What blocked it:
edelauna'sCHANGES_REQUESTEDreview at 2026-10-18T03:49:28Z (review id 5561319525, on the same head). Key sentence, verbatim:The PR was green, so this is a direction change rather than a technical failure:
successon44ca22216—check-translations,platform-unit-test (ubuntu-latest),platform-unit-test (windows-latest),compile,knip,e2e-mock;mutation-diffwas green too.APPROVED(2026-10-18T03:36:21Z, same head). Pre-merge checks: 7 passed / 1 warning, and 0 of 5 review threads open.Nothing from #2051 landed on
main(verified against05d01f33dtoday):scripts/stryker-diff.mjsstill exportsvalidateDisableDirectives(advisory shape check);assertNoDisableDirectivesis absent..github/workflows/mutation-testing.ymlstep is still namedEnforce executable-line scope and run advisory mutation testing.AGENTS.mdstill permits named + justifiedStryker disablecomments.Stryker disabledirectives remain across 10 files:pathUtils.ts11,Task.ts7,lm-studio.ts5,qwen-code.ts5,gemini.ts3,openai.ts2,openai-codex.ts2,NativeToolCallParser.ts2,webviewMessageHandler.ts2,abort-signal.ts1.Direction now pursued instead — discoverability, not prohibition:
mutation-diffjob summary as a per-fileFile | Mutants | Killed | Survived | NoCoverage | Scoretable, sorted worst-first, and points at the existingchanged-code-mutation-reportartifact. It does not change the Stryker suppression policy.main(head3719b9938): the commits carried over from ci(mutation-gate): forbid Stryker suppression comments in source #2051 were dropped, so it no longer touchesAGENTS.md,CONTRIBUTING.mdor the workflow, andscripts/stryker-diff.mjsgains no directive ban. The threegemini.tsdirectives re-added upstream by fix(api): require HTTPS for custom Google Gemini base URLs (CWE-319) #2033 are tracked in mutation-gate: remove the 3 Stryker directives re-added upstream in gemini.ts (#2033) after #2054's evidence snapshot #2147.Effect on this issue's two-PR plan: step 1 (enforcement) is withdrawn in favour of the discoverability work in #2149; step 2 (#2054) now stands on its own as a cleanup of already-dead suppressions. #2051 will not be reopened.