ci: the race job lays its guards out by what each weighs by donislawdev · Pull Request #216 · donislawdev/TestingFilesGenerator

donislawdev · 2026-11-03T17:46:47Z

What changes

The race detector job (ci.yml, job race) lays its guards out over its four parts by what each weighs, instead of by the checksum of each name alone.

.github/race-weights: the 48 guards that took 10 s or more under the race detector, each at the longest time any of eight measured parts took, and an unlisted weight (the mean of the rest).
.github/scripts/race_parts.py split: the weighted guards go heaviest first, each into the lightest part, over the light ones. The light ones stay on the checksum, so adding or removing one moves no other test.
.github/scripts/race_parts.py unlisted: after the tests, a guard missing from the weights that ran 60 s or more fails its part with the line to add.
A weight for a test the binary does not list fails every part.
The race job now also runs when the weights or the script change.

Why

The machine steps added in #215 answered why part 2 ran past its twenty minutes in one run of three. The cause is the processor the runner is given:

part	processor	time
2	AMD EPYC 7763	1202 s (timed out)
2	AMD EPYC 9V45	689 s
3	AMD EPYC 7763, twice	525 and 530 s

Every test of part 2 was slower by the same factor (1.73 to 1.93 for the tests of 5 s or more). Split by the checksum, the parts were about 1250, 1158, 795 and 526 s on the 7763, so two of them sat at the timeout. By the measured times they are now 912 to 961 s on that processor.

How it is checked

internal/guard/raceparts_test.go:

every guard is laid into exactly one part, for 1, 4 and 5 parts, against the binary's own -test.list;
each part is within a tenth of the mean by the weights, and under 85% of the timeout read from ci.yml;
the script refuses a weight for a test that is gone, a malformed or negative weight, a test weighed twice, a missing unlisted line and a part past the parts;
a guard missing from the weights fails its part at 60 s, and a long subtest does not;
the job calls both halves, and lists both files among the ones that make it run.

There are thirteen new mutation entries. The script was also run under Python 3.12, the python3 of ubuntu-latest.

🤖 Generated with Claude Code

Summary by CodeRabbit

Chores
- Automated race-test runs now distribute tests using measured runtimes, helping balance work across parallel parts.
- Runs now flag unlisted tests that take 60 seconds or longer, and reject outdated or invalid test-weight entries.
- Additional checks verify that race-test partitions are balanced and that changes to the allocation setup trigger the relevant checks.

The steps added in #215 named the processor of every race part. The same part 2 took 1202 s on an AMD EPYC 7763, past its twenty minute timeout, and 689 s on an EPYC 9V45, every test of it slower by the same factor, while part 3 took 525 and 530 s on two 7763s. Split by the checksum of each name, the four parts were about 1250, 1158, 795 and 526 s on the 7763. .github/race-weights holds the 48 guards that took 10 s or more, each at the longest time any of eight parts took, and the mean of the rest. .github/scripts/race_parts.py lays those out heaviest first, each into the lightest part, over the light guards, which stay on the checksum so adding one moves no other. By the measured times the parts are now 912 to 961 s on the slowest processor. A weight for a test the binary does not list fails every part, and a test missing from the weights that runs 60 s or more fails its part with the line to add. The race job now also runs when either file changes. Five guards in raceparts_test.go: every guard in exactly one part, checked against the binary's own -test.list; the parts within a tenth of their mean and under 85% of the timeout read from ci.yml; the refusals; the unlisted check; and the job's use of both. Thirteen mutation entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-11-03T17:47:25Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Advanced
Run ID: cb4677f7-4e07-41ea-a941-813e84058287

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

Trigger review

Walkthrough

The race job now assigns tests using recorded weights and checks its log for unweighted tests that run for at least 60 seconds. The change adds the allocation script, race-job integration, weight data, and tests for allocation and CI wiring.

Changes: Weighted race-test partitioning

Layer / File(s)	Summary
Weight data, allocation, and validation `.github/race-weights`, `.github/scripts/race_parts.py`, `internal/guard/mutation/entries.py`, `internal/guard/raceparts_test.go`	The weights file records measured durations for named tests and an `unlisted` weight. The script validates the inputs, assigns weighted tests to the currently lightest part, assigns unweighted tests by name checksum, and reports estimated loads. Tests and mutation entries check allocation, input validation, and detection of slow unweighted tests.
Race-job integration `.github/workflows/ci.yml`, `internal/guard/mutation/entries.py`, `internal/guard/raceparts_test.go`	The race job watches the weights and script, uses the script to assign tests, and checks the race log for slow unweighted tests. CI checks verify the job wiring and check order.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant RaceJob
  participant race_parts.py
  participant GoTestProcess
  participant RaceLog
  RaceJob->>race_parts.py: Provide test list, weights, and part index
  race_parts.py-->>RaceJob: Return tests assigned to the part
  RaceJob->>GoTestProcess: Run assigned race tests
  GoTestProcess->>RaceLog: Write test results
  RaceJob->>race_parts.py: Check race log against weights
  race_parts.py->>RaceLog: Read test results
  race_parts.py-->>RaceJob: Return unweighted-test check result

Merge Risk: Low · previous review 1e0c5

The race job remains mergeable with bounded follow-up: handle the test helpers’ parse errors and make the mutation specifically check missing-weight validation.

Pre-merge checks: 11 · 3

Failed checks: 3 warnings

Check name	Explanation	Resolution
Desktop Robustness	The new `.github/scripts/race_parts.py` can crash on a corrupted log. `TOP` accepts any sequence of digits and periods as the duration (`[0-9.]+`), but `check_unlisted()` calls `float(found.group(2))`…	Validate the captured duration with a strict numeric pattern before conversion, or catch `ValueError` around both duration conversions and call `refuse()` with the log path and line number. Add a test for a malformed matching log line and r…
Safe File Parsing	The new `.github/scripts/race_parts.py` has unsafe parsing paths. `check_unlisted()` applies `TOP = re.compile(... ([0-9.]+)s)` and then calls `float(found.group(2))` without handling conversion error…	Use a strict duration grammar such as `[0-9]+(?:\\.[0-9]+)?`, and wrap duration conversion in `try/except (ValueError, OverflowError)`; call `refuse()` for malformed or non-finite values. Replace `open(...).read().splitlines()` with bounded …
Clear User-Facing Text	The new CLI error path exposes a raw exception. In `.github/scripts/race_parts.py:42-47`, `read_lines` formats `err` directly into `cannot read ...: %s`, so missing, unreadable, or invalid-UTF-8 input…	Replace the raw exception interpolation with a direct message such as: `cannot read the <what> file <path>. Check that it exists, is readable, and uses UTF-8.` Apply this for both `OSError` and `UnicodeDecodeError`; keep exception details o…

Passed checks: 11 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly describes the main change: the race job partitions guard tests by their measured weights. It is specific and not vague.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Tests For Changed Behavior		The PR adds `internal/guard/raceparts_test.go` with coverage for weighted partitioning, part balance, stale/duplicate/invalid weights, unlisted-test runtime checks, and CI wiring. It also adds mutatio…
No Secrets Or Debug Leftovers		The diff adds no private AI-agent files or `.env` files. Added content contains no credentials, tokens, API keys, URLs, e-mail addresses, IP addresses, or local absolute paths. The AMD EPYC references…
No Hardcoded Ui Styling		The pull request changes CI configuration, race-partitioning scripts, weight data, and Go tests. It does not add or change GUI code such as XAML, Slint, Fyne, Tkinter, or WPF code-behind. The hardcode…
No Obvious Performance Problems		No clear performance problem is introduced. The new splitter performs linear name processing plus sorting the 48 weighted tests and a constant-size part selection; it has no UI-thread work or per-item…
System Changes Are Reversible		PASS: The pull request does not add or change code that modifies network filters, proxies, firewalls, system time, process hooks/injection, Windows services, registry, or drivers. The new Python scrip…
No Resource Leaks		No resource leak is introduced. `.github/scripts/race_parts.py` opens files with `with open(...)`, so files close on success and errors. The added Go tests run child processes synchronously with `Run(…
Scope, Duplication And Docs		PASS. The diff is limited to the stated race-job weighting feature: the new weights file and script, CI integration, and their tests and mutation entries. The PR description covers the new files, comm…

Full details: Tests For Changed Behavior

Explanation

The PR adds internal/guard/raceparts_test.go with coverage for weighted partitioning, part balance, stale/duplicate/invalid weights, unlisted-test runtime checks, and CI wiring. It also adds mutation entries for the new behavior. No existing test files were deleted, skipped, or weakened in the PR diff.

Full details: No Secrets Or Debug Leftovers

Explanation

The diff adds no private AI-agent files or .env files. Added content contains no credentials, tokens, API keys, URLs, e-mail addresses, IP addresses, or local absolute paths. The AMD EPYC references are processor models used in CI timing comments, not private machine or user names. The Python print calls produce required splitter output and errors; no debug-only print, debugger, breakpoint, or equivalent remains.

Full details: No Hardcoded Ui Styling

Explanation

The pull request changes CI configuration, race-partitioning scripts, weight data, and Go tests. It does not add or change GUI code such as XAML, Slint, Fyne, Tkinter, or WPF code-behind. The hardcoded UI styling check is not applicable.

Full details: No Obvious Performance Problems

Explanation

No clear performance problem is introduced. The new splitter performs linear name processing plus sorting the 48 weighted tests and a constant-size part selection; it has no UI-thread work or per-item external process. The log check scans the part log once. The CI change replaces the prior per-test cksum process loop with one Python invocation per matrix part.

Full details: System Changes Are Reversible

Explanation

PASS: The pull request does not add or change code that modifies network filters, proxies, firewalls, system time, process hooks/injection, Windows services, registry, or drivers. The new Python script reads files and calculates test partitions. The workflow changes only select and validate race-test partitions and logs; no targeted system-state change is introduced.

Full details: No Resource Leaks

Explanation

No resource leak is introduced. .github/scripts/race_parts.py opens files with with open(...), so files close on success and errors. The added Go tests run child processes synchronously with Run() or Output(), and temporary files use t.TempDir(), which the test framework removes. The CI changes add no persistent handlers, timers, goroutines, queues, or unmanaged handles.

Full details: Scope, Duplication And Docs

Explanation

PASS. The diff is limited to the stated race-job weighting feature: the new weights file and script, CI integration, and their tests and mutation entries. The PR description covers the new files, commands, allocation rules, failure rules, timing basis, and CI watch paths. The repository's CHANGELOG states that it records changes users notice; this PR changes internal CI behavior and does not change the product, public CLI, API, persisted data, or build requirements. Searches of the base revision found no existing race-partition helper or duplicate implementation.

Full details: Desktop Robustness

Explanation

The new .github/scripts/race_parts.py can crash on a corrupted log. TOP accepts any sequence of digits and periods as the duration ([0-9.]+), but check_unlisted() calls float(found.group(2)) without handling ValueError. A line such as --- PASS: TestX (1.2.3s) therefore raises an unhandled exception instead of reporting a controlled input error. Missing files and malformed weight values are handled, but this malformed log case violates the robustness check.

Resolution

Validate the captured duration with a strict numeric pattern before conversion, or catch ValueError around both duration conversions and call refuse() with the log path and line number. Add a test for a malformed matching log line and require a non-zero controlled error without a traceback.

Full details: Safe File Parsing

Explanation

The new .github/scripts/race_parts.py has unsafe parsing paths. check_unlisted() applies TOP = re.compile(... ([0-9.]+)s) and then calls float(found.group(2)) without handling conversion errors. A malformed log line such as --- PASS: Test (1.2.3s) matches the regex and raises ValueError instead of producing a controlled refusal. read_lines() also uses f.read().splitlines() with no size limit, so a huge names, weights, or log file can exhaust memory. The PR introduces this script and invokes it in ci.yml, so the defect is PR-caused.

Resolution

Use a strict duration grammar such as [0-9]+(?:\.[0-9]+)?, and wrap duration conversion in try/except (ValueError, OverflowError); call refuse() for malformed or non-finite values. Replace open(...).read().splitlines() with bounded streaming reads, enforcing maximum file and line sizes before processing. Also cap parts before allocating [0.0] * parts.

Full details: Clear User-Facing Text

Explanation

The new CLI error path exposes a raw exception. In .github/scripts/race_parts.py:42-47, read_lines formats err directly into cannot read ...: %s, so missing, unreadable, or invalid-UTF-8 input shows Python exception text instead of a clear user action.

Resolution

Replace the raw exception interpolation with a direct message such as: cannot read the <what> file <path>. Check that it exists, is readable, and uses UTF-8. Apply this for both OSError and UnicodeDecodeError; keep exception details out of the user-facing message.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Advanced
Run ID: fc9bf2bb-1d1a-4973-b288-92f4bb3b1539

Commits

Reviewing files that changed from the base of the PR and between eceea37 and 1e0c50a.

Files selected for processing: 5

.github/race-weights
.github/scripts/race_parts.py
.github/workflows/ci.yml
internal/guard/mutation/entries.py
internal/guard/raceparts_test.go

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Recent review details

Context from checks skipped due to timeout: 18

GitHub Check: race detector (part 1 of 4)
GitHub Check: race detector (part 2 of 4)
GitHub Check: race detector (part 3 of 4)
GitHub Check: race detector (part 0 of 4)
GitHub Check: part 0
GitHub Check: CodeRabbit
GitHub Check: test on macos-latest
GitHub Check: bill of materials
GitHub Check: linters
GitHub Check: semgrep
GitHub Check: test on windows-latest
GitHub Check: test on ubuntu-latest
GitHub Check: reference tools actually installed
GitHub Check: coverage gate
GitHub Check: known vulnerabilities
GitHub Check: the Chocolatey packages install and leave
GitHub Check: staticcheck
GitHub Check: Analyze (go)

Additional context used

Path-based instructions: 12

Applies to text shown to the user (labels, buttons, tooltips, placeholders, dialogs, errors, status messages, empty states, translations).