Repository navigation
Conversation
Shards 1/8 to 7/8 took 70 to 89 minutes in 14 recent runs, against a 90-minute limit, and the limit cancelled a shard in 5 of them. Nine shards left little room on #2670's and #2840's branches; ten shards on #2840 took 62 to 73 minutes. Shard 10/10 runs the tests that need the dataset, with one worker, as shard 8/8 did. The Makefile, the testing skill, the code-changes workflow and the Ubuntu versus macOS page now give the new count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
left a comment
There was a problem hiding this comment.
Review status: COMPLETE at dbca2c6bec2133cde8669887fb03502ea66f9a4e — no findings or material evidence gaps. Approval is deferred while exact-head CI is unfinished.
The ten-shard wiring preserves the dataset-only last shard with one worker, dynamically numbers ordinary shards 1–9, and leaves the Result gate and required context names unchanged. The release-wheel workflow reuses the same ten-shard suite.
57 focused tests passed (54 existing tests plus three N10 diagnostics), zero failures. A real 53-node pytest sample was assigned exactly once across all ten current conftest shards, with dataset nodes only on shard 10. The actual Result checker accepted complete records and rejected missing/duplicate logs. The weighted and unknown-node fallback paths were also checked. These are routing/gate tests, not a claim that the whole country suite has finished.
Independent technical-source review preserved 29 official GitHub originals covering all eleven cited runs, six explicit 90-minute timeout annotations and the applied branch rules. The cited cancellation examples and rounded nine-/ten-shard timings are corroborated. The broader fourteen-run/81-minute statistic remains attributed rather than independently reconstructed; the review does not promise a future runtime bound.
No country-model formulas, tests, dependency lock or sharding algorithm change. Country-law review is not applicable; no private-data or population run was used. Required current-head checks, including Result and Lint, must pass before approval. Pending CI can be checked later without repeating this completed review.
juaristi22
left a comment
There was a problem hiding this comment.
Approved at dbca2c6bec2133cde8669887fb03502ea66f9a4e after the CI-readiness follow-up.
The completed independent code/source review remains applicable: 57 focused tests passed, including actual ten-shard collection and missing/duplicate outcome-gate checks; no findings or material gaps remain. The commit and code are unchanged, so no tests or model work were repeated.
Current official checks pass at this exact head, including required Lint and Test / Result; the PR is ready and mergeable. The added CI-results paragraph is supported by the two linked run payloads: test steps round to 59–74 minutes (dataset shard 22), and release test steps to 57–69. The current check set has 43 successful contexts plus a skipped Publish context. These measurements remain observations, not a future runtime guarantee.
|
Thanks, María. An independent review of |
|
Thanks, María. An independent review of dbca2c6 approved it, with no finding above P3. Its one P3 predates this PR. In The reviewer couldn't reach GitHub, so I checked each run this PR cites. Every cancellation and timing matches its run within a minute. Shard 3/8 hit the 90-minute limit in 43617721320, 43805734007 and 43814986259, and shard 7/8 in 43752493693 and 43619418570. On #2670, shards 1/9 to 8/9 averaged 72 and 76 minutes (43715128471, 43796151601), and on #2840 shards 1/10 to 9/10 took 62 to 73 (43805104592). |
Shards 1/8 to 7/8 of the Python tests now take 70 to 89 minutes against their 90-minute limit, and the limit cancelled a shard in 5 of the 14 recent runs counted below. This PR spreads the tests over ten shards. It changes no test and no model code.
What happened from 14 to 16 November
Test / Resultfailed.Why ten and not nine
The change
test-suite.yamlruns ten shards. Shards 1 to 9 share the tests that do not need the dataset. Shard 10 runs the tests that do, with one worker, as shard 8 did.Makefile,docs/engineering/skills/testing.md,pr_code_changes.yamlanddocs/book/validation/ubuntu-vs-macos.mdnow give the new count. The last two had said seven.shard_durations.tomlstay as they are. If a shard nears the limit on main, the next step is to regenerate the durations table from ten-shard logs with.github/test_durations.py.Checks
Test / ResultandLint, so the renamedTest / Python K/10checks need no settings change.test_ci_test_shards.pychecks that the matrix numbers its shards from 1 to N and gives the last shard one worker. Both still hold. Four of its properties draw at most eight shards; Let the shard-assignment properties draw up to 16 shards #2920 tracks raising that to 16.test-suite.yaml, the release workflow also builds and tests the wheel on it.CI at
dbca2c6beOpen PRs that touch the same lines
testing.md, word for word, so it merges after this without a conflict there.test-suite.yaml, and two intesting.md. The two PRs conflict there in either order, and ten should win. If Fix structural reforms for their requested periods #2670 lands first, this branch keeps its own side of those lines when it merges main.axiom: n/a: CI test sharding only
🤖 Generated with Claude Code