Repository navigation
Regenerate the shard durations so shard 4/8 stops hitting the time limit - #2907
Conversation
Python shard 4/8 needs 86 to 90 minutes against the job's 90-minute limit. On 11 November the merge queue cancelled it at the limit in four of the six groups that held #2016, and the other two finished in 88.6 and 89.9 minutes. It also hit the limit in two groups without #2016. No test ran away: the 24 October table had drifted. Its 551 tests took 1.31 times their listed seconds and came to 50% of the dealt shards' time, not the three quarters the header gives, and test_mixed_age_couple_invariants alone grew from 498 to 1,044 seconds. .github/test_durations.py --min-seconds 20 regenerates the table from shards 1/8 to 7/8 of pull request run 42936378851, keeping only tests defined on main. Its 987 entries cover 73% of the work. Replaying those logs, the slowest dealt shard falls from 86.8 to 75.3 minutes with that run's times, and from 91.8 to 77.2 with the queue runs' times. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
05e3232 to
768cc49
Compare
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target. Checks failing on both base and head (1)
Counted-once checkThe check raised each of 92 components by £1 per unit in 2026-27 and read every head of its totals household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A response that is not finite breaks every rule. The check measures a head that breaks a rule twice more: once holding every other head at its baseline value, and once holding every other head at its value after the first £1 while the component rises again. The head fails where either measurement still shows the response. The check cannot hold a head at a value that is not finite. In each unit where another head is not finite after the first £1, it holds that head at its baseline value, and the second measurement judges no household the unit belongs to. There a head fails wherever it first broke a rule, or wherever the check could not confirm whether its response contains the component. 0 failed and the check skipped 0.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
juaristi22
left a comment
There was a problem hiding this comment.
Review status: COMPLETE
GitHub disposition: COMMENT — approval deferred pending CI
Still-open critical count: 0
Reviewed 768cc49b82809b313981498ae863b255a1e8fe9a; no findings or material evidence gaps.
- All 987 duration entries exactly reproduce from the official source run, using the generator and current-main test filter.
- 54 focused shard tests passed (zero failures/skips), including partition coverage, fallback placement, dataset isolation and real pytest/xdist outcome checks.
- Model formulas, test assertions and the scheduler are unchanged. The runtime forecasts remain estimates.
Seven Python CI shards and Aggregate impact were still running at the final capture. I am withholding approval until the required checks pass at this exact head; a later CI-only check can finish the approval without repeating this code review.
There was a problem hiding this comment.
Review status: COMPLETE
Review severity: APPROVE
Still-open critical count: 0
Approving unchanged head 768cc49b82809b313981498ae863b255a1e8fe9a after the required Lint and Test / Result checks passed. All eight Python shards and the YAML job passed.
The previous complete review remains valid: 54 focused tests passed, all 987 duration entries matched the official logs, and no findings or material evidence gaps remain. No code tests or model calculations were repeated.
All current CI checks, including Aggregate impact, have now passed at this exact head. This PR changes only shard timing data and its changelog.
#2907 regenerated shard_durations.toml from 11 November's CI logs. Take main's table and drop its line for test_carers_allowance_earnings_limit_properties.py::test_net_earnings_never_fall_as_pay_rises, which this PR replaces with test_net_earnings_never_fall_for_a_rise_of_two_pence_or_more. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python shard 4/8 sits at its 90-minute limit, so the merge queue cancels it on ordinary runner speed. This PR regenerates
policyengine_uk/tests/shard_durations.toml, as its header asks "when a shard drifts well above the others". It changes no test and no model code.What happened on 11 November
test_mixed_age_couple_invariantsalone grew from 498 to 1,044 seconds.The new table
.github/test_durations.py --min-seconds 20built it from shards 1/8 to 7/8 of run 42936378851 (Count other families' children and add the foster and overnight-care bedrooms in the LHA size criteria #2016's own CI, early on 11 November), keeping only tests defined on main.conftest.py's deal. With that run's times the slowest dealt shard falls from 86.8 minutes, as the run logged, to 75.3. With the merge-queue runs' times for the 59% of the work their saved logs cover, it falls from 91.8 to 77.2. These are minutes of tests; the job adds about 1.4. Where a long test falls in a shard's order can still move a shard by a few minutes.Open PRs that touch the table
test_2030_31_rise_changes_only_holders_income_tax_and_class_1a, which the new table lists. It merges without a text conflict, buttest_the_checked_in_shard_durations_name_tests_that_existthen fails in its queue group. Once this lands, Include Class 1A NICs in employer NI incidence #2823 needs main merged in and that line renamed totest_static_2030_31_rise_changes_only_holders_income_tax_and_class_1a[2030].Checks
test_ci_test_shards.pyruns in CI and fails if an entry names a missing test.axiom: n/a: CI test sharding data only
🤖 Generated with Claude Code