Skip to content

Guard against random draws in the engine - #2873

Merged
juaristi22 merged 4 commits into
mainfrom
claude/deterministic-engine-guard-o2b5an
Nov 12, 2026
Merged

juaristi22 merged 4 commits into
mainfrom
claude/deterministic-engine-guard-o2b5an

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Addresses #2874
axiom: n/a: tests only

Why

policyengine-core bans randomness in variable formulas, so the data build has to draw each stochastic input and store it (PolicyEngine/policyengine-core#500). Core's own check looks for numpy and stdlib randomness in formulas, but it misses the splitmix64 hash of entity ids in policyengine_uk/utils/stochastic.py and the seeded numpy draws outside formulas. This PR adds policyengine_uk/tests/code_health/test_no_engine_time_randomness.py, which fails on any new draw. It lists today's draws as a ratchet and the draw in Simulation.subsample as a sampling tool. Later PRs move each ratchet draw into the data build and lower the ratchet. This PR changes no model code and no output.

Where the guard comes from

The file ports the scanner of PolicyEngine/policyengine-us#10447 at 221f2fe44ff265b8911810bcc440b3586fa6fc51 (policyengine_us/tests/code_health/test_no_engine_time_randomness.py, git blob 15b6826c0ebecbec84c6b51f128e5f76226e0e47). The scanner, from the constants down to package_scan, and every generic detector test stay as they were. The port changes only:

  • the package name, policyengine_us to policyengine_uk;
  • RATCHET, RATCHET_CEILING, EXEMPT and OFFLINE_MODULES (empty: the scan finds no offline tool that needs one, so test_model_never_imports_offline_modules passes trivially until a module joins it);
  • SAMPLING_TOOLS, a third list that the pe-us guard does not have, and _summed, which adds two lists' counts per key;
  • the two package tests, test_package_draws_nothing_outside_the_allowlists and test_allowlist_entries_still_match. The pe-us guard passes RATCHET to check_allowlists as its first list, and these pass RATCHET summed with SAMPLING_TOOLS;
  • four new tests for the sampling tool: test_the_subsample_tools_functions_are_draw_helpers, test_a_use_of_a_sampling_tools_function_from_another_scope_fails, test_sampling_tools_lists_only_the_subsample_tool_and_no_key_is_in_two_lists and test_no_package_module_uses_the_subsample_method;
  • the tests that named pe-us modules: test_model_api_passes_on_cores_numpy_and_its_own_helpers now checks a pe-uk variable module and add_for_members, which policyengine_uk/model_api.py passes on; test_splitmix64_hash_and_its_uprating_caller_are_draw_helpers replaces the pe-us start-month test; the offline-import tests check an example name of their own, policyengine_uk.tools.offline, since pe-uk has no tools package;
  • the docstring's and comments' pe-us examples, its sentence on how the dataset build draws inputs, a new limit on the clock, and a sentence on SAMPLING_TOOLS;
  • a module-scoped autouse fixture that clears package_scan's cache when the module's tests finish, as test_test_module_caches_released.py requires of every test module;
  • the four pytest.skip calls for cases that need newer syntax. _on_this_python filters those cases as pytest collects them, and two tests whose cases all need Python 3.12 sit under if sys.version_info >= (3, 12), so no Python collects a case it cannot parse and no run reports a skip. Every case stays.

The PR also adds test_the_draw_neither_reads_nor_moves_the_global_random_streams to policyengine_uk/tests/test_subsample.py.

The lists

EXEMPT holds 21 digest occurrences in 9 files. Each computes a SHA-256 for provenance, cache keys, content checks or an equality check, and none decides who gets something or meets a threshold:

File Scopes
build_metadata.py get_data_build_fingerprint
data/capital_gains_projection.py _fingerprint
data/dataset_sources.py _cached_dataset_path
data/economic_assumptions.py uprating_input_fingerprint
utils/aggregate_impact/__main__.py _sha256
utils/aggregate_impact/dataset.py sha256_file
utils/aggregate_impact/measure.py _digest_tool_files, definitions_digest, model_fingerprint, runtime_identity
utils/aggregate_impact/partition.py load_partition
utils/build_lha_published_rates.py fetch

each with its module's hashlib import.

RATCHET holds 21 occurrences in 10 files, so RATCHET_CEILING = 21:

  • utils/stochastic.py: splitmix64_uniform, the three splitmix64 constants and three xorshift steps;
  • uc_deduction_random_draw.py, uc_deduction_type_random_draw.py, warm_home_discount_random_draw.py and months_since_last_birthday.py: one formula call each to splitmix64_uniform;
  • data/economic_assumptions.py: uprate_student_loan_plans hashes person_id and compares the draw with the take-up rate; apply_single_year_uprating, apply_uprating, extend_single_year_dataset and Simulation._extend_dataset_with_cgt_provenance in simulation.py reach it;
  • dynamics/participation.py: a seeded numpy RandomState in apply_participation_responses;
  • scenarios/uc_reform.py: a seeded numpy Generator in add_universal_credit_reform, and the module-level Scenario that passes it on;
  • scenarios/pip_reform.py: np.random.seed and np.random.random in modify_simulation, and the module-level Scenario that passes it on.

SAMPLING_TOOLS holds the 8 occurrences of one sampling tool in 2 files. Simulation.subsample draws households when a caller calls it, and the draw decides which households the simulation keeps and the weights they carry. It draws from a generator of its own, which it seeds from the caller's seed or the fixed default "policyengine-uk":

  • utils/subsample.py: the hashlib import; _random_state, which seeds a numpy RandomState from the SHA-256 digest of the seed's text, and its return annotation; the choice call in draw_household_counts and that function's call to _random_state; and subsample_simulation's call to draw_household_counts;
  • simulation.py: Simulation.subsample's call to subsample_simulation.

draw_helpers reads EXEMPT alone, so EXEMPT entries would stop _random_state, draw_household_counts and subsample_simulation counting as draw helpers, and a formula could then call _random_state with no finding. RATCHET entries would need RATCHET_CEILING to rise from 21 to 29, which the file says never to do, and RATCHET lists draws that later PRs move into the data build; #2874 names no such step for the subsample. Under SAMPLING_TOOLS the three functions stay draw helpers, and RATCHET, RATCHET_CEILING and EXEMPT keep their values.

The scan over main at 74db8b6793b13810c52e5d33f2bb3437adb79046 reads 1,616 modules, cuts no lookup at the depth limit and finds no other draw pattern. Without EXEMPT, the scan reports 90 occurrences in 22 files: the 21 in RATCHET, the 21 digests, the 8 in SAMPLING_TOOLS, and 40 draw helper calls that reach the digests.

Invariants

  • The guard counts every occurrence, per file, enclosing scope and kind.
  • An occurrence beyond its entry fails test_package_draws_nothing_outside_the_allowlists, including a second call in a scope the lists already name and a new function in a listed file.
  • An entry that allows more than the scan finds fails test_allowlist_entries_still_match, and test_ratchet_only_shrinks fails when RATCHET exceeds RATCHET_CEILING. So RATCHET can't grow unless a change also raises RATCHET_CEILING, which the file says never to do.
  • A use of _random_state, draw_household_counts or subsample_simulation counts as a draw helper call in its scope, under any name the code imports the function as. So a use in a scope with no draw helper call entry fails test_package_draws_nothing_outside_the_allowlists, and so does a use beyond a listed scope's entry. test_the_subsample_tools_functions_are_draw_helpers fails if a change takes any of the three out of the draw-helper set, as EXEMPT entries for the tool would.
  • SAMPLING_TOOLS can name only occurrences in utils/subsample.py and the one call in Simulation.subsample, and no (file, scope, kind) key can sit in two of the three lists. test_sampling_tools_lists_only_the_subsample_tool_and_no_key_is_in_two_lists fails otherwise.
  • No package module outside tests can call or read an attribute named subsample: test_no_package_module_uses_the_subsample_method fails on one.
  • draw_household_counts gives the same counts for a seed whatever numpy's and the standard library's global random streams hold, and leaves both streams as they were (test_the_draw_neither_reads_nor_moves_the_global_random_streams).

The guard reports recognised patterns, so it works as a ratchet, not a proof that no hand-written draw exists. It inherits every limit the docstring lists: it does not follow eval or exec, imports of a computed name, values reached through objects or containers, or methods as draw helpers. For example, Simulation calls universal_credit_july_2025_reform.simulation_modifier(self) through an object, which the scan does not follow. The guard does not scan policyengine-core, so it cannot show that nothing downstream reads numpy's global stream. It also looks for draws and hashes, not reads of the clock (#2875). policyengine_uk/tests/code_health/test_formulas_do_not_read_the_clock.py scans the modules under variables for those.

Three limits apply to the sampling tool:

  • The scan cannot see a call of the method, such as simulation.subsample(n=1000) in a scenario. The test on the attribute name catches that spelling and misses getattr.
  • The lists count occurrences, so they cannot show that the tool seeds its generator or leaves the global streams alone. A _random_state that returns an unseeded RandomState changes no count. policyengine_uk/tests/test_subsample.py tests that behaviour: test_the_same_seed_keeps_the_same_households for the seed and the new test for the global streams.
  • The lists count draw helper calls per scope, not which helper a scope calls. A scope that holds a draw helper call entry, such as each of the four ratcheted formulas, can swap its call for one of the tool's functions with no finding.

SAMPLING_TOOLS has no ceiling. A PR that lists a draw from another file there has to change the test that limits the list to the subsample tool.

Interaction with open work

main gained utils/subsample.py and Simulation.subsample through #2308 on 9 November. This PR merges main and lists the tool's eight occurrences in SAMPLING_TOOLS.

The file says never to raise RATCHET_CEILING, so neither of these PRs can add a RATCHET entry. The guard fails each of them until the PR changes or, for #2862, adds an EXEMPT entry:

  • Break wealth-decile ties by a fixed hash of household_id #2618 adds _fixed_hash in variables/household/wealth/household_wealth_decile.py, a copy of the splitmix64 mix that orders tied households and so decides which decile each falls in: three hash constant, three xorshift, and a draw helper call from the formula. It needs a tie-break without a hash, or a stored draw.
  • Reuse prepared current-law baselines for local H5 scenarios #2862 adds a SHA-256 digest in _core_fingerprint in utils/prepared_baseline.py, which identifies policyengine-core's code and decides no household's outcome. An EXEMPT entry for utils/prepared_baseline.py, ("<module>", "hash library"): 1 and ("_core_fingerprint", "hash use"): 1, clears all ten occurrences the guard reports.

Impact

Impact: none; tests only.

Tests

On commit 697529ba571a36860cc9b1cefadb11a8831c49af, pytest (Python 3.12.14, policyengine-core 3.34.9, Hypothesis 6.170.15) passes all 386 tests in the guard file. On the commit before it, which differs only in one comment line of the guard file, policyengine_uk/tests/test_subsample.py gives 60 passed and 2 failed, the new test's two cases among the passes. The two failures, test_where_capital_gains_inputs_came_from_is_kept and test_a_neutralised_capital_gains_input_still_reads_as_its_default, look up the installed policyengine-uk distribution, and that run used an exported tree with no install. They fail the same way on main's tree in the same environment. CI on the pushed head runs both files on an installed package.

These checks ran on 697529ba5 with Python 3.12.14. Each loads the guard file by its path and calls its functions, and none starts pytest or imports policyengine_uk:

  • The guard's own scan (package_scan, then check_allowlists with the three lists): 1,616 modules, no draw pattern outside the lists, no entry that allows more than the scan finds, no lookup cut at the depth limit, 11 draw helpers. With RATCHET as the first list on its own, the scan reports the tool's eight occurrences.
  • Fourteen of the file's tests, called as plain functions, pass: the two package tests above, the four new tests, five other tests that read the package scan or the lists, and three fixture tests of the lists.
  • Removing any one of the eight SAMPLING_TOOLS entries makes test_package_draws_nothing_outside_the_allowlists fail with one finding. Raising any one to 2 makes test_allowlist_entries_still_match fail with one entry.
  • Nine probe modules, added to the scan in memory: the scan reports the six that use one of the tool's three functions (called directly, in a Variable formula, under an alias and through policyengine_uk.simulation) and a control that draws from numpy directly. It misses a scenario that calls simulation.subsample(frac=0.1), which test_no_package_module_uses_the_subsample_method reports, and a getattr with a computed name on a dynamic import, which nothing reports.
  • With the tool's five base occurrences moved into EXEMPT, the draw helpers fall from 11 to 8 and test_the_subsample_tools_functions_are_draw_helpers fails.

The new test_subsample.py test also passes when called on the source of _random_state and draw_household_counts, compiled outside the package with numpy 2.5.4, and fails on six edits of that source that read or move a global stream.

ruff format --check and ruff check (ruff 0.17.2, the version uv.lock pins) pass on both changed test files at 697529ba5.

🤖 Generated with Claude Code

Port policyengine-us's scanner for engine-time randomness and hash draws
as a code-health test. EXEMPT lists the package's SHA-256 digests, which
fingerprint, key caches or check content; RATCHET lists today's draws:
the splitmix64 helper, the formulas and uprating that call it, and the
seeded numpy draws in participation and two scenarios. Cases that need
newer syntax are filtered at collection, so no Python reports a skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6H1FBndAKeuXdSGuft
@MaxGhenis

MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Head: 0835944512f8632f878168bbe1d2eceba951500d on claude/deterministic-engine-guard-o2b5an (the harness pins this branch name), from main at 47a2817e2d57e6acbc08b76e09fde89fc27b81fa. main had not moved when I opened the PR.

Pre-flight

Comparison with the reference (after formatting)

diff -u shows 13 hunks. Each falls under a listed item, except one:

  1. The docstring (lines 2–15): I dropped "the months a reported annual amount falls in" (a pe-us example) and changed "and tools" to "scenarios, simulation code, … and utilities". Item 4.
  2. Line 61, policyengine_uk.model_api import np as lib. Item 1. pe-uk's model_api star-imports core's, so the example holds.
  3. The lists. Item 2.
  4. Outside the items: _release_package_scan, a module-scoped autouse fixture that calls package_scan.cache_clear() after its yield (just below package_scan). pe-uk's test_test_module_caches_released.py::test_every_test_module_cache_is_released failed without it: every test module must clear its module-level functools caches. The cost is that a worker may rebuild the scan (about 8 s) if it leaves this module and comes back to it.
  5. _on_this_python, plus the re-wrapped parameter lists, the type-alias test and the type-parameter test. Item 5. diff -w shows only the wrapper call, the removed skips and the two if sys.version_info >= (3, 12): blocks.
  6. The mcdonald_rivet comment. pe-uk has no package whose __init__ rebinds a subpackage's name (I checked every __init__.py), so I dropped the parenthetical rather than invent an example.
  7. test_model_api_passes_on_cores_numpy_and_its_own_helpers now uses policyengine_uk.variables.gov.dwp.tax_credits_current_year_income (it star-imports model_api and calls add_for_members) and checks for policyengine_uk.utils.benefit_unit.add_for_members.
  8. test_splitmix64_hash_and_its_uprating_caller_are_draw_helpers replaces the start-month test. It checks the three names the brief gives.
  9. The offline tests: _is_offline(name, offline=OFFLINE_MODULES), plus EXAMPLE_OFFLINE_MODULES = {"policyengine_uk.tools.offline"}, and the import-form cases rewritten from taxsim to offline. OFFLINE_MODULES = set(), so test_model_never_imports_offline_modules passes on an empty set. I kept it so the check is ready if a later change adds an offline module.

The scanner from the constants down to package_scan is unchanged, apart from the package name.

Lists, reconciled

My run on main at 47a2817e: 1,534 modules, no lookup cut at the depth limit. It reports exactly the brief's 21 EXEMPT occurrences in 9 files and 21 RATCHET occurrences in 10 files, entry by entry, so RATCHET_CEILING = 21. The scan finds nothing else.

I read every function behind the digests. All are SHA-256 used for provenance, a cache key or a content check, and none decides who gets something or meets a threshold. The two that the code compares are:

  • uprating_input_fingerprint (data/economic_assumptions.py:302): simulation.py:1101 and :1168 compare it with != to decide whether a cached projection still applies.
  • fetch (utils/build_lha_published_rates.py:102) and verify_file (utils/aggregate_impact/dataset.py:131): both compare a digest with an expected value and raise if they differ.

The ratchet draws I read:

  • utils/stochastic.py:19–22: three constants and three xorshift steps.
  • The four formulas:
    • uc_deduction_random_draw.py:28 (salt 0)
    • uc_deduction_type_random_draw.py:27 (salt 1)
    • warm_home_discount_random_draw.py:29 (household_id, salt 2)
    • months_since_last_birthday.py:45 (person_id, salt 2)
  • data/economic_assumptions.py:929, salt 3, with draws < prob. Its callers are at lines 519, 389 and 363, and at simulation.py:951.
  • dynamics/participation.py:599: np.random.RandomState.
  • scenarios/uc_reform.py:311: default_rng(43), compared with new_claimant_share. The module-level Scenario is at line 358.
  • scenarios/pip_reform.py:7–8: np.random.seed(42) and np.random.random. The Scenario is at line 23.

One possible difference from the brief: without EXEMPT, my run reports 82 occurrences in 21 files, not 80. That count covers every occurrence, including the 21 that RATCHET covers; 61 occurrences in 13 files go beyond RATCHET. I couldn't reproduce 80 under any count.

Observations, not acted on:

  • warm_home_discount_random_draw and months_since_last_birthday both use salt 2. They hash different ids (household_id and person_id).
  • simulation.py:416 calls universal_credit_july_2025_reform.simulation_modifier(self). That is a call through an object, which the scan doesn't follow. The PR body names it as an example of the limits.

Star imports in a possible cycle

The scanner's own graph shows 9 of 1,432 star imports (1,418 of them from package modules) in a possible cycle:

  • 3 importers of model_api
  • entities
  • the system imports in tax_benefit_system, microsimulation and simulation
  • reforms.policyengine.disable_simulated_benefits
  • reforms.conservatives.household_based_hitc

policyengine_uk/__init__.py imports system while it runs. The lazy imports in modules that run during that import put them in a cycle:

  • data/economic_assumptions.py:237, 375, 660: from policyengine_uk.system import system inside functions.
  • simulation.py:416, 488 and utils/scenario.py:224, 234, 338, 739: scenarios/scenarios.uc_reform inside functions.
  • reforms/reforms.py: code there that may run later reaches reforms.policyengine.

Tests (all on 0835944)

  • uv sync --frozen --extra dev, then uv run pytest policyengine_uk/tests/code_health/test_no_engine_time_randomness.py -n 4 (Python 3.13.18): 382 passed.

  • Only pytest 9.2.0, --noconftest -p no:cacheprovider:

    Python Collected Passed Skipped Wall time Peak RSS
    3.11.18 369 369 0 18.8 s 56 MB
    3.12.3 380 380 0 22.7 s 59 MB
    3.13.18 382 382 0 20.3 s 59 MB
    3.14.6 382 382 0 18.8 s 65 MB

    3.11 leaves out the 13 cases that need 3.12 or 3.13 syntax; 3.12 leaves out the 2 that need 3.13.

    Caveat: the brief's bare uvx … pytest <file> --noconftest gives collection errors on every Python. pytest 9 sets up the policyengine_uk Package node above the file, which imports policyengine_uk/__init__.py, and that needs policyengine-core. I ran instead through a scratch plugin (-p skip_package_init) that turns _pytest.python.Package.setup into a no-op. The plugin is not committed. The guard itself imports only the standard library and pytest.

  • The guard file in the locked environment, serial, with conftest: 28.8 s wall, 316 MB peak RSS. Most of that memory comes from the conftest importing the model. The longest single call is the first package_scan (about 7 s; the scan alone takes about 7.8 s for 1,534 modules on 3.13).

  • The wheel: uv build --wheel (sha256 48570c87…), locked environment with --no-install-project, then uv pip install --no-deps the wheel, run from an empty directory against the installed tests directory: 382 passed. The scan reads the same 1,534 modules with identical counts and lines.

  • uv run pytest policyengine_uk/tests/code_health -n 4: 3,544 passed. Before I added the fixture, it gave 1 failed (test_every_test_module_cache_is_released) and 3,543 passed.

  • ruff format --check ., ruff check ., ruff check --select F811,F821 policyengine_uk (all ruff 0.17.2) and git diff --check pass.

Mutation checks (scratch copies; all killed)

  1. import random added to warm_home_discount_random_draw.py: killed. stochastic import (1 found, 0 allowed).
  2. A second splitmix64_uniform call in warm_home_discount_random_draw.formula: killed. draw helper call (2 found, 1 allowed).
  3. A new new_draws in data/economic_assumptions.py that calls splitmix64_uniform: killed. new_draws: draw helper call (1 found, 0 allowed).
  4. One ratchet occurrence removed (the module-level Scenario in pip_reform.py replaced): killed as stale by test_allowlist_entries_still_match.
  5. The xorshift entry raised from 3 to 4, which takes RATCHET to 22 against a ceiling of 21: killed by test_ratchet_only_shrinks and as stale.
  6. The _fingerprint EXEMPT entry removed: killed. The hash use and 7 draw helper calls spread through capital_gains_projection.py, simulation.py and tax_benefit_system.py.

Open work, checked

I ran the guard over each PR's head:

None has merged.

Judgment calls

  • No changelog fragment. check_changelog_fragments.py only warns when a PR adds none, a fragment would trigger a release for a tests-only change, and recent tests-only commits on main (abab81f8b, c78f88649, 8199bc05e, c9397c23e) add none. github-prs.md says every PR needs one, so the hub may want changelog.d/deterministic-engine-guard.changed.md.
  • I added the cache-release fixture (hunk 4 above) to meet pe-uk's own code-health rule.
  • The PR body has no Fixes line, because no issue exists for this work.

CI on 0835944 (checked 05:57Z)

All seven Test / Python shards, Test / YAML and Test / Result pass, so the strict outcome check found no new skip, xfail or XPASS. Changelog fragments, Changes, Documentation build, Fix evidence, Lint, the four install and smoke-import jobs and Release build also pass. Aggregate impact was still running at 05:57Z, after 73 minutes. No push was needed.

Skipped

Nothing needs private data. I didn't run the full suite; CI covers it.


Generated by Claude Code

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Aggregate impact on certified data: pass

Base 74db8b6793 against head 377f34afad, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

  • In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Caseloads

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target.

Checks failing on both base and head (1)
  • VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-52,314,332 in total; 2026-27: 76 units below zero, a weighted £-54,150,722 in total; 2027-28: 76 units below zero, a weighted £-56,043,426 in total
Counted-once check

The check raised each of 92 components by £1 per unit in 2026-27 and read every head of its totals household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A response that is not finite breaks every rule. The check measures a head that breaks a rule twice more: once holding every other head at its baseline value, and once holding every other head at its value after the first £1 while the component rises again. The head fails where either measurement still shows the response. The check cannot hold a head at a value that is not finite. In each unit where another head is not finite after the first £1, it holds that head at its baseline value, and the second measurement judges no household the unit belongs to. There a head fails wherever it first broke a rule, or wherever the check could not confirm whether its response contains the component. 0 failed and the check skipped 0.

  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)
Provenance
Side Commit Runtime Environment hash Partition hash Time
base main 74db8b6793 uk 2.178.2 (files 32d397dd1cb9), core 3.34.9, py 3.13.18 aarch64 d88544666480 f5dab8526157 2730.51s
head #2873 merged into main 377f34afad uk 2.178.2 (files 32d397dd1cb9), core 3.34.9, py 3.13.18 aarch64 d88544666480 f5dab8526157 2764.22s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Final charge on cash interest in non-cash ISAs £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Lifetime ISA withdrawal charge £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Income tax £294.44bn £315.20bn £335.79bn £351.25bn £366.92bn £385.31bn £399.96bn
National Insurance (employee and self-employed) £51.70bn £54.14bn £56.13bn £57.73bn £59.76bn £61.64bn £63.51bn
National Insurance (employer) £148.71bn £155.37bn £160.88bn £165.09bn £172.06bn £177.36bn £182.57bn
Employer NI change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Employer NI change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Capital gains tax £12.60bn £13.15bn £13.60bn £14.11bn £14.67bn £15.24bn £15.80bn
Student loan repayments £7.82bn £8.13bn £8.39bn £8.58bn £8.79bn £9.04bn £9.29bn
VAT excluding private school fees £242.22bn £249.47bn £258.67bn £264.86bn £270.95bn £277.04bn £283.21bn
Change in VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
VAT on private school fees £1.95bn £2.01bn £2.09bn £2.14bn £2.20bn £2.26bn £2.31bn
Change in private school VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Fuel duties £24.71bn £24.13bn £26.00bn £26.83bn £26.51bn £25.78bn £24.63bn
Alcohol duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tobacco duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Vehicle excise duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Betting and gaming duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
TV licence fee £4.26bn £4.40bn £4.41bn £4.42bn £4.43bn £4.44bn £4.45bn
Council tax £49.21bn £52.16bn £54.89bn £57.95bn £61.16bn £64.47bn £67.91bn
High value council tax surcharge £0.00bn £0.00bn £0.00bn £0.48bn £0.49bn £0.50bn £0.51bn
Northern Ireland domestic rates £0.55bn £0.58bn £0.60bn £0.63bn £0.66bn £0.69bn £0.72bn
Business rates £34.68bn £36.20bn £38.00bn £39.46bn £42.27bn £43.25bn £44.29bn
Stamp duty land tax on the household's own purchases £6.88bn £7.32bn £7.66bn £8.06bn £8.47bn £8.91bn £9.35bn
Stamp duty land tax paid by companies £4.72bn £4.57bn £5.29bn £6.03bn £6.37bn £6.92bn £7.45bn
Land transaction tax (Wales) £0.07bn £0.08bn £0.08bn £0.09bn £0.10bn £0.10bn £0.11bn
Land and buildings transaction tax (Scotland) £0.31bn £0.32bn £0.33bn £0.35bn £0.36bn £0.38bn £0.40bn
Unspecified tax change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Unspecified tax change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax excluding the main residence £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Land value tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Carbon tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Spending by benefit and service: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
State Pension £121.68bn £125.92bn £126.72bn £129.48bn £132.98bn £136.65bn £140.41bn
Pension Credit £6.21bn £6.46bn £6.68bn £6.82bn £6.89bn £7.07bn £7.22bn
Winter Fuel Payment £1.86bn £1.85bn £1.79bn £1.77bn £1.78bn £1.78bn £1.78bn
Pension Age Winter Heating Payment (Scotland) £0.19bn £0.20bn £0.20bn £0.21bn £0.21bn £0.22bn £0.22bn
Universal Credit £75.70bn £81.81bn £84.29bn £86.34bn £87.51bn £89.06bn £89.70bn
Housing Benefit £8.57bn £8.52bn £8.85bn £9.06bn £9.29bn £9.57bn £9.84bn
Council Tax Reduction £2.19bn £2.32bn £2.44bn £2.58bn £2.72bn £2.87bn £3.02bn
Employment and Support Allowance (income-related) £0.79bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Employment and Support Allowance (contribution-based) £5.20bn £5.37bn £5.56bn £5.69bn £5.82bn £5.95bn £6.09bn
Jobseeker's Allowance (income-based) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Jobseeker's Allowance (contribution-based) £0.29bn £0.30bn £0.31bn £0.31bn £0.32bn £0.33bn £0.34bn
Income Support £0.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Working Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Incapacity Benefit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Personal Independence Payment £26.95bn £28.03bn £29.05bn £30.09bn £30.75bn £31.51bn £32.21bn
Disability Living Allowance £7.49bn £7.79bn £8.07bn £8.36bn £8.55bn £8.76bn £8.95bn
Attendance Allowance £8.68bn £9.03bn £9.36bn £9.69bn £9.90bn £10.15bn £10.38bn
Carer's Allowance £4.48bn £4.66bn £4.83bn £5.00bn £5.11bn £5.27bn £5.42bn
Carer Support Payment (Scotland) £0.31bn £0.32bn £0.33bn £0.34bn £0.35bn £0.36bn £0.37bn
Scottish Carer Supplement £0.00bn £0.04bn £0.04bn £0.05bn £0.05bn £0.05bn £0.05bn
Severe Disablement Allowance £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn
Industrial Injuries Disablement Benefit £0.50bn £0.52bn £0.54bn £0.55bn £0.56bn £0.58bn £0.59bn
Armed Forces Compensation Scheme £0.61bn £0.63bn £0.65bn £0.66bn £0.68bn £0.69bn £0.71bn
War Widow's or Widower's Pension £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Benefit £16.19bn £16.82bn £17.39bn £18.00bn £18.39bn £18.84bn £19.25bn
Scottish Child Payment £0.49bn £0.52bn £0.53bn £0.55bn £0.56bn £0.57bn £0.56bn
Two Child Limit Payment (Scotland, counterfactual) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Healthy Start vouchers £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn
Maternity Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Maternity Pay £3.30bn £3.41bn £3.54bn £3.62bn £3.70bn £3.78bn £3.87bn
Statutory Paternity Pay £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Sick Pay £1.12bn £1.16bn £1.20bn £1.23bn £1.26bn £1.29bn £1.32bn
Sure Start Maternity Grant £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn
Bereavement Support Payment £0.67bn £0.69bn £0.71bn £0.73bn £0.75bn £0.76bn £0.78bn
Widowed Parent's Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tax-Free Childcare £0.56bn £0.57bn £0.57bn £0.58bn £0.60bn £0.61bn £0.59bn
Extended childcare entitlement £2.99bn £3.50bn £3.59bn £3.71bn £3.78bn £3.86bn £3.93bn
Universal childcare entitlement £2.06bn £2.22bn £2.29bn £2.37bn £2.43bn £2.48bn £2.53bn
Targeted childcare entitlement £0.50bn £0.51bn £0.52bn £0.54bn £0.55bn £0.57bn £0.56bn
Childcare Offer for Wales £0.26bn £0.28bn £0.28bn £0.29bn £0.30bn £0.31bn £0.32bn
Care to Learn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Childcare Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Parents' Learning Allowance £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn
Adult Dependants' Grant £0.06bn £0.06bn £0.04bn £0.04bn £0.04bn £0.04bn £0.04bn
Travel Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
NHS spending £203.89bn £204.30bn £204.56bn £204.91bn £205.34bn £205.85bn £206.35bn
State education spending £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn
Transport subsidy spending £25.05bn £25.10bn £25.14bn £25.18bn £25.23bn £25.30bn £25.36bn
Cost-of-living support payment £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Bills Rebate £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Household Energy Payment £0.00bn £2.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Price Guarantee subsidy £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disability basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Other public spending budget change £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Caseloads: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus 0k 0k 0k 0k 0k 0k 0k
State Pension 11,264k 11,145k 10,819k 10,700k 10,723k 10,750k 10,776k
Pension Credit 1,489k 1,484k 1,476k 1,454k 1,452k 1,457k 1,466k
Winter Fuel Payment 7,844k 7,783k 7,473k 7,375k 7,391k 7,409k 7,427k
Pension Age Winter Heating Payment (Scotland) 803k 802k 800k 797k 799k 801k 802k
Universal Credit 6,343k 6,419k 6,408k 6,369k 6,364k 6,343k 6,295k
Housing Benefit 1,249k 1,182k 1,181k 1,165k 1,164k 1,166k 1,168k
Council Tax Reduction 3,326k 3,344k 3,351k 3,364k 3,376k 3,388k 3,395k
Employment and Support Allowance (income-related) 71k 0k 0k 0k 0k 0k 0k
Employment and Support Allowance (contribution-based) 719k 720k 721k 722k 724k 726k 728k
Jobseeker's Allowance (income-based) 0k 0k 0k 0k 0k 0k 0k
Jobseeker's Allowance (contribution-based) 67k 67k 67k 67k 67k 68k 68k
Income Support 25k 0k 0k 0k 0k 0k 0k
Working Tax Credit 0k 0k 0k 0k 0k 0k 0k
Child Tax Credit 0k 0k 0k 0k 0k 0k 0k
Incapacity Benefit 0k 0k 0k 0k 0k 0k 0k
Personal Independence Payment 3,567k 3,575k 3,579k 3,585k 3,593k 3,602k 3,611k
Disability Living Allowance 1,291k 1,294k 1,296k 1,298k 1,300k 1,304k 1,307k
Attendance Allowance 1,695k 1,698k 1,700k 1,703k 1,707k 1,711k 1,715k
Carer's Allowance 1,034k 1,037k 1,038k 1,040k 1,042k 1,060k 1,062k
Carer Support Payment (Scotland) 71k 71k 71k 72k 72k 72k 72k
Scottish Carer Supplement 0k 71k 71k 72k 72k 72k 72k
Severe Disablement Allowance 4k 4k 4k 4k 4k 4k 4k
Industrial Injuries Disablement Benefit 137k 137k 137k 138k 138k 138k 139k
Armed Forces Compensation Scheme 93k 93k 94k 94k 94k 94k 94k
War Widow's or Widower's Pension 0k 0k 0k 0k 0k 0k 0k
Child Benefit 8,123k 8,120k 8,120k 8,130k 8,145k 8,161k 8,178k
Scottish Child Payment 351k 352k 352k 353k 350k 348k 337k
Two Child Limit Payment (Scotland, counterfactual) 0k 0k 0k 0k 0k 0k 0k
Healthy Start vouchers 200k 200k 200k 201k 201k 201k 202k
Maternity Allowance 0k 0k 0k 0k 0k 0k 0k
Statutory Maternity Pay 333k 333k 334k 334k 335k 336k 337k
Statutory Paternity Pay 0k 0k 0k 0k 0k 0k 0k
Statutory Sick Pay 181k 182k 182k 182k 182k 183k 183k
Sure Start Maternity Grant 26k 26k 26k 26k 26k 26k 26k
Bereavement Support Payment 98k 98k 98k 98k 98k 99k 99k
Widowed Parent's Allowance 0k 0k 0k 0k 0k 0k 0k
Tax-Free Childcare 1,043k 1,027k 1,014k 1,009k 1,011k 1,009k 993k
Extended childcare entitlement 795k 800k 796k 797k 798k 798k 792k
Universal childcare entitlement 1,071k 1,072k 1,072k 1,073k 1,076k 1,079k 1,074k
Targeted childcare entitlement 165k 156k 154k 154k 155k 155k 150k
Childcare Offer for Wales 39k 39k 39k 39k 39k 39k 39k
Care to Learn 0k 0k 0k 0k 0k 0k 0k
Childcare Grant 0k 0k 0k 0k 0k 0k 0k
Parents' Learning Allowance 98k 98k 98k 99k 99k 99k 99k
Adult Dependants' Grant 18k 18k 12k 12k 12k 12k 12k
Travel Grant 0k 0k 0k 0k 0k 0k 0k
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund 0k 0k 0k 0k 0k 0k 0k
Cost-of-living support payment 0k 0k 0k 0k 0k 0k 0k
Energy Bills Rebate 0k 0k 0k 0k 0k 0k 0k
Household Energy Payment 0k 21,065k 0k 0k 0k 0k 0k
Energy Price Guarantee subsidy 0k 0k 0k 0k 0k 0k 0k
Basic income 0k 0k 0k 0k 0k 0k 0k
Disability basic income 0k 0k 0k 0k 0k 0k 0k
Government balance, each component counted once: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Government receipts, each component once £884.82bn £927.24bn £972.81bn £1,008.06bn £1,046.18bn £1,083.32bn £1,116.48bn
Government spending £529.29bn † £545.32bn † £550.39bn † £559.10bn † £566.74bn † £575.46bn † £583.06bn †
Government balance, each receipt once £355.53bn † £381.92bn † £422.42bn † £448.96bn † £479.45bn † £507.87bn † £533.42bn †
Benefits in household_benefits but not in gov_spending £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Model totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
gov_tax £884.82bn £927.24bn £972.81bn £1,008.06bn £1,046.18bn £1,083.32bn £1,116.48bn
household_tax £491.95bn £520.39bn £551.18bn £575.97bn £600.96bn £626.65bn £648.38bn
pre_budget_change_household_tax £484.12bn £512.26bn £542.79bn £566.92bn £591.68bn £617.12bn £638.58bn
gov_spending £529.29bn † £545.32bn † £550.39bn † £559.10bn † £566.74bn † £575.46bn † £583.06bn †
household_benefits £529.29bn † £545.32bn † £550.39bn † £559.10bn † £566.74bn † £575.46bn † £583.06bn †
gov_balance £355.53bn † £381.92bn † £422.42bn † £448.96bn † £479.45bn † £507.87bn † £533.42bn †
Household income totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Household market income £1,637.96bn £1,702.77bn £1,757.35bn £1,806.41bn £1,855.97bn £1,912.43bn £1,970.44bn
Household net income £1,636.62bn † £1,687.50bn † £1,715.11bn † £1,747.12bn † £1,778.27bn † £1,816.57bn † £1,859.18bn †
Household net income (HBAI definition) £1,449.50bn £1,499.51bn £1,530.52bn £1,564.80bn £1,597.47bn £1,635.14bn £1,676.11bn
Population: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
People 69,245k 69,383k 69,473k 69,592k 69,738k 69,912k 70,080k
Benefit units 35,110k 35,181k 35,226k 35,286k 35,360k 35,449k 35,534k
Households 31,046k 31,108k 31,149k 31,201k 31,267k 31,345k 31,420k
Poverty rates: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 16.2% 15.4% 16.3% 16.2% 16.4% 16.4% 16.5%
Absolute poverty, before housing costs: children 24.6% 21.9% 23.1% 22.7% 22.9% 23.0% 23.2%
Absolute poverty, before housing costs: working-age adults 14.0% 13.7% 14.6% 14.6% 14.7% 14.7% 14.9%
Absolute poverty, before housing costs: pensioners 13.7% 13.4% 14.0% 13.9% 14.0% 14.0% 13.9%
Absolute poverty, after housing costs: everyone 21.5% 21.0% 22.0% 22.5% 22.8% 23.0% 23.4%
Absolute poverty, after housing costs: children 33.1% 31.2% 32.1% 32.4% 32.8% 33.2% 34.0%
Absolute poverty, after housing costs: working-age adults 19.9% 19.8% 20.8% 21.5% 21.8% 22.0% 22.2%
Absolute poverty, after housing costs: pensioners 12.7% 12.4% 13.2% 13.2% 13.3% 13.4% 13.6%
Relative poverty, before housing costs: everyone 16.9% 15.9% 16.3% 16.2% 16.3% 16.2% 16.5%
Relative poverty, before housing costs: children 25.8% 22.6% 23.1% 22.7% 22.8% 22.6% 23.2%
Relative poverty, before housing costs: working-age adults 14.4% 14.0% 14.6% 14.6% 14.7% 14.6% 14.9%
Relative poverty, before housing costs: pensioners 14.5% 14.1% 14.0% 13.8% 13.8% 13.9% 13.9%
Relative poverty, after housing costs: everyone 21.4% 20.9% 21.2% 21.3% 21.3% 21.5% 21.7%
Relative poverty, after housing costs: children 32.9% 31.0% 31.1% 31.0% 31.1% 31.2% 31.7%
Relative poverty, after housing costs: working-age adults 19.8% 19.7% 20.2% 20.4% 20.5% 20.6% 20.8%
Relative poverty, after housing costs: pensioners 12.5% 12.4% 12.2% 11.8% 11.7% 11.8% 12.1%
People in poverty: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 11,251k 10,708k 11,337k 11,302k 11,412k 11,452k 11,588k
Absolute poverty, before housing costs: children 3,677k 3,272k 3,456k 3,407k 3,443k 3,465k 3,511k
Absolute poverty, before housing costs: working-age adults 5,950k 5,860k 6,279k 6,333k 6,390k 6,402k 6,493k
Absolute poverty, before housing costs: pensioners 1,624k 1,576k 1,602k 1,562k 1,579k 1,585k 1,584k
Absolute poverty, after housing costs: everyone 14,871k 14,567k 15,264k 15,691k 15,911k 16,096k 16,374k
Absolute poverty, after housing costs: children 4,941k 4,671k 4,815k 4,873k 4,932k 5,015k 5,146k
Absolute poverty, after housing costs: working-age adults 8,426k 8,436k 8,944k 9,325k 9,475k 9,560k 9,684k
Absolute poverty, after housing costs: pensioners 1,504k 1,460k 1,505k 1,493k 1,505k 1,520k 1,545k
Relative poverty, before housing costs: everyone 11,698k 11,036k 11,323k 11,301k 11,352k 11,354k 11,588k
Relative poverty, before housing costs: children 3,849k 3,391k 3,456k 3,407k 3,429k 3,416k 3,511k
Relative poverty, before housing costs: working-age adults 6,131k 5,993k 6,272k 6,333k 6,359k 6,362k 6,493k
Relative poverty, before housing costs: pensioners 1,719k 1,652k 1,594k 1,561k 1,565k 1,576k 1,584k
Relative poverty, after housing costs: everyone 14,802k 14,520k 14,761k 14,796k 14,871k 15,007k 15,219k
Relative poverty, after housing costs: children 4,916k 4,649k 4,663k 4,653k 4,675k 4,703k 4,798k
Relative poverty, after housing costs: working-age adults 8,397k 8,415k 8,702k 8,816k 8,873k 8,962k 9,052k
Relative poverty, after housing costs: pensioners 1,488k 1,455k 1,395k 1,326k 1,324k 1,342k 1,369k
Inequality: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Gini coefficient, equivalised HBAI income before housing costs 0.3077 0.3049 0.3055 0.3054 0.3059 0.3064 0.3080
Mean household net income in each side's own deciles: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Decile 1: mean household_net_income per household £17,748 † £17,980 † £17,927 † £18,181 † £18,317 † £18,537 † £18,742 †
Decile 2: mean household_net_income per household £28,048 † £29,038 † £28,757 † £28,999 † £29,260 † £29,678 † £30,174 †
Decile 3: mean household_net_income per household £34,220 † £35,425 † £36,066 † £36,547 † £37,156 † £37,773 † £38,102 †
Decile 4: mean household_net_income per household £38,926 † £40,769 † £41,716 † £42,763 † £43,387 † £44,167 † £44,499 †
Decile 5: mean household_net_income per household £43,346 † £45,603 † £46,054 † £47,001 † £47,756 † £48,627 † £50,309 †
Decile 6: mean household_net_income per household £47,936 † £49,179 † £50,883 † £52,036 † £52,926 † £53,947 † £54,723 †
Decile 7: mean household_net_income per household £55,669 † £56,647 † £57,489 † £57,670 † £58,788 † £59,703 † £61,102 †
Decile 8: mean household_net_income per household £68,786 † £70,811 † £71,870 † £73,483 † £73,890 † £75,325 † £77,181 †
Decile 9: mean household_net_income per household £78,891 † £81,011 † £80,860 † £82,646 † £85,464 † £86,940 † £89,508 †
Decile 10: mean household_net_income per household £116,864 † £120,162 † £124,545 † £126,420 † £125,756 † £128,269 † £130,694 †
Standard reform costings (change in gov_balance): head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp £6.41bn £6.66bn £6.63bn £6.85bn £7.03bn £7.25bn £7.47bn
Raise higher rate by 1pp £3.61bn £3.87bn £4.00bn £4.22bn £4.44bn £4.71bn £4.83bn
Raise personal allowance by ~800GBP/year £-3.82bn £-3.93bn £-4.01bn £-4.05bn £-4.11bn £-4.24bn £-1.95bn
Raise child benefit by 25GBP/week per additional child £-1.99bn £-1.75bn £-1.55bn £-1.38bn £-1.30bn £-1.18bn £-1.07bn
Reduce Universal Credit taper rate to 20% £-20.83bn £-21.38bn £-21.81bn £-22.02bn £-22.37bn £-22.77bn £-22.96bn
Raise Class 1 main employee NICs rate to 10% £10.89bn £11.30bn £11.71bn £12.01bn £12.31bn £12.62bn £13.00bn
Raise VAT standard rate by 2pp £24.17bn £24.97bn £25.86bn £26.46bn £27.07bn £27.68bn £28.31bn
Raise additional rate by 3pp £2.90bn £3.15bn £3.22bn £3.41bn £3.61bn £3.85bn £4.14bn
Raise the Universal Credit standard allowance by 10% £-3.06bn £-3.23bn £-3.30bn £-3.40bn £-3.48bn £-3.53bn £-3.55bn
Raise the State Pension by 10% £-7.76bn £-7.99bn £-8.22bn £-8.44bn £-8.79bn £-8.52bn £-8.86bn
Standard reforms: change in each reform's target: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp: change in income_tax £6.55bn £6.86bn £6.79bn £7.00bn £7.20bn £7.44bn £7.66bn
Raise higher rate by 1pp: change in income_tax £3.61bn £3.92bn £4.01bn £4.23bn £4.45bn £4.72bn £4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax £-3.94bn £-4.07bn £-4.14bn £-4.18bn £-4.22bn £-4.40bn £-1.76bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit £2.33bn £2.14bn £1.94bn £1.76bn £1.65bn £1.52bn £1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit £20.79bn £21.36bn £21.84bn £22.07bn £22.39bn £22.80bn £23.01bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance £11.15bn £11.63bn £12.00bn £12.29bn £12.61bn £12.94bn £13.31bn
Raise VAT standard rate by 2pp: change in vat £23.98bn £24.77bn £25.66bn £26.24bn £26.85bn £27.46bn £28.07bn
Raise additional rate by 3pp: change in income_tax £2.90bn £3.15bn £3.22bn £3.42bn £3.62bn £3.86bn £4.12bn
Raise the Universal Credit standard allowance by 10%: change in universal_credit £3.06bn £3.22bn £3.32bn £3.41bn £3.48bn £3.54bn £3.55bn
Raise the State Pension by 10%: change in state_pension £10.68bn £11.17bn £11.28bn £11.60bn £12.03bn £12.46bn £12.96bn

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Verdict: REQUEST_CHANGES at 0835944512f8632f878168bbe1d2eceba951500d

The port is faithful, every test passes on every Python, and the lists match the scan entry by entry. Three things need to change before merge: a model output that differs between runs and that neither the lists nor the stated limits cover, a missing changelog fragment whose cited precedent doesn't hold, and advice to four open PRs that the ratchet's own ceiling rules out. None is P1: CI is green and no model output changes.

All runs below used a detached scratch worktree of the head, with HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset.

Findings

P2-1. Formulas read the clock, and neither the lists nor the stated limits say so

policyengine_uk/variables/household/income/inflation_adjustment.py:16 and :38 call cpi(datetime.datetime.now().strftime("%Y-01-01")) inside the formulas of inflation_adjustment and inflation_adjustment_ahc. Four formulas multiply by inflation_adjustment: real_household_net_income, real_household_net_income_ahc and real_hbai_household_net_income (hbai_household_net_income.py:132-134), and real_hbai_household_net_income_ahc (hbai_household_net_income_ahc.py:44-46). The same inputs therefore give different outputs in 2026 and 2027, which breaks core#500's rule that a formula is a pure function of its inputs.

$ grep -rnE "datetime\.now|date\.today|time\.time" --include=*.py policyengine_uk | grep -v tests/
variables/household/income/inflation_adjustment.py:16:        now_cpi = cpi(datetime.datetime.now().strftime("%Y-01-01"))
variables/household/income/inflation_adjustment.py:38:        now_cpi_ahc = cpi(datetime.datetime.now().strftime("%Y-01-01"))
(plus labels in three variable files, and timings in utils/aggregate_impact/measure.py, which don't reach an output)

A mutation confirms the guard can't see this. A new formula returning datetime.datetime.now().microsecond survived: all 20 package tests passed. The guard looks for draws, so it isn't wrong to miss this. But the body says the scan "finds nothing else", and the docstring's list of limits doesn't mention the clock, so later PRs that rely on the guard will take the engine to be deterministic apart from RATCHET.

Replacement text:

  • Body, ## The lists: replace "cuts no lookup at the depth limit and finds nothing else." with "cuts no lookup at the depth limit and finds no other draw pattern."
  • Body, the Limits: paragraph, add at the end: "The guard looks for draws and hashes, not for reads of the clock: inflation_adjustment and inflation_adjustment_ahc (variables/household/income/inflation_adjustment.py:16 and :38) price to the year datetime.datetime.now() returns, so real_household_net_income, real_household_net_income_ahc, real_hbai_household_net_income and real_hbai_household_net_income_ahc change with the year a simulation runs."
  • Docstring, after "or methods as draw helpers.", add: "It looks for draws, not for other reads that differ between runs, such as the clock: inflation_adjustment reads the year from datetime.datetime.now()." (This is a pe-uk example, so it falls under the brief's item 4.)

P2-2. The PR needs a changelog fragment, and the hand-off's precedent says the opposite of what it shows

docs/engineering/skills/github-prs.md:20 reads: "Every PR must include a Towncrier changelog fragment under changelog.d/, except for a dependency-only update". The hand-off cites four tests-only commits without a fragment. Each one is an intermediate commit of a PR whose merge did add a fragment, and one of those PRs is itself tests-only:

$ for c in abab81f8b c78f88649 8199bc05e c9397c23e; do m=$(git log --merges --ancestry-path --format=%h $c..47a2817e2 | tail -1); git log -1 --format="$c -> %h %s" $m; git diff --stat $m^1 $m -- changelog.d | head -1; done
abab81f8b -> ed3342c5 Merge pull request #2589 ...   changelog.d/isa-current-law.fixed.md | 1 +
c78f88649 -> f153a60f Merge pull request #2598 ...   changelog.d/2597.changed.md | 1 +
8199bc05e -> 7e7b1452 Merge pull request #2555 ...   changelog.d/remove-core-reform-api-tests.changed.md | 1 +
c9397c23e -> 7e7b1452 (same PR)
$ git diff --stat 7e7b1452^1 7e7b1452   # #2555: three test files and the fragment, nothing else

.github/check_changelog_fragments.py only warns when a PR adds no fragment, which is why Changelog fragments passed. A changed fragment merged to main starts .github/workflows/versioning.yaml (on.push.paths: changelog.d/**). That workflow bumps the patch version (bump_version.py: changed is neither in MAJOR_TYPES nor MINOR_TYPES), builds the wheel, runs the suite from it (this guard ships in the wheel's tests/) and publishes. It does not trigger Fix evidence, which looks for fixed fragments only.

Add changelog.d/deterministic-engine-guard.changed.md:

The test suite fails on any new random draw, sampling call or hash of entity ids in the package, and lists today's draws as a ratchet that only falls.

P2-3. Three of the four open PRs can't "carry their own entry"

The body says: "The guard fails each of these PRs until it carries its own entry or changes." test_ratchet_only_shrinks fails if RATCHET exceeds RATCHET_CEILING = 21, and the docstring says never to raise the ceiling. So #2618, #2265 and #2308 can't add a RATCHET entry, and EXEMPT is for digests that decide nothing. Only #2862 can pass with an entry.

I merged the guard's head into each spot-checked head (git merge-tree --write-tree, clean) and ran the package tests:

#2618 (47ee0515b + guard): 1 failed, 19 passed
  household_wealth_decile.py:8,9,10 in _fixed_hash: hash constant (3 found, 0 allowed)
  household_wealth_decile.py:11,9,10 in _fixed_hash: xorshift (3 found, 0 allowed)
  household_wealth_decile.py:35 in household_wealth_decile.formula: draw helper call (1 found, 0 allowed)
  (no stale entries once main's warm_home_discount_random_draw.py is present)
#2862 (3886de2d1 + guard): 1 failed, 19 passed; 9 occurrences in simulation.py and utils/prepared_baseline.py
#2862 + EXEMPT {"utils/prepared_baseline.py": {("<module>", "hash library"): 1, ("_core_fingerprint", "hash use"): 1}}: 20 passed

_fixed_hash in #2618 orders households with equal wealth, so it decides which decile a tied household falls in: a draw by the brief's test. _core_fingerprint in #2862 identifies policyengine-core's code and decides nothing. The failure message ("Draw in the dataset build and store the draw as an input variable") tells the authors of #2618, #2265 and #2308 what to do. It misleads #2862's author, who sees nine "draw helper call" lines from a provenance digest and no mention of EXEMPT.

Replace the section's opening sentence and the #2862 bullet with:

`test_ratchet_only_shrinks` keeps `RATCHET` at or below `RATCHET_CEILING`, so none of these PRs can add a `RATCHET` entry. The guard fails each of them until it changes, or, for #2862, until it adds an `EXEMPT` entry:
...
- #2862 adds a SHA-256 digest in `_core_fingerprint` in `utils/prepared_baseline.py`, which identifies policyengine-core's code and decides nothing. An `EXEMPT` entry for `utils/prepared_baseline.py`, `("<module>", "hash library"): 1` and `("_core_fingerprint", "hash use"): 1`, clears all nine occurrences it reports.

and add to the #2618 bullet: "Its hash orders tied households and so decides their decile; #2618 must order ties without a hash, for example by household_id itself."

P3-1. The "82" sentence says the spread reaches 82 occurrences; it reaches 40

The scan without EXEMPT gives 82 occurrences in 21 files, which matches the body and the hand-off. The brief's 80 matches no count I could build. Those 82 are the 21 RATCHET occurrences, the 21 digests and 40 draw helper calls that reach the digests.

$ uv run python scan.py <guard>   # analyse + draw_helpers(analyses, {}) + count_findings
without EXEMPT: occurrences 82 files 21
beyond RATCHET: 61 in 13 files     # 21 digests + 40 draw helper calls

Body replacement: "Without EXEMPT, the scan reports 82 occurrences in 21 files: the 21 in RATCHET, the 21 digests, and 40 draw helper calls that reach the digests. So a stale exemption fails loudly."

P3-2. "So RATCHET and RATCHET_CEILING only fall" claims more than the tests enforce

test_ratchet_only_shrinks compares two numbers in the same file, so a change that raises both passes. Only review stops it. Replacement: "and test_ratchet_only_shrinks fails when RATCHET exceeds RATCHET_CEILING. So RATCHET can't grow unless a change also raises RATCHET_CEILING, which the file says never to do."

P3-3. The UC rebalancing comment misses that every simulation runs it

simulation.py:416-418 imports universal_credit_july_2025_reform and calls its simulation_modifier(self) in Simulation.__init__, and parameters/gov/dwp/universal_credit/rebalancing/active.yaml reads true from 2025. So current law draws default_rng(43) in every simulation, not only in a scenario someone opts into. The body's limits name the call through an object, but the RATCHET comment at test line 222 reads like an optional scenario. Replacement:

    # UC rebalancing: a seeded numpy Generator draws one value per benefit
    # unit and compares it with the new claimant share. Simulation.__init__
    # applies it to every simulation through the Scenario object, a call the
    # scan does not follow; the Scenario at module level passes on the
    # modifier.

P3-4. splitmix64_uniform has four constants, not three

utils/stochastic.py:18-21 uses four constants: 0x632BE59BD9B4E019, which spaces the salts, and the three splitmix64 ones. The scan recognises three (HASH_CONSTANTS). Test lines 173-175 and the body say "its three mixing constants". Replacement for the comment: "The splitmix64 finaliser that maps entity ids to uniform draws on [0, 1): the three splitmix64 constants the scan recognises and three xorshift steps." In the body: "the three splitmix64 constants and three xorshift steps".

P3-5. Docstring lines 5-6 and 12: wrapping and voice

The edits left two over-long lines (97 and 85 characters, against 79 in the rest of the docstring), and "are drawn" is passive. Replacement for lines 4-9:

(policyengine-core#500), and seeding does not exempt a draw. The dataset
build draws stochastic inputs such as take-up flags and stores them as input
variables; the model compares them with its rules. policyengine-core#519
checks formulas for numpy and stdlib randomness when variables register, but
a hand-written integer hash of entity ids passes that check, so this scan
looks for the hash as well.

and rewrap lines 11-13 so that "A name resolves to" starts the next line. "finalizer" (line 173) should read "finaliser".

P3-6. A filtered case changes its node id on Python 3.11

pytest gives some cases index ids (body13), and filtering shifts the index:

$ comm -23 ids_3.11.txt ids_3.13.txt
...::test_a_class_body_that_binds_a_name_reads_it_from_the_module[body11-True-11]
# on 3.13 the same case is [body13-True-11]; [body11-True-12] there is a 3.12-only case

CI runs only Python 3.13 (.github/actions/test-environment: python-version: "3.13", and the collection, shards and check_test_shards.py all run on one interpreter). So ids stay stable across shards and Test / Result is unaffected. If ids should also match across Pythons, pass ids= built from each case's text. Optional.

P3-7. The body's pytest-only runs can't be reproduced as written

"The guard alone with only pytest 9.2.0" omits that a bare pytest <file> --noconftest fails at collection: pytest 9 sets up the policyengine_uk package node and imports policyengine_uk/__init__.py. Replacement: "The guard alone with only pytest 9.2.0, --noconftest, and a plugin that makes _pytest.python.Package.setup a no-op so pytest doesn't import policyengine_uk/__init__.py: …". Also, test_model_never_imports_offline_modules can't fail while OFFLINE_MODULES is empty. The hand-off says so and the body doesn't. Add "(so test_model_never_imports_offline_modules passes trivially until a module is listed)" after "OFFLINE_MODULES (empty: …)".

Checked and sound

  • Reference. Fetched from raw.githubusercontent.com at f36d09d3…: git hash-object gives b8a87d1bd151266469a3b8ac253fd86140499d6d.
  • Port. I compared the reference, with policyengine_us→policyengine_uk substituted, against the head: diff -u gives 11 hunks and diff -u -w gives 13. Every hunk falls under the brief's items 1-5 or is _release_package_scan. From the constants to package_scan, the only change is the package name. _is_offline(name, offline=OFFLINE_MODULES) binds the same set object, so the default changes nothing.
  • Version gating. _on_this_python (sys.version_info >= (3, case[minor])) and the two if sys.version_info >= (3, 12): blocks cover every case that had a skip. 3.11 collects 369, 3.12 collects 380, and 3.13 and 3.14 collect 382 each. 3.11 leaves out 13 cases and 3.12 leaves out 2, and every left-out case needs newer syntax. Every case runs on every Python that parses it.
  • Fixture. No test in the file changes EXEMPT, RATCHET or a package file, and a rebuild reads the same files, so clearing can't produce a wrong scan. With an instrumented copy, pytest policyengine_uk/tests/code_health -n 4 built the scan once. In CI the six tests that call package_scan hash (conftest.shard_of) to shards 1, 3, 4, 6, 6 and 7. Each of those shards builds once, and shard 6 may build a second time (about 7 s) under --dist worksteal.
  • Lists. I reran the scan on Python 3.13.18: 1,534 modules, 7.4 s, 0 cut lookups, 42 occurrences in 18 files, no violations and no stale entries. It reports exactly the 21 EXEMPT occurrences in 9 files and the 21 RATCHET occurrences in 10 files, so RATCHET_CEILING = 21 holds. I read every function behind an EXEMPT entry: get_data_build_fingerprint, _fingerprint, _cached_dataset_path, uprating_input_fingerprint (compared with != at simulation.py:1101 and :1168 to reuse an uprated dataset), _sha256, sha256_file/verify_file, _digest_tool_files, definitions_digest, model_fingerprint, runtime_identity, load_partition and fetch. Each serves provenance, a cache key or a content check, and none decides who gets something or meets a threshold. I also read each RATCHET site, and the comments match the code apart from P3-3 and P3-4.
  • Other sources (step 4). I searched for np.random, numpy.random, random., default_rng, RandomState, hash(, hashlib, crc32, adler32, uuid, time.time, datetime.now, date.today, os.urandom, secrets, shuffle, .sample(, choice(, PYTHONHASHSEED and os.environ, plus iteration over set displays. Apart from P2-1, no hit outside the lists reaches an output:
    • os.environ selects the default dataset URL (simulation.py:219), an input, and filters git's environment (build_metadata.py:205).
    • The set iteration in utils/dated_reforms.py:673 and :703 feeds all-or-nothing checks, so order doesn't matter.
    • time.time only times the aggregate impact tool.
  • Tests on the head. uv sync --frozen --extra dev, then uv run pytest <guard> -n 4 (Python 3.13.18): 382 passed in 21.4 s, 35 s wall including startup. With only pytest 9.2.0 and the Package.setup no-op plugin: 3.11.18 gave 369 passed (17.6 s), 3.12.3 gave 380 (21.2 s), 3.13.18 gave 382 (18.0 s) and 3.14.6 gave 382 (18.0 s). No run skipped or xfailed a test.
  • Wheel. uv build --wheel reproduced the hand-off's sha256 48570c87…. I installed it as test-environment does (--no-install-project, then uv pip install --no-deps) and ran from an empty directory: 382 passed. The scan read the same 1,534 modules with the same counts.
  • Code health and lint. uv run pytest policyengine_uk/tests/code_health -n 4: 3,544 passed. ruff@0.17.2 format --check ., check . and check --select F811,F821 policyengine_uk all pass, and so does git diff --check 47a2817e2..HEAD.
  • Mutations. I ran each on the scratch worktree, then restored it.
    • The hand-off's six were all killed: import random, a second splitmix64_uniform call, new_draws, the pip_reform Scenario removed (stale), xorshift raised to 4 (ceiling and stale), and the _fingerprint entry removed.
    • Mine, killed: import numpy.random as r, from numpy import random as r, splitmix64_uniform as mix re-exported through a new utils/mixer.py and star-imported by a formula, 0x9E3779B97F4A7C15 hand-written in a new formula, hash(person_id) % 100, np.random.default_rng through model_api's np, a module-level draw, a class-attribute draw, a draw in a comprehension, frame.sample(5), a staticmethod helper called through its class, and H.r = np.random (caught at the read).
    • Mine, survived: a Knuth multiplicative hash (ids * 2654435761) & 0xFFFFFFFF and a sin-fract hash np.sin(x * 12.9898) * 43758.5453 of an id. Both fall within the stated limit ("a ratchet over recognised patterns, not a proof"). datetime.datetime.now() in a formula also survived, which is P2-1.
  • Merge and CI. origin/main is still 47a2817e2. git merge-tree --write-tree --name-only origin/main 0835944 gives a clean tree (32ac6a18e…, exit 0). The head has 20 check runs, all completed/success: Test / Python 1/7-7/7, Test / YAML, Test / Result, Lint, Changelog fragments, Fix evidence, Changes, Documentation build, Aggregate impact, Release build records the commit, and Install + smoke-import on 3.11-3.14.
  • Body numbers. The 382/369/380/382/382, the wheel's 382 and 1,534 modules, 3,544, the 21 and 21 and 82 in 21 files, and every entry in the EXEMPT table match my runs. The title and the axiom: and Impact: lines are as the brief requires.

What I couldn't check


Generated by Claude Code

The guard's docstring now says it looks for draws and not for the clock
(#2875). The comments on the splitmix64 and Universal Credit entries say
which constants the scan recognises, which entries reach the hash and how
Simulation.__init__ applies the modifier. Two test rows follow
PolicyEngine/policyengine-us#10447 at 221f2fe44, and a changelog fragment
records the guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 6f4d3313060987611bd00da46d3f392ffda1013e — COMPLETE; no confirmed findings or material evidence gaps.

All 389 focused checks passed (384 randomness-guard checks and five cache-release checks). The manual review covered alias/import resolution, scope handling, exclusions, model inventory and ratchet behavior, and compared the earlier-head author review with the fixes adopted in this head. Conclusions apply to the guard’s documented static-analysis scope and exemptions.

The PR remains draft; this is a review comment, not an approval.

@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 11:22
@MaxGhenis
MaxGhenis requested a review from juaristi22 November 5, 2026 11:22

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review status: COMPLETE — approved at 6f4d3313060987611bd00da46d3f392ffda1013e. No findings or material evidence gaps remain.

The head and description are unchanged from the completed review. I retained the verified 389 prior test passes (384 randomness-guard cases and five module-cache checks), together with the completed resolver/import-graph and author-feedback assessment; no code review or tests were repeated for the readiness change.

The PR is now ready for review. Fresh checks pass at this exact head, including the required Lint and Test / Result contexts. Historical benchmarks and the guard's documented limits retain their original qualifications.

MaxGhenis added a commit that referenced this pull request Nov 6, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 6, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 added this pull request to the merge queue Nov 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 9, 2026
@juaristi22
juaristi22 added this pull request to the merge queue Nov 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 10, 2026
@juaristi22
juaristi22 added this pull request to the merge queue Nov 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 11, 2026
Main gained Simulation.subsample and policyengine_uk/utils/subsample.py
through #2308 on 9 November, after this branch's last CI run. The guard
scans the whole package, so the merge queue ran it over that code and
removed #2873 for a failed check on 11 November.

The merge has no conflict: main does not touch the guard or its changelog
fragment. On the merged tree the guard reports eight draw patterns, all in
the subsample code; the next commit handles them.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On main's code the guard reports eight draw patterns, all in the subsample
tool that #2308 added. Five sit in policyengine_uk/utils/subsample.py, where
it hashes the seed, makes a generator and draws households. The other three
are the calls that reach them, in draw_household_counts, subsample_simulation
and Simulation.subsample.

This commit lists the eight in a third allowlist, SAMPLING_TOOLS. The two
package tests allow RATCHET and SAMPLING_TOOLS summed per key. draw_helpers
still reads EXEMPT alone, so the tool's functions stay draw helpers and a
use of one from any other scope counts there as a draw helper call. RATCHET,
RATCHET_CEILING, EXEMPT and the scanner do not change.

Four new guard tests pin the list. The tool's three functions are draw
helpers. A formula that uses a listed sampler fails. The list names only the
subsample tool and shares no key with RATCHET or EXEMPT. No package module
uses an attribute named subsample, because the scan does not follow method
calls. A new test in test_subsample.py checks that the draw neither reads
nor moves numpy's or the standard library's global random stream.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

@juaristi22, the new head, 697529ba5, changes a rule you approved at 6f4d33130, so it needs your review again.

At 6f4d33130 the guard had two lists: RATCHET for draws that later PRs move into the data build, and EXEMPT for reviewed hashes that draw nothing. Since then #2308 put Simulation.subsample on main, and the guard reports its eight draw patterns, which fit neither list. This head merges main and lists them in a third list, SAMPLING_TOOLS, which the two package tests add to RATCHET per key. EXEMPT, RATCHET and RATCHET_CEILING keep their values. The tool's three functions stay draw helpers, so a call to one from a scope the lists don't name still fails. The body's "The lists" section gives the reasons, and its limits paragraph names what the new list can't catch.

All 386 guard tests pass on this head locally, and CI runs on it now. The merge stays with you.

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review of 697529ba571a36860cc9b1cefadb11a8831c49af, incremental from 6f4d3313060987611bd00da46d3f392ffda1013e.

Code review COMPLETE: no findings or material evidence gaps. The eight SAMPLING_TOOLS occurrences are checked alongside the ratchet without treating them as non-draw digests. All three sampling functions remain draw helpers, so unlisted callers and extra occurrences still fail. RATCHET, EXEMPT and the ceiling retain their values. The documented static-analysis limits remain explicit.

Validation: 18 focused tests passed, including the four new guard tests, both global-stream cases and native UK subsampling/seed regressions (15.65s pytest; 21.20s wrapper). The current 1,616-module scan has no unexpected occurrences or stale entries. The exact upstream commit/blob reference also matches the official source. Unchanged detector behavior was retained from the prior review rather than rerunning the full generic suite.

Approval remains deferred pending CI. At the final evidence capture, Lint passed, but the required Test / Result check was not yet available and seven Python shards were still running. This completes the requested code re-review; the merge hold now waits on required checks at this exact head. I will recheck CI without repeating the completed code review.

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval of 697529ba571a36860cc9b1cefadb11a8831c49af, completing the CI gate after the code re-review.

The review remains COMPLETE with no findings or material evidence gaps. All 21 current CI contexts have passed, including required Lint and Test / Result. The head, description and human review activity are unchanged; the prior 18 focused passing tests and source checks remain valid. No code review, model calculation or test execution was repeated for this approval.

The reviewed sampling-tool fix, completed re-review and successful exact-head CI satisfy the hold requested for this PR.

@juaristi22
juaristi22 added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit f6b7fbd Nov 12, 2026
21 of 22 checks passed
@juaristi22
juaristi22 deleted the claude/deterministic-engine-guard-o2b5an branch November 12, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants