Repository navigation
Update parameter-read tests for children built on first read - #2926
Conversation
policyengine-core#570 makes ParameterNodeAtInstant build each child the first time it is read. Three tests relied on every child being built with the node, so they fail on a Core release containing that change. The read recorder now also patches __getattr__, which returns a child on its first read. The recorder test covers first and repeated attribute reads and fancy indexing served from the cached vectorial node, and the control parameter is built during the recording. The uprating cache test reads a node so that lower caches fill. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target. Checks failing on both base and head (1)
Counted-once checkThe check raised each of 92 components by £1 per unit in 2026-27 and read every head of its totals household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A response that is not finite breaks every rule. The check measures a head that breaks a rule twice more: once holding every other head at its baseline value, and once holding every other head at its value after the first £1 while the component rises again. The head fails where either measurement still shows the response. The check cannot hold a head at a value that is not finite. In each unit where another head is not finite after the first £1, it holds that head at its baseline value, and the second measurement judges no household the unit belongs to. There a head fails wherever it first broke a rule, or wherever the check could not confirm whether its response contains the component. 0 failed and the check skipped 0.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
vahid-ahmadi
left a comment
There was a problem hiding this comment.
Thanks, Anthony. This reads well and does what it says. Nothing blocks.
Reproduced locally on the same five tests, with Core selected through PYTHONPATH and its loaded path confirmed:
- This PR at
acf0564: 5 passed on Coremaster(175e7eec) and 5 passed on PolicyEngine/policyengine-core#570 (f8f77d24). - The base versions of the two test files (
98381bf) on #570: 3 failed and 2 passed. The failures aretest_every_labour_supply_parameter_is_read[gov.simulation.labour_supply_responses],test_recorder_counts_each_kind_of_readandtest_a_failing_probe_leaves_the_system_and_its_caches_alone. - I did not run the #570 + #660 merge.
Checked against #570's ParameterNodeAtInstant:
- Attribute reads on #570. A child that hasn't been read is not in the instance
__dict__. So the original__getattribute__raises, and Python falls back to__getattr__, which resolves the child and puts it in__dict__withsetdefault. Patching__getattr__therefore records the first read, and the__dict__test inrecording_getattributerecords later reads. - Attribute reads on
master. Every child is set in__dict__at build time, so the newkey in __dict__test matches the oldkey in _childrentest. The only other__dict__entries are underscore names, whichrecord_attribute_readskips. - Children built without a name read are not recorded. That covers
__getitem__with a string resolving intostate.resolved, and_materialisesetting only_children. A later attribute read still goes through__getattr__, so it is recorded. _leaves_at_instantnow usesnode._children. That is needed, sinceobject.__getattribute__(node, "_children")would raise on an unmaterialised #570 node. The underscore name keeps it out of the reads.- Building a child doesn't record it. Leaves are built with
Parameter._get_at_instant, which isn't patched, so the control's._childrenbuild can't count it as read.- Building a scale does record its brackets through
bracket.get_at_instant, as the docstring says. - No scales sit under either guarded prefix, so this can't hide an unread parameter there.
- Building a scale does record its brackets through
test_recorder_counts_each_kind_of_readcovers both attribute paths on #570:by_attributethrough__getattr__, andby_attribute_again, read before recording, through__dict__. Building the vectorial node before recording keeps fancy indexing to the__getitem__path.- Uprating test: on #570, reading
.gov.hmrc.income_taxfills_at_instant_cacheon that path, sofilledstays non-empty and the test still checks that a failing probe leaves those caches alone.
The new comments and docstring match the behaviour on both Core versions. CI here runs against released Core, so it covers the master side only. The #570 side rests on the local runs above.
Review drafted with AI assistance and checked against the code at acf0564.
Fixes #2925
What and why
This PR is needed before a policyengine-core release contains PolicyEngine/policyengine-core#570. That change makes
ParameterNodeAtInstantbuild each child the first time it is read, instead of building every child when the node is created. Simulation values do not change, but three tests here depend on the up-front build. policyengine-uk requirespolicyengine-core>=3.34.9with no upper limit, so it would accept such a release.Only tests and a changelog fragment change. No model code changes.
test_labour_supply_parameters_are_read.py): it also patchesParameterNodeAtInstant.__getattr__, which returns a child on its first read. The__getattribute__patch records only names already in the node's__dict__.test_recorder_counts_each_kind_of_readnow covers a first attribute read, a repeated attribute read, and fancy indexing served from the cached vectorial node. It also checks that__getattr__is restored. The control parameter's node builds all its children during the recording, so the control test shows that a built parameter is not counted as read.test_a_failing_probe_leaves_the_system_and_its_caches_alone): it readsparameters("2027-01-01").gov.hmrc.income_tax, so lower caches fill with either Core behavior.Test plan
I ran the five tests that use the changed code, selected by pytest node ID, with Core selected through
PYTHONPATHand its loaded path confirmed:master175e7eecf8f77d24ec5927a7The three failures before this PR were the two recorder tests named in #2925 and the uprating cache test.
I also ran scratch copies of the recorder with deliberate faults on all three Core versions. The tests failed for each fault that can occur on that version: not patching
__getattr__(#570 only, becausemasternever reaches it for a child),__getattribute__recording nothing, fancy indexing recording nothing, counting a node as a read, counting every leaf under a node that was read, and counting children when_childrenis built (#570 only).ruff formatandruff checkon the two changed test files, using ruff 0.17.0 from the local environment. I did not runmake format, because it covers the whole repository.changelog.d/lazy-parameter-node-tests.changed.md, checked with.github/check_changelog_fragments.py.I did not run the full test suite.
Reproducer
Reproducer: n/a: test-only change with no model code
axiom: n/a: test-only change with no policy change
🤖 Generated with Claude Code