Repository navigation
Conversation
Build on the per-formula TracingParameterNode view from #574. The trace now records tax-scale reads with a description of the schedule, reads of which children a node has, parameters a formula reads before choosing an instant, and the parameters core reads for adds, subtracts and uprating. Each trace node also records whether its formula ran. Rebuilt from #543. The tree-level tracing state that PR added is dropped, because #574 replaced it. Co-authored-by: PavelMakarchuk <pavel.ma99@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An output now counts as covered in a file only after the formula it uses at that period has run and the output calculated without an error. A value masked by defined_for, given as an input, or from an output that raised no longer hides later tests. The selection moves into trace_yaml_tests, which reports how many tests it skipped. The fingerprint now covers the whole package and the traced tests. YAML files holding one mapping and .yml files are read as the runner reads them. The filter on gov.abolitions paths is gone, because core's abolition check is not traced. The command, trace_microdata and the selection's effect on edges now have tests. Co-authored-by: PavelMakarchuk <pavel.ma99@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Attributes of an undated parameter such as values_list, children and clone are now recorded at that node's path for all instants, and looking for a child that is absent records the node. dependency-map counts excluded tests and unreadable files, follows nested period mappings, ignores hidden files in the fingerprint, rejects a missing tests root and says when it skipped tests. Co-authored-by: PavelMakarchuk <pavel.ma99@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A country simulation that traces its own parameter root sends core's abolition check through the tracing node, once per variable. Recording absent children for key look-ups stored every abolition name each time, so only attribute look-ups are recorded now. dependency-map leaves gov.abolitions reads out again, because they reach the trace the same way. Co-authored-by: PavelMakarchuk <pavel.ma99@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A key look-up that finds no child should be recorded at the missing path, without the names of the node's other children, so core's abolition check records only the variable's own switch. dependency-map should leave out only that switch, and keep a formula's read of another variable's switch. These tests fail until the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A key look-up that finds no child is recorded again, now at the missing child's path with no value, so a formula that falls back on a default is a reader of the parameter that would change it. Core's abolition check therefore records only the variable's own switch, without the names of the other switches. dependency-map leaves out only that switch, gov.abolitions.<variable>, and keeps a formula's read of another variable's switch: policyengine-us's SSI income helpers make such reads, and turning those switches on changes the reading variable. The tracing node now documents how absent children are recorded, and the docstrings no longer say that key look-ups are not recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A country package can trace its own parameter root. Core's abolition check then reached the trace, once per variable, and dependency-map had to leave those reads out. These tests expect the check to record nothing, whether the switch exists, is missing, or there is no gov node. They also expect trace_microdata to record reads made through a root the package traces with the tracer it hands over, and a miss looked up with a str-based enum member to be recorded at the member's string value. These tests fail until the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acer Core's abolition check now reads the parameter tree without a tracing view, so a country package that traces its own parameter root no longer records one read, or one miss, per calculated variable. In policyengine-us that removed the extra objects that caused a second full garbage collection in traced household calculations. dependency-map no longer needs to leave abolition switches out: every recorded switch read is a formula's. dependency-map's tracing setup kept replacing the simulation's tracer after a country package had handed it to its own parameter root, so reads through that root went to a tracer nobody read. The same tracer now drops values instead. A key that is a str-based enum member is recorded at its string value, the path the look-up uses. The docstrings and changelog say which look-ups that find nothing are recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #541. Fixes #542.
Replaces #543 by Pavel Makarchuk, rebuilt on the per-formula parameter view from #574, following the review of 2026-10-27.
What changes
A traced simulation now records these reads on the node of the variable being calculated:
_children, iterating,in, and an attribute look-up that finds no child), at the node's own path;parameters(period).taxes["future_rate"]caught by the formula, at the missing child's path with no value;parameters.gov.irs.credits.eitc(period).masterrecords none of these, and US uses them in the federal EITC eligibility formulas among others;values_list,children,clone,brackets), at that node's path with periodeternity;adds,subtractsand uprating.Each trace node also records whether its formula ran (
formula_ran).Core's abolition check now reads the parameter tree without a tracing view. When a country package traces its own parameter root, as
policyengine_us.Simulationdoes, the check used to add one read per calculated variable, which made up most of the reads in a US trace. It now adds none.policyengine-core dependency-map -c <package>writes which variables read each parameter and which variables feed which. An output now counts as covered in a file only after the formula it uses at that period has run and the output calculated without an error. Every recorded read becomes an edge, a formula's read of an abolition switch included. The command keeps the tracer that a country package hands to its own parameter root when tracing is switched on, so reads through that root reach the map.Relationship to #543
Kept:
describe_tax_scale,TracingChildren,_record_parameter_read, and thedependency-mapcommand. The command is a separate commit with Pavel as author. Rebuilt commits carry aCo-authored-bytrailer.Dropped, because #574 already does this:
ParameterNode.set_tracing, the tracer write in theSimulation.tracesetter, the save and restore of tracing state in_run_formula, the tree updates inget_branch, and the at-instant cache change.parameter_node.pyis unchanged.Changed from #543:
trace_yaml_tests, withskippedandexcludedcounts;.ymlfiles and nested period mappings are read as the runner reads them;test_tracing_parameter_node_reads_through_to_the_nodefrommasterchanges, becauseview.taxesis now a tracing view ofparameters.taxes.Verification
master(22d06a4a), and merged with #570 (headc9bbd7e8)household_net_income, one three-person US household: (variable, parameter) reads recorded, leaving outgov.abolitions, on1936f11bmaster, 1,201 here, none lostmarginal_tax_rate,household_net_incomeandssi, one three-person CA household, against1936f11bTraced US calculations, 5 runs per version, run in rotating order on a shared machine (medians):
masterhousehold_net_income, CA family of four: timemarginal_tax_rate, one TX adult: time866c25casurtax_rateis read bymd_capital_gains_surtax,credits.ctc.phase_out.ratebymd_ctcandcredits.cdcc.ratebyco_cdccTwo adversarial review rounds ran. Round 2 found no missing read and no changed value, but found that a round 1 fix made US traces very large. A third review, of that fix (
1936f11b), confirmed it and found two more bugs, both fixed here with tests:dependency-mapleft out everygov.abolitionsread, including another variable's switch. US SSI income helpers read such switches, and turning one on changes the reading variable.A fourth round reviewed those fixes. It found that recording core's abolition check as a miss for every variable slowed traced US household calculations by about 24%, through an extra full garbage collection, and that
dependency-mapreplaced the tracer a country package had handed to its parameter root, so reads through that root never reached the map. Both are fixed here, by taking core's check out of traces and by keeping that tracer, together with the path of a key that is astr-based enum member. The fixes from the fourth round have had no further adversarial review.Not run: the full test suite, which is left to CI;
dependency-mapover the whole US suite;trace_microdataon US microdata.Default
dependency-mapselection compared with--every-testEach state's baseline test directory only, on current
policyengine-us:The default can still miss a formula below an output whose own formula has run. Other test directories should supply many of the paths in the last column, but that was not measured. The command says when it skipped tests and points to
--every-test.Reads this PR does not record
gov.abolitions.<variable>). A formula's read of a switch is recorded.simulation.tax_benefit_system.parametersrather than the formula'sparametersargument. In US:foreign_earned_income.py,tools/marital_units.py,ny_ctc_pre_2024.pyandbehavioral_response_measurements.py.get_childand a bracket position past the end of a scale.Known limits left open
These came from review. None is reached by current
policyengine-uscode:2017.5, or a test file that is not UTF-8, stopsdependency-map;testsdirectory gets a traceback;vars(),dir()andrepr()on a tracing view differ from the plain node or record nothing;.namewhen read at an instant;node["by_zone.east"]that finds nothing is recorded atby_zone.eastbelow the node, although the look-up does not descend into that path.🤖 Generated with Claude Code