Repository navigation
Conversation
policyengine-us declares in_denver, in_yonkers, in_wilmington and pa_philadelphia_wage_tax_resident as inputs with no formula. The registry maps each to its 2020 county (Denver 08031, Philadelphia 42101) or place (Yonkers 36/84000, Wilmington 10/77580), checked against the 2020 Gazetteer, P.L. 94-171 P1_001N and block ladders 7ba39b95 and 6840b990. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The block-ladder frame wrapper derives all four flags from the block's county and place. The PUMA ladder and the pooled ACS line derive the county-rule flags (in_denver, pa_philadelphia_wage_tax_resident) from the final county and remove the place-rule flags (in_yonkers, in_wilmington), which no place rung can decide there. Summaries record each flag as derived or unavailable, and both ladder gates fail a present flag that disagrees with its rule. The pool manifest moves the county-rule flags to geography_derived_jurisdiction_input (assembled) and the place-rule flags to unavailable_place_jurisdiction_input, and relabels county_fips and congressional_district_geoid, which the stacked PUMA operator writes, as preserved_stacked_engine_input. tools/repin_us_pool_engine_contracts.py re-derived the provision counts, the manifest digest and the engine ABI lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-derived with the repo tools after the derivation change: repin_us_spec_engine_contracts.py (--legacy-only, then full), generate_us_bundle_from_constants.py (US spec_sha256 0b5c29e4...) and spec_engine_coverage.py (42305/42305 fields, 41/41 checks); --check passes on all three and on repin_us_pool_engine_contracts.py. Tests: Hypothesis properties for the iff rule, block-population conservation, row-order invariance and person broadcast, plus registry pin, gate, PUMA and pooled-line cases (engine_free); an engine contract that each flag is an input-only boolean its consumers read; and an engine scenario in which Yonkers, Wilmington, Denver and Philadelphia residents placed by the block ladder get their local tax or benefit and their neighbours do not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
November 2, 2026 13:40
…, record the no-ladder pool receipt Answers the round-1 review of b7b3e21: - rederive_us_local_jurisdiction_flags reads the rungs an existing artifact's own geography supports (county when county_fips is on every row; place only when place_fips and a 15-digit block_geoid are too), refuses carried household or person flags that disagree, and writes the flags. export_us_l0_refit_h5 and the fiscal refresh release builder call it on the loaded base and record the receipt. - Both ladder gates fail an absent flag whose rule the line's own rung decides (block: county and place; PUMA: county), so a missing column can no longer reach the engine as an all-False default. - build_optional_acs_multispine records the pool's flag receipt in provenance on both the ladder and no-ladder paths. Also classifies local_jurisdictions.py for the spine-blindness test, repins the engine ABI digest, and reruns the US spec repin chain (seed-kernel attestations hash geography_ladder.py and puma_ladder.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nifest tools/graph_us_post_transfer_fixture.py rewrote the pinned graph's remaining_stage_manifest_sha256 (d4fb9de9 -> 6e70bc74) after this PR moved the pool manifest. Sources and expected.csv are byte-identical; the tool's own graph-parity check passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 finding 1: the rungs belong to the whole household table, so a pooled base whose ACS rows carry no block decides only the county-rule flags, and a frozen support or L0 support that keeps only block-anchored donors would export without in_yonkers and in_wilmington even though its own geography decides them. The fiscal refresh builder now derives the flags on the loaded base, again right after frozen-support selection (before target materialization), and again on the export frame once the L0 or full-pool weights attach, before any export gate or the H5 write. The calibration diagnostics record each receipt under support_value_repairs.local_jurisdiction_flags, keyed base, frozen_support and export. rederive_us_local_jurisdiction_flags returns the same frame when it already carries exactly the flags its rungs decide, as boolean columns, so the repeat calls copy no tables. Property tests check that deriving a selection of a derived support equals deriving the kept rows from scratch and leaves every decidable flag, and that the unchanged return equals a rewrite. The commit also adds us_local_jurisdiction_artifact_receipt, which refuses an artifact that lacks or contradicts a flag its own geography decides and otherwise returns the receipt a derivation would give. The selection property test uses it as its completeness oracle; the annual static-aging lane uses it in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 finding 2: the annual lane copies its parent's base year byte for byte and keeps the parent's columns in every projected year, and the annual publication validator requires exactly those columns. A parent built before the local-jurisdiction flags therefore produced annual files without in_yonkers or in_denver, and the engine defaulted every resident to False. The lane cannot write the flags without breaking that preservation contract (the base-year file is the certified release's own base artifact), so it now requires them. build_annual_static_aging reads the parent before the model provenance check and calls us_local_jurisdiction_artifact_receipt, which refuses a parent that lacks a flag its own geography decides, carries one that disagrees with it, or carries one its rungs cannot decide. The candidate manifest records the receipt as base.local_jurisdiction_flags. Engine-free tests write parents with block, county-only and no geography: six refusal cases fail before the model provenance check and before the output directory exists, and complete parents reach it. Against the previous lane the six refusal cases and the three receipt cases fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 nit 3: an assembled checkpoint from before the local-jurisdiction flags holds a passing PUMA-ladder receipt but no in_denver. The live gate now fails it, and the rejection said only that the receipt changed from live output, so the operator could not see the repair it needs. _validate_stacked_geography_assignment_receipt now appends the live gate's failure lines (in_denver: absent, but this table's county rung decides it, ...) when the live gate fails, and says that the live gate passes but its details differ otherwise. It still fails closed in both cases. Engine-free tests build an assembled pool with every column the live gate reads, with and without in_denver, and check that the message names the missing flag only when it is missing; a third test covers the passing, changed-details branch. All three fail against the previous message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in main at 3a9c808. Both sides moved the US seed-protocol digests: main through housing_inputs (microcosm#1444), this branch through geography_ladder and puma_ladder. The four conflicts were all digest pins, so none keeps either side's value: tools/repin_us_spec_engine_contracts.py recomputed EXPECTED_HASHES seed_map and seed_protocol, the loader golden spec_sha256 and the pool-tool run_config spec_sha256, and tools/spec_engine_coverage.py rewrote docs/evidence/spec-engine/ us-f0-coverage.json (42210/42210 configuration fields, 41/41 inventory checks). A comment above the seed pins names both causes. same_edit.py found no edit both sides made identically. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrites six passive comments and docstring lines from the support re-derivation commit so each names its actor (_main, the derivation, the release engine). Comments only; the builder tests pass unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in main at 1e88319 (1,023 commits since the merge base 3a9c808). Seven files conflicted: - us_annual_static_aging.py: the module docstring. Kept both paragraphs: the branch's flag requirement, then main's dataset names (#1570). - docs/us-annual-static-aging.md: the manifest `base` bullet. Took main's wording (base build ID `parent_release`) and appended the branch's `local_jurisdiction_flags` receipt clause. - us_runtime/l0_refit_export.py: the import block. Main's #1532 widened the input_mass import (us_input_mass_parity_gate, us_line_32_carriers); the branch added the local_jurisdictions import beside it. Kept both; the function bodies merged cleanly, so the export re-derives the flags and then runs main's Schedule E line 32 input-mass gate. - spec_engine/inventory_coverage.py, docs/evidence/spec-engine/ us-f0-coverage.json, tests/engine_free/shared/test_spec_engine_loader.py and tests/engine_workflow/us/test_us_multispine_pool_tool.py: every hunk is a digest that a tool computes. Took main's side and kept the branch's comment sentences naming #1444 and #1453 above seed_map. Neither side's seed and spec digests are right for the merged tree; a later commit recomputes them with tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py. Known red until the following commits: the transport binding-identity count (B1) and the digests above. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The annual static-aging lane checks its parent's flags before any model
work, but it imported the check from microcosm.build.us_runtime. Importing
any us_runtime module runs that package's __init__, which builds
US_SPINE_AGREEMENT_REGISTRY (spine_agreement.py), which loads the take-up
contract, which calls assert_engine_abi_lock_current. With policyengine-us
installed that compares a lock derived from the installed engine against
us/engine_abi.lock.json (2.2.1), so under any other engine version the lane
raised inside its import and refused every parent, flags or not (gate-lens
review S1).
- us_runtime/local_jurisdictions.py and us_local_jurisdictions.json move to
microcosm/build/us_local_jurisdictions.py and .json. The module imports
only the standard library, numpy, pandas and microcosm.frame; the one
change besides its docstring is files("microcosm.build") for the registry.
The registry bytes and US_LOCAL_JURISDICTION_REGISTRY_SHA256 are unchanged.
- us_runtime/local_jurisdictions.py re-exports the same names (same
__all__), so geography_ladder.py, puma_ladder.py and the other importers
keep their import lines and their attested bytes.
- The annual lane imports microcosm.build.us_local_jurisdictions.
- tools/verify_us_local_jurisdiction_registry.py and the registry test read
the JSON at its new path; US_LOCAL_JURISDICTIONS.md and the changelog
fragment say where it lives and why.
- New engine-free test test_us_annual_static_aging_gate_imports.py: a fresh
interpreter runs build_annual_static_aging up to the model-provenance
step on a parent that carries its flags and on one that lacks them, and
asserts that no microcosm.build.us_runtime module is in sys.modules. It
must be a fresh interpreter because the test process has usually loaded
us_runtime already. A second test checks that the re-export hands out the
same objects.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The transport battery's binding identity reaches every module the bindings import. The us_runtime package imports geography_ladder and puma_ladder, which now import the flags through us_runtime.local_jurisdictions, the re-export of microcosm.build.us_local_jurisdictions. Computed with the test's own describe() loop on this tree: 246 modules against the pinned 198 + 23 + 23 = 244, and the two extra are exactly those two modules (the branch's red engine-free legs at 1818486, assert 221 == 197 + 23, were the same count before main split the set out). Both join BINDING_MODULES_ADDED_AFTER_BASE with a comment naming #1453. test_us_spine_blindness.py: local_jurisdictions.py stays a classified us_runtime module, now as the re-export. The pool tool's runtime graph counts us_runtime files only, so it still reaches 82 modules; the comment says the implementation sits outside that graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main (1e88319) and this branch both moved the US seed and spec digests: main through its ACS local stages, the student-loan stages, the gate calibration and the housing gate; this branch through geography_ladder and puma_ladder (attested kernel modules) and the multispine pool, which write the local-jurisdiction flags. The merge took main's values, so each is recomputed here by its tool on this tree: - tools/repin_us_spec_engine_contracts.py --legacy-only --check: current (source_stages.json is unchanged). - tools/generate_us_bundle_from_constants.py --check (policyengine-us 2.2.1): current; US bundle spec_sha256 aa7433b70fa4... - tools/repin_us_spec_engine_contracts.py, run twice (its first pass measures coverage against the old seed pins and writes covered_item_count 39; the second restores 41): EXPECTED_HASHES full_checkpoint 47d4474df59d..., seed_map 17c6d8d9d5a8..., seed_protocol 780bc4112001...; the loader golden spec_sha256 4253e04fc13a...; the engine_workflow pool-tool spec_sha256 aa7433b70fa4.... - tools/spec_engine_coverage.py: docs/evidence/spec-engine/ us-f0-coverage.json (42394/42394 configuration fields, 41/41 inventory checks), including the stacked_checkpoint_pool_code_exact digest 3c5e14bef2c3.... - All three --check modes exit 0 afterwards. ruff format then changed only the quote style of the rewritten lines, and a comment line above the loader golden and the pool-tool pin names #1453. The same two checks pass on a copy of this tree with the pre-move module layout, so the move in 3a6d247 moves none of these pins. The pool's engine contracts are current under policyengine-us 2.2.1 (remaining-stage manifest 6e70bc749ab7..., 1,059 rows), so us/engine_abi.lock.json, the engine-contract test and the H3 post-transfer fixture keep their values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's #1022 test (test__given_receipt_anchors__then_fs_overrides_the_ transfer_before_pooling, from 44a4528) stands in for with_optional_acs_spine with a bare object(). On this branch build_optional_acs_multispine reads the pooled frame's household table and weights to record the pool's local-jurisdiction receipt, so the merged test raised AttributeError before its assertions. The branch gave its own older pool fake the same shape when it added the receipt; this does the same for main's test and checks that the anchored path records the receipt too (rungs [] without a PUMA ladder). No behaviour changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_us_local_jurisdiction_engine_contract.py opened the registry JSON
with files("microcosm.build.us_runtime"), which 3a6d247 moved to
microcosm/build, so the engine-us collection failed with
FileNotFoundError. It now imports from microcosm.build.us_local_jurisdictions
and reads files("microcosm.build"), like the module itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained #1600, #1605, #1590, #1047 (US household location v1, the opt-in block_v1 rule), #1592, #1598 and #1545 since the 1e88319 this branch last merged. Seven files conflicted: - us_runtime/acs_multispine.py: the base_pool import block only. Union: main's ACS_POOL_BLOCK_LOCATION_METADATA_KEY and this branch's acs_pool_local_jurisdiction_rungs. The function body merged cleanly. - us_runtime/base_pool.py: __all__ only. Union of main's two new names and this branch's one. - tests/engine_free/us/test_us_spine_blindness.py: the pool tool's runtime-graph count. Main counted 83 (81 plus block_location.py and cps_source_geography.py), this branch 82 (81 plus local_jurisdictions.py). The test's own graph helper reaches 84 on the merged tree, with all three modules in it and none unclassified; the comment keeps both explanations. - spec_engine/inventory_coverage.py, tests/engine_free/shared/ test_spec_engine_loader.py, tests/engine_workflow/us/ test_us_multispine_pool_tool.py and docs/evidence/spec-engine/ us-f0-coverage.json: digests only. Each conflicted hunk takes main's value and main's comment, and keeps this branch's comment sentences beside it. The values are stale until the pin tools rewrite them in a later commit. Functions both sides changed and git merged without conflict: build_optional_acs_multispine, with_optional_acs_spine, the fiscal refresh builder's _main and the pool tool's _validate_stacked_geography_assignment_receipt. The last two are independent edits. The first two are not: under block_v1 the merged pool still picks the flag rungs from the PUMA ladder alone, so it removes every flag from a block-located pool, and the receipt can name rungs the pool did not use. A later commit fixes that. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
#1047's opt-in --location-rule block_v1 puts each household on one drawn 2020 block and looks county and place up from it. This pull request derives four engine inputs from county and place, and its ladder gates refuse a table that lacks one. Merged, the two did not meet: - with_household_us_block_location (the base-h5 line and the stacked pool) wrote no flag, so the base-h5 line's geography-ladder gate and the stacked pool's PUMA-ladder gate refused every block_v1 draw (17 of #1047's stacked-pool tests failed at the merge commit); - with_optional_acs_spine picked its flag rungs from the PUMA ladder alone. A block_v1 pool takes none, so it removed every flag, while build_optional_acs_multispine, which staging also hands a PUMA ladder, recorded the county-rule flags as derived. Now every line that draws derives all four flags from the drawn or kept block's county and place (BLOCK_GEOGRAPHY_RUNGS), after the draw, and overwrites any flag the frame carried before it: - with_household_us_block_location applies the flags to its tables once the block's geography is written, as the block ladder does; - acs_pool_local_jurisdiction_rungs takes the location rule and returns the block rungs under block_v1; the pool and the multispine receipt both pass it, and the block_v1 geography record carries the block-rung receipt in place of the PUMA summary's; - the stacked pool's block-location summary records local_jurisdiction_flags, as its legacy summary does. The default legacy rule is unchanged: acs_pool_local_jurisdiction_rungs returns what it did when location_rule is left at its default, and no legacy ladder, gate or receipt is touched. Tests: a shared synthetic ladder carrying Denver, Philadelphia, Yonkers, Wilmington and a decoy for each place code, and an expectation written out in the test support, not read from the registry. On each line (kernel with 1 and 3 clones, base-h5, stacked pool, ACS pool, ACS multispine) a frame that carries flags from before the draw comes out with flags that equal the rules on the drawn geography, and the line's receipt names the county and place rungs with matching counts. The module docstring, US_LOCAL_JURISDICTIONS.md and docs/us-location-v1.md now say this; they named a block-location draw that did not exist when the branch was written (review D1). The pool's static remaining-stage manifest still lists the place-rule flags as unavailable for every pool; its comment and the document say that this describes the default rule. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…w M1) The local-jurisdiction flags moved out of us_runtime so the annual lane's gate imports no us_runtime module; us_runtime/local_jurisdictions.py re-exports them. test_us_spine_blindness.py scans us_runtime by glob and the pool tool's import graph by us_runtime imports, so after the move its source-spine-blindness scan, its frame-metadata scan and the pool tool's graph scan read only the re-export. The 900-line implementation, which the ladders and the pool run, was scanned by none. - A reviewed registry names each out-of-runtime implementation and the us_runtime module that re-exports it. The two glob scans read every us_runtime module and every registered implementation. - The pool tool's import graph follows a registered re-export into its implementation's imports, and the build-path scan reads the tool, the graph and the implementations behind it. The implementation is not a runtime module, so the pinned count stays 84. - A new test finds every us_runtime module that re-exports microcosm.build code (imports a name and lists it in __all__) and requires the registry to name exactly those, with each target present, outside us_runtime and among the scanned files. Moving more code behind a re-export, or dropping the entry, now fails instead of shrinking the scans silently. Mutation checks in a scratch clone, by direct calls of the tests: a source-spine read planted in the implementation fails both spine scans (and passes them with the test file as it was); a Frame rebuilt there without metadata fails the metadata scan; removing the registry entry, or adding an unregistered re-export, fails the registry test. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
test_us_annual_static_aging_gate_imports.py asserted that the annual
lane's parent-flag check leaves microcosm.build.us_runtime unloaded. The
documents cite it for more: that the check never asserts the engine ABI
lock, and that it runs before any model work. us_runtime is only the
lock assertion's usual carrier. microcosm.build.country_spec, which the
package __init__ loads, imports spec_engine.engine_abi inside the
take-up projection it compiles, so a check that called that projection
would assert the lock with us_runtime still unloaded, and the test
passed on exactly that mutant (the reviewer's M3).
The fresh-interpreter probe now also reports the loaded modules at or
under microcosm.build.spec_engine.engine_abi and policyengine_us, and
the test asserts both lists are empty after the gate ran on a flagged
and an unflagged parent.
Mutation checks in a scratch clone: a call to
country_spec.load_country_take_up_contract_projection("us") inside
us_local_jurisdiction_artifact_receipt now fails the test on the
engine_abi assertion alone; the progress log records the engine-side
and re-export mutants.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nits) Review D2 (test only). test_builder_derives_flags_after_every_support_ reduction pinned that the export derivation's result is assigned, but not the base's or the frozen support's. A call whose return value is dropped would leave the unflagged frame in use for target materialization while its receipt says the flags were derived, and both such mutants passed. The test now requires each of the three calls that names a support to be the one assigned to its frame. Run against mutated copies of _main, the base and frozen return-discarded mutants now fail; both passed the test as it stood. Wording: - G-N1: docs/us-annual-static-aging.md, the annual module docstring and US_LOCAL_JURISDICTIONS.md name the third refusal: a parent that carries a flag its geography cannot decide. - G-N2: the re-export serves the us_runtime importers and three tools (the release builder, the registry verifier and now the pool tool), not only the geography lines. - D4: the changelog says the pooled ACS line writes the county-rule flags with its PUMA ladder; without one it writes none. - US_LOCAL_JURISDICTIONS.md names what the gate-import test now pins. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
CI on bcb0194 failed test_us_post_export_scoring.py::test_main_has_no_calibrated_target_frame_at_the_export_write with NameError: name 'local_jurisdiction_flags' is not defined (run 43479440014, engine-free 3.13 and 3.14). That test executes _main's export and cleanup statements in a namespace it builds by hand. This branch put the export support's flag derivation among those statements, where it belongs: it must run before _without_calibrated_frames, which binds the export frame an exact-K outcome keeps. The derivation reads two names _main binds earlier, the receipts dict and the progress reporter, and the branch never gave them to the test. A miss of this branch, not a misplaced statement. The namespace now holds both. The fixture's households get a county, half of them Denver's, so the statements run the real derivation and it writes columns on a new frame. The write check asserts that the written frame carries in_denver as its county decides and no Philadelphia resident, and (as before) that it is the frame the cleanup keeps; the test then asserts the export receipt: county rung, nothing carried, a nonzero Denver count equal to the written column's. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The merge of main 68bbff7 took main's values for every conflicted pin hunk, and the block_v1 change then edited attested modules, so the pins were stale by design until the tools ran. Each value below was written by its tool on this tree; none is hand-edited. tools/repin_us_spec_engine_contracts.py (two passes; the first writes a covered_item_count of 40 into test_spec_engine_coverage_tool.py and the second restores 41, so that file is unchanged): - inventory_coverage.py EXPECTED_HASHES seed_map adb7a1d9..., seed_protocol e3bd670e... - test_spec_engine_loader.py golden spec_sha256 cc830059... - engine_workflow test_us_multispine_pool_tool.py spec_sha256 1fe859c1..., equal to what generate_us_bundle_from_constants.py --check prints under policyengine-us 2.2.1 tools/spec_engine_coverage.py (two passes): docs/evidence/spec-engine/ us-f0-coverage.json, the same seed map, seed protocol and spec digests. ruff format restored double quotes on the three Python pin lines (the repinner writes repr()). Checked after the writes: the repinner's --legacy-only --check and --check, spec_engine_coverage.py --check, the bundle --check, and ci_test_plan.py verify all exit 0. No engine pin moves: the pool's remaining-stage manifest is still 6e70bc74... over 1,059 rows (repin_us_pool_engine_contracts.py --emit-json under policyengine-us 2.2.1), which is what multispine_pool.py, the engine ABI lock and its test pin. full_checkpoint (47d4474d...) and the stacked pool's code digest keep the values this branch already carried. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A PUMA redraw through with_household_us_puma_ladder kept the block, tract, place, state-legislative-district and CBSA columns an earlier block line wrote, beside the county it had just redrawn. The rung inference then called the table block-anchored, and a re-derivation wrote in_yonkers on a Bronx household whose kept block was in Yonkers. - with_household_us_puma_ladder removes each column only a block decides before it draws (US_PUMA_LADDER_BLOCK_ANCHORED_COLUMNS and the per-plan district columns of a block_v1 draw). assign_us_puma_ladder keeps them: the pooled ACS line restores a block-line donor's own district and county after it. The stacked pool's input is operator-free, so its output is unchanged. - us_local_jurisdiction_rungs_present raises on a table in which any row's 15-digit block_geoid does not begin with the row's county_fips, naming the count and the first pairs, and the place rung of us_local_jurisdiction_flag_values raises the same way. So the L0/refit export, the release builder, the annual lane and both ladder gates stop on such a table. Every line writes county_fips as a 2020 county, Connecticut included, so no table a line writes is refused. - US_LOCAL_JURISDICTIONS.md and docs/us-annual-static-aging.md no longer say every parent that predates the flags needs a rebuild: one whose geography decides no flag (no county on some household) passes when it carries none. - The spine-blindness test pins that each registered re-export and implementation declares one literal, unannotated __all__, the only form its re-export detector reads. The spec-engine pins follow in the next commit. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
21ec4d9 edited puma_ladder.py (a seed-attested module) and us_local_jurisdictions.py (in a binding-identity module set), so the pins were stale by design until the tools ran. Each value below was written by its tool on this tree; none is hand-edited. tools/repin_us_spec_engine_contracts.py (two passes; the first writes a covered_item_count of 40 into test_spec_engine_coverage_tool.py and the second restores 41, so that file is unchanged): - inventory_coverage.py EXPECTED_HASHES seed_map adb7a1d9... -> 307b79f2..., seed_protocol e3bd670e... -> 74a41eec... - test_spec_engine_loader.py golden spec_sha256 cc830059... -> d092ce03... - engine_workflow test_us_multispine_pool_tool.py spec_sha256 1fe859c1... -> 4cc1be3c... tools/spec_engine_coverage.py (two passes): docs/evidence/spec-engine/ us-f0-coverage.json, the same seed map, seed protocol and spec digests; 42394/42394 configuration fields, 41/41 inventory checks. ruff format restored double quotes on the three Python pin lines (the repinner writes repr()). Checked after the writes: the repinner's --legacy-only --check and --check and spec_engine_coverage.py --check exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_no_sequence_of_lines_leaves_a_flag_or_block_outside_its_county now always runs two fixed sequences besides its generated ones: a block line, a PUMA redraw and the artifact re-derivation on New York households whose PUMAs span three counties (the review's sequence), and a block_v1 draw followed by a PUMA redraw with a tract and a pre-flag artifact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…usal The round-1 check that a block sits in its county now refuses a tampered pool inside build_optional_acs_multispine, when the pool's flag receipt reads the block rungs. test__legacy_tool__block_v1_gate_failure_is_hard tampered before its harness computed that provenance, so the check fired in the harness and the tool's own block-location gate was never reached. The harness now tampers after the provenance, so that test still exercises the gate, and a new test runs the real build on a pool whose first household's county is moved off its block and asserts the refusal's message, count and (block, county) pair. Also answers verify's wording findings: the PUMA-wrapper sentence in US_LOCAL_JURISDICTIONS.md no longer says a kept block always lies outside the new county or brings a flag back, a stray line break is reflowed, the two module docstrings name the per-plan congressional-district columns the wrapper also drops, and the annual lane's refusal prefix names a remedy that also covers a block outside its county (microcosm#1453, review round 1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
puma_ladder.py is a seed-attested module, and its module docstring now names the per-plan congressional-district columns, so its attested bytes moved. Written by the tools, not by hand: tools/repin_us_spec_engine_contracts.py (seed_map, seed_protocol, the loader golden and the engine-workflow US spec_sha256) and tools/spec_engine_coverage.py (us-f0-coverage.json), in the order repinner, coverage, repinner, coverage; ruff format then normalized the repinner's quotes. Every --check exits 0, and under policyengine-us 2.2.1 generate_us_bundle_from_constants.py --check prints the new spec_sha256 (microcosm#1453, review round 1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained 47 commits since the 68bbff7 this branch last merged: #1612 (a test of the build-folder publisher), #1596 (the US bundle generator's tombstone renewal), #1603 (UK local-area releases enter build folders through the registry), #1601 (US: one copy of each shared helper, #1585) and #1597 (UK stock cells, #1524). Four files conflicted, all of them pin files. No conflicted hunk takes a whole-file side: each takes main's text, keeps this branch's comment sentences, and leaves the value to the tools. - spec_engine/inventory_coverage.py, one hunk at seed_map and seed_protocol: main's seven comment lines byte for byte (#1585), then this branch's two sentences (#1444 and #1453) on three new lines. - tests/engine_free/shared/test_spec_engine_loader.py, one hunk: the golden value. This branch's two comment lines had merged cleanly. - tests/engine_workflow/us/test_us_multispine_pool_tool.py, one hunk: this branch's two comment lines, whose first is main's line, and the spec_sha256. - docs/evidence/spec-engine/us-f0-coverage.json, five hunks, digests only. One conflict git did not report, in tests/engine_free/us/test_us_spine_blindness.py. #1601 made us_runtime/support_mix.py re-export microcosm.build.holdout's hash_holdout_uniform as hash_uniform. This branch's test_out_of_runtime_implementations_are_exactly_the_runtime_re_exports requires the registry of out-of-runtime implementations to name every such re-export, so it fails on the merged tree though each parent passes alone. The registry gains ("support_mix.py", "microcosm.build.holdout"), so the runtime scans read holdout.py. The pool tool's import graph does not reach support_mix.py (only the support-mix bake-off tool imports it), so the pool test now requires the graph to reach exactly the re-exports in the new _POOL_BUILD_RE_EXPORTS (local_jurisdictions.py), each of which must be registered. The pinned graph size stays 84. main's support_mix.py is unchanged. Both sides changed five definitions, none of them in its logic. Three hold a pin (EXPECTED_HASHES and the two pinned tests above): with the digests masked, the merged AST equals main's and this branch's. The other two are unions. BINDING_MODULES_ADDED_AFTER_BASE in test_transport_evidence_kernels.py holds main's stochastic_assignment and this branch's two modules, 26 names. The imports of tools/build_us_fiscal_refresh_release.py hold main's recert and this branch's rederive_us_local_jurisdiction_flags. Every other definition in the files both sides changed was changed by one side, and the merged AST equals that side's. In tools/build_us_fiscal_refresh_release.py main re-bound US_POPULATION_AGGREGATE_VARIABLES, _signed_component and _engine_known_periods to the shared rules in microcosm.data.compatibility_recertification, and this branch changed _main and added _with_local_jurisdiction_flags. In test_us_fiscal_refresh_builder.py each side added tests. Pins on the merged tree, each written by its tool: - tools/repin_us_spec_engine_contracts.py: seed_map a402868b and seed_protocol 27fd0f86 in inventory_coverage.py; the golden 7a2c31e2 in shared/test_spec_engine_loader.py; the US spec_sha256 3ec2d79d in engine_workflow test_us_multispine_pool_tool.py. Its covered_item_count in engine_contract test_spec_engine_coverage_tool.py ends at main's 41. - tools/spec_engine_coverage.py: us-f0-coverage.json (the seed map and protocol, the seed-stream implementation digest and spec_sha256, each twice); 42395/42395 configuration fields and 41/41 inventory checks. - Unmoved by the merge, as the coverage tool observes them again: full_checkpoint 47d4474d and the stacked pool's code digest 3c5e14be, this branch's values on lines main did not touch. The calibrate parity fixture (direct.csv, graph.json, inputs.csv, pins.json) is byte-identical to main's; this branch changes nothing under microcosm.calibrate. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nd 2) In a legacy ACS pool the block donors sit beside ACS rows with no block, so the base decides only the county-rule flags. When the L0 support drops every ACS row, its geography decides in_yonkers and in_wilmington too, and the export step derived them after selection and refit. That changed engine inputs the weights were fit to: a retained Wilmington itemizer gets the city's earned income tax, a larger SALT deduction and a lower federal income tax, while the refit and the fit gates still report the targets computed without the flag. Under policyengine-us 2.2.1 a Wilmington worker with $100,000 of wages and $20,000 of cash gifts owes $275 less 2026 federal income tax that way. The export step now writes nothing. us_local_jurisdiction_artifact_receipt takes rungs=, the rungs whose flags the artifact must carry (default its own, so the annual lane's parent check is unchanged). It refuses rungs the artifact's own geography does not supply and a carried flag beyond them, and records each flag its own rungs decide beyond them as unavailable, saying why. The builder's export calls it with the rungs of the last support derived before the targets were materialized (the frozen support, else the base) and returns the frame it was given, and it refuses an export with no earlier receipt. The base and frozen-support derivations are unchanged, so the targets still see every flag a frozen support decides. A build whose export rungs equal its calibrated support's returns the same frame object and the same receipt as before. That covers the full-pool path, a base or frozen support that already decides the place rung, and any L0 support that keeps one row without a block. In the hit case the release carries an ordinary pooled release's flags; its own geography still decides the place-rule flags, so the annual lane refuses it as a parent. Rematerializing and refitting on the L0 support would avoid that, at the cost of another engine pass and a second refit inside calibration. Tests: - test_builder_export_writes_no_input_its_targets_were_not_materialized_with: a Yonkers and a Wilmington block donor beside an ACS Denver row; the L0 weights keep the two donors. Every table of the export equals the kept rows of the base, the receipt holds the base's ["county"] rungs, and in_yonkers and in_wilmington read unavailable. It fails on 59d539e. - Three module tests: a property test over pools and selections (a selection held to its support's rungs carries that support's flags, and its receipt equals the unheld one when no rung is added) and one test for each refusal. - The frozen/L0 test now draws the L0 support from the frozen support and checks the export refusal; the stale-block test starts each support from a clean base receipt; the post-export AST test and the diagnostics harness of test_main_writes_diagnostics_before_post_calibration_gate_failure derive or fake the base receipt as _main does. US_LOCAL_JURISDICTIONS.md, the module docstring and the changelog fragment describe the export. No pin moves: the repinner's --legacy-only --check and --check, spec_engine_coverage.py --check and ci_test_plan.py verify exit 0 (microcosm#1453, review round 2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e fit to An engine-scenario regression for review round 2 of microcosm#1453, under policyengine-us. The pool holds a Wilmington itemizer on a block ($100,000 of wages, $20,000 of cash gifts) beside a Denver ACS row with no block, so the base derivation decides only the county rung. The test runs the builder's own base and export steps around attach_l0_refit_entity_weights, which keeps only the Wilmington row, and materializes income_tax, salt_deduction and in_wilmington on both supports. The written support's values equal the pool's first row, the ones the refit used. So that the parity is not vacuous, it also materializes the kept row with its own flags derived again (what the export wrote before the fix): in_wilmington is True, the city's earned income tax is $1,250, the SALT deduction is larger and federal income tax is lower. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
export_us_l0_refit_h5 derived the local-jurisdiction flags on the rows the saved weights keep. When those weights keep only the block-line rows of a pooled base whose ACS rows have no block, the kept rows decide in_yonkers and in_wilmington, so the export wrote inputs the weights were never fit with; the input-mass gate does not fail a column only the export carries. The export now derives the flags on the whole loaded base, the frame the weights were fit on, applies the weights, and holds the kept rows to the base's rungs through us_local_jurisdiction_artifact_receipt, the rule the release builder's export follows since round 2. A flag only the kept rows decide is recorded as unavailable, with the reason. The geography-ladder gate requires every flag a block decides, so by default it refuses that export, and the refusal now says which flags the export withholds and why. A base whose block lies outside its county is refused whichever rows the weights keep. Tests: two engine-free export tests (input parity with the fitted base on the reviewer's Wilmington case; the refusal's message) and an engine scenario test (the written H5 gives the income tax the weights were fit to; writing in_wilmington would lower it). Docs and the changelog fragment follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd 3) Every pool the held-selection property drew had a county on every row, so the support always supplied the county rung and the only flags a selection could add were the place-rule ones. A mutant that gave the special withheld reason to place-rule flags only passed the property. The property's strategy now adds, in half its draws, one or two rows with a state and no county, place or block, so the support can supply no rung and a selection can widen to the county rung or to both. A new explicit example and a named test take a Wilmington donor from a pool whose other row has no county: all four flags are unavailable, each with the reason that the kept row's own geography decides it. The other two properties that share the strategy draw as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iew round 3) Since round 2 the release builder's export stage derives nothing: it keeps the local-jurisdiction flags of the support the targets were materialized from and checks them against the export support's geography. Its telemetry message still said "Derived local-jurisdiction flags from the export support's geography." The export stage now says it kept and checked the flags of the base or frozen support; the base and frozen-support stages keep their message. The structured fields (rungs, carried flags, each flag's status) are unchanged. The support-sequence builder test now reads each stage's message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained 39 commits since 085672f, this branch's last merge: #1615 (US and UK build simulations stop at the year they read), #1618 and #1624 (reference documents trimmed to the keys their kernels read), #1096 (solve's opt-in softmax parametrization projects exactly onto the capped simplex) and #1602 (Belgium's Child rules bindings and Axiom input closure, still inactive). Four files conflicted, all of them pin files. Each conflicted hunk takes main's text, keeps this branch's comment sentences after main's, and leaves the value to the tools. - spec_engine/inventory_coverage.py, one hunk at seed_map and seed_protocol: main's three #1096 comment lines byte for byte, then this branch's three #1444 and #1453 lines byte for byte. - tests/engine_free/shared/test_spec_engine_loader.py, one hunk: main's #1096 comment line, then this branch's two #1453 lines, then the golden value. - tests/engine_workflow/us/test_us_multispine_pool_tool.py, one hunk: main's "attested bytes." line and its #1096 line, then this branch's #1453 sentence on two new lines (same words), then the spec_sha256. - docs/evidence/spec-engine/us-f0-coverage.json, five hunks, digests only, each to main's side. Git merged three other files that both sides changed: tools/build_us_fiscal_refresh_release.py, tests/engine_free/us/test_us_fiscal_refresh_builder.py and tests/engine_free/us/test_us_post_export_scoring.py. In each, every definition outside the pins was changed by one side, and the merged AST equals that side's. In the tool, main changed seven helpers (among them _dataset_from_frame and _materialize_base_simulation_columns) and this branch changed _main, _with_local_jurisdiction_flags and LOCAL_JURISDICTION_FLAG_SUPPORTS. The tool's imports hold main's us_dataset_through and this branch's local-jurisdiction import. Main touched no us_runtime module and no binding-identity list. Pins on the merged tree, each written by its tool: - tools/repin_us_spec_engine_contracts.py: seed_map 5230ab2f and seed_protocol 4ac422f9 in inventory_coverage.py; the golden 23b97b69 in shared/test_spec_engine_loader.py; the US spec_sha256 97f2b938 in engine_workflow test_us_multispine_pool_tool.py. Its covered_item_count in engine_contract test_spec_engine_coverage_tool.py ends at main's 41. - tools/spec_engine_coverage.py: us-f0-coverage.json (the seed map and protocol, the seed-stream implementation digest and spec_sha256, each twice); 42395/42395 configuration fields and 41/41 inventory checks. - Unmoved by the merge, as the coverage tool observes them again: full_checkpoint 47d4474d and the stacked pool's code digest 3c5e14be, this branch's values on lines main did not touch. tools/generate_us_bundle_from_constants.py --check reports the same US spec_sha256, 97f2b938. tools/repin_us_pool_engine_contracts.py --emit-json gives this branch's pinned manifest, 6e70bc74 over 1,059 rows. The release input coverage manifest built on the merged tree equals the committed one. The calibrate parity fixture (direct.csv, graph.json, inputs.csv, pins.json) has main's blob ids; this branch changes nothing under microcosm.calibrate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…w round 4) The changelog said the L0/refit export's written dataset still gives the values its saved weights were fit to. On a base that carries no flags it does not: the export derives the flags the whole base decides, and a fit that read that H5 unchanged saw False. On a base whose households all sit on blocks, a Wilmington itemizer gains in_wilmington, and its 2026 federal income tax trails the value the fit computed by $275. The changelog now claims what both exports enforce: neither writes a flag that only the rows its weights keep decide. The export's docstring and US_LOCAL_JURISDICTIONS.md drop the same broader claim and say that on a base that predates the flags, the derived flags can change a value the fit computed. No code changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d 4) The round-4 verify found three sentences that still told the reader the L0/refit export's saved weights come from a fit on the base it loads. `load_l0_refit_npz` checks that pairing by household count alone. On a base that carries no flags, the export writes the flags the whole base's geography decides, which a fit on the stored base never read. - F1: the engine scenario module docstring now says the export withholds `in_wilmington` when the rows its saved weights keep decide it and the base it loads does not. - F2: the geography-gate refusal note says the export writes only the flags its whole base's geography decides; the pinned test string follows. - F3: US_LOCAL_JURISDICTIONS.md says the export assumes that a fit on the base it loads produced its saved weights, and that `load_l0_refit_npz` checks that assumption by household count alone. - The two `us_local_jurisdictions.py` docstrings that stated the same pairing as fact now say the L0/refit export assumes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
policyengine-us 2.2.1 declares four local-tax jurisdiction flags as inputs with no formula, so each stays
Falseunless the dataset writes it:in_denverin_yonkersny_yonkers_income_tax(resident surcharge)in_wilmingtonde_wilmington_earned_income_taxpa_philadelphia_wage_tax_residentpa_philadelphia_wage_tax(resident rate)No Microcosm build wrote any of the four, and
us/engine_abi.lock.jsonlisted each asdeclared_absent_engine_input. On every built dataset, policyengine-us therefore charged no Yonkers surcharge or Wilmington earned income tax and gave no Denver relief.Change
Each US geography line now derives the flags its own geography decides, from a registry whose SHA-256 the code pins. The L0/refit export and the release builder derive them again from an existing artifact, and the annual lane requires them of its parent. Neither export writes a flag that only the rows its weights keep decide. A PUMA redraw removes every column that only a block decides, and no line or step reads a place from a table in which a household's block lies outside its county.
Main's annual lane now refuses a parent without the flags
build_annual_static_agingcopies the parent's base year byte for byte and keeps the parent's columns in every projected year, so it cannot add a flag. It now reads the parent and checks its flags before any model work:county_fipson every household and lacksin_denverorpa_philadelphia_wage_tax_resident.place_fipsand a 15-digitblock_geoid, it refuses a parent that lacksin_yonkersorin_wilmington.block_geoiddoes not begin with that household'scounty_fips, whatever flags it carries.No commit on main before this change writes those columns. Main's annual lane therefore refuses every certified release that predates this change and has a county on every household, and its message tells the operator to rebuild the parent through a geography line. A parent with no
county_fipscolumn, or with a null county on some household, decides no flag. The lane accepts such a parent when it carries no flag and no block outside its county, and its receipt then records no rung and no carried flag.The lane records the parent's receipt as
base.local_jurisdiction_flagsinannual_manifest.json. An annual cut from a commit before this change runs no flag check and records no receipt. Neithertools/assemble_us_annual_release_dir.pynorvalidate_annual_projection_extensionreads that key, so assembly and publication accept such a cut. Where its parent lacks the flags, every annual file lacks them too, and policyengine-us readsFalsefor every resident. #1622 asks the publication validator to require the receipt.The build rule
This change alters how main builds the US dataset. Under the default
legacylocation rule, a build now writes engine inputs that no earlier build wrote:tools/build_us_puf_support_base.pywrites all four flags.in_denverandpa_philadelphia_wage_tax_resident.The first release after this change can therefore differ from its predecessor in
co_denver_property_tax_reliefand in the totals that include it. Where every household of the support a step derives on sits on a block,ny_yonkers_income_taxandde_wilmington_earned_income_taxcan move too. The release builder derives the flags before it materializes targets, so a target that reads one of these variables can take a new value and move the calibrated weights.pa_philadelphia_wage_taxstays at zero until a line supplies taxable wages.docs/us-release-build-rule.mdasks a pull request that changes how main builds the dataset for a from-scratch build and certification receipt from its own tree. This pull request carries none, so it merges only on a waiver from Max.Registry and derivation
microcosm/build/us_local_jurisdictions.jsonmaps each flag to one 2020 geography, andmicrocosm/build/us_local_jurisdictions.pypins its SHA-256 (45035eee) and holds the derivation.in_denverfor county 08031 and the Philadelphia flag for county 42101, and each county covers the same blocks as its city.in_yonkersfor place 36/84000 andin_wilmingtonfor place 10/77580. Place codes repeat across states, so the rule also matches the state fromcounty_fips[:2].person_household_id.tools/verify_us_local_jurisdiction_registry.pychecks each code against the 2020 Gazetteer, the 2020 CensusP1_001Ncounts and any block ladder. The registry records the result for two ladders, each of which sums to the national 331,449,281:in_denverin_wilmingtonin_yonkerspa_philadelphia_wage_tax_residentBoth files sit outside
us_runtime, so that the annual lane can check a parent under any engine version. Importing anyus_runtimemodule runs that package's__init__, which builds the spine-agreement registry, loads the take-up contract and asserts the engine ABI lock in any environment that has policyengine-us. The lock names policyengine-us 2.2.1, so a parent check that went throughus_runtimewould refuse every parent under any other engine version. The lane imports the check frommicrocosm.build.us_local_jurisdictions, which imports the standard library, numpy, pandas andmicrocosm.frame.us_runtime/local_jurisdictions.pyre-exports the module's 24 public names, and sevenus_runtimemodules and three tools import them there.Geography lines
with_household_us_geography_ladder) writes all four flags from the block's county and place.with_household_us_puma_ladder, which the stacked pool runs) writesin_denverand the Philadelphia flag from the county it draws. It never writesin_yonkersorin_wilmington, removes any stale column, and records both asunavailablein its summary. Because it draws every household's district and county again, it first removes each column that only a block decides:block_geoid,tract_geoid,place_fips,sldu,sldl,cbsa_codeand anycongressional_district_geoid__<plan>column. Withassign_tract=Trueit then writes a fresh tract.with_optional_acs_spine) derives the county-rule flags for every row from the final county and removes the place-rule flags. Without a PUMA ladder it removes every flag, since ACS rows then have no county, andbuild_optional_acs_multispinerecords the pool's receipt asprovenance["local_jurisdiction_flags"]on both paths. With its PUMA ladder, this line calls the kernelassign_us_puma_ladder, which keeps the block columns, and then gives each block-line donor back its own district and county.block_v1draw (with_household_us_block_location, US household location v1: one census block per household, every geography derived #1047, now on main) derives all four flags from the county and place of the block it draws. It derives them after the draw and overwrites any flag the frame carried. The base-h5 line and the stacked pool both run that draw, and underblock_v1the pooled ACS line also derives all four for every row.The derivation raises on missing, null or malformed geography, so no line writes
Falsefor a household whose county or place it does not know. Under the defaultlegacyrule, the flags and receipts equal what the branch gave before it merged #1047. A differential run on a synthetic ladder gave byte-identical flag columns, receipts and gate results at59d539e0and atbcb01948, the head before that merge. The PUMA ladder's column removal changes no production output, because its one production caller, the stacked pool, hands it source frames that carry no block column.Gates
Both ladder gates now fail a flag that disagrees with its rule, a null or non-boolean flag, and a place-rule flag on a table with no complete
place_fips. They also fail a place-rule flag on a table in which a household's block lies outside its county. Each gate also fails a missing flag that its line's own rung decides: the block-ladder gate requiresin_denver,in_yonkersandin_wilmington, and the PUMA-ladder gate requiresin_denver. Underblock_v1the base-h5 line runs the block-ladder gate on the households it draws, and the stacked pool runs the PUMA-ladder gate.Existing artifacts
A base H5 that predates this change carries no flags, so the two steps that start from an existing artifact both call
rederive_us_local_jurisdiction_flags:export_us_l0_refit_h5calls it on the whole loaded base, whose household count must equal the number of saved weights, and then keeps the rows those weights select. Before the geography-ladder gate, it callsus_local_jurisdiction_artifact_receipton the kept rows with the base's rungs, and it records that receipt aslocal_jurisdiction_flagsin its summary.tools/build_us_fiscal_refresh_release.pycalls it on the loaded base or prepared pool, and again on the frozen support when a selection source reduces it, both before it materializes its targets. For the export frame, the rows that the L0 or full-pool weights keep, the builder callsus_local_jurisdiction_artifact_receiptwith the rungs of the last support it derived. That call writes nothing and requires the frame to carry that support's flags and no others. The builder records each receipt undersupport_value_repairs.local_jurisdiction_flagsin the calibration diagnostics, keyedbase,frozen_supportandexport, and reports each as a telemetry stage.The helper reads the rungs from the artifact's own geography: the county rung needs
county_fipson every row, and the place rung also needs a non-nullplace_fipsand a 15-digitblock_geoidon every row. The rungs belong to the whole table, so a selection that drops every row without a block makes the place-rule flags decidable. The builder therefore derives again on the frozen support, and its targets read every flag that support decides. A household or person flag that disagrees with the artifact's geography stops the step.The L0 calibration chooses its support after the builder materializes the targets. When the L0 support of a pooled base keeps only block-line donors, its geography decides
in_yonkersandin_wilmington, but the builder writes neither. Writingin_wilmingtonwould add the city's earned income tax to a retained household's SALT deduction and so change an itemizer's federal income tax, while the refit and the fit gates report targets computed without it. The export receipt records both flags asunavailable, with a reason that names the export's own rungs and those of the last support derived.us_local_jurisdiction_artifact_receiptalso refuses a requested rung that the frame's own geography does not supply, and a carried flag that the requested rungs do not decide.The L0/refit export holds the rows its saved weights keep to its base's rungs in the same way. When those weights keep only the block-line donors of a pooled base whose other rows lack a block, the kept rows' geography decides
in_yonkersandin_wilmington, and the export writes neither. The geography-ladder gate requires both flags where every household sits on a block, so by default the export refuses, and its message names the withheld flags and the reason. With--allow-geography-ladder-gate-failures, it writes the kept rows with the base's flags, and its summary records the gate's failures and the receipt's reasons. Because the export now derives on the whole base, a carried flag that contradicts the base's geography, or a block outside its county, stops the export whichever rows the weights keep.Blocks outside their county
A 2020 block geoid begins with the five digits of its 2020 county. The block ladder and the
block_v1draw writecounty_fipsfrom those five digits, and the pooled ACS line checks each donor's restored county against the donor's tract. Every line puts Connecticut's planning regions incounty_fips_2023, socounty_fipsholds a 2020 county on every line. A household whose 15-digitblock_geoidbegins with another county kept its block, and the block's place, from before its county changed._require_blocks_in_their_countiesrefuses a table with any such household and names the count and the first five (block, county) pairs. It compares each row that carries a 15-digit text block and a 5-digit text county. Each step that reads the place rung from a table runs the check:us_local_jurisdiction_rungs_presentand the place-rung derivation raiseValueError. Both ladder gates turn that error into a failure for each place-rule flag the table carries, and the block-ladder gate also fails such a table on its own check ofcounty_fipsagainst the block.rederive_us_local_jurisdiction_flagsreads the rungs before any flag and raises, so the L0/refit export and the release builder stop.block_v1,build_optional_acs_multispineraises at the pool's flag receipt, before it writes any provenance. The block-location gate oftools/_legacy/build_us_acs_multispine_base.pystays behind it as a second stop.No table that a line writes trips the check, Connecticut included (
test_connecticut_blocks_sit_in_their_county_fips_on_every_line).Engine ABI lock and pool manifest
tools/repin_us_pool_engine_contracts.pywrote the pins of the pool's remaining-stage manifest and regeneratedus/engine_abi.lock.json, which reclassifies six inputs that it listed asdeclared_absent_engine_input:in_denver,pa_philadelphia_wage_tax_residentgeography_derived_jurisdiction_input, available atassembledin_yonkers,in_wilmingtonunavailable_place_jurisdiction_input, with fallbackpolicyengine_default_no_place_rungcounty_fips,congressional_district_geoidpreserved_stacked_engine_input: the stacked PUMA operator writes bothIn
POOL_PROJECTION_INPUT_PROVISION_COUNTS, declared-absent inputs fall from 762 to 756 and preserved inputs rise from 4 to 6, and the manifest digest moves fromd4fb9de9to6e70bc74over 1,059 rows. The manifest describes the default rule, so for ablock_v1pool it still lists the two place-rule flags as unavailable although that pool writes them. The stacked pool's checkpoint identity includes the manifest's receipt, so the pool tool resumes no stacked checkpoint that predates this change.Pins
The pin tools wrote these values, each of which moved against main
e555b4d8:inventory_coverage.pyEXPECTED_HASHES:full_checkpointecf64f2b→47d4474dtools/repin_us_spec_engine_contracts.pyinventory_coverage.pyEXPECTED_HASHES:seed_map8d0542f7→5230ab2finventory_coverage.pyEXPECTED_HASHES:seed_protocol3ab23e82→4ac422f9test_spec_engine_loader.pygoldenspec_sha256fac42090→23b97b69test_us_multispine_pool_tool.pyspec_sha256fa75bd68→97f2b938docs/evidence/spec-engine/us-f0-coverage.json731c8c33→3c5e14betools/spec_engine_coverage.pymultispine_pool.pymanifest digest and provision counts, andus/engine_abi.lock.jsond4fb9de9→6e70bc74tools/repin_us_pool_engine_contracts.pyus_post_transfer.jsond4fb9de9→6e70bc74tools/graph_us_post_transfer_fixture.pyThe seed and spec digests move because
geography_ladder.pyandpuma_ladder.pychanged, and the seed protocol attests their bytes. The first review round changedpuma_ladder.pytwice, in its code and then in a docstring, and the last two merges with main brought main's own seed and spec values, the second from #1096's change tosolve. Each time,repin_us_spec_engine_contracts.pyrewrote the seed map, the seed protocol, the loader's golden and the engine-workflowspec_sha256, andspec_engine_coverage.pyrewrote their copies inus-f0-coverage.json: in6ed3ccd0,d7a90ecc,59d539e0ande4f52d92. The full checkpoint, the pool's code digest, the manifest and the graph fixture kept their values through the three review rounds and both of those merges.Three more test pins follow from the code.
test_spec_engine_engine_abi.pyasserts the new manifest digest,BINDING_MODULES_ADDED_AFTER_BASEgains the two new module names, and the spine-blindness test counts 84 modules in the pool tool's runtime graph. The registry pin45035eeekept its value when the files moved.At
59d539e0, and again atdb83dde8, the--checkmodes ofrepin_us_spec_engine_contracts.py(with and without--legacy-only) andspec_engine_coverage.pyexit 0, as doesci_test_plan.py verify. Under policyengine-us 2.2.1, at both commits,generate_us_bundle_from_constants.py --checkexits 0 and prints3ec2d79d, andrepin_us_pool_engine_contracts.py --emit-jsonreturns6e70bc74over 1,059 rows. The second review's commits,77984c91anddb83dde8, move no pin.The third review's commits move no pin either, and under policyengine-us 2.2.1 the bundle check at
68effae8still prints3ec2d79d. With4a9859db's changes in the tree, both--checkmodes ofrepin_us_spec_engine_contracts.py,spec_engine_coverage.py --checkandci_test_plan.py verifyexit 0. The two later commits change two test files and the release builder, which no pin attests, and they add, move or remove no test file and add no engine guard.At the merge commit
e4f52d92, which holds every commit of the third review, both--checkmodes ofrepin_us_spec_engine_contracts.py,spec_engine_coverage.py --checkandci_test_plan.py verifyexit 0. Under policyengine-us 2.2.1 the bundle check prints97f2b938, thespec_sha256the tools wrote, and the pool tool's--emit-jsonreturns6e70bc74over 1,059 rows.87a03db1and59e14835, which answer the fourth review, move no pin. At each, both--checkmodes ofrepin_us_spec_engine_contracts.py,spec_engine_coverage.py --checkandci_test_plan.py verifyexit 0.What the datasets still do not do
in_yonkersorin_wilmington. Each receipt records both asunavailable, and the engine readsFalsefor every row. The place rung belongs to the whole table, so one household without a block withholds both flags from every household, including donors whose stored place lies in Yonkers or Wilmington.Falsefor every flag. The L0/refit export derives the flags that base's geography decides, and they can change a value the fit computed. For example, on a base whose households all sit on blocks, the export writesin_wilmingtonfor the second review's Wilmington itemizer, whose 2026 federal income tax then trails the fit's value by $275. Check that L0/refit saved weights match their base and the flags their fit read #1636 asks each writer of saved weights to record the flags its fit read, so that the export can refuse to write any other.pa_philadelphia_wage_tax_taxable_wages, so policyengine-us still computes zero Philadelphia wage tax on built datasets. Derive Philadelphia taxable wages and record the work-location inputs as unavailable #1462, a draft, derives those wages.Merges with main
The branch merged main four times after
1818486a:1c65d4d8merges main1e883190with seven conflicted files, in which the annual module's docstring keeps both sides' paragraphs andl0_refit_export.pykeeps both imports. The annual doc'sbasebullet takes main's wording plus the receipt clause.8a5b3aecmerges main68bbff77and so brings US household location v1: one census block per household, every geography derived #1047, again with seven conflicted files. The import block ofacs_multispine.pyand the__all__ofbase_pool.pykeep both sides' names, and the spine-blindness count becomes 84, the number the test's own graph helper reaches.59d539e0merges main085672fa(Test that the build-folder publisher publishes the SCF education-debt fit tables #1612, Renew the US bundle generator's migration tombstone to 16 February 2027 (#1585) #1596, Admit a UK local-area release to build folders only once the registry names it #1603, US: one copy of each shared helper (#1585 items 4, 5 and 8.3) #1601, UK stock cells: homes and the land under them in England and Wales (#1524) #1597), with four conflicted files, all pins. US: one copy of each shared helper (#1585 items 4, 5 and 8.3) #1601 madeus_runtime/support_mix.pyre-exportmicrocosm.build.holdout'shash_holdout_uniformashash_uniform. So the spine-blindness registry gains that pair, and the pool-graph test now checks only the re-export the pool reaches,local_jurisdictions.py.e4f52d92merges maine555b4d8(Stop US and UK build simulations at the year they read #1615, Trim reference documents to the keys their kernels read #1618, Project the softmax parametrization exactly onto the capped simplex each step #1096, Tighten the reference-trim tests and wording from #1618's review #1624, Add Belgium's Child rules bindings and Axiom input closure (PR 4a), still inactive #1602), with four conflicted files, all pins. Intools/build_us_fiscal_refresh_release.py, which git merged, main changed seven helpers and this branch changed_mainand_with_local_jurisdiction_flags. No function changed on both sides, and the tool's imports hold both sides' names. Main's Stop US and UK build simulations at the year they read #1615 changes how the US engine adapter'smaterializebuilds its dataset, and this branch's engine scenario test callsmaterialize. The fourth review reported that main's change preserves the scenario's meaning and that a real-engine test passed on the merged head, with fitted and written income tax both at $11,240.82.In each merge the pin files took main's side, and the pin tools then rewrote them. The first two merge commits differ from git's automatic merge in their seven conflicted files alone,
59d539e0differs in its four and intest_us_spine_blindness.py, ande4f52d92differs in its four alone. Git mergedwith_optional_acs_spineandbuild_optional_acs_multispinewithout conflict in the second merge, yet underblock_v1the merged pool still chose its flag rungs from the PUMA ladder and the draw wrote no flag.56b63ebbmakes the rungs follow the location rule and derives the flags after the draw.Review
A first review at
d8e1490dfound two problems:in_yonkerson a household the redraw had moved to the Bronx.21ec4d95removes those columns in the PUMA wrapper and refuses any table with a block outside its county, and4030c014adds that sequence as examples of the line property. The new check then failedtest__legacy_tool__block_v1_gate_failure_is_hard, whose harness changed a county before it computed the pool's provenance.42c021bbchanges the county after the provenance and tests the build's own refusal.docs/us-annual-static-aging.mdandUS_LOCAL_JURISDICTIONS.mdtold the operator to rebuild every parent that predates the flags.21ec4d95limits that, in both, to a parent with acounty_fipson every household. Both now say the lane accepts a parent with nocounty_fipscolumn, or with a null county on some household, when it carries no flag.6ed3ccd0andd7a90ecchold the pins the tools rewrote after21ec4d95and42c021bb.A second review at
59d539e0found one problem. In a pooled base whose ACS rows carry no block, an L0 support that kept only block-line donors gained the place rung, and the export step then wrotein_yonkersandin_wilmingtonafter the fit. The written dataset no longer gave the targets the refit had matched. For example, under policyengine-us 2.2.1, a retained Wilmington worker with $100,000 of wages and $20,000 of cash gifts to charity owed $275 less 2026 federal income tax, while the fit gates reported the fitted value.77984c91makes the export keep the flags of the support the targets came from and record the others asunavailable, anddb83dde8adds the engine regression. Rematerializing the targets and refitting on the L0 support would also keep the dataset and its targets consistent, at the cost of another engine pass and a second refit inside calibration.A third review at
db83dde8found three problems:export_us_l0_refit_h5still derived the flags on the rows its saved weights kept, so it could addin_wilmingtonafter the fit, as the release builder had before the second review. The input-mass gate passes a column that only the export carries, so nothing stopped it.4a9859dbmakes the export derive on the whole base and hold the kept rows to the base's rungs. It adds two engine-free tests, which fail on the old export, and an engine test on the review's two Delaware itemizers.46ea7d76adds households with a state and no county to that property's draws, an example whose support has no rung while its selection has both, and a test that checks the reason for all four flags.68effae8changes that message to say the stage keeps the flags of the support the targets came from and checks them, and the builder test now reads the frozen-support and export messages.A fourth review read the third review's answers and the merge with main at
e4f52d92, and approved them. It found the export problem closed, killed four mutants of that fix in memory and checked the answers to the other two problems. It rebuilt the merge in memory and found thate4f52d92departs from git's own merge only in pin digests and in comment lines the branch kept.It raised one problem. The changelog promised that each written dataset still gives the values its fit computed. On a base that carries no flags and whose households all sit on blocks, the L0/refit export writes
in_wilmington, which a fit on the stored base never read. The second review's Wilmington itemizer then owes $275 less 2026 federal income tax than the fit computed.87a03db1and59e14835narrow that promise to what the code does, in the changelog, the export's docstring and refusal message,US_LOCAL_JURISDICTIONS.md, the engine test's docstring and two docstrings ofus_local_jurisdictions.py. #1636 takes the two remaining gaps: the export ties saved weights to their base by household count alone, and on a base without flags it writes flags a fit on that base never read.Invariants
Each invariant holds for every input its test generates or lists:
county_fipsequals the registry county and for no other, and a place-rule flag for the households whose state and place both match.test_flag_is_true_iff_county_or_state_and_place_matchcompares both with codes that the test writes out.test_flag_weighted_block_population_equals_the_jurisdictions_blocks).test_derivation_commutes_with_household_row_order,test_person_flags_carry_their_household_value_in_any_person_order).test_a_line_without_a_place_rung_never_writes_a_place_flag,test_unknown_or_malformed_geography_raises_instead_of_writing_false).test_rederiving_a_pre_flag_artifact_restores_the_block_line_flags,test_rederiving_after_a_selection_equals_deriving_the_kept_rows).test_gate_requires_every_flag_its_rungs_decide).test_parent_without_its_flags_is_refused_before_model_work).us_runtimemodule, nospec_engine.engine_abiand nopolicyengine_us(test_the_annual_parent_check_leaves_us_runtime_unloaded).test_builder_derives_flags_after_every_support_reduction).us_runtimealso read each implementation that aus_runtimemodule re-exports (test_out_of_runtime_implementations_are_exactly_the_runtime_re_exports).block_v1draw, each flag equals its rule on the block the draw chose, whatever the frame carried before (test_frame_location_writes_the_flags_its_drawn_block_decides, with one and three clones).block_v1draw leaves no column that only a block decides, with or without a fresh tract (test_a_puma_redraw_leaves_no_column_only_a_block_decides,test_a_puma_redraw_of_a_block_v1_frame_leaves_no_block_lookup).Trueon a household whose county holds none of the flag's jurisdiction, and no household carries a block outside its county (test_no_sequence_of_lines_leaves_a_flag_or_block_outside_its_county).test_rungs_are_never_read_from_a_table_with_a_block_outside_its_county).unavailable(test_builder_export_writes_no_input_its_targets_were_not_materialized_with). Under policyengine-us 2.2.1 the written support gives theincome_tax,salt_deductionandin_wilmingtonthat the targets used, though writingin_wilmingtonwould lower that household's income tax (test_a_block_only_l0_export_keeps_the_income_tax_its_weights_were_fit_to). The L0/refit export writes no flag that only the rows its weights keep decide. Each table it writes equals the kept rows of its base, with the flags the whole base decides, and its receipt records each withheld flag asunavailable(test_export_writes_no_flag_the_base_its_weights_were_fit_on_did_not_decide). For two Delaware itemizers whose base decides no place-rule flag, the export's written rows give the base'sincome_tax,salt_deductionandin_wilmingtonunder the same engine (test_a_block_only_saved_weight_export_keeps_the_income_tax_they_were_fit_to).unavailable. With no rung added, its receipt equals the unheld one, and the test draws supports in which some household has no county (test_a_selection_held_to_its_supports_rungs_keeps_that_supports_flags). When the support has no rung and the selection has both, each of the four flags gets a reason that names both lists of rungs (test_a_selection_held_to_a_support_with_no_rung_says_why_for_every_flag).Each of these mutations fails its test:
us_runtimeand fails test 8. Calling the take-up projection inside the receipt function loadsspec_engine.engine_abiand fails the same test.test_pool_build_tool_import_graph_is_source_spine_blind.test_a_puma_redraw_cannot_bring_a_place_flag_back, the first review's sequence.county_fips_2023fails the Connecticut test.test__multispine__block_v1_refuses_a_pool_block_outside_its_county, and a block-location gate that ignorescounty_fipsfailstest__legacy_tool__block_v1_gate_failure_is_hard.test_builder_flags_follow_the_support_through_frozen_and_l0_selection.test_a_selection_cannot_be_held_to_rungs_its_geography_does_not_supply. Giving the withheld flags the generic reason fails test 16 and the builder test of invariant 15.db83dde8did, failstest_export_writes_no_flag_the_base_its_weights_were_fit_on_did_not_decideandtest_export_says_which_flags_it_withholds_when_the_geography_gate_refuses.test_a_selection_held_to_a_support_with_no_rung_says_why_for_every_flag, and test 16 on its no-rung example and on its draws.test_builder_flags_follow_the_support_through_frozen_and_l0_selection.Tests
Four batches ran at
d8e1490d, before the first review:tools/build_us_fiscal_refresh_release.py, withtest_us_post_export_scoring.pyThe three expected failures sit in
test_us_release_base_input_invariant.py, which this branch leaves unchanged.The eight engine files hold the engine contract and scenario tests, the annual-lane, pool-tool and frame-serializer workflow tests, the ABI and coverage-tool contract tests, and the annual import test with the engine installed. After the first review's fix, the contract test (8 passed) and the scenario test (5 passed) ran under policyengine-us 2.2.1 at
4030c014. At59d539e0, under the same engine, the coverage-tool, inventory-coverage and ABI contract tests gave 7, 2 and 1 passed, the whole pool-tool workflow test 31 passed, and the graph's post-transfer parity test 1 passed.The engine contract test asserts that each flag stays an input-only boolean on its entity, with default
False, and that each consumer the registry lists reads it. The engine scenario test places synthetic households withwith_household_us_geography_ladderand runs them throughPolicyEngineUSEngine.materialize. It asserts that the Yonkers surcharge, the Wilmington earned income tax and Denver renter relief come out nonzero for the resident and zero for the neighbour. Once the test strips the flags, each resident falls to the neighbour's result. With taxable wages in the fixture, the Philadelphia resident pays the resident rate and the Delaware County neighbour the nonresident rate.From the first review's fix on, every local run behind these counts took one test file per process. At
4030c014, 14 engine-free files that call the changed code gave 1,285 passed and 9 skipped, andtest_us_acs_block_location.pygave 65 passed atd7a90ecc. At59d539e0, the engine-free files that read what the merge combined gave these results:test_us_spine_blindness.pytest_us_fiscal_refresh_builder.pytest_transport_evidence_kernels.pytest_spec_engine_inventory_coverage.py,test_spec_engine_seeds.pytest_spec_engine_loader.pytest_us_compatibility_recertification_parity.py,test_us_spec_bundle.pyAt
db83dde8, the files that hold the new tests, or call the code that77984c91changed, gave these results:test_us_spine_blindness.pytest_us_fiscal_refresh_builder.pytest_us_local_jurisdictions.pytest_us_l0_refit_export.pytest_us_post_export_scoring.pytest_us_annual_static_aging_parent_flags.pytest_us_local_jurisdiction_flags_engine.py, under policyengine-us 2.2.1With
4a9859db's changes in the tree, these files ran one per process:test_us_l0_refit_export.pytest_us_spine_blindness.pytest_us_local_jurisdiction_flags_engine.py, under policyengine-us 2.2.1With only the two new export tests added to
db83dde8, those two failed and the file's four other flag tests passed. A differential run then compared the old and new exports on 600 random pools, saved weights and base flags. In the 521 cases where the kept rows decide no rung beyond their base's, both exports wrote equal tables, weights and summaries. In the 51 cases where the kept rows decide more, the new export wrote the base's flags and withheld those the old one added. Of 28 bases with a household whose block lies outside its county, both exports refused 18, and the new export alone refused the 10 in which the weights dropped that block's household.After a disk limit stopped local pytest, one Python process loaded each test module that
46ea7d76and68effae8change and called its tests directly, with no test runner. Every test it called passed, including test 16 over 80 draws that reach all six pairs of support and selection rungs. With the third review's mutant swapped into the same process,test_a_selection_held_to_a_support_with_no_rung_says_why_for_every_flag, test 16's no-rung example and test 16's draws failed, while 80 draws limited to supports with a county still passed. With the old telemetry message swapped in,test_builder_flags_follow_the_support_through_frozen_and_l0_selectionfailed, andtest_builder_export_writes_no_input_its_targets_were_not_materialized_with, which reads no message, passed. CI givestest_us_local_jurisdictions.pyandtest_us_fiscal_refresh_builder.pytheir first whole run since those commits.At
59e14835, these files ran one per process:test_us_l0_refit_export.pytest_us_spine_blindness.pyIn the engine scenario test and in
us_local_jurisdictions.py,59e14835changes only docstrings, and CI runs that test andtest_us_local_jurisdictions.pyon the pushed head.No batch ran locally after
d8e1490d, so CI tests the pushed head. Its engine-free job runs 805 files on Python 3.13 and 3.14, and its US engine job runs 90 files under policyengine-us 2.2.1. Between them the two jobs hold every file of the four batches.axiom: n/a: data build and release tooling; no policy rule changes.
🤖 Generated with Claude Code