Repository navigation
Conversation
Materialize observed household rosters, exclude ACS group quarters, and let solver mass fit the ACS household total across release solve arms. Keep SOI status targets in the existing tax-unit universe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retire the two national Historic Table 2 credit aggregates in favor of the newer filing-status partition. Require all four status amount cells and apply the existing critical credit fit bounds to the new rows. Register the household measurement universe and verify exclusion boundaries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add eight scoped pairs and exactly 3,094 facts from Chronicle 60266a3. The full rebuild preserves every prior row byte for byte. Record per-cell parity decisions, including the two superseded national CTC aggregates, and document the resulting net 22-target increase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Commit the exact feed comparison and added-row listing. Follow both reviewed source namespaces in the existing anchor proof chain, retaining the prior refusal checks and testing cross-source and out-of-scope additions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Distinguish the congressional-district population-age branch from the new national household-type targets in the fact-to-target guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main moved 544 commits. Two additions collided: the release builder now imports both the household-type materializer and main's wider input-mass helpers, and the transport-evidence module list keeps both this branch's household-type modules and main's microcosm.data modules. Main still pins Chronicle feed 41b2fb2, which this branch's feed 60266a3 builds on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's 88 commits conflict in one place: both sides add modules to BINDING_MODULES_ADDED_AFTER_BASE in the transport evidence test, and the merge keeps both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…880) The per-target exact-critical guard refused recertification of every current parent: the national SOI Table 4.3 itemized-deductions row carries ledger_filter_income_percentile_range="all", and the scorer rejected any ledger filter key it did not list. The release tool skips a filter whose value selects everything, so the scorer now does the same, with the release tool's own no-op rule. Rows the release tool narrows by EITC (the positive-EITC domain, or a child count the row names or its record set or measure implies) stay unscored, so accepting no-op filters cannot score a row the scorer does not reproduce. Both guards stay. On the target surface the pinned feed compiles (33,272 specs), no exact critical row is now unscored; before, the Table 4.3 row was. The only definition added is the Table 4.3 metric, equal to its Historic Table 2 and Table 2.1 siblings apart from its ids. Drifts certified before this change, for parents that carry the row, attest definitions that leave it unscored, so they no longer recompute; the doc says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frame built before the HRHTYPE carry classifies its ASEC households by the roster, so its ASEC group quarters count as housing households, and nothing recorded that. The fallback now warns, and the release records a household-universe receipt in its calibration diagnostics (build.target_compilation.household_universe): households and initial weight classified by TYPEHUGQ, HRHTYPE and the roster fallback, and each household-type target's fallback part. Only the release's _load_or_materialize_target_frame attaches it, after the checkpoint write, so scorers that digest a compilation per household slice are unchanged. A frame that carries HRHTYPE now refuses a missing code on an ASEC-arm household or on any household without TYPEHUGQ, and a household carrying both codes. The arm comes from the first provenance tag present, spine before support channel, as the SPM universe producer reads it. Frames with no HRHTYPE column, such as today's certified frame, keep the roster fallback with unchanged numeric output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 3 found that the recertification scorer's no-op ledger filter rule and its EITC mask mirror copy release-tool logic with no committed test binding them. The parity module now compares each copy with the tool over generated values and rows, and checks that the record-set suffixes and measure prefixes the tool reads as an EITC child count match the scorer's lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
One more step before #1610 merges. #1668 (at #1668's doc records what exists so far. The 17 November check compiled |
Review round 4 raised five P2s and one P3. This commit answers the four P2s that concern its files. The fifth asks to confirm #1610's follow-up outside this PR, and the P3's attribution fix goes in the PR body. - P2, the reviewer's check could clear a rerun it should not. From packages/microcosm-build, the documented git diff printed nothing, though from the root it listed 101 files. The reviewer now fetches the change's head and main, naming origin/main as main's destination, and stops if the fetch fails. They write both commits as full shas, the head's from gh pr view, and go on only if git merge-tree exits 0. They run git diff with :(top) paths and --exit-code, take only exit 0 with no output as a match, and check again if either sha moves before the merge. - P2 (inferred), the evidence could describe another environment. The operator runs both compiles from one clean checkout of the tested commit, in the environment that uv sync --all-packages --locked --extra us builds. In a receipt, the operator records the interpreter, the imported packages' versions, the runner's and comparator's sha256, both feed digests, where the compile imported the lane's tool and each workspace and overridden package from, and git status --porcelain, and lists each departure. The reviewer checks the digests against the two pins, checks that git status printed nothing, and checks each import path against the checkout and the departures. A commit that records the result changes only the changelog and the doc's paragraph on the allowed cut, and reports a result from the runner, comparator and inputs the receipt names. - P2, a non-string cut passed all three guards. The full-id guard now requires a str before its length and character checks, and the malformed cases gain a tuple of the cut's characters and the cut's bytes. - P2, the pin-state matrix let two weaker county-pin guards pass. It gains a main pin that matches the cut in all but its last character, and main and county pins at two different later commits with no cut. The county-pin guard alone refuses each. Four copies of the test file each carry one fault: county-pin guards that compare seven- or ten-character prefixes, a county-pin guard that accepts any main pin but 41b2fb2 once the county pin moves and the cut holds None, and a full-id guard without the str check. On the engine-free harness, each copy fails one new case and passes the other 24 pin-state cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
As of its round-3 and round-4 fixes ( In a receipt in the evidence, the operator records the tested commit, the interpreter version, the resolved versions of the packages the compile imports, and both feeds' digests. The receipt also carries the sha256 of the script that runs the compiles (the runner) and of the script that compares the lane's selections (the comparator). The operator records where the compile imported the lane's tool ( A later commit that records the result changes nothing but the changelog and the paragraph of #1668's doc that starts "Today the US feed owner allows". It reports a result from the runner, the comparator and the inputs the receipt names. That paragraph exists only on #1668's branch today, so #1610 gains it with the merge of main. If main moves after the compiles, or #1610 gains any other commit, the operator compiles again. The reviewer can skip that rerun on finding no compile-input difference between the tested commit and the merge tree, the tree that main's tip and #1610's head merge to. That check covers The earlier comment put the compiles at #1610's head after its last merge of main and named no rerun if main moved again. It let a later commit keep them if it touched no file under |
WIP (draft). Review has finished. This PR still waits on two things:
test_the_county_pin_moves_with_the_next_main_pin, from Pin US county facts beside the main feed for the county rung #1614 for County-calibrated County Files (3,144 counties, 2026 + 2027): build epic #922). This branch's feed (Chronicle60266a3) is cut on the split pin41b2fb2, before the county pin's3e9d3a0, so CI on the merge with main fails that test. Re-cutting on Chronicle main would change 20 other scope packages. The US feed owner chose to keep this cut. Allow one feed cut on the split pin until the combined re-pin #1668 (draft) allows it as main's one feed cut on41b2fb2until a later re-pin moves both pins together. This branch merges main once Allow one feed cut on the split pin until the combined re-pin #1668 is on it.spm-20260915) as a base: it lacks the congressional-district vintage attributes and the PUF capital-gains tail column. So the receipt needs another frame; the 3 Nov Route A base is proposed. That choice is the maintainer's call.PolicyEngine/chronicle#430 merged (b428468d), and the feed this branch pins is pushed as branch
feed/us-41b2fb2-household-880with tagmicrocosm-us-feed-880-v1(60266a3). Its five packages are byte-identical to what #430 merged.Review round 1 (11/13) asked for changes. Three commits answer its findings:
555e38dc4(P1): the household cells counted ASEC group quarters as housing households. The ASEC pool now carries the native HRHTYPE by source household id, and HRHTYPE 9 or 10 adds zero to every ACS housing-household cell, while its people and weights stay. The pinned 2024 file holds 59 such records, each with one householder, so the householder check couldn't exclude them.759541cec(P1): compatibility recertification left the eight status-level credit rows unscored, socombine_driftrefused. The SOI metric builder now scores the four national filing-status slices, and every unscored exact critical target refuses recertification.a7aa3abb2(P2): the free-mass test read syntax only. A 24-record housing-plus-group-quarters fixture now runs the release's own solve arms (dense, L0 with refit, and exact-k on the subset and the full pool) and checks the household, person, fiscal and group-quarters totals.Review round 2 (11/15) found round 1's findings closed and asked for changes on two new P1s. Two commits answer them:
527b8a900: the rule that every unscored exact critical target refuses recertification refused every current parent, because the national SOI Table 4.3 itemized-deduction row carriesledger_filter_income_percentile_range="all", which the scorer rejected. The scorer now treats a filter value that selects everything as no filter, with the release tool's own rule, and refuses rows the release tool narrows by EITC, which it doesn't reproduce. On the target surface the pinned feed compiles (33,272 specs), no exact critical row is left unscored; the definitions gain only the Table 4.3 metric. Drifts certified before this change, for parents that carry that row, no longer recompute. One such child is published (populace-us-2024-b5db384-recert-pe-us-2.90.4-r085672fa9); round 2's head already refused it, and strict validation refuses it once producer files change.3a950a6f9: frames without HRHTYPE used the roster silently, and missing values in a present HRHTYPE column slipped through. The fallback now warns, and the release records a household-universe receipt atbuild.target_compilation.household_universein its calibration diagnostics: households and initial weight by TYPEHUGQ, HRHTYPE and roster fallback, and each household-type target's fallback part. A frame that carries HRHTYPE refuses a missing code on an ASEC-arm household or on any household without TYPEHUGQ, and a household carrying both codes. The arm comes from the first provenance tag present, spine before support channel, as the SPM universe producer reads it.One limit carries forward. Today's frame has no HRHTYPE, so on it the materializer falls back to the roster and still counts those 59 records as housing households (48 of them also as living alone). A re-solve can't recover the field. On that frame every ASEC household falls back, so the receipt above bounds what the fallback reaches but can't isolate the 59. A from-scratch build carries the field and excludes them, so the re-solve receipt will say so, and the next certified build removes them.
Review round 3 (11/17), a scoped round on round 2's two findings and the commits above, approved: both findings are closed. It raised one P2: the scorer's no-op filter rule and EITC mask mirror copy release-tool logic with no committed test binding them.
e03aab364adds that test. The parity module compares each copy with the tool over generated values and rows, and checks the record-set suffixes and measure prefixes the tool reads as an EITC child count against the scorer's lists. Each of the seven behaviour-changing mutants tried on the two copies fails it.One overlap stays bound, as it was before this change. The 51 state Historic Table 2 TY2022 CTC cells split the same credit vectors as the new Table 1.3 TY2023 status cells. Their claims sum to 37,955,640, against 38,949,527 for the status cells (2.6% apart). Aged to 2024, their amounts sum to $92.56 billion, against $91.78 billion (0.85% apart, the other way). The ACS-local lane selects the same state cells, so this PR keeps them, and their retirement or rebase moves to the re-pin that moves both Chronicle pins together. The doc paragraph that records this lands with the next merge of main. The re-solve receipt reports both partitions' fits. It also reports the eight Table 1.3 CTC cells against their 15% critical-fit limit; as new rows, they have no incumbent escape.
The household-type rule (exactly one householder per housing household) refuses today's certified frame on its own: 297 of its 57,240 households (180 PUF-channel, 117 ASEC-channel; weight 621,264) have no member with
A_EXPRRP1 or 2. The 3 Nov Route A base, built from scratch at9434c504b, has none among its 352,932 households, so builds from the current pipeline pass it.Merges of main:
f47e1ee8d(544 commits; two collisions, both sides kept) andda1576a5(88 commits; one collision in the transport evidence test's module list, both sides kept). Main has since moved pasta21fa3b9b. The branch merges witha21fa3b9bwithout conflicts, but CI on that merge (run 44096261674) failstest_the_county_pin_moves_with_the_next_main_pin. That test reached main after this branch's last merge of main; see the feed pin item above. That is the run's only failing test. On Python 3.13 and 3.14 the engine-free jobs passed 19,136 tests and skipped 156, and every other test job passed. Main still pins Chronicle feed41b2fb2, which this branch's feed builds on.Artifacts: the round-1 fix build's report, commands and logs, and the round-2 fix workflow's progress files and logs, are in the operator's review folder (
artifacts-r2/,fix-r3/).Refs #880.
What changes
The US build has too many married couples with children and too few people
living alone (#880). Bind eight national ACS 2024 household cells and sixteen
IRS SOI TY2023 Table 1.3 filing-status cells by default. Route A's
national_statesurface grows from 6,102 to 6,124 targets; the full surfacegrows from 33,250 to 33,272. These are 24 additions minus two superseded national HT2 CTC
aggregates, a net increase of 22 rather than the initial expectation of 24.
Every other target's name, value and metadata stays identical.
The ACS cells are B11001 001, 003, 005, 006 and 008, and B11003 003, 010 and
016. They constrain household count, married couples, other families by
householder sex, living alone and families with own children under 18. State
estimates, Puerto Rico, margins of error, implied cells and child-age splits
receive reviewed exclusions.
Table 1.3 binds single, married filing jointly/surviving spouse, married filing
separately and head of household for returns, AGI, nonrefundable child and
other dependent credit returns, and credit amount. Its all-status returns
and AGI repeat Table 1.2; its CTC totals remain implied by the bound status
cells. The pinned Table 1.2 has no CTC totals. The existing TY2022 national HT2
CTC count and amount measure the same credit as the new status sums, with
different totals, so reviewed exclusions retire those older aggregates. The status cells use the
existing SOI tax-unit universe with no filer mask. Whether the whole SOI
family should count only filed returns remains a separate modeling question.
The household materializer reads the frame's carried person roster, requires
one householder per housing household and excludes native group quarters:
ACS TYPEHUGQ 2/3 and ASEC HRHTYPE 9/10. Frames built before the HRHTYPE carry
classify ASEC households by the roster, with the warning and receipt above;
missing householders refuse. All release solve arms use free
mass so B11001 001 can fit the ACS housing-unit total even when group-quarters
placeholders carry positive weights. The starting person-population check and weight caps still apply. The
profile gate requires the four status amount cells, and the existing default 15%
CTC critical-fit tolerance covers each status's count and amount, retaining
the existing incumbent-improvement rule.
Feed and parity
Re-pin Chronicle to
60266a3088319c909cf7fc89c9d00d3b055dc6e1(
microcosm-us-feed-880-v1). The feed adds exactly 3,094 rows: 3,074 ACSestimates/margins of error and 20 Table 1.3 cells. Removing those eight new
(record set, period) pairs reproduces all 40,642 old rows byte for byte.
The new feed has 43,736 rows; SHA256
dce5c32e7f0f386260638547438b1d2fafd08a890d09f3099c22b7fca2a95e2d.It retains
chronicle.consumer_fact.v3and excludes the separate CensusVintage 2025 re-pin. The regenerated parity resources identify 34 compiled
families and 64 reviewed exclusions and account for every new cell.
Invariants and validation
For every valid generated roster:
indicators partition households.
Synthetic compile tests verify the bound cells, metadata and exclusion
reasons. Materializer tests cover missing data, invalid relationships, group
quarters and batched release wiring. The release mass regression checks every
solve arm. Focused suites passed: 17 compile, 33 materializer, and 4 population
and solver-policy tests. Ruff and test-plan verification passed.
On 11/12, before the two merges of main and rounds 2 and 3, the whole engine-free job ran locally with four pytest workers: 26,838 passed,
280 skipped, three xfailed and three failed in 4,683.63 seconds. All 50 new
tests passed. One source-sweep failure came from generated scratch files and
passed after cleanup. A focused recheck with a workspace Hugging Face cache
confirmed that the existing live US
latest.jsonreturns HTTP 404; the otherremaining failure is the sandbox's denial of
psin the existing credentialwrapper security test. Those loader, registry and wrapper paths are unchanged.
The live pointer and the security check need ordinary CI/operator follow-up.
The doc-linked staging frame expands to 19.4 GB, above the optional 8 GB
cross-check limit. No calibration, from-scratch build or release ran here.
Under the US release build rule, Max waived
this PR's from-scratch receipt: the re-solve receipt follows review, and the
next certified US build on main supplies the from-scratch run. B11007 and
B12002 remain follow-ups from #1212.
axiom: n/a: calibration targets and feed re-pin
🤖 Generated with Claude Code