Skip to content

Calibrate federal student loan balances to Federal Student Aid's portfolio (#1074) - #1357

Merged
MaxGhenis merged 40 commits into
mainfrom
fsa-student-loan-calibration-targets
Nov 7, 2026
Merged

MaxGhenis merged 40 commits into
mainfrom
fsa-student-loan-calibration-targets

Conversation

@DTrim99

@DTrim99 DTrim99 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

The receipt: 130 targets and the L2 grid on main (2026-11-06)

#1246 squash-merged into main (b563fbb6f), and Max merged main into this branch (2a8d4070) and named the FSA targets module in BINDING_MODULES_ADDED_AFTER_BASE (493a09e7). This push puts #1525 and the round below on top of 493a09e7, with the ed_fsa parity wording and one fix the engine-free run found (below), and refits on that tree and the feed main pins (Chronicle 41b2fb2b, sha256 1d45a89c…, 40,642 rows, rebuilt in the container as pinned).

The receipt is Modal run 20261106T142002Z-8a9d335a7: experiments/1074-fsa-calibration-receipts.md, with the JSON in experiments/receipts/. It binds 6,102 non-FSA targets and the 130 FSA targets, drops none, and solves every arm with the release's 1,500 epochs, learning rate, 5x weight cap and conserved mass.

Release A B A + record L2 B + record L2 A + chi-square L2 B + chi-square L2 B, counts x0 B + record L2, counts x0
FSA targets within 5% (130) 0% 1% 95% 0% 76% 0% 59% 57% 45%
FSA mean error 48.7% 48.6% 0.8% 48.7% 4.0% 49.2% 8.7% 9.6% 15.0%
Balance targets within 5% (65) 0% 2% 98% 0% 92% 0% 80% 95% 80%
Count targets within 5% (65) 0% 0% 91% 0% 60% 0% 38% 18% 9%
Non-FSA capped error (6,102) 5.30% 3.30% 3.66% 3.85% 3.86% 3.93% 4.18% 3.42% 3.84%
Non-FSA within 5% 85.2% 92.0% 90.5% 90.4% 90.1% 89.8% 87.7% 91.5% 90.4%
Effective sample size (ESS) 9,174 4,149 2,932 6,435 4,484 8,487 6,222 3,346 5,302
Median household's share of its published weight 100.0% 2.1% 0.045% 13.3% 1.2% 81.2% 0.00024% 0.23% 5.9%
Households with federal education debt (SCF 18.2%) 16.4% 16.6% 33.9% 16.6% 32.4% 16.3% 28.0% 30.4% 25.7%
Borrowers (FSA 44.31M) 21.10M 21.54M 44.27M 21.34M 42.42M 21.00M 36.61M 39.68M 33.72M
Mean balance per borrower (FSA $37,894) $40,688 $41,125 $37,886 $40,101 $39,359 $40,319 $43,996 $42,244 $49,164
Parent PLUS share (FSA 6.8%, diagnostic) 10.7% 10.8% 10.2% 10.8% 10.7% 10.5% 11.3% 10.3% 10.8%
Borrower age profile vs FSA (TVD) 0.035 0.040 0.001 0.038 0.020 0.038 0.031 0.032 0.032

Share of households with federal education debt by household income quintile, at the release arm's cut points. Every household keeps the quintile it has at the published weights, so each column holds the same households in every arm:

Q1 Q2 Q3 Q4 Q5 Q5 - Q1, points Bottom two quintiles' share of borrower households
Release 14.3% 15.8% 14.7% 23.1% 14.2% +0.0 36.6%
A 14.9% 16.8% 15.6% 22.2% 13.8% -1.1 35.4%
B 36.6% 37.8% 28.2% 41.4% 27.1% -9.4 40.5%
A + record L2 14.5% 16.1% 15.4% 22.7% 14.4% -0.1 35.2%
B + record L2 32.1% 34.1% 27.7% 41.8% 27.0% -5.1 38.8%
A + chi-square L2 14.0% 16.0% 14.9% 22.8% 14.2% +0.3 36.2%
B + chi-square L2 24.7% 27.2% 24.0% 38.7% 25.5% +0.8 36.2%
B, counts x0 30.3% 33.4% 26.6% 37.9% 24.3% -6.0 38.5%
B + record L2, counts x0 23.7% 26.2% 22.5% 35.0% 21.7% -2.0 36.8%

The income is the household income the imputation conditions on, over the SCF primary economic unit; the pilot cuts tax-unit AGI over all tax units and counts borrowers as people, so these are not its income tables. The receipt also has each arm's own cut points (the SCF's row there: 11.4%, 17.1%, 17.8%, 26.6%, 17.9%, +6.5 points, 31.4%) and each fixed quintile's share of household weight. Q4 tops every row, so Q5 - Q1 is an endpoint contrast rather than a slope.

What it shows:

  • Chi-square 0.03 moves the fit, so the grid needs no second lambda. Against B at lambda 0, B + chi-square L2 binds 59% of the FSA targets within 5% (B: 95%), costs 0.52 points of non-FSA error, and raises ESS from 2,932 to 6,222. The household share falls from 33.9% to 28.0%.
  • It does not anchor most households to the design. In B + chi-square L2 the median household keeps 0.00024% of its published weight (B: 0.045%; B + record L2: 1.2%). The chi-square cost of sending a weight to zero is bounded by the household's share of the anchor, so at 0.03 the solve drops most households and spreads the rest: ESS rises while three quarters of the published support go to about zero. A + chi-square L2 keeps its median household at 81% of its weight, so the collapse comes with the FSA pull.
  • The record penalty keeps the FSA fit closer. B + record L2 binds 76% of the FSA targets within 5% and 92% of the balances, at no non-FSA cost against A + record L2 (+0.01 points), with ESS 4,484 and 32.4% of households with debt.
  • Without a penalty, the FSA family costs 0.36 points of non-FSA error (TANF +2.3, SSI federal recipients +1.6, JCT tax expenditures +1.4, BEA proprietors' income +1.3, SOI capital gains +1.0). With the record penalty it costs 0.01; with the chi-square one, 0.25.
  • Income. At the release's cut points, B lifts the debt share in the bottom quintile from 14.3% to 36.6% and in the top from 14.2% to 27.1%, so Q5 - Q1 goes from 0.0 to -9.4 points against the SCF's +6.5 (own cuts). B + record L2: -5.1; B + chi-square L2: +0.8; the x0 arms: -6.0 and -2.0. The A controls move no quintile by more than 1.0 point.
  • Parent PLUS, now unbound, stays at 10.2-11.3% of the host's balance in every arm, against FSA's 6.8%, a likely lower bound.

Decided (Max, 2026-11-06):

  1. The release keeps lambda 0, Route A's setting (arm B), so this PR adds no --l2-basis flag. B fits the 130 FSA targets more closely than B + record L2 (95% within 5%, against 76%) and the 6,102 others too (3.66% capped error, against 3.86%). These arms re-solve the published 57,240 households, while Route A's whole-base dense solve on 10/24 ended at an ESS of 72,166 across 352,932 households at lambda 0, so the receipt can't size a penalty on the release's support. The release's calibrated fit table (us_scf_education_debt_calibrated_fit.json) measures the income contrast on the release itself.
  2. Widening the stage to adults outside the SCF primary economic unit waits for its own PR under Carry SCF education debt into the US build and allocate federal student loan balances to borrowers #1073, after the release.

Checks (Modal, all passed, at the pushed head 7e203d0bc unless noted):

  • the CI engine-free job's 739 files at 8a9d335a7 (run 20261106T141933Z-8a9d335a7, -n 7, no --maxfail): 26,743 passed, 262 skipped, 13 failed. Four failures were this PR's: test_us_post_export_scoring runs _main's export and cleanup statements on their own, and the calibrated fit-table write sat inside them; 7e203d0bc moves it after the cleanup. The other nine fail only in the Modal image: no node (four orrery tests), Python 3.14.2's worker-recycling guard (two), root permissions and subprocess timeouts (three test_us_trade_imdb_bulk tests, and test_route_a_driver's child-environment test);
  • at 7e203d0bc (run 20261106T151208Z-7e203d0bc): the post-export scoring and transport-binding suites in full, the whole fiscal-refresh builder suite, and the FSA, fiscal-target, spine-blindness, chronicle-feed, parity and label suites: 1,339 passed, 11 skipped; ruff check . and ruff format --check on this PR's files;
  • the spec repin sequence at 8a9d335a7 (run 20261106T141948Z-8a9d335a7): repin --legacy-only, the generator, the repinner twice and the coverage tool change no file, and every --check passes: bundle spec_sha256 ffa31ce0… (main's), 42,394/42,394 configuration fields, 41/41 inventory checks. The commits after it change the receipts, the doc, and where the release writes its calibrated fit table;
  • ci_test_plan.py verify passes, and GitHub CI passes 11/11 at 7e203d0bc.

Max's review of 2026-11-05: diagnostic loan-type rows, the L2 grid and the income tables

The code, doc and receipt-builder changes for it, now refit above.

  • Keep FSA loan-type rows diagnostic and refuse changed FSA tables #1525 is merged in (aa103ec6c, two commits by Max), unchanged:
    • Both loan-type dollar rows are diagnostic, so the family binds 130 targets: 65 on balances and 65 on borrower counts.
    • _require_reviewed_rows refuses a location, age or debt-size table whose rows at the reference quarter differ from FSA's set, a repeated row included.
    • The feed-refusal test reads _main's syntax tree, and two Hypothesis properties check the family totals and the band partitions.
    • One note on its history: the first commit's message cites 34 CFR 685.220(a) and says FSA's Parent PLUS row covers only unconsolidated loans; the second commit retracts that, and the code and docstrings carry the second reading.
  • Parent PLUS. Under 34 CFR 685.220(f), the Secretary pays off each loan that a Direct Consolidation Loan consolidates. FSA's notes don't say where a consolidated Parent PLUS loan goes, and the stage's draw does not read consolidation status (scf_education_debt.py:838-880). Our inference, not FSA's statement: since the Parent PLUS loan is paid off, FSA likely files the new loan under "Consolidation". If so, its Parent PLUS row ($117.9B, 6.8% of the loan-type table's dollar rows) is a lower bound on the stage's federal_parent_plus_loan_balance, which holds the balances the stage classifies as Parent PLUS, consolidated or not. Every other family pins the total, so a target on either side of FSA's split would still constrain the split. The receipt reports the host's Parent PLUS share against FSA's diagnostic row, which the fit now reads from the diagnostic facts rather than a target. The doc says the same (docs/us-fact-to-target.md, "Parent PLUS").
  • Borrower counts stay at full weight (Max's call). At the host's 1.05 borrowers per borrower household, FSA's 44.31M implies debt in 34.3% of the host's households, and B lands at 33.7%. Holding the SCF's 18.2% would take 1.99 borrowers per borrower household, against 1.11 in the SCF. The x0 arms stay in the grid as a measurement, not a release option: no release flag reproduces them, since --target-family-loss-multiplier ed_fsa= scales balances and counts together (target_loss_weights.py:164-176). The "bind count shares" option is gone; the student debt pilot's scope names FSA's balances and borrower counts as totals to match.
  • L2 is a concentration penalty. The + L2 arms call calibrate with its default record basis, which spreads weight without anchoring to the design (solve.py:55-67), as the --l2-lambda help calls it. The grid for the rerun (SOLVED_ARMS in fit_1074_fsa_calibration.py):
Arm FSA family Penalty Borrower-count weight
Release the published weights
A, B without, with none 1
A, B + record L2 without, with record, lambda 0.03 1
A, B + chi-square L2 without, with chi_square, lambda 0.03 1
B, counts x0 with none 0
B + record L2, counts x0 with record, lambda 0.03 0
  • The chi-square arms pass calibrate's l2_basis. If 0.03 barely moves the fit, --extra-chi-square-lambda (or launch --fit-args) adds A and B at a second lambda without a code change. The ACS local L2 study measures lambda in units of its own loss, so 0.03 may not carry over.

  • Release constraint, not implemented here. The pilot builds with Route A's --dense-default-dataset (tools/route_a/route_a.env.example:46 at main). There --refit-l2-lambda refuses at start (build_us_fiscal_refresh_release.py:1982-1985), and --l2-lambda passes no basis, so the release can take lambda 0 or the record penalty today. A chi-square release would need a pass-through flag (an --l2-basis the dense path hands to calibrate).

  • Income tables. The receipt now shows every household-income quintile for every arm, the A controls included, beside the SCF:

    • at each arm's own cut points;
    • at the release arm's cut points applied to every arm, so each quintile holds the same households in every arm;
    • each fixed quintile's share of household weight;
    • Q5 - Q1, and the bottom two quintiles' share of borrower households.

    The income is the one the imputation conditions on, over the SCF primary economic unit. The pilot counts borrowers by person and cuts tax-unit AGI over all tax units, so these rows are not its income tables.

  • Fit script.

    • It reads FSA's Parent PLUS figure from the diagnostic loan-type facts (parent_plus_diagnostic).
    • The JSON writes a missing value as null (allow_nan=False); the old receipt's bare NaN was the household share's FSA target.
    • It refuses a feed that binds no FSA target.
    • Every table covers every arm.
  • Runner. modal_1074.py now builds the feed the commit pins by default (--feed pin): it rebuilds us/chronicle_feed.json's Chronicle commit and checks the sha256 and the row count. --feed candidate keeps the earlier splice for a pre-Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506 base. A fit_smoke step runs the grid at 20 epochs, with its receipt kept in the run's results; a lint step runs ruff.

  • The release's own fit table. A --scf-education-debt release now writes us_scf_education_debt_calibrated_fit.json beside us_scf_education_debt_fit.json. It holds the same us_scf_education_debt_fit_table, computed on the frame the release exports, at the weights it ships, before post-export scoring and the terminal gates. That holds on the dense path and on the L0 path. New test: test_scf_education_debt_fit_is_written_at_calibrated_weights_before_scoring pins the payload and, from _main's syntax tree, that the write sits under the flag alone, after the export frame takes its weights and before the scoring plan.

  • Missing ages. A person with no numeric age counts in the youngest band (FSA_MISSING_AGE = -1.0), so the age bands still partition every borrower. The constant, the materializer docstring and the doc now say so. New test: test_a_missing_age_counts_in_the_youngest_band covers NaN, None and a non-numeric string.

  • Reference date. The frame's age stays unshifted. The doc now says the materializer groups the stored, unaged age into FSA's March 2026 bands. FSA's largest change in a band's share between March 2024 and March 2026, at ages 25 to 34, is about 1.4 points for borrowers and 2.6 for balances.

  • Wording.

    • The stage misses a financially independent adult in someone else's household, not adults living alone, who draw as a household's reference person. These receipts don't separate the causes of the host's shortfall.
    • FSA says Not Reported includes FFEL and Perkins loans whose lenders need not report an address.
    • The loan-type table's $1,723.8B comes from its dollar rows; only its counts measure recipients.
    • The re-pin is done (Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506).
    • The stale 42,368 field count is gone.
    • The run table no longer calls 20261023T191527Z-4b6bfce9a "the receipt".
  • Parity wording. The ed_fsa families' reviewed-exclusion reason, in tools/build_us_target_parity_manifest.py and its eleven manifest entries, now lists the loan-type table among the diagnostic ones. It moves no spec identity value (the repin sequence changes no file).

  • Checks of this round on the old base (runs 20261105T220255Z-8a314f103, 20261105T214457Z-66b37d37c and the smoke 20261105T214517Z-66b37d37c) passed; the checks above, on main, replace them.

Fixture stamp and a refusal for debt builds without FSA facts (2026-11-04)

Max's two asks on 0ff293b1a:

  • The fixture is restamped v3. Its 244 rows now carry chronicle.consumer_fact.v3, as every row of the feed Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506 pins does (41b2fb2b, 40,642 rows). The stamp is the only change. Modal run 20261104T184919Z-ff4530bcb rebuilt that feed with main's tools/build_us_chronicle_feed.py at the pinned scope (sha256 1d45a89c… and 40,642 rows, as pinned, every row v3), and each of the 244 rows is one of its 461 FSA rows, byte for byte. The 461 rows of chronicle#347's export at its merge commit acbedb5 (sha256 58774e11…), restamped v3, are exactly the feed's FSA rows, so the stamp is their only difference. The "Schema versions" line below now says so.
  • A --scf-education-debt build on a feed with no ed_fsa facts now refuses. At 0ff293b1a, plan_fsa_student_loan_targets returned an empty plan for such a feed, and the tool compiled no FSA target and raised nothing. The release tool's new _check_us_feed_carries_fsa_facts stops the build before the compile, with an error that names the flag, the ed_fsa source and its four bound tables.
    • It sits in the release tool beside its other feed checks (_check_committed_us_ledger_feed_pin, _check_us_esi_anchor_against_feed), which name the flags they enforce. The runtime modules name none.
    • The empty plan stays a valid answer for callers that don't opt in, and the receipt's no_fsa_facts status is unchanged.
    • The us_source_coverage.json receipt now reuses the check's plan instead of planning again.
    • New test: test_release_refuses_a_debt_build_on_a_feed_without_fsa_facts. It pins the refusal, the inert path without the flag, the plan on the fixture feed, and that _main runs the check before the compile.
  • Follow-up, not in this PR: the order of the debt-size bands on the dashboard. The dashboard would list the nine bands alphabetically ("100K to 200K", "10K to 20K", "200K+", …), because HierarchyDimension carries no value order (hierarchy.py:66-73) and the dashboard falls back to sorting labels. If microcosm carried each value's source order there and in _inherited_dimensions (ledger_targets.py:1578-1610), the dashboard could sort them. Label the ED Federal Student Aid provider and test its target family calibration-diagnostics#276 describes the dashboard's side.
  • Base. Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506 merged on 2026-11-04, so main's pin now carries the FSA rows. Neither scf-2025-education-debt nor main is merged in here yet; Calibrate federal student loan balances to Federal Student Aid's portfolio (#1074) #1357 picks both up from Impute federal student loan balances from the 2025 SCF (#1073) #1246's final head, and the rerun reads main's pin.
  • Checks at ff4530bcb (Modal run 20261104T184939Z-ff4530bcb): all passed.
    • the tests step: 816 passed, 8 skipped, plus 51 builder tests; engine-contract: 37;
    • generator and coverage --check: 42,489/42,489 configuration fields, and the bundle spec_sha256 is still be81ebbc…, so neither change moves a spec identity value. Then 253 spec-engine tests;
    • ruff check ., ci_test_plan.py verify, the graph acceptance burndown, and the FSA, builder, spine-blindness and register suites: 1,004 passed, 2 skipped.

Down-weighted borrower counts (2026-11-02)

Six new arms bind the 67 FSA balance targets at full loss weight. They keep the 65 borrower-count targets (state, age band, debt-size band) at 0.25, 0.1 or 0 of their weight, renormalized to mean 1, each with and without the L2 anchor. At 0 the counts only validate. The existing arms are unchanged, and the release tool still binds every FSA target at full weight. No release flag reproduces these arms: --target-family-loss-multiplier ed_fsa= scales balances and counts together.

Modal run 20261102T174003Z-88d7a950d at 88d7a950d (receipt 0ff293b1a; full table, with every age band, in experiments/1074-fsa-calibration-receipts.md):

FSA Release B x0.25 x0.1 x0 B + L2 x0.25 x0.1 x0
Balance targets within 5% (67) 0% 99% 96% 96% 96% 94% 88% 88% 87%
Count targets within 5% (65) 0% 92% 43% 31% 17% 62% 17% 12% 9%
Non-FSA capped error 5.30% 3.62% 3.50% 3.47% 3.45% 3.85% 3.84% 3.85% 3.84%
Households with federal education debt (SCF 18.2%) 16.4% 33.7% 32.7% 31.7% 30.2% 32.3% 28.2% 26.7% 25.7%
Borrowers 44.31M 21.18M 44.26M 42.98M 41.71M 39.75M 42.67M 37.27M 35.38M 34.11M
Parent PLUS share of the balance 6.8% 10.7% 6.8% 6.8% 6.8% 6.8% 6.9% 6.9% 6.9% 6.9%
Borrowers aged 24 and younger 13.8% 12.2% 13.9% 12.7% 12.1% 11.6% 12.9% 11.6% 11.3% 11.0%
Borrowers aged 35 to 49 33.5% 37.2% 33.5% 34.9% 35.8% 36.0% 34.8% 36.1% 36.2% 36.2%
Borrower age profile vs FSA (TVD) 0.037 0.001 0.030 0.041 0.044 0.025 0.036 0.035 0.035

What it shows:

  • Down-weighting the counts does not undo the doubling. With the balances bound alone, 30.2% of households hold debt (25.7% with the anchor). That compares with 33.7% at full count weight and 18.2% in the SCF. The host holds $0.86T against FSA's $1.68T, so reaching the balances still takes far more borrower-household weight: a mean weight ratio of 1.84 at x0 (1.57 with the anchor), against 2.05 in B.
  • Without the counts, the solver reaches the balances through larger balances per borrower. The mean is $42.2K at x0 ($48.8K with the anchor), against FSA's $37.9K. So the debt-size bands under $10K fall short (under $5K: -38%, and -54% with the anchor), and the age profile drifts back toward the release's: 35 to 49 is over-represented, and the under-25s fall short.
  • The rest of the surface barely moves. Non-FSA error is 3.45% at x0 against 3.62% in B (A: 3.30%). With the anchor it stays at 3.84-3.85%. Parent PLUS stays at 6.8-6.9%.
  • So most of the doubling comes from FSA's balance level, not from the count targets. Adding borrowers the SCF misses, the Carry SCF education debt into the US build and allocate federal student loan balances to borrowers #1073 follow-up on adults outside the primary economic unit, is untested here.

Also in this push:

  • Base. Impute federal student loan balances from the 2025 SCF (#1073) #1246's updated head 74837b4d7 (its merge with main, re-derived spec identity values and rerun receipts) merged cleanly in e6a026bbe. Nothing moved: the existing arms reproduce all 4,363 numbers of the 4b6bfce9a receipt.
  • Reference date. policyengine-us#10415 merged on 2026-10-30, so the model now holds the stored balances at their value in every year. The receipt's reference-date sentence says so.
  • Feed. Main had re-pinned the US feed to cf9ece2 (microcosm#454). The experiment kept the 51aa40f candidate: cf9ece2 keeps all 40,178 of its rows and adds only the three B4923C facts, which compile into no target. (Since Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506, main's pin carries the FSA rows, and the rerun reads it.)
  • Checks at 88d7a950d (Modal run 20261102T174020Z-88d7a950d), all passed:
    • the tests step: 815 passed, 8 skipped, plus 51 builder tests; engine-contract: 37;
    • generator and coverage --check: 42,489/42,489 configuration fields, and the bundle spec_sha256 is be81ebbc…, as on Impute federal student loan balances from the 2025 SCF (#1073) #1246, so this PR still moves no spec identity value. Then 253 spec-engine tests;
    • ruff check ., ci_test_plan.py verify, the graph acceptance burndown, and the builder, spine-blindness and register suites: 939 passed, 2 skipped.

Rebased on #1246's review round (2026-10-23)

  • History. Squashed onto Impute federal student loan balances from the 2025 SCF (#1073) #1246's head 3eb32320b as two commits: the feature (4b6bfce9a) and the refreshed receipt and docs (3b2886597).
  • What Impute federal student loan balances from the 2025 SCF (#1073) #1246 changed under it. A loan for a child's education now goes to a co-resident adult child who fits it. It stays the reference person's Parent PLUS when that child could have borrowed it as a dependent student and the undergraduate borrowing exceeds the dependent limit, or when no listed child fits an undergraduate loan. A graduate loan with no fitting child becomes the reference person's own, since Parent PLUS lends only for a dependent undergraduate (34 CFR 685.200(c)(2)(i)); the X8000 codes of loans 4 and 6 are swapped back; Parent PLUS is drawn first. This PR moved no spec identity value at that base: the bundle and coverage --checks passed on Impute federal student loan balances from the 2025 SCF (#1073) #1246's re-derived values.
  • One fix the rebase run surfaced. The US spine-blindness test requires every us_runtime module to be classified. fsa_student_loan_targets.py is now a reviewed non-registry module, and its band sort uses a named key instead of a lambda subscript the scanner cannot resolve. Nothing else changes.
  • chronicle#347 merged on 2026-10-22. The runner now builds the FSA rows at its merge commit acbedb5 on PolicyEngine/chronicle main, not the fork head. The export is byte-identical (sha256 58774e11…), so the candidate feed (d8e3d0fd…, 40,639 rows) and the test fixture are unchanged.
  • Refit at 4b6bfce9a (Modal run 20261023T191527Z-4b6bfce9a; a run at the pre-fix commit gave the same numbers):
Before (c94516ee1) After
FSA targets within 5%: B / B + L2 92% / 76% 95% / 78%
FSA mean error: B / B + L2 1.5% / 4.7% 0.7% / 3.6%
Non-FSA capped error: A / B / A + L2 / B + L2 3.30% / 3.66% / 3.85% / 3.83% 3.30% / 3.62% / 3.85% / 3.85%
Households with federal education debt: release / B / B + L2 (SCF 18.2%) 16.4% / 33.0% / 31.5% 16.4% / 33.7% / 32.3%
Parent PLUS share: release / B / B + L2 (FSA 6.8%) 16.8% / 6.8% / 7.6% 10.7% / 6.8% / 6.9%
Borrowers: release / B / B + L2 (in-scope control 44.31M) 21.22M / 43.78M / 41.95M 21.18M / 44.26M / 42.67M
Borrowers under 25: release / B / B + L2 (FSA 6.13M) 1.06M / 3.72M / 3.43M 2.58M / 6.13M / 5.52M
Borrower records under 25 (unweighted) 561 1,246

What moved, in plain words:

  • The routing moves balances from parents to their adult children in the same household. The host gains young borrowers and its Parent PLUS share falls, but it holds the same number of borrowers (21.2M) and borrower households (16.4%) at the published weights.
  • So the doubling of borrower households does not shrink: it is 33.7% after calibration, against 33.0% before. The SCF misses a financially independent adult in someone else's household (an adult living alone draws as the reference person), and the routing does not add such borrowers. These receipts don't separate the causes of the shortfall against FSA's count.
  • With young borrowers in the support, B meets every age band (under 25 was -39%), and B + L2 is 10% short under 25 (was -44%). B + L2 now falls short at 50 to 61 (-9%) and 62 and older (-13%), the ages the moved loans left.
  • Parent PLUS needs less pull (10.7% to 6.8%, not 16.8% to 6.8%), so B + L2 now meets it too. The rest of the surface is unchanged within 0.05 points.

Part of #1074 (the student debt pilot, PolicyEngine/us-student-debt#1). Draft, on main since #1246 merged (b563fbb6f). PolicyEngine/chronicle#347 merged on 2026-10-22, and #1506 re-pinned the US feed to Chronicle 41b2fb2b with its FSA rows on 2026-11-04; nothing here pins a fork. No release is built, published or certified.

What binds

A new runtime module, us_runtime/fsa_student_loan_targets.py, turns Federal Student Aid's portfolio facts into 130 targets on the two person columns #1246 imputes. A person's federal balance is federal_student_loan_balance + federal_parent_plus_loan_balance, and a borrower is a person whose federal balance is positive. FSA's borrower tables count a Parent PLUS borrower as the parent, which is where the stage puts those balances.

Family FSA table Model measure Targets
By state Portfolio by Location balance and borrowers in each of the 50 states and DC 102
By age Portfolio by Age balance and borrowers in 5 bands of the person's age 10
By debt size Portfolio by Debt Size balance and borrowers in 9 bands of the person's federal balance (lower edge inclusive) 18

Diagnostic only, each with its reason in the receipt:

  • Repayment plans (all six sheets). The SCF records only an income-driven yes/no, the build has no plan column, and FSA counts plan recipients loan by loan. The receipt keeps FSA's Direct Loan dollars by plan at FY2026 Q3 for context: SAVE 28.7%, IBR 18.2%.
  • Portfolio summary. Recipient totals by loan program, which the balances do not record.
  • Loan-type dollar rows, Parent PLUS included (34 CFR 685.220(f); see the review section above).
  • Loan-type recipient counts. These are loan-level, so a person with two loan types counts twice.
  • Stafford (Combined). A subtotal of rows already counted.

Reference period

Every family binds at FY2026 Q2 (31 March 2026), the latest quarter all four tables publish:

  • the location workbook publishes only that date;
  • age and debt size publish Q1 and Q2;
  • loan type publishes Q1 to Q3.

Holding every table to one date makes every family describe one portfolio on one day. The Q1 and Q3 rows stay in the receipt as other_quarter.

The build period is the H5's, 2024. The FSA levels are stocks, applied unaged:

  • Target aging gains a declared alignment, factor policy 5: period_alignment = stock_at_source_reference_date. It is neither aged nor a period-contract violation, and only targets that declare it are affected.
  • The reasons: no income-growth series describes a loan stock, and the SCF draws are already 2025 survey-date dollars.
  • Specs sit at period 2024 with source_period = 2026-03.

Reference date of the stored balances. The stored balances are the March 2026 FSA-calibrated stock, written at the build period, 2024: the H5 records one period for every column. policyengine-us#10415 (merged 2026-10-30) holds them at that value in every year, so a 2024 or a 2026 read returns each person's stock. policyengine-us also raises household weights by population growth, ×1.0165 from 2024 to 2026 at the pilot's pin, so a perfect fit at $1,679.1B and 44.31M reads $1,706.8B and 45.04M in 2026, before the pilot's borrower mask. us-student-debt#16 says the same in the pilot's METHODS. The age bands group the stored, unaged age, which the stage's draw and the pilot's age table also read.

Not Reported, Puerto Rico and Other: one control total

The location table has no total row. The state targets are scaled to the national in-scope total:

  • Not Reported is allocated to every reported location in proportion to that location's reported value. It holds $52.2B and 1.95M borrowers; FSA says it includes FFEL and Perkins loans whose commercial lenders need not report an address.
  • Puerto Rico and Other leave the total, with their share of Not Reported. The host is the 50 states and DC that the CPS samples, and "Other" is an address outside the listed states and territories.
Balance Borrowers
50 states and DC, as published $1,627.4B 42.38M
Puerto Rico and Other $16.7B 433,400
Not Reported $52.2B 1.95M
Allocation factor 1.0317 1.0456
In-scope control $1,679.1B 44.31M

FSA's tables disagree slightly: the borrower tables total $1,696.3B, and the loan-type table's dollar rows $1,723.8B (only its counts measure recipients). So the age and debt-size bands bind as their table's share of the in-scope control, control x row / partition, the #940 pattern. All three families then sum to the same total. The age table's own "not reported" row is in no partition, so the reported bands absorb it.

Units, all from Chronicle's declared scales:

  • dollars are published in $ billions;
  • location counts are in thousands, the other tables' counts in millions;
  • location, age and debt size count borrowers, while the summary and loan type count recipients. Recipient counts never bind.

How it is wired

  • Compile. compile_us_fiscal_target_registry(..., student_loan_targets=True) binds the facts; ed_fsa facts never bind through dynamic dispatch. The release tool sets the flag only with --scf-education-debt, the only builds that carry the columns. Default compiles and releases are unchanged.
  • Materializer. fsa_stored_student_loan_balances reads the stage's stored person columns (fsa_student_loan_household_values), because the pinned policyengine-us does not define them yet.
  • Restated-filter guard. It accepts the FSA age bands, through the existing age rule with the FSA materializer added, and the debt-size bands, through a new student_loan_balance_band rule. Both are accepted only on that materializer.
  • Parity contract. The opt-in family stays off the one-surface target-parity contract (_without_opt_in_student_loan_targets), so a default build and a --scf-education-debt build cannot fail each other.
  • Receipt. us_source_coverage.json gains fsa_student_loan_target_receipt, with every FSA fact's disposition and the controls.
  • Reviewed rows. The plan refuses a location, age or debt-size table whose rows differ from FSA's set at the reference quarter (Keep FSA loan-type rows diagnostic and refuse changed FSA tables #1525's _require_reviewed_rows).
  • Calibrated fit table. A --scf-education-debt release writes us_scf_education_debt_calibrated_fit.json, the stage's fit table at its shipped weights, before scoring.
  • Labels. ED · Federal Student Aid and four categories (federal_student_loan_balance, federal_student_loan_borrowers, parent_plus_loan_balance, loan_balance_excluding_parent_plus) join the calibration label registries.
  • Docs. docs/us-fact-to-target.md has the rules, the measured limits and the pin.
  • Spec identity: unchanged. generate_us_bundle_from_constants.py --check and spec_engine_coverage.py --check pass: 42,394/42,394 configuration fields (see the checks above).

Fit

experiments/1074-fsa-calibration-receipts.md holds the receipt, with the JSON in experiments/receipts/. The setup:

  • Host: the published default populace-us-2024-spm-20260915, 57,240 households, with the Impute federal student loan balances from the 2025 SCF (#1073) #1246 stage run at seed 0.
  • Feed: the pin on main, Chronicle 41b2fb2b (sha256 1d45a89c…, 40,642 rows), rebuilt in the container as pinned.
  • Targets: the release's own compile, Medicaid substitutions, national_state surface, materializer (one engine batch), loss weights and cap. That is 6,102 non-FSA targets plus the 130 FSA targets, with 0 dropped.
  • Solve: calibrate with the release's 1,500 epochs, learning rate, 5x weight cap and conserved mass.
  • Arms: the release (the published weights); A (a dense refit without the FSA family) and B (with it), each at lambda 0, with the record concentration penalty at 0.03 and with the chi-square penalty at 0.03; and B with the borrower counts at none of their loss weight (x0), at lambda 0 and with the record penalty.

It is a re-solve of the published support, not a release build; a Route A build solves the whole base.

FSA targets. Mean absolute relative error, with the share of targets within 5% in parentheses:

FSA family Targets Release A B A + record L2 B + record L2 A + chi-square L2 B + chi-square L2 B, counts x0 B + record L2, counts x0
State balance 51 47.0% (0%) 46.7% (2%) 0.2% (98%) 47.0% (0%) 1.1% (94%) 47.6% (0%) 2.8% (84%) 0.2% (98%) 2.7% (82%)
State borrowers 51 49.4% (0%) 49.9% (0%) 1.2% (90%) 49.3% (0%) 6.8% (61%) 49.7% (0%) 12.6% (41%) 19.4% (14%) 26.1% (4%)
Age-band balance 5 46.4% (0%) 44.5% (0%) 0.1% (100%) 46.8% (0%) 0.3% (100%) 47.4% (0%) 3.4% (60%) 0.1% (100%) 0.6% (100%)
Age-band borrowers 5 54.8% (0%) 54.0% (0%) 0.1% (100%) 54.4% (0%) 6.1% (40%) 54.9% (0%) 19.3% (0%) 13.1% (20%) 25.2% (0%)
Debt-size balance 9 51.1% (0%) 50.6% (0%) 0.6% (100%) 51.2% (0%) 4.5% (78%) 51.6% (0%) 12.8% (67%) 8.7% (78%) 18.8% (56%)
Debt-size borrowers 9 49.4% (0%) 49.7% (0%) 2.4% (89%) 49.9% (0%) 5.5% (67%) 50.1% (0%) 13.7% (44%) 11.5% (44%) 20.6% (44%)
All 130 48.7% (0%) 48.6% (1%) 0.8% (95%) 48.7% (0%) 4.0% (76%) 49.2% (0%) 8.7% (59%) 9.6% (57%) 15.0% (45%)

B's largest misses, mostly on borrower counts: Delaware -18.8%, $200K+ borrowers +11.1%, Maine -10.5%, Colorado -8.0%, Connecticut -7.0% (borrowers); Louisiana's balance -6.7%.

The rest of the surface. Mean absolute relative error over the 6,102 non-FSA targets, each miss capped at 100% as the loss caps it, is in the table at the top. Adding the family costs 0.36 points without a penalty, 0.01 with the record penalty and 0.25 with the chi-square one.

How calibration closes the SCF gap, and what it does to the borrower share and the Parent PLUS mix

The SCF-imputed host holds about half of FSA's portfolio at the published weights: $858.4B and 21.10M borrowers, against $1,679.1B and 44.31M. The stage gives balances to reference persons, spouses and co-resident adult children, and Parent PLUS to the reference person. So the host is missing borrowers rather than balances: its mean balance per borrower, $40,688, is already close to FSA's $37,894.

Calibration does not change any balance. It reweights:

FSA Release B B + record L2 B + chi-square L2
Federal balance $1,679.1B $858.4B $1,677.3B $1,669.7B $1,610.5B
Borrowers 44.31M 21.10M 44.27M 42.42M 36.61M
Mean balance per borrower $37,894 $40,688 $37,886 $39,359 $43,996
Households with federal education debt (SCF 18.2%) 16.4% 33.9% 32.4% 28.0%
Borrower households' mean weight ratio 2.07 1.97 1.71
Other households' mean weight ratio 0.79 0.81 0.86
Parent PLUS share of the balance (FSA diagnostic) 6.8% 10.7% 10.2% 10.7% 11.3%
Effective sample size (release 9,174) 2,932 (A: 4,149) 4,484 (A: 6,435) 6,222 (A: 8,487)
  • Borrower share. At the host's 1.05 borrowers per borrower household, FSA's 44.31M implies debt in 34.3% of the host's households, and B lands at 33.9%. Holding the SCF's 18.2% would take 1.99 borrowers per borrower household, against 1.11 in the SCF. The SCF misses a financially independent adult in someone else's household (an adult living alone draws as the reference person), and these receipts don't separate the causes of the shortfall. Household-level tables (share of households with debt, payments per borrowing household) inherit the doubling.
  • Borrowers by age. The host has 1,262 borrower records under 25 among 59,179 people that age. B meets every FSA age band. B + record L2 is short under 25 (-11.5%), at 50 to 61 (-9.5%) and at 62 and older (-7.0%); B + chi-square L2 is short in every band (-10% to -29%).
  • Parent PLUS mix. With both loan-type rows diagnostic, nothing binds the split, and the host's Parent PLUS share stays at 10.2% in B (10.7% at the published weights), against FSA's 6.8%. FSA likely files a consolidated Parent PLUS loan under "Consolidation" (34 CFR 685.220(f) has the Secretary pay it off; FSA's notes don't say), so its share is likely a lower bound on the stage's column.
  • Weight concentration. A dense refit of an L0-selected support without a penalty concentrates weight, A as well as B: the median household keeps 2.1% of its published weight in A and 0.045% in B. The record penalty spreads weight (1.2% in B + record L2) rather than anchoring to the design. At 0.03 the chi-square penalty keeps A near the design (81%), but under the FSA pull it lets most households go to about zero (0.00024%) while ESS rises.

Racial composition of borrowers, before and after calibration

Borrower households by the reference person's race (SCF racecl4; the SCF column pools the five implicates):

SCF 2025 Release B B + record L2 B + chi-square L2 B - release (pp)
White non-Hispanic 57.6% 60.8% 57.8% 60.2% 59.9% -3.0
Black non-Hispanic 18.6% 17.4% 18.6% 18.9% 18.0% +1.2
Hispanic 9.2% 12.9% 14.5% 12.4% 13.2% +1.6
Other or multiple 14.7% 9.0% 9.1% 8.5% 8.9% +0.1

Borrowers as people, by their own race, release to B: White non-Hispanic 60.4% → 57.4%, Black non-Hispanic 16.7% → 18.2%, Hispanic 13.3% → 14.5%, other 9.5% → 9.8%.

So calibration moves the composition by up to 3 points:

  • The White and Black shares move toward the SCF.
  • The Hispanic share moves away from it without a penalty.
  • "Other or multiple" stays about 5.5 points under the SCF before and after. That gap is in the host's population, not the calibration: 8.0% of host households are in the group against 13.7% of SCF families, while the share of them with debt is close (18.4% against 19.5%).

The levels move much more than the composition. The share of households with debt roughly doubles in every group, as the borrower share does: Black non-Hispanic households go from 24.9% to 48.9% (SCF 24.3%) and White non-Hispanic from 15.3% to 30.6% (SCF 17.2%). Race tables of levels inherit the borrower-share issue above. Race tables of composition are less affected.

These tables go to PolicyEngine/us-student-debt#4 before any race breakdown.

Targets that do not compile or bind

  • Materialization dropped none of the 130. A build without the stage's columns does not compile them at all, because the flag is off.
  • Facts that do not bind, from the 461-row FSA bundle:
Disposition Facts Reason
diagnostic 257 repayment plans (227), portfolio summary (24) and the loan-type dollar rows (6), reasons above
other_quarter 58 Q1 or Q3 of the age, debt-size and loan-type tables
excluded 8 loan-type recipient counts (6); Stafford (Combined) (2)
allocated 4 Not Reported, by location (2) and by age (2)
out_of_scope 4 Puerto Rico and Other (2 each)

For review

  1. Parent PLUS. Decided: both loan-type rows are diagnostic (Keep FSA loan-type rows diagnostic and refuse changed FSA tables #1525).
  2. Borrower counts. Decided: full weight, the baseline. Widening the stage to adults outside the SCF primary economic unit is the Carry SCF education debt into the US build and allocate federal student loan balances to borrowers #1073 follow-up; whether it can wait depends on the release's calibrated fit table and its coverage checks.
  3. The release's L2 setting. Open; see "For decision" at the top.

Dashboard

The dashboard (calibration-diagnostics) builds its families from a release's calibration_diagnostics.json hierarchy, with no registry to edit there. The rows carry:

  • provider ed_fsa, labelled "ED · Federal Student Aid";
  • categories ed_fsa.federal_student_loan_balance, …borrowers, …parent_plus_loan_balance and …loan_balance_excluding_parent_plus;
  • the state as geography and the age or debt band as a dimension.

So the family appears with its release fit as soon as a published release carries it, which is Max's call. Arm B's schema-8 calibration_diagnostics.json is on the Modal volume microcosm-1074-cache (results/20261023T191527Z-4b6bfce9a/diagnostics/, sha256 380c3221…). It could be staged to populace-us-staging for the staging page; I have not uploaded it.

How the dashboard reads the balances. The calibration pages read the release's calibration_diagnostics.json, whose estimates are the stored balances times the weights, so they show the unaged stock; they never read the H5 through policyengine-us. The variable explorer does compute from the H5, but it serves only the 2024 SPM build (populace-us-2024-spm-20260915), which doesn't carry these columns yet, and only at 2024. When a build it serves carries them, it reads them at 2024.

Chronicle and the pin

Modal runs (experiments/modal_1074.py, app microcosm-1074-experiments, volume microcosm-1074-cache)

Run Commit What
20261106T142002Z-8a9d335a7 8a9d335 The receipt: the review's grid on main and the pinned feed (22 min)
20261106T151208Z-7e203d0bc 7e203d0 Lint and the suites this PR touches, after the export-order fix: all passed
20261106T141948Z-8a9d335a7 8a9d335 Spec repin sequence and every --check: no file changed, all passed
20261106T141933Z-8a9d335a7 8a9d335 The CI engine-free file list without --maxfail: four failures from this PR, fixed in 7e203d0; nine environment-only
20261106T015951Z-a65c6eb4d a65c6eb (old base) The same grid on the pre-squash base; superseded
20261105T214517Z-66b37d37c 66b37d3 (old base) 20-epoch smoke of the grid on the candidate feed
20261104T184939Z-ff4530bcb ff4530b Restamp and refusal: tests, contract, spec and lint, all passed
20261104T184919Z-ff4530bcb ff4530b The fixture against a rebuild of the feed #1506 pins: rebuilt at 41b2fb2b as pinned (sha256 1d45a89c…); all 244 rows match, byte for byte
20261023T191527Z-4b6bfce9a 4b6bfce Fit with A, B, A+L2 and B+L2 on #1246's head then (13 min); the merged #347 export matched
20261023T200336Z-3b2886597 3b28865 Final head: tests, ruff, test-plan verify, spine-blindness and FSA suites, all passed
20261023T191540Z-4b6bfce9a 4b6bfce tests,spec passed; its extra probe stalled on the environment-only tests and was cancelled
20261023T181926Z-2468fba75 2468fba (pre-fix) The same fit, identical numbers
20261023T182913Z-2468fba75 2468fba (pre-fix) Lint, test-plan verify, bundle and coverage --check, and the engine-free shared and US suites; found the spine-blindness classification fixed in 4b6bfce
20261023T182014Z-2468fba75 2468fba (pre-fix) The CI engine-us job (tools.run_engine_test_categories): 448 passed, 6 skipped
20261023T181958Z-2468fba75 2468fba (pre-fix) tests,contract: passed
20261022T194252Z-c94516ee1 c94516e The previous receipt: fit with A, B, A+L2 and B+L2 (13.5 min; reform passes from the volume cache; 88 GB peak)
20261022T184056Z-fd7b279ce fd7b279 First full fit, A and B (55 min); same A and B numbers
20261022T164311Z-399e6fffe 399e6ff tests,contract,spec, all passed
20261022T170743Z-0ed9d7eef 0ed9d7e Fit preempted mid-materialization, cancelled; the runner is now non-preemptible
20261022T162443Z-9772bd15b, 20261022T162425Z-9772bd15b, 20261022T163137Z-4af1cab30 Earlier failures, each fixed: a formula-owned host column (has_marketplace_health_coverage), and the two test pins below

The runs before 2026-10-23 are on the pre-rebase history (#1246's older head), which the squash replaced. The receipt commit after 4b6bfce changes only the receipt, docs, the changelog fragment and a test docstring.

Tests

  • New: test_us_fsa_student_loan_targets.py (26 tests). They pin:
    • the reference quarter, the controls and each family's values;
    • every fact's disposition;
    • refusals of partial feeds and gapped bands;
    • the materializer on a synthetic frame;
    • the release guard, both accepting and refusing;
    • the release tool's refusal of a --scf-education-debt build on a feed with no FSA facts;
    • the parity filter.
  • Updated: the label tests pin ed_fsa. The compile-signature pin admits student_loan_targets, the restated-rule pin admits the balance band, and the US spine-blindness inventory classifies fsa_student_loan_targets.py.
  • Modal, after the rebase:
    • final head 3b2886597 (run 20261023T200336Z-3b2886597): the tests step (786 passed, 7 skipped, plus 51 builder tests), ruff check ., ci_test_plan.py verify, and the spine-blindness and FSA suites (523 passed);
    • 4b6bfce9a (run 20261023T191540Z-4b6bfce9a): generator and coverage --check (41/41 inventory checks), plus 252 spec-engine tests;
    • 2468fba75, before the spine-blindness fix: tests,contract passed (engine-contract 37); the CI engine-us job passed (448, 6 skipped); the engine-free shared and US suites (about 14,300 tests) failed only the three spine-blindness checks now fixed and six tests that cannot run in the Modal image (Python 3.14.2's worker-recycling guard, no ps binary, running as root, subprocess timeouts under 7 workers), in files this PR does not touch. The UK and node-backed orrery tests were not run.
  • GitHub CI runs now that the PR targets main: 11/11 checks pass at 7e203d0bc, both engine-free legs included.

🤖 Generated with Claude Code

DTrim99 and others added 21 commits October 12, 2026 13:47
Add the scf_education_debt runtime module. It carries federal education
debt from the 2025 SCF through two weighted QRFs and allocates it to the
SCF-style reference person and spouse as four person inputs:
federal_student_loan_balance and federal_parent_plus_loan_balance
(policyengine-us#9721), is_income_based_repayment_new_borrower and
federal_student_loan_graduate_share (policyengine-us#9724).

- Only loans flagged federal count; X7179 is spread over loans 1-6 in
  proportion to their balances; whose-education code 3 is Parent PLUS
  on the reference person, codes 6, 7 and -7 the reference person's own.
- scf_wealth declares the two pinned 2025 files as artifacts 3 and 4
  with a new scf_2025 vintage. Net worth, liquid assets and auto loans
  stay on 2022.
- The fiscal-refresh tool runs the stage only with --scf-education-debt.
  The four columns join the stored-input register with the pending-
  engine reason the parent ids carry; the coverage manifest is left
  until the engine pin defines them.
- Regenerate sources, vintages and calibration YAML and re-pin the
  frozen source_stages.json and calibration projection digests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Weighted fit of the stage against SCF 2025: source checks against the
Fed's Alternate Table 13 (19.92%, $25,500, matched exactly), a five-fold
family hold-out with a predictor ablation, and the stage imputed onto
the published default populace-us-2024-spm-20260915 at its weights.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Codes the CPS recipient side the way the SCF defines it: married includes
cohabiting partners, children count the partner's children, and the
reference person follows the householder-couple rule. These move Parent
PLUS from 21.5% to about 16% of imputed federal balances (SCF 13.8%) but
leave the borrower-household share at about 14.2% (SCF 18.2%).

Adds diagnosis and experiment scripts and their first receipts. The
predictor-shift and remedy runs still need to run (on Modal; local memory
is too low); the fit receipt and PR body are not updated yet.

Part of #1073.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first host fit drew federal education debt for 14.4% of CPS
households against 18.2% in SCF 2025, and gave Parent PLUS 21.5% of
balances against 13.8%, while the SCF hold-out reproduced the SCF. The
cause is covariate shift: the CPS host carries tax-record imputations of
interest and dividends and of Social Security and pensions, which put
42% and 18% of 25-34 households above zero against 16% and 6% of SCF
families. Mapping each to its SCF distribution at equal rank moves the
host's borrower share most of any predictor; all together reach the
hold-out level (receipts/1073-predictor-shift-diagnosis.json).

- SCF_EDUCATION_DEBT_CANDIDATE_PREDICTORS keeps every predictor both
  sides build; SCF_EDUCATION_DEBT_PREDICTORS, used by both QRFs, leaves
  out the two shifted income components, each with its reason in
  SCF_EDUCATION_DEBT_EXCLUDED_PREDICTORS.
- The recipient definition sums income over the reference person,
  spouse or partner and their children, the CPS counterpart of the SCF
  primary economic unit, on top of the SCF-matching codings.
- The stage spec drift check now covers both QRFs; the stage notes,
  module docstring and changelog say why.
- The receipt builder reports the build, the first receipt and the
  all-candidates setting side by side and records commit and run id.
- experiments/modal_1073.py runs the heavy steps on Modal, adds generate
  and spec steps, and collects every tracked file a run changed.

Re-pins the frozen source_stages.json digest; the generated mirrors are
regenerated on Modal in a follow-up commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…edictors (#1073)

Output of tools/generate_us_bundle_from_constants.py (run on Modal at
b9f3d67, run 20261020T154855Z-b9f3d6719): both education-debt QRFs drop
interest_dividend_income and social_security_pension_income, the stage
notes follow the module, and the stage-asset digest moves.

The Modal experiment runner now strips the Modal runtime's PYTHONPATH from
step environments; inheriting it put Modal's own packages beside the venv's
and the worker-identity check refused the duplicate idna.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_a_spouse_draw_without_a_spouse_moves_to_the_reference_person checks
where a spouse's draw lands. With the synthetic donor's random incidence
and a 30-household frame, the SCF-comparable predictor set draws no debt
for any couple on 3 of 20 seeds (seed 11 among them), while the routing
assertion holds on all 20 for both predictor sets (Modal probe run
20261020T161732Z-5d047f687). Every donor family now borrows in this test.

The Modal runner gains --script for an uncommitted local probe, recorded
by sha256 in the run log.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… loan register entries (#1073)

- experiments/1073-scf-education-debt-receipts.md and its JSON are the
  receipt of Modal run 20261021T133110Z-e35c4b0f9 at e35c4b0 (build,
  first-receipt and all-candidates settings on the published default).
- experiments/modal_1073.py gains `launch`, which spawns the run and
  exits, so a stopped local client cannot cancel it, and `--script` for
  an uncommitted probe.
- test_support/microcosm_data/stored_inputs.py lists the four federal
  student loan inputs among the producer-bound register entries, next to
  the parent ids: no file examined for #1026 stores them, and
  test_us_stored_input_register.py binds them to their live producer.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ves (#1073)

The scf_wealth education-debt declaration adds 145 configuration fields to
the US bundle (141 authored, 4 resolved). Modal probe run
20261021T145336Z-b1a461f19 printed every value below from the code; none
is derived by hand. It also diffed the field universe against origin/main:
all 159 added and 14 removed pointers belong to exactly one claim, and no
field is unclaimed or claimed twice.

- field_usage: 42,329 configuration / 32,545 authored / 9,784 resolved
  (was 42,184 / 32,404 / 9,780); claim pins source_stages 1,845 (the two
  2025 artifacts and six operations of scf_wealth), vintage_records 120
  and resolved_vintage_authorities 67 (scf_2025), calibration 316 (the four
  person outputs in the QRF tail-concentration list), each with its
  observed digest; ledger mode and generation-0 effect counts in the
  field-usage and coverage-tool tests.
- inventory_coverage: source_manifest, late_resource_semantics, tail and
  full_checkpoint digests; tail control fields 938 and resolved
  references 336.
- The pool tool's bundle spec_sha256 (a6f254a8...).
- experiments/modal_1073.py: the spec step first runs the bundle generator
  and the coverage attestation in --check mode.

The coverage tool's pointer-inventory digest and the regenerated
docs/evidence/spec-engine/us-f0-coverage.json follow in the next commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…1073)

Modal probe run 20261021T150509Z-fcbd704c9, with the previous commit's
field-usage and inventory pins in place, printed the coverage tool's
pointer-inventory digest (37ce78d6...) and regenerated
docs/evidence/spec-engine/us-f0-coverage.json with tools/spec_engine_coverage.py;
--check then reported 42329/42329 configuration fields and 41/41 inventory
checks. The bundle spec_sha256 stays a6f254a8....

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#1073)

test_support/microcosm_build/spec_engine_inventory_coverage.py keeps its own
copy of the reviewed inventory counts. Modal run 20261021T151444Z-05856e6ca
reported the two the stage moves, matching inventory_coverage.py and probe
run 20261021T145336Z-b1a461f19: resolved references 336 (the scf_2025
vintage) and tail control fields 938 (the four outputs in the QRF
tail-concentration list). Everything else in that run passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main moved the seed protocol, the compiled seed map and the bundle
spec_sha256 (microcosm#104, #1124, #1301, #1332); this branch moved the
source-manifest, resource-semantics, tail and full-checkpoint digests.
The conflicts keep both sides' changes; the merged tree's identity values
are re-derived on Modal in the next commit, and the F0 attestation is
regenerated there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Modal run 20261021T183247Z-97bb3af92 regenerated the bundle mirrors on the
merge commit (no change) and re-derived every identity value there: the
field-usage counts and claim pins, the ledger counts, the inventory
digests and counts, and the pointer-inventory digest all match the
committed values. The only moved value is the bundle spec_sha256, now
90fa3260... (main's kernel changes plus this branch's stage), which the
pool tool pins. tools/spec_engine_coverage.py regenerated the attestation
and --check reported 42329/42329 configuration fields and 41/41 inventory
checks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reran build_1073_scf_education_debt_receipt.py on Modal at 2a6e65a (run
20261022T153020Z-2a6e65a73), after the merge with main brought in #1332's
QRF change. Every fit figure is unchanged; only the receipt's commit, run
and date move.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…docstring

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… swap X8000 codes on loans 4 and 6 (#1073)

Parent PLUS is drawn for itself before the own shares, the IBR new-borrower
share is labelled an upper bound, and the receipt benchmarks Parent PLUS and
borrowers against Federal Student Aid.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…1073)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Regenerated on Modal (run 20261023T143414Z-b93cc3ec0): sources.yaml, the
field-usage counts and source_stages claim pin, three inventory digests, the
coverage pointer-inventory digest, the bundle spec_sha256 and the F0
attestation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… against FSA

Modal run 20261023T152015Z-60255dd98 at 60255dd (tests, spec and receipt
steps all passed). Parent PLUS falls from 13.8% to 9.1% of SCF federal
balances and from 16.8% to 10.7% on the host; FSA's Direct Parent PLUS share
is 6.7%, a floor.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d portfolio facts (#1074)

A new runtime module, fsa_student_loan_targets, binds Federal Student Aid's
portfolio tables (the six packages/ed_fsa packages of chronicle#347) to the
SCF education-debt stage's person columns, federal_student_loan_balance and
federal_parent_plus_loan_balance: the balance and borrower count of each
state and DC, of five borrower age bands and of nine debt-size bands, and
the Parent PLUS balance beside the balance of every other loan type. Every
family binds at FY2026 Q2 (31 March 2026), the latest quarter all four bound
tables publish, as a stock unaged at the 2024 build period (target aging's
new declared stock_at_source_reference_date alignment). Not Reported is
allocated to the reported locations, Puerto Rico and Other leave with their
share, and the age, debt-size and loan-type rows bind as shares of the one
in-scope control ($1,679.1B, 44.31M borrowers). Repayment plans, the
portfolio summary and loan-type recipient counts stay diagnostic, each with
its reason in the receipt.

compile_us_fiscal_target_registry binds the facts only with
student_loan_targets, which the fiscal-refresh release tool sets with
--scf-education-debt; the release materializes the rows from the stored
person columns, guards their restated age and balance bands, records every
fact's disposition in us_source_coverage.json and keeps the opt-in family
off the target-parity contract. Default compiles and releases are unchanged.
The ED Federal Student Aid provider joins the calibration label registries.

experiments/fit_1074_fsa_calibration.py refits the published default with
and without the family (plus L2-anchored arms) on Modal through
experiments/modal_1074.py, against the pinned feed plus chronicle#347's FSA
rows, now built at its merge commit on Chronicle main.

Squashes #1357's earlier history onto #1246's review-round head.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…chronicle#347 (#1074)

Modal run 20261023T191527Z-4b6bfce9a at 4b6bfce. chronicle#347's export
at its merge commit (acbedb5) is byte-identical to the PR head's (sha256
58774e11...), so the candidate feed and the test fixture are unchanged.

With loans for a child's education on co-resident adult children, the host
gains young borrowers (1,246 borrower records under 25, from 561) and its
Parent PLUS share falls from 16.8% to 10.7%, but holds the same borrowers
(21.2M) and borrower households (16.4%) at the published weights. Arm B now
meets 95% of the FSA targets within 5% (92% before) and every age band;
B + L2 78% (76%). Borrower households' share of weight after calibration is
33.7% (B) and 32.3% (B + L2), against 33.0% and 31.5% before and 18.2% of
SCF families: the routing moves balances within households, so it does not
shrink the doubling. Non-FSA capped error: B 3.62% (3.66%), B + L2 3.85%
(3.83%), A unchanged at 3.30%.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DTrim99
DTrim99 force-pushed the fsa-student-loan-calibration-targets branch from 61803e7 to 3b28865 Compare October 23, 2026 20:22
@MaxGhenis

Copy link
Copy Markdown
Contributor

A note on the reference date, from building the student debt pilot's measures pipeline (us-student-debt, private).

The body says the March 2026 stock is carried "unaged at the 2024 build period", and that "the analysis reads the columns in 2026". In policyengine-us, though, both balance variables declare uprating = "gov.bls.cpi.cpi_u":

  • policyengine_us/variables/gov/ed/student_loans/federal_student_loan_balance.py
  • policyengine_us/variables/gov/ed/student_loans/federal_parent_plus_loan_balance.py

So when a dataset holds the balance at 2024 and a simulation reads it in 2026, the model raises it by CPI-U from 2026 over 2024. At policyengine-us 035a1570 that factor is 326.588 / 309.794 = 1.0542. As a check, a one-person Microsimulation on a 2024-dated dataset with a $30,000 balance reads $31,626 in 2026 (measures/tests/test_counterfactuals.py::test_loan_balances_held_at_build_values in that repo). The FSA stock would end up aged after all: about 5.4% high in any 2026 read, the dashboard included. Payments that read the balance move with it: the standard payment amortizes it (standard_repayment_payment.py:25), and IBR is capped at the standard payment (income_based_repayment_payment.py:47).

Three ways to resolve it:

  1. Write the two balance columns at 2026 in the build, so nothing uprates them.
  2. Drop uprating from the two balance variables in policyengine-us. A loan stock isn't a price-indexed flow. This would also stop aging for any other dataset that relies on it, so check those first.
  3. Leave the build as it is, and have each consumer hold the 2024 values in 2026. The measures pipeline does this now with set_input, but the dashboard and other users wouldn't.

Option 1 or 2 fixes it for every consumer. @DTrim99, your call which. Whichever you choose, it would help to note in the PR body how the dashboard reads these columns.

DTrim99 and others added 6 commits November 2, 2026 11:21
Conflicts: the scf_wealth stage notes (main's source-impute citation and the
education-debt rules both kept), the release tool (both new helpers kept), the
stored-input register (county_fips_2023 beside the four student loan inputs),
and the spec identity values, which the next commit re-derives on Modal.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Regenerated on Modal (run 20261102T162227Z-3c4cfe441): sources.yaml, the
field-usage counts (42,489 configuration fields, 32,704 authored and 9,785
resolved), the source_stages and calibration claim pins, four inventory
digests (source_manifest, late_resource_semantics, tail, full_checkpoint),
the resolved-reference and tail-control counts (337, 942), the coverage
pointer-inventory digest, the calibration projection digest and the bundle
spec_sha256. The source_stages.json byte digest is the merged file's. The F0
attestation is from run 20261102T163748Z-c012b92c9 (write, then --check and
the generator's --check: 42,489/42,489 configuration fields, 41/41
inventory checks).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Modal run 20261102T172355Z-3297aefd8 at 3297aef. No reported number moved:
the fit JSON differs from the 60255dd receipt only in its provenance and in
the last digit or two of 14 floats (floating-point summation order).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n-targets (#1074)

Brings #1246's merge with main (e226192), its re-derived spec identity
values and its rerun receipts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Six arms bind the 67 FSA balance targets at full loss weight and the 65
borrower-count targets (state, age band, debt-size band) at 0, 0.1 and 0.25
of their weight, renormalized to mean 1, with and without the L2 anchor. At 0
the counts only validate. The existing arms are unchanged and the receipt
keeps their tables; a new section compares the arms on the FSA balance and
count fit, the non-FSA surface, the share of households with debt, total
borrowers, the Parent PLUS share and the borrower age profile against FSA.

The receipt's reference-date sentence now says what the stored balances are:
the March 2026 stock written at the 2024 build period, which
policyengine-us#10415 (merged 2026-10-30) holds at its stored value in every
year. The runner and fit docstrings say the candidate keeps the 51aa40f feed,
the pin before main's cf9ece2 re-pin, which adds only the B4923C facts that
bind no target.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…1074)

Modal run 20261102T174003Z-88d7a950d at 88d7a95. The merge with #1246's
updated head moved nothing: all 4,363 numbers of the existing arms are
identical to the 4b6bfce receipt.

Binding the balances with the borrower counts at 0.25, 0.1 and 0 of their
weight lowers the share of households with debt from 33.7% to 32.7%, 31.7%
and 30.2% (B + L2: 32.3% to 28.2%, 26.7% and 25.7%; SCF 18.2%), and the
borrowers from 44.26M to 42.98M, 41.71M and 39.75M (B + L2: 42.67M to 34.11M
at 0). Balance targets within 5%: 99% in B, 96% in each down-weighted arm (B +
L2 94%, 88% to 87%); count targets within 5%: 92% to 43%, 31% and 17%. The
solver reaches the balances with larger balances per borrower, so the bands
under $10K fall short. Non-FSA capped error 3.45% to 3.50% (B 3.62%, A
3.30%); Parent PLUS 6.8% to 6.9% throughout.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, David. All 234 feed rows from FSA's location, age, debt-size, loan-type and summary workbooks match their cells, and the body's fit tables match the receipt. A trial merge with main (db5a1c54) compiles 33,249 targets with main's digest without --scf-education-debt, and adds 132 with it. #1525 targets your branch with the module and test changes for Parent PLUS and the refusals, so you can merge it or take what you want from it.

Parent PLUS

Please make both loan-type rows diagnostic, which leaves 130 targets. The body already calls FSA's row a lower bound on the stage's column. B binds it at full weight, taking the host's 10.7% of the balance down to 6.8% while the SCF shows 9.1%. "Excluding Parent PLUS" goes too, since every other family pins the total and that row would still constrain the split. A total-only row would repeat the control those families bind.

FSA's notes don't say where a consolidated Parent PLUS loan goes, and the stage's draw doesn't read consolidation status (scf_education_debt.py:838-880). Under 34 CFR 685.220(f), the Secretary pays off each loan that a Direct Consolidation Loan consolidates. Please cite that in the doc (docs/us-fact-to-target.md:261-264) and the body, and give FSA's classification, with the lower bound that follows from it, as an inference.

Borrower counts

Please keep them at full weight as the baseline. At the host's 1.05 borrowers per borrower household, FSA's 44.31M implies debt in 34.3% of the host's households, and B lands at 33.7%. Holding the SCF's 18.2% of those households would take 1.99 borrowers per borrower household, against 1.11 in the SCF.

The x0 arms give up the published count (39.75M and 34.11M against 44.31M) and raise the mean balance to $42.2K and $48.8K against FSA's $37.9K. They miss under-$5K balances by 38.5% and 54.3%, and they keep 80% and 59% of the rise in households with debt that B and B + L2 show. No release flag reproduces them, since --target-family-loss-multiplier ed_fsa= scales balances and counts together (target_loss_weights.py:164-176), so please drop that option from the body. The student debt pilot's scope names FSA's balances and borrower counts as totals to match, so binding count shares instead would change what the pilot calibrates to.

L2

The +L2 arms call calibrate with its default record basis (experiments/fit_1074_fsa_calibration.py:1065-1076), and solve.py:55-67 says that basis spreads weight without anchoring to the design, while chi_square measures distance from it. In B + L2 the median record keeps 1.1% of its published weight, against 0.07% in B, and ESS rises from 3,045 to 4,424. So please call these arms a concentration penalty in the body, as the --l2-lambda help does, and add a chi_square arm at 0.03 through calibrate's l2_basis argument. The ACS local L2 study recommends that setting on its own loss and measures lambda in units of the loss (experiments/us-acs-local-l2-basis-20260928/README.md:79-91), so add a second lambda if 0.03 barely moves the fit.

The pilot builds with Route A's --dense-default-dataset (tools/route_a/route_a.env.example:46 at main). There --refit-l2-lambda refuses at start (build_us_fiscal_refresh_release.py:1982-1985) and --l2-lambda passes no basis, so the release can take lambda 0 or the record penalty today, and chi-square would need a pass-through flag.

Income

The fit JSON holds by_income_quintile tables that the body and the Markdown receipt leave out. Here's the share of households with federal education debt, by each arm's own household income quintile:

Arm Q1 Q2 Q3 Q4 Q5 Q5 − Q1, points
SCF 2025 11.4% 17.1% 17.8% 26.6% 17.9% +6.5
Release 14.3% 15.8% 14.7% 23.1% 14.2% 0.0
A 15.7% 15.7% 15.8% 22.0% 13.9% −1.8
B 37.2% 36.4% 28.0% 40.7% 26.2% −11.0
x0 31.1% 32.2% 25.7% 38.5% 23.5% −7.6
A + L2 14.9% 15.8% 15.1% 22.4% 14.7% −0.2
B + L2 32.9% 33.9% 27.0% 41.3% 26.5% −6.4
x0 + L2 24.7% 26.3% 21.1% 35.1% 21.6% −3.0

Q4 tops every row, so Q5 − Q1 gives an endpoint contrast rather than a slope. The release arm starts flat against the SCF's +6.5, and the FSA family moves the contrast by about 9 points without L2 (A to B) and about 6 with it. The bottom two quintiles hold 31.4% of the SCF's borrower households, 36.6% of the release arm's, 43.7% of B's and 41.4% of B + L2's.

Each arm recomputes its own cut points. The pilot counts borrowers by person and cuts tax-unit AGI over all tax units (measures/definitions.py:50-52 and measures/config.py:498-503 in us-student-debt), so these rows don't give the pilot's income tables. Please add all five quintiles and the A controls to the receipt, with a table at fixed cut points.

Rerun

Please rerun on #1246's final head and the feed main pins (1d45a89c…, 40,642 rows). The current receipt (20261102T174003Z-88d7a950d) comes from a run at #1246's 74837b4d on candidate feed d8e3d0fd…, and the 130 targets change what binds. The rerun needs the release arm, A and B at lambda 0, record 0.03 and chi-square 0.03, and x0 at lambda 0 and record 0.03. Please report FSA fit, non-FSA fit, ESS and the income tables for each arm, since the release's L2 setting depends on them.

fit_1074_fsa_calibration.py:1163-1167 reads FSA's Parent PLUS figure from the target row, so after #1525 it needs the diagnostic fact. Line 4201 of the fit JSON holds a bare NaN, so please write null there and pass allow_nan=False. The body's run table still calls 20261023T191527Z-4b6bfce9a "The receipt", and the new run can take that label.

These arms re-solve the published 57,240 households while Route A solves the whole base. fit_1074 reads its host through --h5 with the release's loader (fit_1074_fsa_calibration.py:1001), so the same arms on a Route A base would come closer to the release. At the release's gates, us_scf_education_debt_fit.json reports the fit at pre-calibration weights (build_us_fiscal_refresh_release.py:12728-12743), and build_1073 re-imputes the stage onto any H5 it reads. Could the release also write us_scf_education_debt_fit_table on its calibrated frame before scoring? Whether widening the stage can wait for the #1073 follow-up depends on that table and the release's coverage checks.

Refusals and tests

The docstring at fsa_student_loan_targets.py:269-273 says a partial table can't bind silently, and at ff4530bc 11 single-row drops still bind and move targets. For example, dropping a Not Reported row from the location table moves targets by about 3.1% for balances and 4.4% for borrowers. _require_reviewed_rows in #1525 refuses any location, age or debt-size table whose rows differ from FSA's set, including a table that repeats a row.

The refusal test searches _main's text and still passes if someone comments out the check, while #1525's test reads the syntax tree. #1525's two Hypothesis properties check the family totals and the band partitions on sampled row values, balances and ages.

Reference date

The stage's draw, the materializer and the pilot's age table all read the frame's age with no shift (scf_education_debt.py:1373-1375, fsa_student_loan_targets.py:499-505, measures/extract.py:398). Please keep that, since a shift in the materializer alone would set calibration and the pilot's table apart. Please also say in the doc that the materializer groups the stored, unaged age into FSA's March 2026 bands. FSA's largest change in a band's share between March 2024 and March 2026, at ages 25 to 34, comes to about 1.4 points for borrowers and 2.6 for balances.

A 2024 or a 2026 read returns each person's stock, as the body says, and policyengine-us also raises household weights by population growth, ×1.0165 from 2024 to 2026 at the pilot's pin. So a perfect fit at $1,679.1B and 44.31M reads $1,706.8B and 45.04M in 2026 before the pilot's borrower mask. Please say so in the body's reference-date paragraph, as us-student-debt#16 does in the pilot's METHODS.

Body and doc lines

  • The body's "independent adults living on their own or with others" and the doc's "live on their own or in someone else's household" (docs/us-fact-to-target.md:252-255) take in adults living alone. The stage's docstring (scf_education_debt.py:147-150) names adults in someone else's household, and an adult living alone draws as the reference person (:124-126, :1350-1369). These receipts don't separate the causes of the host's shortfall against FSA's borrower count.
  • "What waits on chronicle#347", the opening's "the US feed re-pin" and the doc's "When the facts reach the pin" (docs/us-fact-to-target.md:265-270) describe the re-pin as ahead, and #1506 merged it with the parity resources.
  • The reference-date paragraph's "Once" and "Until then" sentences can go, since policyengine-us#10415 merged on 10/30.
  • "FFEL and Perkins loans whose commercial lenders need not report an address": FSA says Not Reported includes those loans and doesn't say they fill it.
  • "recipient-based loan-type table": its $1,723.8B comes from dollar rows, and its counts alone measure recipients.
  • The "Spec identity: unchanged" bullet repeats an older field count (42,368), and the top of the body has 42,489.
  • A missing age lands in the youngest band through fillna(-1.0) (fsa_student_loan_targets.py:502), and no test pins that.

#1246 landed on main as a squash (b563fbb), while this branch's history still
holds its older commits, so a plain merge of main conflicts in about 20 files.

This merge takes main's version of every file only #1246 changed and applies
this PR's own diff since 74837b4, 19 files, on top. git merge-tree computed it
with --merge-base=74837b4d7. Fifteen of the 19 files match this branch's blobs,
and the other four (three test files and the release tool) keep the same
changed lines.

The engine-free tests for the FSA targets, fiscal targets, refresh builder,
spine blindness, labels, SCF stage and spec pins pass on this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor

David, I merged main into this branch (2a8d4070) so your rerun can start from #1246's final head. #1246 landed as a squash while this branch still held its older commits, so a plain merge conflicted in about 20 files. Your 19 files keep their diff from 74837b4d, every other file takes main's version, and #1525 still merges cleanly on top. Please pull before your next push.

us_runtime.fiscal_targets imports the new fsa_student_loan_targets module, so the
transport bindings' identity reaches one more module. test_the_batterys_binding_identity_adds_only_the_leaf_and_its_imports
names it in BINDING_MODULES_ADDED_AFTER_BASE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor

David, engine-free failed on the merge because us_runtime.fiscal_targets now imports your new targets module, which adds one to the transport bindings' module count. I named the module in BINDING_MODULES_ADDED_AFTER_BASE (493a09e7). That job stops at its first failure, so the rerun may show more, and you'll want to pull before your next push.

DTrim99 and others added 7 commits November 6, 2026 09:17
…tudent-loan-calibration-targets (#1074)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…elease fit table (#1074)

Max's review of 2026-11-05, on top of #1525's diagnostic loan-type rows:

- The fit runs the review's grid: A and B at lambda 0, record L2 0.03 and
  chi-square L2 0.03 (calibrate's l2_basis), and B with the borrower counts
  at x0, at lambda 0 and record 0.03. --extra-chi-square-lambda adds A and B
  at another lambda. Every receipt table covers every arm.
- It reads FSA's Parent PLUS figure from the diagnostic loan-type facts,
  writes strict JSON (null, allow_nan=False) and refuses a feed that binds no
  FSA target.
- The receipt adds every household-income quintile for every arm, at each
  arm's own cut points and at the release arm's applied to every arm.
- A --scf-education-debt release writes us_scf_education_debt_calibrated_fit.json,
  the stage's fit table on the exported frame at its shipped weights, before
  post-export scoring; a test pins the write and its place in _main.
- A missing age counts in the youngest band (FSA_MISSING_AGE), now named,
  documented and tested.
- The doc and changelog cite 34 CFR 685.220(f) and give FSA's filing of a
  consolidated Parent PLUS loan as an inference, say the materializer groups
  the stored, unaged age, and drop the re-pin as a future step (#1506).
- The Modal runner builds the feed the commit pins by default and gains
  fit_smoke and lint steps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The receipt writes a flat Q5 - Q1 as +0.0, not -0.0, and a negative income
as -$945,807. The FSA_MISSING_AGE comment says the plan, not FSA, spreads the
"not reported" ages, and the doc's Parent PLUS share reads as a share of the
loan-type dollar rows. The fit_smoke run 20261105T214517Z-66b37d37c showed both
signs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Run 20261105T215640Z-f83241311 passed ruff check and every test, and its
format check flagged only eleven us_runtime modules this PR leaves untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
families' reviewed-exclusion reason no longer lists the loan-type table among
the tables a --scf-education-debt build binds. The reason changes in the
builder and in the eleven manifest entries alike.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Modal run 20261106T142002Z-8a9d335a7 at 8a9d335, on the feed main pins
(Chronicle 41b2fb2b, sha256 1d45a89c..., 40,642 rows, rebuilt as pinned):
6,102 non-FSA and 130 FSA targets, none dropped, every arm of the review's
grid solved. B binds 95% of the FSA targets within 5% at ESS 2,932 and 33.9%
of households with debt; B + record L2 76% at 4,484 and 32.4%; B +
chi-square L2 59% at 6,222 and 28.0%. Chi-square 0.03 moves the fit, so no
second lambda. The doc's measured limits take the new levels.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_us_post_export_scoring executes _main's export and cleanup statements
on their own, without args, so the calibrated fit-table write inside that
span failed it with a NameError. The write now follows the ladder cleanup,
still before any scoring or terminal gate, and the builder test pins that
order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor

Thanks, David. The body's fit and income tables match the receipt JSON, CI passed all 11 jobs at 7e203d0, and another reviewer reads that head next.

On the two items under "For decision":

  1. Let's keep lambda 0, Route A's setting, for the pilot's release. B fits the 130 FSA targets more closely than B + record L2 (95% within 5%, against 76%), and the 6,102 others too (3.66% capped error, against 3.86%). The record penalty raises ESS from 2,932 to 4,484 and moves Q5 − Q1 from −9.4 to −5.1 points at the release arm's cuts, still opposite in sign to the SCF's +6.5. These arms re-solve the published 57,240 households, where A drops the median household to 2.1% of its published weight with no FSA target. Route A's whole-base dense solve on 10/24 ended at an ESS of 72,166 across 352,932 households at lambda 0. So the receipt can't size a penalty on the release's support, and this PR needs no --l2-basis flag. Your calibrated fit table measures the income contrast on the release itself.

  2. Widening the stage past the SCF primary economic unit can wait for its own PR under Carry SCF education debt into the US build and allocate federal student loan balances to borrowers #1073, after the release.

@DTrim99

DTrim99 commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks, Max, for merging main into this branch and for the bindings fix (493a09e). Both decisions are now recorded in the body: the release keeps lambda 0 (arm B), so there's no --l2-basis flag here, and widening the stage past the SCF primary economic unit waits for its own PR under #1073, after the release.

One item from "For review" stays open: Parent PLUS. With nothing binding it, it holds 10.2–11.3% of the balance in every arm, against 6.8% in FSA's diagnostic row. We read that row as a lower bound under 34 CFR 685.220(f), since a consolidated Parent PLUS loan likely files under "Consolidation". The release's calibrated fit table will show where it lands at the shipped weights. I'll mark this ready once the second review is in.

@MaxGhenis

Copy link
Copy Markdown
Contributor

Thanks, David. The second reviewer compiled the default build on main and on this head and got the same target registry, b8d571e91c22. They ask for one correction to how the PR describes the Parent PLUS column and for two smaller fixes, and they found nothing else that blocks the merge.

Parent PLUS

The module docstring (fsa_student_loan_targets.py:31), _REASON_LOAN_TYPE_DOLLARS (:279-280), docs/us-fact-to-target.md:219-220 and the Parent PLUS bullet under my 11/5 review in the body say that federal_parent_plus_loan_balance holds every loan for a child's education. Since #1246, the stage gives a loan for a child's education to a co-resident adult child who fits it (scf_education_debt.py:974). It keeps the loan as the head's Parent PLUS only when that child could have borrowed as a dependent student and the undergraduate borrowing exceeds the dependent limit. It makes a graduate loan with no fitting child the head's own (:973). Please replace "every loan for a child's education" in those four places with the balances the stage classifies as Parent PLUS. Please also add the graduate rule to the body's "What #1246 changed under it" line. The 34 CFR 685.220(f) inference and the rest of each passage can stay. The receipt's disposition table quotes the old reason (experiments/1074-fsa-calibration-receipts.md:228), and it can stand as the run's record.

On the share, let's leave it diagnostic, as you propose. Every other family binds the total, FSA's row likely understates the column, and the calibrated table reports parent_plus_total at the weights the release ships (scf_education_debt.py:2453). The manifest's artifacts list neither fit file, though (build_us_fiscal_refresh_release.py:11137), and Route A's publisher call passes no --extra-file (route_a.sh:614), so a publish with those arguments would leave both on the run host. Registering them can go in its own PR.

Two smaller fixes

  • The body's "Spec identity: unchanged" line cites 42,489/42,489 configuration fields, while the latest spec checks, at 8a9d335a7, report 42,394/42,394.
  • docs/us-fact-to-target.md:280-281 describes the calibrated table as reporting the share of households with debt "that the FSA family moved". The release tool's comment (build_us_fiscal_refresh_release.py:15398-15399) says the table reports "the borrower-household share it moved". The table gives shares at the shipped weights. Separating the FSA family's effect from the other targets' would take an A control on the release's support, so please drop the "moved" clause in both places.

Release checks

Route A's standalone preflight compiles the registry without student_loan_targets (release_gate_preflight.py:957) and never receives RELEASE_EXTRA_ARGS (route_a.sh:551), so a green preflight doesn't test the FSA family. The integrated --dry-run-gates-report compiles and stages the family but stops before the solve (build_us_fiscal_refresh_release.py:15024), so we first see its fit at the release's own gates.

Once you've pushed these, please mark it ready, and we'll check that head.

…p the "moved" clause (#1074)

Since #1246 the stage gives a loan for a child's education to a fitting
co-resident adult child, keeps it as the head's Parent PLUS only past the
dependent limit or when no child fits an undergraduate loan, and makes a
graduate loan with no fitting child the head's own. The module docstring,
the loan-type reason and the doc now say the column holds the balances the
stage classifies as Parent PLUS. The calibrated fit table reports shares at
the shipped weights, not the FSA family's effect, so the doc and the release
tool's comment no longer say it reports what the family moved.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DTrim99
DTrim99 marked this pull request as ready for review November 7, 2026 21:35
@DTrim99

DTrim99 commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks, Max, and thanks to the second reviewer. Pushed in b6a9805, and marked ready:

  • Parent PLUS column. The module docstring, _REASON_LOAN_TYPE_DOLLARS, docs/us-fact-to-target.md and the body's Parent PLUS bullet now say the column holds the balances the stage classifies as Parent PLUS. The body's "What Impute federal student loan balances from the 2025 SCF (#1073) #1246 changed under it" line now gives the full rule: a fitting child's loan stays Parent PLUS only past the dependent limit, an undergraduate loan with no fitting child is the head's Parent PLUS, and a graduate loan with no fitting child is the head's own (34 CFR 685.200(c)(2)(i)). The 685.220(f) inference is unchanged, and the receipt's disposition table stays as the run's record.
  • Spec identity line. Now 42,394/42,394, matching the checks at 8a9d335.
  • "Moved" clause. Dropped from the doc and from the release tool's comment: the calibrated table reports shares at the shipped weights.

Parent PLUS stays diagnostic. Registering both fit files in the manifest and passing them through Route A's publisher can go in their own PR, as you suggest.

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, David. At b6a98053 the docstring, _REASON_LOAN_TYPE_DOLLARS, the doc and the body's Parent PLUS bullet describe the column as the balances the stage classifies as Parent PLUS. The #1246 line now gives the graduate rule, the spec line reads 42,394, and neither the doc nor the release tool's comment says the FSA family moved the share. An independent review of this head found nothing that blocks the merge.

The follow-up PR that registers the two fit files for publication can also add "consolidated or not" to _REASON_LOAN_TYPE_DOLLARS (fsa_student_loan_targets.py:277-279) to say why the column can exceed FSA's row. In the body, the #1246 line reads a little narrower than the code (scf_education_debt.py:951-974). The stage keeps a loan for a child's education as Parent PLUS whenever no listed child fits it, unless the loan financed a graduate program. It counts a child as possibly dependent when the child's age band allows an age under 24 in the loan's year.

@MaxGhenis
MaxGhenis merged commit 3db6515 into main Nov 7, 2026
11 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor

GPT-6.1 Sol's review of 7e203d0b asked for the Parent PLUS wording and two smaller fixes, and an independent Opus 5.5 review of b6a98053 approved them. CI run 42416516550 passed all 11 checks at that head, and the PR landed as squash 3db65154.

MaxGhenis added a commit that referenced this pull request Nov 11, 2026
Main moved from 7080ba8 to 27e305d (microcosm#1571). Two conflicts,
both where each side adds its own keyword to
compile_us_fiscal_target_registry:

- us_runtime/fiscal_targets.py: this branch adds state_age_band_targets
  and main's #1357 adds student_loan_targets, in the signature, the
  docstring and the first statements of the body. The merge keeps both.
  reference_facts (this branch) still feeds the dynamic references, and
  student_loan_plan (main) still reads materialized_facts.
- test_us_fiscal_targets.py: the signature test's expected parameter set
  lists both keywords.

Every one of the 25 files this PR changes carries exactly the PR's
changed lines on top of main. tools/build_us_target_parity_manifest.py
on the e86a83b feed rewrites target_parity_manifest.json and
target_parity_feed_families.json byte for byte (32 compiled families, 65
reviewed exclusions).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 13, 2026
… bands sort naturally (#1598)

* Carry each hierarchy dimension value's source order (value_rank)

The calibration dashboard lists FSA's nine debt-size bands alphabetically
("100K to 200K", "10K to 20K", "200K+", ...) because HierarchyDimension
carries no value order and the dashboard falls back to sorting labels
(Max Ghenis on #1357; PolicyEngine/calibration-diagnostics#276).

- HierarchyDimension gains an optional value_rank, the value's position
  from 0 in the order its source lists the dimension's values.
  _inherited_dimensions takes it from layout.groupby_ordinal for the
  group-by dimension when every selected fact gives the same non-negative
  integer; derived facts and other dimensions give none, and a Medicaid
  enrollment substitution drops the template state's rank.
- to_dict leaves out an absent rank, so TargetSpec.to_dict, the UK local
  surface and the diagnostics rows encode a rank-less dimension as before
  and registries written before this keep their version.
- Diagnostics schema 8 dimensions carry value_rank; schema 9 catalog
  values carry rank when no value of the dimension is ranked two ways, and
  expansion gives it back to the rows. Row references stay {id, value_id}.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Rank a catalog dimension only when no category ties two values

Probe on the pinned US feed: the USDA SNAP state tables give several
states one row number, so ranks shown together in one category would not
order them. Also re-pin the national_state surface version the ranks move
(3aa2a0830e7d -> c698e70d33ce) and give the ledger rank tests' facts their
group-by dimension label.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Say how the SNAP state tables number states; build the FSA diagnostics frame in one concat

The USDA SNAP state tables number the states within each region, so seven
states share row 0; the changelog and the hierarchy doc said only that the
tables list states by region. The schema 9 export test now adds the FSA
measures to the household table with one concat instead of one column at a
time.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Re-pin what the value ranks move, and regenerate the diagnostics JSON Schemas

On this branch's merge of main (c9e2211):

- The schema 8 and 9 diagnostics JSON Schemas gain value_rank and rank
  (tools/generate_calibration_diagnostics_json_schema.py).
- The US F0 spec pins. us_runtime/medicaid_take_up.py is an attested seed
  kernel module, so its edit moves the seed map, the seed protocol and the
  US bundle's spec_sha256 (3dabec5c to ab3f3aa9), and the loader's golden
  vector with them (repin_us_spec_engine_contracts.py,
  generate_us_bundle_from_constants.py, spec_engine_coverage.py, in the
  order the repinner documents; all three pass --check afterwards).
- calibrate.adam@1's H1 parity pin. The kernel hashes microcosm.calibrate
  sources this branch edits, so its implementation hash and node key move;
  direct.csv does not. tools/graph_parity_repin.py refuses off the one
  pinned platform (arm64/darwin/py3.14), and this ran on x86_64/linux. The
  case is numeric "bitwise", so its key carries no platform and
  test_h1_kernel_parity asserts the authoring platform's key and bytes
  everywhere. The tool's repin() therefore ran unchanged with only the
  fingerprint it uses to choose its own pin entry set to the authoring
  platform: the direct call's bytes, the dependency versions, the old pin
  against its recorded hash and the executed key against the derived one
  were all checked on Linux.

The pinned-feed national_state surface stays at c698e70d33ce after the
merge (6,102 of 33,250 targets); compiled with the rank lookup turned off
it is main's 3aa2a0830e7d.

Run on Modal (Linux, Python 3.14.2, uv.lock with the US engine). Afterwards
on the re-pinned tree: ruff check; the spec-engine, calibrate, diagnostics,
graph parity, ledger, fiscal and FSA target tests (1,656 passed, none
skipped, the pinned feed present); the engine-contract coverage tool and
inventory tests (9 passed); the engine-workflow multispine pool tool tests
(31 passed).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Answer the review of the value ranks: scope, registry format 4, the Medicaid test

- Ranks order values within one category, not a dimension's whole list:
  say so in HierarchyDimensionValueEntry, the catalog module and the
  hierarchy doc, and tell readers to put unranked values last instead of
  sorting only where every value has a rank.
- A rank is the position of the value's row among its source table's rows.
  HierarchyDimension (both packages) and the changelog said "in the order
  the source lists the dimension's values".
- The Medicaid substitution drops the template's rank because that rank is
  the template state's row, not because the substituted state has no row
  in the CMS table. The labelled template in the substitution test now
  carries a rank, so the test fails without value_rank=None.
- Registry files are format 4; formats 2 and 3 stay readable. A reader
  from before the field refuses a new file by its revision instead of
  failing on the unexpected value_rank keyword.
- The changelog says that UK targets gain ranks too, and which diagnostics
  readers refuse the new fields.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Re-pin the US spec identity and regenerate the diagnostics JSON Schemas after the review edits

On 081c025, which merges main at d5a9322:

- The US F0 spec pins again. The corrected comment in
  us_runtime/medicaid_take_up.py changes an attested seed kernel module's
  bytes, so the seed map, the seed protocol, the loader's golden vector
  and the US bundle's spec_sha256 move (ab3f3aa9 to 98d58365), with the
  same three tools in the repinner's order; all three pass --check.
- The schema 8 and 9 diagnostics JSON Schemas, whose descriptions follow
  the reworded model docstrings.

Not moved: calibrate.adam@1's parity pin (its pin tests pass before any
re-pin; the kernel hashes neither hierarchy.py nor registry.py), and the
pinned-feed national_state surface, still c698e70d33ce and 3aa2a0830e7d
with the rank lookup off. The registry format number is outside the
content version.

The hierarchy doc now gives the pinned feed's counts for its two examples,
read from the compiled registry: age has 21 values on 18 ranks across the
Census population and SSI recipient categories, and the CMS Medicaid
state dimension ranks 51 of its 52 values.

Run on Modal (Linux, Python 3.14.2, uv.lock with the US engine), on the
re-pinned tree: ruff check, ruff format --check and ci_test_plan verify;
the spec-engine, calibrate, diagnostics, graph parity, ledger, fiscal,
Medicaid take-up and FSA target tests with the pinned feed present (2,155
passed, none skipped); the engine-contract coverage tool and inventory
tests (9 passed); the engine-workflow multispine pool tool tests (31
passed). With value_rank=None deleted from the substitution, the
substitution test fails ("assert 4 is None"). The previous re-pin
commit's "1,656 passed" undercounted that run, which passed 2,153.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 13, 2026
* Publish a --scf-education-debt release's two fit tables

The release tool wrote us_scf_education_debt_fit.json and
us_scf_education_debt_calibrated_fit.json into its artifacts directory,
and the release manifest listed neither, so the publisher left both on
the run host. Write them into the release directory and list them as
diagnostics artifacts when the build ran with --scf-education-debt, and
only then, so the publisher uploads them under releases/<id>/ without
--extra-file. A flagged build missing either table fails before it
writes a manifest. Route A's publisher call needs no --extra-file; its
comment and runbook now say why (#1074, #1357 review).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Say "consolidated or not" in the loan-type reason and rewrap the Parent PLUS bullet (#1074)

The two edits from #1546 that #1545 didn't carry, as the comment closing
#1546 suggested.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Say the build-folder publisher puts the fit tables in the build's folder

Since #1571 the US releases publish through microcosm-publish-release-build,
which puts each build in releases/<release>/<build>/ and publishes every
artifact the release manifest lists. The docs, changelog, builder comment
and test docstring said the publisher uploads the tables under
releases/<id>/. The builder comment also says a build missing a fit table
writes neither manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Max Ghenis <mghenis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants