Repository navigation
Fill IRS SOI uprating series gaps with Publication 1304 actuals - #10128
Conversation
The national-total uprating series under calibration.gov.irs.soi had no values between 2015 and their next year, so each held its 2015 value through the gap. Their per-capita siblings drifted down with population and then jumped, and any input dated in or before the gap was uprated past it by the wrong factor. - Fill every missing year from IRS SOI Publication 1304 Table 1.4 (all returns) or Table 1.4A, using the same column as the series' existing values, and cite each tax year's file. - Correct values keyed to the wrong tax year or misread: partnership and S corporation income at 2022 was the tax-year 2021 total; Social Security at 2021 was the tax-year 2020 total; farm rental income at 2021 subtracted the number of returns with a loss, not the loss. - Restate rounded totals (partnership 2015, rental 2015 and 2021, unemployment 2021) and alimony 2015 exactly. - Replace the short-term capital gains placeholder (a PUF figure and a backcast) with Table 1.4A net short-term capital gain less loss (Schedule D line 7). - 2018 pensions: the 2018 Form 1040 combined IRA distributions with pensions, so SOI has no separate figure; estimate and say how. - Fill gov.ssa.uprating for 2015-2021 with third-quarter CPI-W; 2016 keeps the 2014Q3 level because no COLA was payable. - Guard: every series an input follows has a January 1 value for every year from its first to its last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The long-term capital gains series uses Table 1.4A's net long-term capital gain: returns with a gain, net losses left out. The short-term series netted gains against losses and carryovers, a large negative total that moves against gains (x4.1 from 2021 to 2022, the year gains fell 63%). Use the gain column, so both series measure the same thing and the index stays positive. List the 2018 taxable pension estimate alongside the tax-exempt one in the citation test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Picking this up for the US hub. I'll post a hand-off comment here when it's ready. Generated by Claude Code |
Main rewrote gov.ssa.uprating for the 2027 COLA (#10225), so the reference list conflicted. Main's list already cites 42 U.S.C. 415(i) and the BLS CPI-W series (CWUR0000SA0) that the 2015-2021 third-quarter averages come from, so it is kept as is; the branch's 2015-2021 values merge cleanly above main's 2027 actual. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every series this branch fills or corrects now has a YAML case: an input dated in one year and read in a later one, whose expected value is the SOI table total (or the third-quarter CPI-W average) over population, computed by hand and linked to its workbook. The cases cover the corrected 2021 farm rental, 2020-2021 Social Security and 2021-2022 partnership totals, both 2018 pension estimates, the added 2022 unemployment total, the 2016 no-COLA hold and the 5.9% COLA for 2022. Each fails on main and on mutants that undo the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An uprating factor is a ratio of the series, so a series that crossed zero would turn each input projected across it from an income into a loss. This branch replaced the short-term capital gains placeholders (negative) with SOI net short-term capital gain (positive); the guard checks every series an input follows over its whole span. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Ready for hub review Head: Starting head was Land #10114 first. This PR's generalized guard covers CBO total AGI, which has no 2016-2018 values until #10114 lands. At this head that one case fails (below); it passes once #10114 is merged and main is merged here. Still needed?Yes. On
No other open PR does this. #10132 is stacked on this branch and extends it to TY2023. Correct?
Merge main
Main added ReviewsThe PR had no reviews or comments. The original body reported two rounds of external review; this adoption didn't rely on them, since every value was re-derived independently. TestsEach new YAML case's expected value was computed by hand: the SOI totals from the workbooks over the population series, or the CPI-W averages. The comment beside each case shows the arithmetic and links the workbook. At
The Python files above other than Changed rules and their cases: MutantsEach ran in its own process, and the file was restored after each.
All five were killed, and each file was restored with Correction to commit Parameter tree and blast radius
Judgment calls
Overlaps
Follow-up issues I'd propose
Generated by Claude Code |
|
US + core hub merge audit, us#10128 at
🤖 Generated with Claude Code |
Fixes #4281
What changes
The national totals under
calibration/gov/irs/soi/had no values between 2015 and their next year (2020, 2021, 2022 or 2023).parameter(instant)held the 2015 value through the gap, so each per-capita sibling (total ÷ population, built bytools/per_capita_uprating.py) drifted down with population and then jumped. The model then uprated a dollar input dated in a gap year, or read in one, by the wrong factor. Employment income per capita, for example:This PR:
gov.ssa.uprating(the cost-of-living index that veterans' benefits follow) for 2015-2021 with third-quarter CPI-W.Sources
Every SOI value is the "All returns, total" row of Publication 1304 Table 1.4 (
https://www.irs.gov/pub/irs-soi/YYin14ar.xls) for income items, or Table 1.4A (YYin14acg.xls) for capital gains, in thousands of dollars times 1,000. Each parameter file cites one workbook per tax year. The workbooks are XLS files, so no page anchor applies. For example, TY2016 Table 1.4 and TY2016 Table 1.4A.I re-extracted every value from the IRS workbooks, locating each column by its header label, never by position. Layouts change: TY2018 combined IRA distributions with pensions, TY2021 split partnerships from S corporations, and TY2022 renamed wages. All 115 table values match their cells. The two 2018 pension estimates reproduce the method each file states.
Corrections
Social Security cost-of-living index, 2015-2021
gov.ssa.upratingstarted in 2022, sobackdate_parametersheld 2022's value back to 2015. A veterans' benefit dated 2021 got no uprating into 2022, when the COLA was 5.9%. The new values are third-quarter averages of BLS CPI-W, not seasonally adjusted (CWUR0000SA0), the method the file already uses. All seven match BLS's monthly values. 2016 keeps the 2014Q3 level, because the 2015Q3 average (233.278) fell below 2014Q3 (234.242). Under 42 U.S.C. 415(i)(1)(B), a computation quarter is a third quarter "with respect to which the applicable increase percentage is greater than zero". Subparagraph (i)(1)(D) measures the next increase from "the most recent cost-of-living computation quarter", so 2016Q3 ÷ 2014Q3 − 1 = 0.35% reproduces the 0.3% COLA for 2017. Main's owntest_cpi_w_third_quarter_averages_reproduce_ssa_upratingencodes that rule independently, and it now checks these seven values too.Methodology choices
short_term_capital_gainscan hold a loss, and a loss now moves with gains: −$10,000 dated 2021 reads −$3,710 in 2022.gov.ssa.upratingcarries the 2014 third-quarter level through 2016. The 2015Q3 average (233.278) fell below 2014Q3 (234.242), so SSA paid no COLA for 2016 and 2014Q3 stayed the latest computation quarter (42 U.S.C. 415(i)(1)(B)). The file holds 234.242 for 2015 and for 2016, so a benefit dated 2015 keeps its amount in 2016 and rises 0.35% in 2017. With the 2015Q3 average, it would fall 0.4% in 2016 and rise 0.8% in 2017. The index holds unrounded CPI-W averages, as main's values from 2022 do, so a ratio can differ from the rounded COLA: 2021 to 2022 gives ×1.0592, and the COLA was 5.9%.Other judgment calls
start_instant. It existed because CBO's outlays start in 2022, a year after the old last SOI value. core'suprate_parameterprojects fromstart_instant, or else from the last authored value. With TY2022 in the file, those are the same instant, so this PR removes the override.gov.ssa.upratingreferences. Main (Update Social Security and SSI parameters for the 2027 COLA #10225) rewrote the list. It already cites 42 U.S.C. 415(i) and the BLS CPI-W series these values come from, so this branch keeps main's list. It drops its own SSA CPI-W page reference, which duplicates the BLS values, and ssa.gov refuses automated fetches.Effect
Parameters. I dumped every leaf parameter after system init at main (
3386304d28) and at this branch (ee4a74f1c9). 35 differ: the 17 SOI totals, their 17 per-capita siblings andgov.ssa.uprating(2015-2021). Nothing else moves, including the SSI and state supplement amounts that uprate bygov.ssa.uprating. The variable-to-series map stays the same. core's parameter uprating projects a series from its last authored value, so:extend_single_year_datasetgrows each year from the dataset's base year, so a dataset based in 2023 or later reads only unchanged ratios.Household inputs dated 2015-2022 and read in a later year are what change. Factor to 2025 for an input dated January 1 of year Y, main to this PR:
Unemployment compensation was the worst case: main carried pandemic-year totals forward, so it multiplied a 2019 input by 8.6 by 2025.
Blast radius. I scanned every YAML test for an input of a variable that follows a changed series, dated year X, in a case whose period is a later year T where index(T) ÷ index(X) changes. Two existing cases match, and neither asserts a value that moves:
medicare_irmaa_magi_two_years_prior.yamlCase 5 readstax_exempt_interest_incomedated 2015 at 2015 (two years before 2017), so the model projects nothing.medicare_cost.yaml"unit test 5 - 2021 data, premium-free", the input is 0.Tests
Pull request run 43547862788 at
ee4a74f1c9passed all 38 jobs on GitHub's merge with main045d1aee97.I computed each new case's expected value by hand from the SOI totals (or the CPI-W averages) and the population series. The comment beside it gives the arithmetic and links the workbook.
alimony_income,tip_income(employment),farm_operations_income(farm, a series of net losses),farm_rent_income(corrected 2021);long_term_capital_gains_on_collectibles,short_term_capital_gains,qualified_dividend_income,non_qualified_dividend_income;partnership_income(corrected 2021 and 2022),partnership_self_employment_net_earnings(self-employment),rental_income,social_security_retirement_reported(corrected 2020 and 2021);tax_exempt_interest_income,taxable_interest_income,tax_exempt_private_pension_income(from the 2018 estimate),taxable_private_pension_income(into the 2018 estimate),unemployment_compensation(added 2022);veterans_benefits: 2015 to 2016 unchanged (no COLA) and 2021 to 2022 ×1.0592 (the CPI-W ratio behind the 5.9% COLA).tests/core/test_uprating_series.py:test_series_has_a_january_value_for_every_year, over every series an input follows;test_soi_series_follows_its_projection_from_its_last_authored_year;test_series_keeps_one_sign.tests/test_parameter_files.py: every SOI value cites its own tax year's workbook, and the corrected values hold their table totals.Invariants
gov.ssa.upratingholds the third-quarter CPI-W of the latest cost-of-living computation quarter, and so never falls. Main'stest_cpi_w_third_quarter_averages_reproduce_ssa_upratingchecks this against BLS monthly values.test_every_per_capita_series_is_its_total_over_populationchecks this.Overlaps
tests/core/test_uprating_series.py. Fill CBO total AGI for 2015-2018 with IRS SOI actuals #10114'stest_default_series_has_a_value_for_every_yearnow repeats this PR's guard case for CBO AGI. This PR leaves it in place, and a follow-up can delete it.c96ccb7ba4:employment_income.yamland the metadata comment inunemployment_compensation.yaml.tests/test_parameter_files.pynext to this PR's SOI tests.ee4a74f1c9) before it leaves draft.Microsimulation impact
We estimate that this branch changes none of the outputs below in 2026 on the default dataset. We compared
mainat3386304d28with this branch atee4a74f1c9. The PR changes parameter values dated 2015 to 2022 only.employment_incometaxable_pension_incomesocial_securitylong_term_capital_gainsadjusted_gross_incomestate_income_taxincome_taxhousehold_benefitshousehold_net_incomeimpact: none measured in 2026
axiom: n/a: calibration and uprating data (IRS SOI totals and third-quarter CPI-W levels); this PR changes how household inputs grow across years, and no provision's amount changes
🤖 Generated with Claude Code