Repository navigation
Conversation
Historic Table 2 and the congressional-district file publish business (Schedule 1 line 3), partnership/S-corp (Schedule E line 32) and rental/royalty (Schedule E line 26) income as one signed column each: the amount nets losses against income and the count is every return with an entry. The materializer read them as positive parts counted when > 0. Add three net-income-(less-loss) concepts that keep the sign and count a nonzero tax-unit total, and retarget the six measure ids to them. Schedule C reads total_self_employment_income, because the frame moves SSTB businesses' whole amount out of self_employment_income; rental reads rental_income, because line 26 leaves out Form 4835 farm rental. A net-income-(less-loss) id from any family but HT2 or the district file refuses the compile. The Table 1.4 legs stay positive/negative parts: rent_and_royalty_net_income becomes a positive part like every other income leg, and the business legs read total_self_employment_income. No pinned fact uses them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1228) Compile tests for HT2 US/state and district rows of all six ids, a refusal test for unreviewed families, a Hypothesis property that binding depends only on the record-set family, and a pinned-feed invariant: every business, partnership and rental row binds its line's signed concept, nothing else does, no row binds a one-sided Table 1.4 leg, national_state carries 312 of them and each family's 51 states sum to at most the US row. The pinned compile moves national_state e1f9906bf0e0 -> 38c1db8880da (full f5bd921d6b21 -> 9f6c65fe8734) with the same 6,102 / 33,250 specs and no value change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prepare.py streams the Route A release's target-frame checkpoint into a 5,694-row CSR (checked: it reproduces every recorded initial and published estimate exactly), rebuilds the 312 business, partnership and rental rows from person inputs on the release's concepts (max gap 5.4e-8 against the checkpoint) and on the #1228 concepts, and recomputes each arm's loss weights (the release arm reproduces the recorded ones exactly). prepare_rebase.py adds the capital-gains-style arm: HT2 shares of Table 1.4 net-income controls. resolve.py re-solves one arm with the release's settings; modal_resolve.py runs the five arms on Modal; analyze.py compares them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Five arms re-solved on Modal from the release checkpoint. The release-binding replay reproduces the release (loss 0.028420 vs 0.028426, weight correlation 0.9999). The concept-exact binding lands the business, partnership and rental nets and nonzero counts within 0.3-7.3% of HT2 with no collateral cost (other targets' fit unchanged, ESS -0.6%), against a release Schedule C net of $821B (IRS ~$460-472B) and rental net at 2.6x target. The capital-gains style rebase gets Schedule C profits right and the net 1.3x high; dropping the rows lets Schedule C reach $1.3T. The frame's missing Schedule C losses are the remaining gap for every arm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add a hand-computed test through the SOI slice on the fake engine: a unit whose members net to a Schedule C loss, an SSTB-only unit and a farm-rent-only unit, for all three signed concepts and their legs. It fails if business reads self_employment_income. - Tighten the district-file family match to congressional_district[_YYYY]. - Doc and SOI_VARIABLE_MAP: the frame's partnership/S-corp column comes from the PUF donor's nonpassive fields, not E26270; cite Publication 1304 for the "(less loss)" count convention and for Table 1.4's rental total being Schedule E lines 26 + 39 + 40; qualify the leg identity; call the states-sum check a data pin; correct the full-surface row attribution (2,928 district-file rows and 54 single-district-state HT2 rows). - Changelog: the rental amount was already signed. - Experiment: five arms, cache keyed on inputs and resolve.py digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vahid-ahmadi
left a comment
There was a problem hiding this comment.
engine-free fails on the incumbent-package guard (docs/us-soi-net-income-less-loss-concepts.md:97). The new doc names policyengine-us-data ("policyengine-us-data datasets/puf/puf.py"). test_us_plan.py::TestUsSources::test_no_incumbent_data_package_references_in_live_tree forbids that name in the live tree. In CI run 38279638458, both engine-free lanes fail on it with ('docs/us-soi-net-income-less-loss-concepts.md', 'policyengine-us-data'). Each lane stops at that first failure ("stopping after 1 failures"; 10,557 passed on 3.13), so the rest of engine-free hasn't run at this head.
Suggested fix: describe the source without the package name (for example "the incumbent processed-PUF build's puf.py"), then rerun to get the full engine-free result.
Review drafted with AI assistance and checked against the code at ec3df20.
Fixes #1228. Chronicle side: PolicyEngine/chronicle#346.
This changes what 312
national_statetargets bind, so it waits on Max's ruling (d1432) before merge.Summary
Historic Table 2 (HT2) and the IRS congressional-district file publish business (N/A00900), partnership/S-corp (N/A26270) and rental/royalty (N/A25870) income as signed "net income (less loss)" columns: the amount nets losses against income, and the count is every return with an entry. The target compiler bound them to one-sided model concepts:
Two more gaps sat on the same rows:
self_employment_income, which in this frame excludes SSTB Schedule C income;On the certified Route A release, these rows drive the calibrated Schedule C net to $821B, against IRS ~$460–472B, and the rental net to 2.6× its target.
This PR binds each column to the signed tax-unit total of its line and counts nonzero totals. A re-solve of Route A from its own checkpoint shows the effect:
What each IRS column counts
The 2022/2023 HT2 and 2022 district-file doc guides define:
schedule_c_income_*partnership_scorp_income_*rental_royalty_income_*Identities against Table 1.4's split "net income" / "net loss" columns (feed-pinned workbooks):
What changes
fiscal_targets.pySOI_NET_INCOME_LESS_LOSS_VARIABLESholds three signed concepts:business_net_income_less_loss→total_self_employment_income(SE + SSTB SE);partnership_and_s_corp_net_income_less_loss→tax_unit_partnership_s_corp_income;rent_and_royalty_net_income_less_loss→rental_income.partnership_s_corp_incomefrom the nonpassive fields only ((E26190 − E26180) + (E25980 − E25960), not the line-32 total E26270), so the frame starts at $315B net against a $1,158B target and calibration triples those carriers, as it already did under the release binding (US frame partnership/S-corp income is nonpassive-only: the donor builds it from E25980-E25960 + E26190-E26180, not line 32 (E26270) #1239).soi_source_concept(its return line).UnregisteredSoiNetIncomeLessLossFamilyError, because Table 1.4 splits these lines into one-sided columns. Capital-gains rebase control is picked by feed order and crosses IRS concepts (HT2 N01000 vs Table 1.4 col 37) #1035 is the capital-gains case of one id naming two IRS measures.business_net_profits/_losses,partnership_and_s_corp_income/_losses,rent_and_royalty_net_income/_losses) stay reserved for Table 1.4's columns (chronicle#346). No pinned fact uses them. Two changed so a future leg binds correctly:total_self_employment_income;rent_and_royalty_net_incomeis now a positive part like every other income leg (it keeps farm rent, as Table 1.4's total does).tools/build_us_fiscal_refresh_release.py:_signed_componentkeeps the sign for the registered concepts, and_soi_component_rowcounts a nonzero total for them.variable_labels.py: labels for the three concepts.docs/us-soi-net-income-less-loss-concepts.md.experiments/1228-net-income-less-loss/.Effect on the pinned feed
consumer_facts_us_51aa40f, compiled at 2024 with aging, the packaged CD crosswalk and Medicaid substitution:national_state(6 ids × 51 states + US). The other 2,982 sit onfullonly: 2,928 district-file rows (each id's US, 51 state and 436 district rows) and 54 HT2 rows of the nine single-district states, whichfullclasses as district rows.national_statemovese1f9906bf0e0→38c1db8880da(6,102 specs);fullmovesf5bd921d6b21→9f6c65fe8734(33,250).source_measure_id, so it is unchanged.national_stateeach row is its own concept group and no value changes. Recomputed and checked equal.Effect on the Route A calibration (re-solve from the release's own checkpoint)
experiments/1228-net-income-less-loss/:IRS = Table 1.4 TY2022 aged ×1.12035 (the rows' own factor); counts are not aged.
Concept-exact state fit: 82–100% of each measure's state rows land within 10%; the release binding's state rows, read on these concepts, land 0–57%. No other target family's mean capped error rises by more than 0.0025.
The remaining gap is the frame, not the binding (#1238). The
puf_tax_detailstage listsself_employment_income_before_lsrundernonnegative_outputs, although the donor's E00900 is signed. So the frame carries 0.39M weighted Schedule C loss units before calibration, against IRS 8.39M.Once the frame carries losses and Chronicle carries the legs, concept-exact rows plus national Table 1.4 profit/loss targets can get both. Both are proposed as follow-ups.
Invariants
Leg conservation (Hypothesis, 300 examples), for every signed tax-unit vector and nonnegative weights:
For business these are the published identities N00900 = col 31 + col 33 and A00900 = col 32 − col 34, and for partnership/S-corp in TY2020's combined columns. For rental (Table 1.4's legs are lines 26 + 39 + 40) and TY2021+ partnership/S-corp (legs per entity type), the property checks the model's own decomposition of one column.
End to end:
test_net_income_less_loss_rows_materialize_signed_totals_and_nonzero_countsmaterializes all six concepts through the SOI slice on the fake engine. It covers a unit whose members net to a Schedule C loss, an SSTB-only unit and a farm-rent-only unit, hand-computed; it fails if business readsself_employment_income.Bounds and sign symmetry (same property): the less-loss count ≥ the net-income count, and the amount ≤ the net-income amount. Negating every value keeps the count and negates the amount.
Family-only binding (Hypothesis, 200 examples): for any id, tax year and record-set shape, a fact binds its line's signed concept exactly when its family is HT2 or the district file, and refuses otherwise.
Pinned feed (
test_pinned_feed_net_income_less_loss_rows_bind_signed_concepts):national_statecarries exactly 312;Determinism and parity: the release-concept rebuild (5.4e-8) and loss-weight reproduction (exact) in the experiment.
Tests
_soi_source_familyshapes (the district family iscongressional_district[_YYYY]only).test_pinned_feed_national_state_surface_restores_the_fences._reviews/.../out/mutations.txt):51aa40f;ruff checkandruff format --checkare clean on touched files.Review
The Codex review lanes were held by Subfleet's disk floor, so the independent review ran as an in-session Opus 5.5 adversarial reviewer (
_reviews/soi-net-income-less-loss-20261012/review-1-opus.md).Round 1 (head 9efe451): REQUEST CHANGES, on documentation and test coverage only. It confirmed:
It asked for five changes: row attribution, the partnership lineage caveat, a qualified leg identity, the changelog, and an end-to-end test. All five are addressed in ec3df20, along with its nits. The review's own row attribution (312 TY2023 HT2 rows) was itself off; the dump shows 54 single-district-state HT2 rows.
Round 2 (head ec3df20): APPROVE. All required changes and nits are resolved, and the review found no new defects.
Interactions
_soi_record_set_familywith the same semantics as_soi_source_familyhere; whichever merges second should dedupe.national_statein the same test; the second re-pins again.Not changed here (follow-ups)
nonnegative_outputs). Without it, no binding can match both Schedule C profits and net.axiom: n/a: calibration-target concept mapping (microsim data), no policy rule change
🤖 Generated with Claude Code