Skip to content

Value share-like capital by asset type for the sale-expense deduction - #2131

Merged
MaxGhenis merged 26 commits into
uc-capital-sale-expensesfrom
sale-expense-share-split
Nov 16, 2026
Merged

MaxGhenis merged 26 commits into
uc-capital-sale-expensesfrom
sale-expense-share-split

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

This PR values share-like capital by asset type in the means tests. It stacks on #1969 (head a418a0040), where PolicyEngine deducts 10% for expenses of sale. Through #1969 it includes main at 83c2554b2, and it includes #1974. This PR adds the asset split and its tests, updates the CI shard table and main's stored-totals test, and documents secured-debt uprating.

Reproducer: policyengine_uk/tests/test_isa_balances.py::test_person_isa_balances_count_in_full_in_every_means_test

What changes

Under #1969, PolicyEngine takes 10% off all of share_like_capital. DWP tells decision makers to value its parts differently in ADM chapter H1:

Holding Guidance 10%
Quoted shares ADM H1665; DMG 29671, 52671, 84763; HBGM W1.513 Yes
Unit trusts Bid price, with no costs of sale even through a broker: ADM H1673-H1674; DMG 29680-29681, 52680-52681, 84771-84772; HBGM W1.530-W1.531 No
ISAs Withdrawal value, and no ISA paragraph applies the 10%: ADM H1656-H1658; DMG 29662-29664, 52662, 84748-84750 No

This PR splits share_like_capital in two:

  • PolicyEngine calculates share_like_capital_with_sale_expenses by subtracting the holdings in gov.dwp.share_like_capital.without_sale_expenses from share_like_capital, with a floor at nil. The list contains unit and investment trusts and stocks and shares ISAs. For households that itemise nothing, PolicyEngine keeps all share-like capital in this part and retains Value means-tested capital less 10% sale expenses and secured debt #1969's flat 10%.
  • PolicyEngine puts the remainder in share_like_capital_without_sale_expenses, or nil when the household itemises no holding.

All six capital lists name both parts beside savings and cash ISAs: Universal Credit, Housing Benefit, Income Support, income-based JSA, income-related ESA and Pension Credit. Their sale-expense lists name the first part and exclude savings and cash ISAs.

Stocks and shares ISAs

On main, PolicyEngine sums cash_isa and stocks_and_shares_isa from each person's balance (#2712), and reads stocks_and_shares_isa as an itemised holding. It puts stocks and shares ISAs in the part without sale expenses, whether a dataset stores household totals or person balances. Each means test counts the ISA at its withdrawal value.

Under #1969, PolicyEngine takes 10% off the ISA, so this PR changes the expectation in test_isa_balances.py. The stored case supplies £1,000 of savings, a £3,000 cash ISA and a £2,000 stocks and shares ISA for 2025. UC capital in 2027 rises from (£1,000 + £3,000 + 90% × £2,000) × 1.07224 = £6,218.99 to (£1,000 + £3,000 + £2,000) × 1.07224 = £6,433.44. Per capita GDP growth of 4.0% in 2026 and 3.1% in 2027 compounds to 1.07224. A new test gives a single adult the same amounts and supplies their own ISA balances. It checks £6,000 of capital in all six means tests.

main now also counts each member's stocks and shares ISA balance in the holder's own benefit unit (stocks_and_shares_isa_countable_capital), and subtracts it from the household source before it allocates the pool. On main that source is share_like_capital. This PR names the two parts in every capital list in its place, so the merge maps the ISA person source to share_like_capital_without_sale_expenses (HOUSEHOLD_SOURCE_OF), the part that holds the ISA because gov.dwp.share_like_capital.without_sale_expenses lists stocks_and_shares_isa. No programme's sale-expense list names that part, so each means test counts the member's balance at face value in the holder's unit. Main's person ISA tests expected 90% of each balance; 65cebd3f6 and df8c1ea79 move them to face value (see Merge notes).

The mapping is fixed in code and doesn't follow the parameter. After a reform that removes stocks_and_shares_isa from gov.dwp.share_like_capital.without_sale_expenses, a single adult with £1,000 of savings and a £2,000 stocks and shares ISA balance gets UC capital of £2,800, the ISA at 90%, as the reform intends. We haven't tested how such a reform allocates a member's ISA between the benefit units of one household.

Main's test_stored_household_totals_grow_past_the_projection_horizon reaches this PR through #1969. Its household stores the same three amounts, and #1969 makes it expect UC capital of 5.8 × savings. This PR makes the test expect savings, the cash ISA and the stocks and shares ISA at face value, 6 × savings in every year from 2025 to 2040, and drops its check that capital stays below face value.

Regulation 77 company holdings

An owner-manager holds company shares that UC disregards under reg 77(2), and selling shares incurs expenses. This PR moves HOLDINGS_SOURCE from share_like_capital to share_like_capital_with_sale_expenses. PolicyEngine budgets company holdings against that source in the UC pool, so it never deducts them from unit trusts or ISAs. It removes the holding from the pool and any complete override at 90%, matching UC's valuation of shares. On data that don't itemise, PolicyEngine calculates the same value for both sources.

CI shard table

Main's shard_durations.toml names three share-component tests that this PR replaces, and test_ci_test_shards.py fails when a line names a test that doesn't exist. This PR points those lines at the successors and keeps their measured durations.

Growth factors page

main's growthfactors.md now lists the three secured-debt inputs this PR documented (owned_land_secured_debt, other_residential_property_secured_debt and non_residential_property_secured_debt) in its per capita GDP list, so the merge takes main's page and this PR no longer edits it.

Methodology

The Wealth and Assets Survey asks about unit and investment trusts in one question. Investment trusts trade as quoted shares; we infer that decision makers apply H1665's 10% deduction to directly held investment trusts. HMRC's ISA statistics, Table 9.6, put investment trusts at 8.2% of pooled funds in stocks and shares ISAs at 5 April 2022 and 6.9% at 5 April 2025. Those figures describe ISA holdings, and we use that composition as a proxy when treating the combined trust bucket outside ISAs as unit trusts. The survey does not separate the two trust types.

We give the whole trust bucket the treatment of unit trusts and count ISAs at their withdrawal value with no deduction. Under H1656, decision makers value an ISA at what its holder would get by withdrawing it; the paragraph does not mention the 10%. PolicyEngine retains the 10% for directly held shares and all of an unitemised corporate_wealth.

A parameter edit can put a whole holding such as stocks and shares ISAs back under the 10%: remove it from gov.dwp.share_like_capital.without_sale_expenses. Applying the 10% to part of a holding, such as the investment trusts in the bucket, would require a weight parameter and a formula change.

Impact

CI compared #1969's head 3a3241cf6 with 2f1051e9f on the certified Enhanced FRS (enhanced_frs_2024_25.h5, policyengine-uk-data 1.58.1). It found no change in any household's net income from fiscal year 2025-26 to 2031-32, or in receipts, the government balance, poverty or inequality.

policyengine-uk-data (main at e1acdf966) imputes one corporate_wealth. It sums WAS non-defined-benefit pension wealth, employee shares and options, UK shares, investment ISAs, and unit and investment trusts. It doesn't impute the three holdings separately, so PolicyEngine retains #1969's flat 10% for every household in that dataset.

Microcosm's UK build writes directly_held_shares, unit_and_investment_trusts and stocks_and_shares_isa so that they sum to corporate_wealth (was_wealth.py at c0513dbae). On data from that build, PolicyEngine excludes trusts and stocks and shares ISAs from the 10% deduction. CI's aggregate report doesn't cover those data.

Invariants

test_capital_share_components.py uses Hypothesis to check these properties on pounds-and-pence inputs that the engine stores as float32:

  1. share_like_capital keeps the supplied corporate_wealth unless the itemised holdings exceed it beyond float32 rounding, and then equals their sum.
  2. With nothing itemised, the part with sale expenses equals share_like_capital and the other part holds nil. Otherwise the first equals share_like_capital less trusts and ISAs, never below nil, and the two parts sum to share_like_capital.
  3. On unitemised inputs, every programme's assessable capital and UC eligibility equal those under the lists before itemising, exactly, for 2026, January 2026, 2027 and January 2027.
  4. For a consistent split, the property checks that UC capital rises by 10% of a working-age adult's trusts and ISAs. It allows the greater of £0.05 and a relative tolerance of 1e-5.
  5. Given only itemised holdings, with the ISAs as person balances, all six means tests count savings, cash ISAs, trusts and stocks and shares ISAs in full. They count directly held shares less 10%. The property checks each total after the programme's rounding rule, to within £0.011.

The capital property suite from #1969 (test_capital_valuation_properties.py) keeps its oracle. The oracle specifies which assets take the 10% from the law, independently of the parameters. It maps the part with sale expenses to the household's corporate_wealth and the part without to nil. It still raises a KeyError for any source it doesn't map.

test_stored_household_totals_grow_past_the_projection_horizon checks that a household's UC capital equals savings plus its stored cash and stocks and shares ISA totals, and keeps one ratio to savings, in every year from 2025 to 2040.

Fixed cases pin these results:

  • For £399,999.984375 of corporate_wealth beside £200,000 of trusts and a £199,999.984375 ISA, PolicyEngine leaves nil with sale expenses. It rounds both totals to £400,000 in float32 and sums the trusts and ISA before subtracting them;
  • For £16,000 of corporate_wealth beside £8,000.01 of trusts and an £8,000 ISA, PolicyEngine sums the holdings in float32 to £16,000.009765625. It leaves nil with sale expenses and £16,000 without;
  • With an empty exempt list, PolicyEngine deducts 10% from £12,000 of share-like capital whether the household itemises or not. UC counts £11,800 beside £1,000 of savings;
  • Income Support and income-based JSA count £1,000 of savings and £12,000 of corporate_wealth as £11,800 in 2012;
  • UC denies eligibility when a household holds a penny of shares beside £16,000 of savings. PolicyEngine preserves corporate_wealth when float32 rounding pushes the components' sum above £16,000 or £16,384.01;
  • PolicyEngine removes a £30,000 company holding from the UC pool at £27,000;
  • UC values £1,000 of savings, £4,000 of shares, £3,000 of trusts and a £5,000 ISA at £12,600;
  • UC and Pension Credit count itemised holdings in a monthly period without proration (£12,300, in test_stock_capital_periods.py).

Validation

CI runs only on pull requests into main, so it skips this PR while it targets #1969's branch, and we start it by hand. Run 43252893454 at df8c1ea79 passed the YAML job, lint, the documentation and release builds, the smoke imports and seven of the eight Python shards. Shard 3 failed one test (1 failed, 1,970 passed), which 3fe76ee3e fixes (see Merge notes). Run 43444525978 at 3fe76ee3e passed all 18 jobs: 13,482 Python tests passed (61 skipped and 2 xfailed, as allowlisted), and the YAML job, lint, the documentation and release builds and the smoke imports on Python 3.11 to 3.14 passed.

Merge notes

2a1725e50 merges #1969's head a418a0040, which merges main at 83c2554b2, into ce91e92ea. Git stopped on three files:

  • shard_durations.toml: the merge takes main's regenerated table and keeps this PR's three renamed share-component tests at the same durations.
  • growthfactors.md: the merge takes main's page (see Growth factors page).
  • uc_assessable_capital.py: the merge keeps this PR's sale_expenses.sources wording within main's prose, and the stocks and shares ISA at its withdrawal value.

The merge maps the ISA person source to share_like_capital_without_sale_expenses (see Stocks and shares ISAs); without that, counted_person_sources would have dropped it, because no capital list names share_like_capital any more. Two commits move main's person ISA tests to face value:

  • 65cebd3f6 (person_isa_capital.yaml): each counted ISA rises by a tenth. Cases 13 to 15 test a boundary, so their inputs move to keep it: a £30,600 ISA in Case 13, and household totals of £16,000 and £6,000 in Cases 14 and 15.
  • df8c1ea79 (test_person_isa_capital.py): the helpers value each part by whether the programme's sale_expenses.sources lists it, so the producer-shape households give UC capital of £10,900, £8,950, £14,000 and £11,000, and the list-reform test drops share_like_capital_without_sale_expenses, the source that holds the ISA.
  • 3fe76ee3e (test_capital_share_components.py): test_itemised_holdings_are_valued_by_type_in_every_means_test checked each total to within £0.011. The engine holds totals in float32. Its sum can round a penny under the exact total, which the test allows, and it then stores that figure to float32's step. CI at df8c1ea79 drew a household the test expects at £16,834.13: the engine gives £16,834.12, held as 16,834.119, which is £0.0109 under, and the test's own float32 subtraction read £0.0117. The check now subtracts in float64 and allows a penny plus one float32 step at the expected value. It pins that household and one expected at £37,747.34, which the engine holds as 37,747.328, £0.0119 under. With the £0.011 bound the second pinned household fails; with the new bound the test passes.

We ran these on the code at df8c1ea79, one process at a time with a 2 GB memory cap, HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset:

Command Result
policyengine-core test on 17 YAML files: capital_valuation.yaml, capital_valuation_share_split.yaml, person_isa_capital.yaml, isa_balances.yaml, isa_subscription_limits.yaml, the ESA, HB, IS and JSA capital files, uc_property_capital.yaml, uc_company_analogous_business.yaml, uc_unearned_income.yaml, is_UC_eligible.yaml, pension_credit_income.yaml, council_tax_reduction_pension_credit.yaml, private_pension_wealth.yaml and wealth_tax.yaml 227 passed (9 of main's person ISA cases failed before 65cebd3f6)
test_person_isa_capital.py, each test in its own process 32 passed
test_isa_balances.py 8 passed
test_stored_household_totals.py 42 passed
test_capital_valuation_properties.py 18 passed
code_health/test_ci_test_shards.py, the check that the shard table names existing tests 1 passed

The figures in the rest of this section come from ce91e92ea and earlier commits.

We ran these at ce91e92ea9d2d7c7025d932ba613438bf4a70a8d, after uv sync --frozen --extra dev, with HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset:

Command Result
uv run pytest -n 4 on test_stored_household_totals.py 42 passed
test_isa_balances.py 8 passed
test_capital_valuation_properties.py 18 passed
test_stock_capital_periods.py 3 passed
test_uc_company_owner_properties.py 7 passed
test_uc_company_override_properties.py 11 passed
test_uc_property_capital.py 65 passed
test_uc_property_capital_mock.py 26 passed
uv run pytest test_capital_share_components.py::test_unitemised_inputs_match_the_lists_before_itemising, on its own 1 passed (1,791 s)
uv run pytest -n 4 test_capital_share_components.py, deselecting that node 21 passed
uv run pytest -n 4, one file at a time, on the other test files that name share_like_capital, corporate_wealth, stocks_and_shares_isa, uc_assessable_capital or sale_expenses: test_uc_capital_allocation.py 1, test_benunit_reported_capital_properties.py 7, test_corporate_land_value.py 4, test_corporate_receipts_statistics.py 141, test_dataframe_dataset.py 90, test_isa_accounting.py 12, test_lifetime_isa_capital.py 3, test_mixed_age_couple_properties.py 1, test_pension_credit_reported_capital_properties.py 4, test_process_time_reforms.py 71, test_sdlt_counted_once_properties.py 1, test_uc_income_from_capital_properties.py 6, test_uc_legacy_closure_properties.py 4, test_uc_single_claim_properties.py 8, test_wealth_components.py 3, test_wealth_tax_net_base_properties.py 2, test_weighted_incidence.py 19 377 passed
uv run policyengine-core test -c policyengine_uk on capital_valuation.yaml 26, capital_valuation_share_split.yaml 10, cash_isa_capital.yaml 8, lifetime_isa_capital.yaml 24, isa_balances.yaml 2, and the *capital*.yaml files under ESA 13, HB 22, IS 13, JSA 12, UC 22, PC 9 and household/wealth 6 167 passed
uv run pytest -n 4 policyengine_uk/tests/code_health 3,113 passed
.github/check_fix_pr.py on this title and body, with the changes from b4e872273 to the head Fix evidence: PASS, "Changed executable tests: 6"
uvx ruff@0.17.2 format --check . / check . / check --select F811,F821 policyengine_uk 2,180 files already formatted / all checks passed / all checks passed
git diff --check 6a0d26d61..HEAD clean
git merge-tree --write-tree --name-only HEAD origin/main (52b30809f) clean
git merge-tree --write-tree --name-only HEAD af3758c64 (#2779) conflict in docs/book/assumptions/growthfactors.md only

We ran each mutant with Hypothesis shrinking off:

Mutant Commit Result
Stocks and shares ISAs back under the 10% 2f1051e9f killed: all 7 selected ISA-balance tests and all 3 selected share-component tests fail
HOLDINGS_SOURCE on all of share_like_capital 2f1051e9f killed: the regulation 77 test fails
Income-based JSA's capital list drops the part without sale expenses 2f1051e9f killed: the JSA list test, the valuation-by-type property (£7,600 against £15,600) and the JSA ISA-balance test (£4,000 against £6,000) fail
No floor at nil 6a0d26d61 killed: the floor case and its @example find −£0.009765625 with sale expenses
No branch for households that itemise nothing 6a0d26d61 killed: UC capital of £5,999.00 against £5,999.20 for £6,001 of savings and −£2 of corporate_wealth
stocks_and_shares_isa dropped from gov.dwp.share_like_capital.without_sale_expenses, run on the stored-totals node (a fixed case, no Hypothesis) ce91e92ea killed: assert np.float64(5800.0) == 6000.0 ± 0.06, 2025

Runs at earlier commits

Check Commit Result
CI Python tests, six shards (run 41246639348) 6a0d26d61 11,425 passed, 61 skipped, 2 xfailed
CI YAML tests, same run 6a0d26d61 5,451 passed
CI lint, documentation build, release build and smoke imports on Python 3.11 to 3.14, same run 6a0d26d61 passed
CI Python and YAML tests (run 41225440409) 2f1051e9f 11,424 passed, 61 skipped, 2 xfailed; 5,451 YAML cases passed
Aggregate impact against 3a3241cf6 (run 41225446226) 2f1051e9f no change in any year
Differential against 3a3241cf6 for 400 households that don't itemise: each programme's assessable capital and award, UC eligibility and the UC company-holding variables in 2025, 2026 and 2027, and UC capital, eligibility and award in January 2026 (58 outputs, 27,616 values) 2f1051e9f identical
test_isa_balances.py, test_wealth_components.py, test_isa_accounting.py, test_lifetime_isa_capital.py, test_uc_property_capital_mock.py, test_uc_company_override_properties.py, test_uc_company_owner_axiom_parity.py ce360946a 67 passed
code_health/test_ci_test_shards.py, test_stock_capital_periods.py 6a0d26d61 54 passed
YAML: capital_valuation_share_split.yaml, cash_isa_capital.yaml, isa_balances.yaml, lifetime_isa_capital.yaml 6a0d26d61 44 passed
ruff format --check, ruff check, --select F811,F821, git diff --check 6a0d26d61 clean

ce360946a differs from 2f1051e9f only in shard_durations.toml, and 6a0d26d61 adds the floor case to test_capital_share_components.py. CI at 6a0d26d61 passed every case in these files:

File Passed
test_capital_share_components.py 22
test_capital_valuation_properties.py 18
test_uc_property_capital.py 65
test_uc_company_owner_properties.py 7
test_uc_tax_free_childcare_choice.py 20
test_input_records.py 11
test_fiscal_partition.py 43

CI also passed the 70 cases in capital_valuation.yaml, lifetime_isa_capital.yaml, capital_valuation_share_split.yaml, cash_isa_capital.yaml and isa_balances.yaml at 6a0d26d61.

axiom: TheAxiomFoundation/rulespec-uk#395 queued

We added the asset-type distinction and hand-computed companion cases to the issue (TheAxiomFoundation/rulespec-uk#395 (comment)).

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 4, 2026 06:42
…eduction

corporate_wealth is now itemised in the capital sources lists (#1974), so the
sale-expense lists name directly_held_shares, which DWP values less 10% for
costs of sale (ADM H1665; DMG 29671/52671/84763; HBGM W1.513), and
unitemised_corporate_wealth, which keeps the previous flat treatment on
datasets that do not itemise. Unit trusts have no costs of sale even when sold
through a broker (ADM H1673-H1674; DMG 29680-29681, 52680-52681,
84771-84772; HBGM W1.530-W1.531), and an ISA is valued at what the holder
would get by withdrawing it, with no paragraph applying the 10% (ADM
H1656-H1658; DMG 29662-29664, 52662, 84748-84750). The survey bucket of
unit and investment trusts is treated as having no costs of sale; investment
trusts were 8.2% and 6.9% of the pooled funds in stocks and shares ISAs at
5 April 2022 and 2025 (HMRC ISA Table 9.6).

The property tests keep their own legal asset set and assert each
programme's list against it, and gain the split invariants: the residual,
backward compatibility with every component nil, conservation at a zero rate,
split effect = rate x (trusts + ISA), and no double counting. #1974's
itemising-invariance test no longer holds by design and is replaced by them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Max. Reviewed at head b1a66004.

The "+10.5k lines" is the main merge carried in through #1974. The real diff over #1969 + #1974 (merged at their heads) is 11 files, +710/−125:

  • the six sale_expenses.sources lists;
  • the new capital_valuation_share_split.yaml;
  • the property tests;
  • test_stock_capital_periods.py;
  • a changelog fragment.

That's what I reviewed.

Guidance:

  • ADM H1674 is explicit: unit trusts have "no costs of sale. This applies even if persons use an agent, such as a stockbroker."
  • H1665 applies the 10% to quoted shares.
  • H1656 values an ISA at "what people would get if they withdrew their investment".

The lists now apply the 10% to directly_held_shares and the unitemised residual only, which is right for shares and unit trusts. SPC Sch V and UC Sch 10 have no disregard that changes this.

Probes:

  • savings £5,000, shares £4,000, trusts £3,000, ISA £5,000 (corporate_wealth £12,000) gives £16,600 for UC, HB and IS. The same holdings with only the components entered also give £16,600.
  • An unsplit corporate_wealth of £12,000 still gives #1969's £15,800.
  • Your monthly case (£12,300) matches.

Tests:

  • At the head: the capital-valuation, share-component, wealth-component, stock-period and PC reported-capital property tests, 26 passed. DWP, family and household YAML, 1,064 passed.
  • Merged into current main (the merge is clean): YAML, 1,389 passed. Properties: 19 passed and 2 failed. The two failures are the same stale applies() oracle as on #1969 (here at tests/test_capital_valuation_properties.py:240), not the new share-split properties. They'll clear when #1969's fix is carried through.
  • There's no CI on this PR, because its base isn't main, so these local runs are the only test evidence so far.

Findings:

  1. Should fix (merge order). Retarget to main as soon as #1969 merges, and get CI green on the retargeted head before merging. Until then, nothing hosted has run on these lists. This PR must land in the same window as #1969 and #1974 (see my notes there): with those two in and this one out, the 10% on shares disappears.

  2. Should fix. parameters/gov/dwp/universal_credit/means_test/capital/sale_expenses/sources.yaml:20-22, and the same sentence in the other five lists. The ISA exemption is weaker than the unit-trust one:

    • H1674 says outright that unit trusts have no costs of sale. For ISAs, the guidance is silent and you're inferring from H1656 ("withdrawal value").
    • A stocks-and-shares ISA holding quoted shares has to sell them through the ISA manager. That is a broker sale under H1605, and reg 49(1)(a) applies wherever "there would be expenses attributable to sale".

    I'd keep ISAs exempt, since most ISA money is in funds, but describe it as a judgement, like the investment-trust one, rather than "no ISA paragraph applies the 10%". Give the directly-held-shares share of stocks-and-shares ISAs from HMRC Table 9.6 alongside it.

  3. Nit. variables/input/directly_held_shares.py:9. Employee share options now take the 10% as "shares". An unexercised option isn't sold through a broker and is often not realisable at all. It's fine as an approximation, but say so in the list descriptions next to the investment-trust caveat.

  4. Nit. tests/test_capital_share_components.py. You removed the itemising-invariance test on purpose. A one-line comment pointing to the replacement differential in test_capital_valuation_properties.py would stop someone restoring it.

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Independent review (GPT-6.1 Sol, UK hub sweep, 2026-10-12): REQUEST_CHANGES at b1a660045. The full review is below. Blockers 1 and 2 are methodology calls the ADM leaves open; they are queued for Max as d1396. Findings 3–6 are for whoever picks this up after uk-data#501 lands (the triage deferred this PR until after 28 Oct).

VERDICT: REQUEST_CHANGES
Head reviewed: b1a6600

  1. [blocker] Investment-trust treatment requires Max’s methodology decision. UC sale_expenses/sources.yaml:14 exempts the combined unit-and-investment-trust bucket across all six programmes. ADM H1665/H1674 distinguishes quoted shares, which receive 10%, from unit trusts, which have no sale costs. Fix: obtain Max’s explicit decision; separate investment trusts or implement an approved proxy.

  2. [blocker] The ISA exclusion is an unresolved interpretation. UC sale_expenses/sources.yaml:20 infers exemption from the ISA paragraphs’ silence about 10%. ADM H1656 specifies withdrawal value without expressly overriding the general sale-cost rule. The input describes only “amount held”:7. Fix: have Max settle whether the input represents net withdrawal proceeds and how sale costs apply, then align all six lists and tests.

  3. [should-fix] Reconcile main’s reported-capital changes. Against local origin/main 3f174d0, merge-tree reports conflicts in growth-factor documentation, uprating indices, and uc_assessable_capital.py:10. Main Record benefit-unit capital once; derive UC and Pension Credit reported capital from it #2201 introduced shared reported-capital handling. Fix: update the stack while preserving its reported-capital formulas, documentation, and uprating entries.

  4. [should-fix] Clean merges still change newer regression expectations. pension_credit_assessable_capital.py:33 values unitemised £20,000 corporate wealth at £18,000. Main’s new CTR regression:660 expects £20,000. Fix: recalculate its capital, deemed-income and savings-credit expectations, and test the newer HB/CTR routes after reconciliation.

  5. [should-fix] Impact evidence is missing. The PR’s Impact section defers measurement to the data PR. test_capital_valuation_properties.py:336 establishes compatibility when every component is zero; it does not establish enhanced-FRS impact against main. Fix: provide an actual Microsimulation comparison with dataset/version, simulation year, both SHAs, commands and traceable outputs, including any measured zero result.

  6. [should-fix] The stacked diff includes protected and unrelated changes. Its actual base-to-head diff spans 164 files, including simulation.py:235, tests/conftest.py:19, and unrelated tax, earnings and housing changes inherited from main. Fix: refresh the prerequisites/base so the final review diff isolates the exemption changes.

The stated mechanisms and invariants match the code under the chosen methodology. Property tests cover the load-bearing logic; replacing the old itemising-invariance test is justified. The exemption commit adds no skips or xfails. Ruff formatting and lint passed for its three changed Python files.

Shell testing was attempted below the 30 GB compressor threshold. pytest -n 0 lacked xdist; three property files failed collection because Hypothesis was missing. The serial subset collected six tests but was interrupted before any ran. No independent test passes are claimed.

GB regulations and programme manuals were reviewed; several NI references and the HMRC percentages remain unverified. Requested gh commands failed network access, so web and local Git evidence were used. No files were changed.

@MaxGhenis MaxGhenis added the ci-recheck Re-triggers Fix evidence without changing the PR (UK hub, 2026-10-12) label Oct 12, 2026
MaxGhenis and others added 3 commits October 12, 2026 10:59
…ale-expense-share-split

# Conflicts:
#	docs/book/assumptions/growthfactors.md
#	policyengine_uk/data/uprating_indices.yaml
#	policyengine_uk/tests/test_capital_valuation_properties.py
…le-expense-share-split

# Conflicts:
#	docs/book/assumptions/growthfactors.md
#	policyengine_uk/data/uprating_indices.yaml
#	policyengine_uk/tests/test_capital_share_components.py
#1974 replaced the components-plus-residual design with one share_like_capital
block that keeps the supplied corporate_wealth exactly. This follow-up now
splits that block in two:

- share_like_capital_with_sale_expenses: share_like_capital less the itemised
  holdings in gov.dwp.share_like_capital.without_sale_expenses (unit and
  investment trusts, stocks and shares ISAs), floored at nil; all of it when
  nothing is itemised.
- share_like_capital_without_sale_expenses: the rest, exactly nil when nothing
  is itemised.

The six capital lists name both parts and the six sale-expense lists the
first, so datasets that carry only corporate_wealth keep #1969's flat 10%
exactly (differential test, annual and monthly, against the lists before
itemising, including negative aggregates and a value whose 90% is a fraction
of a penny over the limit). The exempt holdings are a parameter list, so the
methodology call on investment trusts and ISAs (d1396) is a one-line change.

Merged the refreshed #1969 (54ed063) and #1974 (7784193). #1969's
property oracle reads the share-like sources from an unitemised household's
corporate_wealth; the split invariants (definition of both parts, the
unitemised differential, split effect = 10% of trusts plus ISAs on UC
capital, components-only valuation) are in test_capital_share_components.
Dropping the nothing-itemised branch from the formula fails the differential.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis changed the title Exempt unit trusts and stocks and shares ISAs from the sale-expense deduction Value share-like capital by asset type for the sale-expense deduction Oct 12, 2026
MaxGhenis and others added 5 commits October 13, 2026 08:50
…uping

Review of b7e6bc5:
- gov.dwp.share_like_capital.without_sale_expenses started in 2013, after
  the legacy capital lists (JSA from 1996-10-07), so a 2012 simulation with
  unitemised corporate_wealth raised ParameterNotFoundError; it now starts at
  1996-10-07, and Income Support and JSA capital in 2012 are tested (both
  fail with the 2013 start).
- An empty exempt list made add() return None; the exempt holdings are now
  summed from a numeric zero, and an empty-list reform (every share-like
  holding less 10%) is tested.
- The property oracle now follows the engine's float32 grouping (components
  left to right, exempt holdings summed before subtraction) and compares
  exactly; the reviewer's example (trusts £200,000, ISA £199,999.984375) is
  pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le-expense-share-split

# Conflicts:
#	policyengine_uk/tests/test_capital_share_components.py
…le-expense-share-split

# Conflicts:
#	policyengine_uk/tests/test_capital_share_components.py

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs changes before merge. The split itself is right and well tested. But this branch carries both CI failures from #1969 and #1974, it inherits the #1965 holding problem in both forms, and nothing hosted has run on it.

@MaxGhenis, this is a delta review. My last review was at b1a66004; this one is at 6a5a13a4. Since then the split has been rebuilt on #1974's block: share_like_capital_with_sale_expenses and _without_sale_expenses, with an exempt-holdings parameter. I reviewed the diff against #1969 + #1974 merged at their current heads: 21 files, +678/−186.

Status of my earlier findings:

  • F1 (retarget to main, CI): still open. See finding 2.
  • F2 (ISA exemption described as a judgement): partly addressed. The body now has a methodology note, but the parameter text still states it as settled. See finding 3.
  • F3 (employee share options caveat): not taken. See finding 5.
  • F4 (invariance test comment): addressed. The module docstring now points to the differential.

Findings:

  1. must (carried in from #1969 and #1974). This head contains both CI failures and the #1965 interaction:

    • test_projection_horizon.py:181 fails, as on #1974. The ISA variables are now mapped in uprating_indices.yaml.
    • test_savings_credit_only_receipt_properties.py:508 fails, as on #1969. It expects 601.02 where reg 19 gives 621.82.
    • Company holdings, probed on this head merged into current main. Take a single UC claimant who is sole owner of a trading company, with a £10,000 holding and £5,000 savings:
      • with corporate_wealth £10,000, UC capital is £4,000 (expected £5,000);
      • with only directly_held_shares £10,000, UC capital is £14,000 (expected £5,000), because the pool cap still reads corporate_wealth.
    • On this head merged into main, the same 8 uc_company_analogous_business.yaml cases and 3 test_uc_company_owner_properties.py properties fail as on #1969. Five of those YAML cases are the holding problem; three are the plain 90% revaluation and only need new expectations (see my #1969 review).
    • Here the holding should come off share_like_capital_with_sale_expenses at 90%, since an owner's shares are directly held shares.

    Once the fixes land on #1969 and #1974, merge them through and re-run.

  2. must. No CI has run on this PR, because its base is uc-capital-sale-expenses. The body's "Local runs" line is still "pending". Retarget to main and get the full suite green before anything in the stack merges. If this PR lands alone, it carries #1969 and #1974 with no window where the 10% on shares disappears. On a #1969 + #1974 tree without this PR, £5,000 savings + corporate_wealth £12,000 gives £17,000 for UC; this head gives £15,800.

  3. should. parameters/gov/dwp/share_like_capital/without_sale_expenses.yaml:10-12 still says "no ISA paragraph applies the deduction", as if that settled it. The changelog fragment states the ISA exemption flatly too.

    • ADM H1656 and DMG 29662 value an ISA at "what people would get if they withdrew their investment" and say nothing about costs of sale.
    • The body parks this, and the investment-trust question, for a decision (d1396).
    • Until it's decided, the parameter text should call the exemption a judgement, as it already does for investment trusts. I'm not suggesting which way to decide.
  4. nit. The six capital/sources.yaml descriptions each have an unwrapped line of about 175 characters. An example is universal_credit/means_test/capital/sources.yaml:8 ("share_like_capital_with_sale_expenses and share_like_capital_without_sale_expenses split ..."). Rewrap to match the rest.

  5. nit. share_like_capital_with_sale_expenses.py:3 hardcodes COMPONENTS for the "is anything itemised" gate, while the exempt list is a parameter. A reform adding a new exempt variable outside COMPONENTS would be ignored for a household that itemises only that variable. Consider a comment saying the exempt list must be a subset of the components, or a test asserting it. Separately, employee share options still take the 10% as "directly held shares" with no caveat (my F3). One clause in without_sale_expenses.yaml would cover it.

What I checked:

  • Guidance, re-read from the current PDFs:

    • ADM H1656-H1658 (ISA at withdrawal value; nothing on costs of sale);
    • ADM H1673-H1674 and DMG 29680-29681 (unit trusts: "no costs of sale ... even if persons use an agent");
    • DMG 29662 and 29664 (ISA), and DMG 29671 (10% on shares).

    The paragraph numbers in the parameters match. I didn't check the NI or HBGM numbers, or the HMRC Table 9.6 percentages.

  • Probes on this head:

    • unsplit £5,000 + corporate_wealth £12,000 gives £15,800 (UC, IS; PC at 70), as on #1969;
    • split shares £4,000, trusts £3,000, ISA £5,000 gives £16,600;
    • components only with £1,000 savings gives £1,000 + 90% × £4,000 + £3,000 + £5,000 = £12,600, matching the body.
  • Revert check. capital_valuation_share_split.yaml run on #1969 + #1974 without this PR's source changes: 10 failed. At this head: 10 passed.

  • Tests. I ran the same property files and YAML directories as on #1969 and #1974, plus #1965's company-owner tests.

    • At this head: 2 failed (the two carried CI failures) and 2,360 passed; YAML 1,889 passed.
    • Merged into current main (clean): 5 failed (those two plus the three #1965 properties) and 2,405 passed; YAML 8 failed (company cases) and 1,980 passed. Current main 714b1438c on the same set: all 2,363 pass, and YAML 1,964 pass.
  • Lint. ruff check and ruff format --check on the 11 changed Python files: clean.

  • Repo process. The .fixed fragment and the Reproducer and axiom lines are present, and the named reproducer case exists. The branch also carries #1974's .added fragment, which is fine if it lands as one unit.

  • Overlap with my open PRs. #2382 merges cleanly with it.

  • Not checked. Any dataset run, or uk-data #501.

🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

UK hub takeover for the Budget fix sprint (Max, 21 Oct). The owner session local_b8fb4700 has been idle since mid-October. The hub woke it on 21 Oct at 23:58Z, and it did not start. To land this before the Budget model locks (the first pe-uk release cut by about 12:00Z Thu 22 Oct), the hub has taken it over as build 20261021-211254-sprint-1974-2131, which merges main, re-tests, pushes as a fast-forward and marks it ready. The hub then reviews it and lands it through the merge queue.

If the owner session wakes, please don't push to this branch; comment here instead, so there's only one writer.

🤖 Generated with Claude Code

When trusts and an ISA exceed corporate_wealth only by float32 rounding,
share_like_capital keeps corporate_wealth and subtracting the exempt
holdings goes below nil. £16,000 of corporate_wealth with £8,000.01 of
trusts and an £8,000 ISA leaves -£0.009765625 without the floor. No fixed
case reached the floor before: the grouping case lands at exactly nil
either way. A fixed test and a matching @example on the property pin it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

We've moved this back to draft. This branch contains #1969's head, so it fails main's stored-totals test the same way (details on #1969). Once #1969's fix reaches its branch, we'll restack this one on it and resolve its conflict with #2779 in docs/book/assumptions/growthfactors.md.

@MaxGhenis
MaxGhenis marked this pull request as draft November 2, 2026 20:26
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Picking this up for the UK hub. Reply here if you're mid-change and I'll stop.


Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

PR_TITLE.txt

Value share-like capital by asset type for the sale-expense deduction

Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

PR_BODY.md

This PR values share-like capital by asset type in the means tests. It stacks on #1969 (head `b4e872273`), where PolicyEngine deducts 10% for expenses of sale. Through #1969 it includes main at `52b30809f`, and it includes #1974. This PR adds the asset split and its tests, updates the CI shard table and main's stored-totals test, and documents secured-debt uprating.

Reproducer: `policyengine_uk/tests/test_isa_balances.py::test_person_isa_balances_count_in_full_in_every_means_test`

## What changes

Under #1969, PolicyEngine takes 10% off all of `share_like_capital`. DWP tells decision makers to value its parts differently in [ADM chapter H1](https://assets.publishing.service.gov.uk/media/6a5e0ac7a2662d4563fe39e0/adm-ch-h1.pdf):

| Holding | Guidance | 10% |
|---|---|---|
| Quoted shares | ADM H1665; DMG 29671, 52671, 84763; HBGM W1.513 | Yes |
| Unit trusts | Bid price, with no costs of sale even through a broker: ADM H1673-H1674; DMG 29680-29681, 52680-52681, 84771-84772; HBGM W1.530-W1.531 | No |
| ISAs | Withdrawal value, and no ISA paragraph applies the 10%: ADM H1656-H1658; DMG 29662-29664, 52662, 84748-84750 | No |

This PR splits `share_like_capital` in two:

- PolicyEngine calculates `share_like_capital_with_sale_expenses` by subtracting the holdings in `gov.dwp.share_like_capital.without_sale_expenses` from `share_like_capital`, with a floor at nil. The list contains unit and investment trusts and stocks and shares ISAs. For households that itemise nothing, PolicyEngine keeps all share-like capital in this part and retains #1969's flat 10%.
- PolicyEngine puts the remainder in `share_like_capital_without_sale_expenses`, or nil when the household itemises no holding.

All six capital lists name both parts beside savings and cash ISAs: Universal Credit, Housing Benefit, Income Support, income-based JSA, income-related ESA and Pension Credit. Their sale-expense lists name the first part and exclude savings and cash ISAs.

### Stocks and shares ISAs

On main, PolicyEngine sums `cash_isa` and `stocks_and_shares_isa` from each person's balance (#2712), and reads `stocks_and_shares_isa` as an itemised holding. It puts stocks and shares ISAs in the part without sale expenses, whether a dataset stores household totals or person balances. Each means test counts the ISA at its withdrawal value.

Under #1969, PolicyEngine takes 10% off the ISA, so this PR changes the expectation in `test_isa_balances.py`. The stored case supplies £1,000 of savings, a £3,000 cash ISA and a £2,000 stocks and shares ISA for 2025. UC capital in 2027 rises from (£1,000 + £3,000 + 90% × £2,000) × 1.07224 = £6,218.99 to (£1,000 + £3,000 + £2,000) × 1.07224 = £6,433.44. Per capita GDP growth of 4.0% in 2026 and 3.1% in 2027 compounds to 1.07224. A new test gives a single adult the same amounts and supplies their own ISA balances. It checks £6,000 of capital in all six means tests.

Main's `test_stored_household_totals_grow_past_the_projection_horizon` reaches this PR through #1969. Its household stores the same three amounts, and #1969 expects UC capital of 5.8 × savings. This PR makes the test expect savings, the cash ISA and the stocks and shares ISA at face value, 6 × savings in every year from 2025 to 2040, and drops its check that capital stays below face value.

### Regulation 77 company holdings

An owner-manager holds company shares that UC disregards under reg 77(2), and selling shares incurs expenses. This PR moves `HOLDINGS_SOURCE` from `share_like_capital` to `share_like_capital_with_sale_expenses`. PolicyEngine budgets company holdings against that source in the UC pool, so it never deducts them from unit trusts or ISAs. It removes the holding from the pool and any complete override at 90%, matching UC's valuation of shares. On data that don't itemise, PolicyEngine calculates the same value for both sources.

### CI shard table

Main's `shard_durations.toml` names three share-component tests that this PR replaces, and `test_ci_test_shards.py` fails when a line names a test that doesn't exist. This PR points those lines at the successors and keeps their measured durations.

### Growth factors page

This PR adds three secured-debt inputs to the per capita GDP list on `growthfactors.md`: `owned_land_secured_debt`, `other_residential_property_secured_debt` and `non_residential_property_secured_debt`. PolicyEngine already uprates them by per capita GDP through `uprating_indices.yaml`.

#2779 rewrites this page, and Git can't merge the two edits to it. #2779's per capita GDP list already names `non_residential_property_secured_debt` and `other_residential_property_secured_debt`. Once #2779 lands on main, this PR's edit shrinks to adding `owned_land_secured_debt` after `owned_land`, and the page names no count that changes.

## Methodology

The Wealth and Assets Survey asks about unit and investment trusts in one question. Investment trusts trade as quoted shares; we infer that decision makers apply H1665's 10% deduction to directly held investment trusts. [HMRC's ISA statistics, Table 9.6](https://www.gov.uk/government/statistics/annual-savings-statistics-2026), put investment trusts at 8.2% of pooled funds in stocks and shares ISAs at 5 April 2022 and 6.9% at 5 April 2025. Those figures describe ISA holdings, and we use that composition as a proxy when treating the combined trust bucket outside ISAs as unit trusts. The survey does not separate the two trust types.

We give the whole trust bucket the treatment of unit trusts and count ISAs at their withdrawal value with no deduction. Under H1656, decision makers value an ISA at what its holder would get by withdrawing it; the paragraph does not mention the 10%. PolicyEngine retains the 10% for directly held shares and all of an unitemised `corporate_wealth`.

A parameter edit can put a whole holding such as stocks and shares ISAs back under the 10%: remove it from `gov.dwp.share_like_capital.without_sale_expenses`. Applying the 10% to part of a holding, such as the investment trusts in the bucket, would require a weight parameter and a formula change.

## Impact

CI compared #1969's head `3a3241cf6` with `2f1051e9f` on the certified Enhanced FRS (`enhanced_frs_2024_25.h5`, policyengine-uk-data 1.58.1). It found no change in any household's net income from fiscal year 2025-26 to 2031-32, or in receipts, the government balance, poverty or inequality.

policyengine-uk-data (`main` at `e1acdf966`) imputes one `corporate_wealth`. It sums WAS non-defined-benefit pension wealth, employee shares and options, UK shares, investment ISAs, and unit and investment trusts. It doesn't impute the three holdings separately, so PolicyEngine retains #1969's flat 10% for every household in that dataset.

Microcosm's UK build writes `directly_held_shares`, `unit_and_investment_trusts` and `stocks_and_shares_isa` so that they sum to `corporate_wealth` (`was_wealth.py` at `c0513dbae`). On data from that build, PolicyEngine excludes trusts and stocks and shares ISAs from the 10% deduction. CI's aggregate report doesn't cover those data.

## Invariants

`test_capital_share_components.py` uses Hypothesis to check these properties on pounds-and-pence inputs that the engine stores as float32:

1. `share_like_capital` keeps the supplied `corporate_wealth` unless the itemised holdings exceed it beyond float32 rounding, and then equals their sum.
2. With nothing itemised, the part with sale expenses equals `share_like_capital` and the other part holds nil. Otherwise the first equals `share_like_capital` less trusts and ISAs, never below nil, and the two parts sum to `share_like_capital`.
3. On unitemised inputs, every programme's assessable capital and UC eligibility equal those under the lists before itemising, exactly, for 2026, January 2026, 2027 and January 2027.
4. For a consistent split, the property checks that UC capital rises by 10% of a working-age adult's trusts and ISAs. It allows the greater of £0.05 and a relative tolerance of `1e-5`.
5. Given only itemised holdings, with the ISAs as person balances, all six means tests count savings, cash ISAs, trusts and stocks and shares ISAs in full. They count directly held shares less 10%. The property checks each total after the programme's rounding rule, to within £0.011.

The capital property suite from #1969 (`test_capital_valuation_properties.py`) keeps its oracle. The oracle specifies which assets take the 10% from the law, independently of the parameters. It maps the part with sale expenses to the household's `corporate_wealth` and the part without to nil. It still raises a `KeyError` for any source it doesn't map.

`test_stored_household_totals_grow_past_the_projection_horizon` checks that a household's UC capital equals savings plus its stored cash and stocks and shares ISA totals, and keeps one ratio to savings, in every year from 2025 to 2040.

Fixed cases pin these results:

- For £399,999.984375 of `corporate_wealth` beside £200,000 of trusts and a £199,999.984375 ISA, PolicyEngine leaves nil with sale expenses. It rounds both totals to £400,000 in float32 and sums the trusts and ISA before subtracting them;
- For £16,000 of `corporate_wealth` beside £8,000.01 of trusts and an £8,000 ISA, PolicyEngine sums the holdings in float32 to £16,000.009765625. It leaves nil with sale expenses and £16,000 without;
- With an empty exempt list, PolicyEngine deducts 10% from £12,000 of share-like capital whether the household itemises or not. UC counts £11,800 beside £1,000 of savings;
- Income Support and income-based JSA count £1,000 of savings and £12,000 of `corporate_wealth` as £11,800 in 2012;
- UC denies eligibility when a household holds a penny of shares beside £16,000 of savings. PolicyEngine preserves `corporate_wealth` when float32 rounding pushes the components' sum above £16,000 or £16,384.01;
- PolicyEngine removes a £30,000 company holding from the UC pool at £27,000;
- UC values £1,000 of savings, £4,000 of shares, £3,000 of trusts and a £5,000 ISA at £12,600;
- UC and Pension Credit count itemised holdings in a monthly period without proration (£12,300, in `test_stock_capital_periods.py`).

## Validation

No CI runs on this PR while its base is #1969's branch.

Local runs at `ce91e92ea9d2d7c7025d932ba613438bf4a70a8d`, with `uv sync --frozen --extra dev` and `HUGGING_FACE_TOKEN` and `POLICYENGINE_UK_DEFAULT_DATASET` unset:

| Command | Result |
|---|---|
| `uv run pytest -n 4` on `test_stored_household_totals.py` | 42 passed |
| `test_isa_balances.py` | 8 passed |
| `test_capital_valuation_properties.py` | 18 passed |
| `test_stock_capital_periods.py` | 3 passed |
| `test_uc_company_owner_properties.py` | 7 passed |
| `test_uc_company_override_properties.py` | 11 passed |
| `test_uc_property_capital.py` | 65 passed |
| `test_uc_property_capital_mock.py` | 26 passed |
| `uv run pytest test_capital_share_components.py::test_unitemised_inputs_match_the_lists_before_itemising`, on its own | 1 passed (1,791 s) |
| `uv run pytest -n 4 test_capital_share_components.py`, deselecting that node | 21 passed |
| `uv run pytest -n 4`, one file at a time, on the other test files that name `share_like_capital`, `corporate_wealth`, `stocks_and_shares_isa`, `uc_assessable_capital` or `sale_expenses`: `test_uc_capital_allocation.py` 1, `test_benunit_reported_capital_properties.py` 7, `test_corporate_land_value.py` 4, `test_corporate_receipts_statistics.py` 141, `test_dataframe_dataset.py` 90, `test_isa_accounting.py` 12, `test_lifetime_isa_capital.py` 3, `test_mixed_age_couple_properties.py` 1, `test_pension_credit_reported_capital_properties.py` 4, `test_process_time_reforms.py` 71, `test_sdlt_counted_once_properties.py` 1, `test_uc_income_from_capital_properties.py` 6, `test_uc_legacy_closure_properties.py` 4, `test_uc_single_claim_properties.py` 8, `test_wealth_components.py` 3, `test_wealth_tax_net_base_properties.py` 2, `test_weighted_incidence.py` 19 | 377 passed |
| `uv run policyengine-core test -c policyengine_uk` on `capital_valuation.yaml` 26, `capital_valuation_share_split.yaml` 10, `cash_isa_capital.yaml` 8, `lifetime_isa_capital.yaml` 24, `isa_balances.yaml` 2, and the `*capital*.yaml` files under ESA 13, HB 22, IS 13, JSA 12, UC 22, PC 9 and `household/wealth` 6 | 167 passed |
| `uv run pytest -n 4 policyengine_uk/tests/code_health` | 3,113 passed |
| `.github/check_fix_pr.py` on this title and body, with the changes from `b4e872273` to the head | `Fix evidence: PASS`, "Changed executable tests: 6" |
| `uvx ruff@0.17.2 format --check .` / `check .` / `check --select F811,F821 policyengine_uk` | 2,180 files already formatted / all checks passed / all checks passed |
| `git diff --check 6a0d26d61..HEAD` | clean |
| `git merge-tree --write-tree --name-only HEAD origin/main` (`52b30809f`) | clean |
| `git merge-tree --write-tree --name-only HEAD af3758c64` (#2779) | conflict in `docs/book/assumptions/growthfactors.md` only |

We ran each mutant with Hypothesis shrinking off, except where the table says otherwise:

| Mutant | Commit | Result |
|---|---|---|
| Stocks and shares ISAs back under the 10% | `2f1051e9f` | killed: all 7 selected ISA-balance tests and all 3 selected share-component tests fail |
| `HOLDINGS_SOURCE` on all of `share_like_capital` | `2f1051e9f` | killed: the regulation 77 test fails |
| Income-based JSA's capital list drops the part without sale expenses | `2f1051e9f` | killed: the JSA list test, the valuation-by-type property (£7,600 against £15,600) and the JSA ISA-balance test (£4,000 against £6,000) fail |
| No floor at nil | `6a0d26d61` | killed: the floor case and its `@example` find −£0.009765625 with sale expenses |
| No branch for households that itemise nothing | `6a0d26d61` | killed: UC capital of £5,999.00 against £5,999.20 for £6,001 of savings and −£2 of `corporate_wealth` |
| `stocks_and_shares_isa` dropped from `gov.dwp.share_like_capital.without_sale_expenses`, run on the stored-totals node (a fixed case, no Hypothesis) | `ce91e92ea` | killed: `assert np.float64(5800.0) == 6000.0 ± 0.06`, 2025 |

Earlier runs, each labelled with its commit:

| Check | Commit | Result |
|---|---|---|
| CI Python tests, six shards ([run 41246639348](https://github.com/PolicyEngine/policyengine-uk/actions/runs/41246639348)) | `6a0d26d61` | 11,425 passed, 61 skipped, 2 xfailed |
| CI YAML tests, same run | `6a0d26d61` | 5,451 passed |
| CI lint, documentation build, release build and smoke imports on Python 3.11 to 3.14, same run | `6a0d26d61` | passed |
| CI Python and YAML tests ([run 41225440409](https://github.com/PolicyEngine/policyengine-uk/actions/runs/41225440409)) | `2f1051e9f` | 11,424 passed, 61 skipped, 2 xfailed; 5,451 YAML cases passed |
| Aggregate impact against `3a3241cf6` ([run 41225446226](https://github.com/PolicyEngine/policyengine-uk/actions/runs/41225446226)) | `2f1051e9f` | no change in any year |
| Differential against `3a3241cf6` for 400 households that don't itemise: each programme's assessable capital and award, UC eligibility and the UC company-holding variables in 2025, 2026 and 2027, and UC capital, eligibility and award in January 2026 (58 outputs, 27,616 values) | `2f1051e9f` | identical |
| `test_isa_balances.py`, `test_wealth_components.py`, `test_isa_accounting.py`, `test_lifetime_isa_capital.py`, `test_uc_property_capital_mock.py`, `test_uc_company_override_properties.py`, `test_uc_company_owner_axiom_parity.py` | `ce360946a` | 67 passed |
| `code_health/test_ci_test_shards.py`, `test_stock_capital_periods.py` | `6a0d26d61` | 54 passed |
| YAML: `capital_valuation_share_split.yaml`, `cash_isa_capital.yaml`, `isa_balances.yaml`, `lifetime_isa_capital.yaml` | `6a0d26d61` | 44 passed |
| `ruff format --check`, `ruff check`, `--select F811,F821`, `git diff --check` | `6a0d26d61` | clean |

`ce360946a` differs from `2f1051e9f` only in `shard_durations.toml`, and `6a0d26d61` adds the floor case to `test_capital_share_components.py`. CI at `6a0d26d61` passed every case in these files:

| File | Passed |
|---|---:|
| `test_capital_share_components.py` | 22 |
| `test_capital_valuation_properties.py` | 18 |
| `test_uc_property_capital.py` | 65 |
| `test_uc_company_owner_properties.py` | 7 |
| `test_uc_tax_free_childcare_choice.py` | 20 |
| `test_input_records.py` | 11 |
| `test_fiscal_partition.py` | 43 |

CI also passed the 70 cases in `capital_valuation.yaml`, `lifetime_isa_capital.yaml`, `capital_valuation_share_split.yaml`, `cash_isa_capital.yaml` and `isa_balances.yaml` at `6a0d26d61`.

axiom: TheAxiomFoundation/rulespec-uk#395 queued

We added the asset-type distinction and hand-computed companion cases to the issue (https://github.com/TheAxiomFoundation/rulespec-uk/issues/395#issuecomment-5979237567).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Head ce91e92ea9d2d7c7025d932ba613438bf4a70a8d on branch claude/restack-2131-isas-growth-y23axr (the harness pins pushes to claude/...). It starts from 6a0d26d618e018d673e23adaaf777cbd2a498d35 and adds two commits:

I pushed nothing to sale-expense-share-split or uc-capital-sale-expenses, and opened no PR, so no CI has run. The title and body above are the hub documents for this head.

Before starting (07:41Z), I checked the branches and comments. sale-expense-share-split was last pushed on 31 Oct at 21:53Z. The last comment on #2131 was the draft note from 2 Nov at 20:26Z. On #1969, the latest activity was the hub's merge at 06:15Z and the review at 07:23Z, and no comment there says anyone is mid-change.

Findings and resolutions

1. Merge of #1969 (ee325c5d). Git merged b4e872273 without conflict. Here is what each of #1969's first-parent commits in 3a3241cf6..b4e872273 changes, against the split's inputs:

  • ba4de95d6 (merges main at 0080274d7). It brings in main's test_stored_household_totals.py, and Simulation growing stored household totals past PROJECTION_END_YEAR. It rewrites only the documentation strings of cash_isa.py, stocks_and_shares_isa.py and household_lifetime_isa_balance.py; no formula changes. It also adds owned_land_secured_debt under per_capita.gdp in uprating_indices.yaml. It touches none of these:

    • the capital sources or sale_expenses/sources lists;
    • share_like_capital or valued_capital;
    • the UC company-holding variables;
    • test_isa_balances.py, test_capital_share_components.py or test_capital_valuation_properties.py.

    git diff --stat 3a3241cf6 b4e872273 over parameters/gov/dwp, utils/capital_valuation.py, variables/gov/dwp/universal_credit, share_like_capital.py, those three tests and shard_durations.toml lists only state_pension files.

    The new stored-totals test reads stocks_and_shares_isa as a stored total, and this PR's split moves that total into share_like_capital_without_sale_expenses. That interaction is the one inconsistency the merge left (finding 2).

  • c11d7c1d9 changes only test_stored_household_totals.py, to expect the 10% off the ISA. Finding 2 covers it.

  • b4e872273 (merges main at 52b30809f) changes no file the split reads or tests. Its only hand-resolved hunk sits in supplied_inputs.py.

The merge kept this branch's shard_durations.toml. It names test_unitemised_inputs_match_the_lists_before_itemising (line 16), and code_health passes.

2. Stored-totals test (ce91e92e, policyengine_uk/tests/test_stored_household_totals.py:219-233).

  • The UC capital assertion now expects savings plus the stored cash ISA plus the stored stocks-and-shares ISA at face value. It also expects 6 × savings in every year from 2025 to 2040.
  • I dropped the read of sale_expenses.rate and the capital < face_value - 0.01 check.
  • The ratio check stays, unchanged.
  • The comment now gives this PR's rule and its source: an ISA counts at its withdrawal value, with no deduction for expenses of sale (ADM H1656), through share_like_capital_without_sale_expenses. That is the source test_isa_balances.py:66 cites.
  • from policyengine_core.periods import period stays, because other tests in the file use it.

3. #2779's growth-factors page. git merge-tree --write-tree --name-only HEAD af3758c64 conflicts in docs/book/assumptions/growthfactors.md only.

Against uprating_indices.yaml at the head, all three names sit under gov.economic_assumptions.yoy_growth.obr.per_capita.gdp, at lines 135, 136 and 140.

4. Body.

Tests at ce91e92e

I ran these with uv sync --frozen --extra dev, with HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset, using -p no:cacheprovider. Every run passed with no failures, errors or skips.

Command Result
pytest -n 4 test_stored_household_totals.py 42 passed (184 s)
test_isa_balances.py 8 passed
test_capital_valuation_properties.py 18 passed (276 s)
test_stock_capital_periods.py 3 passed
test_uc_company_owner_properties.py 7 passed
test_uc_company_override_properties.py 11 passed
test_uc_property_capital.py 65 passed
test_uc_property_capital_mock.py 26 passed
test_capital_share_components.py::test_unitemised_inputs_match_the_lists_before_itemising, on its own, under timeout 2400 1 passed (1,791 s); did not stop
test_capital_share_components.py, deselecting that node, -n 4 21 passed (571 s)
Other files naming the five variables, -n 4 per file: test_uc_capital_allocation.py 1, test_benunit_reported_capital_properties.py 7, test_corporate_land_value.py 4, test_corporate_receipts_statistics.py 141, test_dataframe_dataset.py 90, test_isa_accounting.py 12, test_lifetime_isa_capital.py 3, test_mixed_age_couple_properties.py 1, test_pension_credit_reported_capital_properties.py 4, test_process_time_reforms.py 71, test_sdlt_counted_once_properties.py 1, test_uc_income_from_capital_properties.py 6, test_uc_legacy_closure_properties.py 4, test_uc_single_claim_properties.py 8, test_wealth_components.py 3, test_wealth_tax_net_base_properties.py 2, test_weighted_incidence.py 19 377 passed
policyengine-core test: capital_valuation.yaml 26, capital_valuation_share_split.yaml 10, cash_isa_capital.yaml 8, lifetime_isa_capital.yaml 24, household/wealth/isa_balances.yaml 2 70 passed
policyengine-core test, the other *capital*.yaml files: ESA esa_income_capital 10 and legacy_capital 3; HB housing_benefit_capital 13, housing_benefit_residential_capital 7 and legacy_capital 2; IS income_support_capital 10 and legacy_capital 3; JSA jsa_income_capital 9 and legacy_capital 3; UC uc_income_from_capital 7, uc_assessable_capital 6 and gov/dwp/universal_credit/uc_property_capital 9; PC pension_credit_reported_capital 9; household/wealth/benunit_reported_capital 6 97 passed
pytest -n 4 policyengine_uk/tests/code_health 3,113 passed (234 s)
.github/check_fix_pr.py on the posted title and body, label ci-recheck, changes git diff --name-status --no-renames -z b4e872273 HEAD, --head-sha ce91e92e… Fix evidence: PASS — "Changed executable tests: 6."
uvx ruff@0.17.2 format --check . / check . / check --select F811,F821 policyengine_uk 2,180 files already formatted / All checks passed / All checks passed
git diff --check 6a0d26d61..HEAD clean
git merge-tree --write-tree --name-only HEAD origin/main (52b30809f, unchanged since the brief) clean, tree 427e7bb71
git merge-tree --write-tree --name-only HEAD af3758c64 conflict in docs/book/assumptions/growthfactors.md only

test_isa_balances.py gives 8 here. The brief's review counted 8 on its scratch merge, and #1969's head has 2.

Mutation check

I made a scratch worktree at ce91e92e and removed - stocks_and_shares_isa from policyengine_uk/parameters/gov/dwp/share_like_capital/without_sale_expenses.yaml. I confirmed that policyengine_uk.__file__ resolved to the worktree. test_stored_household_totals_grow_past_the_projection_horizon then failed: AssertionError: 2025, assert np.float64(5800.0) == 6000.0 ± 0.06. I restored the file with git checkout, and git status --short came back empty.

Judgment calls

Skipped

  • Nothing needed a survey dataset.
  • I didn't rerun the earlier aggregate-impact or differential checks. The body keeps them, labelled 2f1051e9f.
  • I didn't run the full suite. CI hasn't run on this head, because the branch has no PR.

Generated by Claude Code

MaxGhenis and others added 4 commits November 12, 2026 09:15
Resolve three conflicts and keep main's person-level ISA allocation:

- shard_durations.toml: main's regenerated table, with this PR's three
  renamed share-component tests (same durations).
- growthfactors.md: main's text, which already lists the three
  secured-debt inputs this PR documents and moves main_residence_value and
  other_residential_property_value to house prices.
- uc_assessable_capital.py: this PR's sale_expenses.sources wording with
  main's prose, and the stocks and shares ISA at its withdrawal value.

Main (#2712, through #1969) moves each stocks and shares ISA balance out of
share_like_capital into its holder's unit. This PR replaces
share_like_capital in every capital list with its two parts, so
counted_person_sources would have dropped the ISA person source. The ISA
now maps to share_like_capital_without_sale_expenses, which holds it
because gov.dwp.share_like_capital.without_sale_expenses lists
stocks_and_shares_isa; no programme's sale-expense list names that part, so
each means test counts the balance at face value. The parameter and
variable descriptions name the new household source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch values stocks and shares ISAs without the deduction for
expenses of sale (ADM H1656-H1658), and the ISA person source now comes out
of share_like_capital_without_sale_expenses. Main's person ISA cases
expected 90%, so each counted ISA rises by a tenth. Cases 13 to 15 test a
boundary, so their inputs move to keep it: a 30,600 ISA in Case 13, and
household totals of 16,000 and 6,000 in Cases 14 and 15.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's tests valued all of share_like_capital, and each stocks and shares
ISA, at 90%. On this branch only share_like_capital_with_sale_expenses takes
the deduction, and the ISA person source comes out of
share_like_capital_without_sale_expenses at face value. The helpers now
value each part by whether the programme's sale_expenses.sources lists it,
and the hand-worked cases follow: the producer-shape households give UC
capital of 10,900, 8,950, 14,000 and 11,000, and the list-reform test drops
share_like_capital_without_sale_expenses, the source that holds the ISA.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test checked each total to within £0.011, and CI at df8c1ea failed it.
The engine holds totals in float32. Its float32 sum can round a penny under
the exact total, which the test's docstring allows, and it then stores that
figure to float32's step, which the £0.011 bound didn't allow for above about
£16,000. CI drew a household the test expects at £16,834.13: the engine gives
£16,834.12, held as 16,834.119, which is £0.0109 under, and the test's own
float32 subtraction read £0.0117.

The check now subtracts in float64 and allows a penny plus one float32 step at
the expected value. Two households are pinned: CI's, and one expected at
£37,747.34 that the engine holds as 37,747.328, which is £0.0119 under and
fails the old bound in float64 too.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaxGhenis, following up on my review at 6a5a13a4 at 3fe76ee.

Done.

  • Company holding. household_uc_company_holding_pool_scale.py:6 now takes the holding off share_like_capital_with_sale_expenses, and test_capital_share_components.py:398-403 pins it.
  • Long lines. This PR's diff no longer adds any parameter line over 120 characters.
  • Exempt list. test_capital_share_components.py:136-137 now pins the baseline exempt list to the two itemised components. That covers the COMPONENTS point at baseline; a comment at share_like_capital_with_sale_expenses.py:3 would still help a future reform.

1. Land this with #1969. My #1969 review asks for exactly that, and this head does the restoring:

  • _person_capital_pool.py:15 maps stocks_and_shares_isa_countable_capital to share_like_capital_without_sale_expenses, which is not on any sale list.
  • Person route. person_isa_capital.yaml Cases 2, 7 and 9 are back to main's values: £9,000 (:82), PC £9,000 (:224) and £11,000 (:284). #1969 alone gives £8,400, £8,800 and £10,700.
  • Household route. test_isa_balances.py and test_stored_household_totals.py now count the stocks and shares ISA through share_like_capital_without_sale_expenses, at full value.
  • The PR is still a draft on #1969's branch with no checks. Please take it out of draft and merge it into #1969's branch (or fold it in) before #1969 merges, so CI runs on the combined tree.

2. should (still open): the ISA wording.

  • parameters/gov/dwp/share_like_capital/without_sale_expenses.yaml:10-12 still reads "no ISA paragraph applies the deduction", and changelog.d/sale-expense-share-split.fixed.md states the ISA treatment flatly.
  • Full value is the reasonable reading of ADM H1656, which values an ISA at "what people would get if they withdrew their investment" (ADM H1, p.55).
  • Please have the parameter say that it rests on H1656's silence on costs of sale, as the PR body already does.

3. nit (still open): employee share options.

  • directly_held_shares includes "employee shares and share options" (variables/input/directly_held_shares.py:9), so they take the 10%.
  • One clause in without_sale_expenses.yaml noting this as an approximation would cover it.

Review drafted with AI assistance and checked against the code at 3fe76ee.

MaxGhenis and others added 3 commits November 16, 2026 05:31
The without_sale_expenses description and the changelog fragment now say
the model counts a stocks and shares ISA at its full withdrawal value
because H1656 says nothing about costs of sale, not that a paragraph
exempts it. The description also notes that employee share options, which
the survey asks about with employee shares, take the deduction as an
approximation. A comment on COMPONENTS says every name on the exempt list
must be one of them, since the formula reads only COMPONENTS to decide
whether a household itemises.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
valued_capital reads only each programme's capital.sources, so a name on
capital.sale_expenses.sources outside them is never valued. The property
now asserts the sale list is a subset of the counted sources for all six
programmes, which the intersection check alone could not see.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…omments

#1969's changelog fragment said the 10% reached all of share_like_capital,
unit trusts included, "until the data separate them". It now names what
takes the deduction after the split: land, property, directly held shares
and any corporate_wealth a dataset does not itemise.

The ISA docs named share_like_capital as the source the stocks and shares
ISA balances leave. They now name share_like_capital_without_sale_expenses,
and say every means test counts that part at withdrawal value.

The test comments and the parameter description say the model reads ADM
H1656's silence on costs of sale as no deduction, rather than saying H1656
rules one out. The COMPONENTS comment notes that share_like_capital sums the
same names, and invariant 7 allows float32's step as well as a penny.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 8f755b9 into uc-capital-sale-expenses Nov 16, 2026
@MaxGhenis
MaxGhenis deleted the sale-expense-share-split branch November 16, 2026 11:52
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Thanks for the follow-up at 3fe76ee. #1969's branch now carries this PR's commits up to 8f755b981, so GitHub closed this PR as merged into it. CI runs on the combined tree in #1969.

  • 2. ISA wording. 690a56222 makes without_sale_expenses.yaml say the model counts an ISA's withdrawal value in full, a reading that rests on ADM H1656 saying nothing about costs of sale. 8f755b981 gives the changelog fragment and the four test comments that cite H1656 the same wording.
  • 3. Employee share options. 690a56222 adds to the same description that directly held shares include employee shares and share options. The Wealth and Assets Survey records them as one total, so share options take the deduction, an approximation.
  • COMPONENTS comment. 690a56222 adds it at share_like_capital_with_sale_expenses.py:3: every exempt name must be one of COMPONENTS. 8f755b981 adds that share_like_capital sums the same names, so a reform that adds a holding must add it there too.

Any further points on the combined change are best raised on #1969.

MaxGhenis added a commit that referenced this pull request Nov 16, 2026
#1969 now values unit trusts and stocks and shares ISAs at full value in
every means test, through share_like_capital_without_sale_expenses, and
takes the 10% only from share_like_capital_with_sale_expenses. This branch
held #1969 at 59a1c4a, before that split.

Each of the six sale-expense lists keeps #2688's overseas and other real
estate and names the part with sale expenses, and Pension Credit's capital
sources name both parts. test_isa_balances.py and
test_stored_household_totals.py take #1969's full-value ISA expectations,
which reverses e041345. test_property_wealth_components.py now checks
the two parts, and the property-wealth docs say which part takes the 10%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-recheck Re-triggers Fix evidence without changing the PR (UK hub, 2026-10-12)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants