diff --git a/changelog.d/marriage-allowance-rate-test-limit.fixed.md b/changelog.d/marriage-allowance-rate-test-limit.fixed.md new file mode 100644 index 0000000000..4badff7cb7 --- /dev/null +++ b/changelog.d/marriage-allowance-rate-test-limit.fixed.md @@ -0,0 +1 @@ +The search for the least-tax share of reliefs and allowances now applies the Marriage Allowance rate test at the statutory limits, instead of with room below them (about 3.9p near the £37,700 basic rate limit), so a recipient whose only qualifying share leaves income at a limit gets the reduction. The search keeps the room where float32 could hold income entered over a limit at the limit: where an amount the test reads reaches £65,536, or where the model cannot bound the float32 error in the amounts behind it, such as pay before expenses, under the least amount the entries can sit over a limit. An amount supplied as an input in place of its formula, such as pay set through Simulation.set_input or in a YAML test, counts as an entry: the model bounds that amount's own float32 error, with no bound from £131,072, and does not read the amounts its formula would. diff --git a/policyengine_uk/tests/policy/baseline/gov/hmrc/income_tax/allowances/received_allowances/least_tax_allocation.yaml b/policyengine_uk/tests/policy/baseline/gov/hmrc/income_tax/allowances/received_allowances/least_tax_allocation.yaml index 0d4bc2a9b1..df889a64dd 100644 --- a/policyengine_uk/tests/policy/baseline/gov/hmrc/income_tax/allowances/received_allowances/least_tax_allocation.yaml +++ b/policyengine_uk/tests/policy/baseline/gov/hmrc/income_tax/allowances/received_allowances/least_tax_allocation.yaml @@ -552,3 +552,124 @@ meets_marriage_allowance_income_conditions: [true, true] marriage_allowance_tax_reduction: [252, 0] income_tax: [7_088, 0] + +- name: Case 24, a recipient's only qualifying share leaves pay exactly at the basic rate limit + period: 2026 + absolute_error_margin: 0.01 + input: + people: + recipient: + age: 40 + employment_income: 40_200 + savings_interest_income: 9_000 + dividend_income: 1_570 + partner: + age: 40 + benunit: + members: [recipient, partner] + is_married: true + output: + # Income after the allowance is 50,770 - 12,570 = 38,200, above the basic + # rate limit, so the savings allowance is £500 in every share (s. 12B). + # The rate test (s. 55B(2)(b) and (ba)) needs no dividends left, at most + # £500 of savings left and at most £37,700 of pay left, so one share meets + # it: £2,500 against pay, £8,500 against savings and £1,570 against + # dividends. Its tax is 37,700 x 20% = 7,540, less 252: £7,288. Without + # the reduction the least tax is £7,440 (£11,500 against pay and £1,070 + # against dividends). + received_allowances_earned_income: [2_500, 0] + received_allowances_savings_income: [8_500, 0] + received_allowances_dividend_income: [1_570, 0] + income_tax_pre_charges: [7_540, 0] + meets_marriage_allowance_income_conditions: [true, true] + marriage_allowance_tax_reduction: [252, 0] + income_tax: [7_288, 0] + +- name: Case 25, a recipient's share leaves pay 2p below the basic rate limit + period: 2026 + absolute_error_margin: 0.01 + input: + people: + recipient: + age: 40 + employment_income: 40_199.98 + savings_interest_income: 9_000 + dividend_income: 1_570 + partner: + age: 40 + benunit: + members: [recipient, partner] + is_married: true + output: + # As Case 24 with 2p less pay. The same deductions leave £37,699.98 of pay + # and £500 of savings at the savings nil rate: 37,699.98 x 20% = + # 7,539.996, less 252: £7,287.996. + received_allowances_earned_income: [2_500, 0] + received_allowances_savings_income: [8_500, 0] + received_allowances_dividend_income: [1_570, 0] + meets_marriage_allowance_income_conditions: [true, true] + marriage_allowance_tax_reduction: [252, 0] + income_tax: [7_287.996, 0] + +- name: Case 26, a recipient sets the allowance against dividends to stop pay at the basic rate limit + period: 2026 + absolute_error_margin: 0.01 + input: + people: + recipient: + age: 40 + employment_income: 50_000 + savings_interest_income: 500 + dividend_income: 270 + partner: + age: 40 + benunit: + members: [recipient, partner] + is_married: true + output: + # The old order leaves £37,430 of pay, £500 of savings at the savings nil + # rate and £270 of dividends within the dividend allowance, for the least + # Step 5 tax, 37,430 x 20% = 7,486. Its £270 of dividends end above the + # basic rate limit, so it fails the rate test (s. 55B(2)(ba)). Setting + # £270 against dividends and £12,300 against pay leaves £37,700 of pay and + # £500 of savings: 37,700 x 20% = 7,540, less 252: £7,288. + received_allowances_earned_income: [12_300, 0] + received_allowances_savings_income: [0, 0] + received_allowances_dividend_income: [270, 0] + income_tax_pre_charges: [7_540, 0] + meets_marriage_allowance_income_conditions: [true, true] + marriage_allowance_tax_reduction: [252, 0] + income_tax: [7_288, 0] + +- name: Case 27, a recipient sets a penny against dividends to stop pay at the basic rate limit + period: 2026 + absolute_error_margin: 0.01 + input: + people: + recipient: + age: 40 + employment_income: 50_269.99 + savings_interest_income: 500 + dividend_income: 0.01 + partner: + age: 40 + benunit: + members: [recipient, partner] + is_married: true + output: + # Income after the allowance is 50,770 - 12,570 = 38,200, above the basic + # rate limit, so the savings allowance is £500 in every share (s. 12B). + # The old order leaves £37,699.99 of pay, £500 of savings at the savings + # nil rate and 1p of dividends within the dividend allowance, for the + # least Step 5 tax, 37,699.99 x 20% = 7,539.998. Its 1p of dividends ends + # above the basic rate limit, so it fails the rate test (s. 55B(2)(ba)). + # Only one share meets the test: 1p against dividends and £12,569.99 + # against pay, which leaves £37,700 of pay and £500 of savings: 37,700 x + # 20% = 7,540, less 252: £7,288. + received_allowances_earned_income: [12_569.99, 0] + received_allowances_savings_income: [0, 0] + received_allowances_dividend_income: [0.01, 0] + income_tax_pre_charges: [7_540, 0] + meets_marriage_allowance_income_conditions: [true, true] + marriage_allowance_tax_reduction: [252, 0] + income_tax: [7_288, 0] diff --git a/policyengine_uk/tests/test_income_tax_allocation.py b/policyengine_uk/tests/test_income_tax_allocation.py index cd34ba7a18..ab830a9c1c 100644 --- a/policyengine_uk/tests/test_income_tax_allocation.py +++ b/policyengine_uk/tests/test_income_tax_allocation.py @@ -32,26 +32,44 @@ than the model's share. 10. Where the search moves a share and counts it as meeting the rate test, the model's own variables meet the test at that share, through float32 rounding. + The test also checks that the model's answer for every recipient is the one + the search priced the stored share with (``_rate_test_on_stored``). That + check restates the variable's formula, so it guards the wiring between the + variable and the search, not the law. 11. The vertex search finds the least Step 5 tax less the reduction a share wins, checked against a dense grid and the thin sets on which the rate test can hold: no dividends left, and savings left at the savings allowance. 12. Fast path: invariant 7 holds with recipients among the people. """ +from types import SimpleNamespace + import numpy as np import pytest from hypothesis import HealthCheck, example, given, settings from hypothesis import strategies as st from policyengine_core import periods +from policyengine_core.reforms import Reform from policyengine_uk import Simulation from policyengine_uk.utils.income_tax_allocation import ( + LOW_PRECISION, + RATE_TEST_ROOM, + RATE_TEST_ROOM_FRACTION, + TIE, _build, + _can_hide_an_excess, _context, + _float32_meets_rate_test, + _held_error, + _inputs, _least_tax_share, _meets_rate_test, _old_order_is_least, + _rate_test_on_stored, + _share_left, _shares, + _stored_share, default_order_deductions, least_tax_deductions, old_order, @@ -62,6 +80,7 @@ marriage_allowance_rate_limits, meets_marriage_allowance_rate_test, ) +from policyengine_uk.utils.supplied_inputs import set_model_value pytestmark = pytest.mark.usefixtures("cloned_uk_tax_benefit_system") @@ -899,13 +918,24 @@ def check_recipients(couple_sims, year, regions, rows): # Invariant 10: the model meets the test where the search counts on it. x, s, v = _least_tax_share(c) counted = _meets_rate_test(c, x, s, v)[:, 0] - moved = ( - law.calculate("received_allowances_savings_income", year) != default.savings - ) | ( - law.calculate("received_allowances_dividend_income", year) != default.dividends + stored = ( + law.calculate("received_allowances_savings_income", year).astype(float), + law.calculate("received_allowances_dividend_income", year).astype(float), ) + moved = (stored[0] != default.savings) | (stored[1] != default.dividends) meets = law.calculate("meets_marriage_allowance_income_conditions", year) assert meets[moved & counted].all() + # And it gives the search's own answer at the share it stores: the test in + # float64 from the stored amounts, or in float32 as the model has always + # applied it (the old order's own answer where the share did not move). + held = stored[0][:, None], stored[1][:, None] + single = np.where( + moved, + _float32_meets_rate_test(c, *held)[:, 0], + c.old_order_meets_rate_test[:, 0], + ) + expected = _meets_rate_test(c, *_share_left(c, *held))[:, 0] | single + assert np.array_equal(meets, expected) @pytest.mark.parametrize("year", sorted({year for year, _, _ in RECIPIENTS})) @@ -968,7 +998,7 @@ def test_rate_test_survives_float32_rounding(couple_sims): def with_marriage_allowance(income_tax, year, inputs, default, seed): """``inputs`` with a reduction at stake for about half the people, the rate test's limits for them, and the old order's answer to the test, in float32 - as the model computes it.""" + as the model computes it (``_old_order_meets_rate_test``).""" rng = np.random.default_rng(seed) count = len(inputs["savings"]) f32 = np.float32 @@ -993,13 +1023,18 @@ def with_marriage_allowance(income_tax, year, inputs, default, seed): non_savings_left, np.maximum(0, savings - default.savings), np.maximum(0, dividends - default.dividends), - limits, + SimpleNamespace( + non_savings=np.asarray(limits.non_savings).astype(f32), + basic_rate=np.asarray(limits.basic_rate).astype(f32), + starting=limits.starting, + savings_allowance=limits.savings_allowance, + ), ) return inputs | dict( marriage_allowance_reduction=rng.choice([0.0, 2.0, 100.0, 252.0], count), rate_test_non_savings_limit=np.asarray(limits.non_savings, dtype=np.float64), rate_test_savings_allowance=np.broadcast_to( - np.asarray(limits.savings_allowance, dtype=np.float64), savings.shape + np.asarray(limits.savings_allowance, dtype=np.float64), (count,) ), old_order_meets_rate_test=meets, ) @@ -1080,11 +1115,9 @@ def test_vertex_search_weighs_the_marriage_allowance(system, year): def test_search_keeps_a_recipient_the_test_passes_exactly_at_the_limit(system): """Pay of £38,570 and savings of £11,700 in 2026-27. The old order leaves £26,000 of pay and £37,700 in all, exactly the basic rate limit, which the - rate test allows (s. 10(2)): 7,340 less the £252 reduction, 7,088. The - search's room below the limit doesn't count that share as meeting the test, - though a share that stops savings at the £1,000 savings allowance does, for - the same 7,088. So the old order must answer for itself, or the search would - move the recipient for no saving.""" + rate test allows (s. 10(2)): 7,340 less the £252 reduction, 7,088. A share + that stops savings at the £1,000 savings allowance gives the same 7,088, so + the search must keep the old order rather than move the recipient for no saving.""" income_tax = system.parameters("2026").gov.hmrc.income_tax inputs, default = one_person(savings=11_700, net_income=50_270) inputs = with_marriage_allowance(income_tax, 2026, inputs, default, 0) @@ -1094,3 +1127,1486 @@ def test_search_keeps_a_recipient_the_test_passes_exactly_at_the_limit(system): inputs, income_tax, default.savings, default.dividends, search_all=True ) assert to_savings[0] == 0 and to_dividends[0] == 0 + + +def test_rate_test_on_stored_counts_shares_that_meet_the_test_only_in_float64( + system, +): + """Two 2026-27 recipients whose pay, savings and dividends come to £50,270, + so after the £12,570 Personal Allowance every share leaves income at the + £37,700 basic rate limit, which meets the rate test (s. 10(2)). The model + holds each amount in float32. For pay of £37,401.34, savings of £5,081.08 + and dividends of £7,787.58, the old order sets the allowance against pay. + Taking the savings and dividends from net income in float32 leaves + £24,831.34375 of pay, so the float32 test reads £37,700.0039 and fails. For + pay of £32,747.81, savings of £7,271.86 and dividends of £10,250.33, a share + that sets £6,771.86 against savings and £5,798.14 against dividends leaves + £500 of savings. Summing what it leaves in float32 gives £37,700.0039 too. + On the amounts the model holds, each share leaves £37,700 in float64, so + ``_rate_test_on_stored`` counts both.""" + income_tax = system.parameters("2026").gov.hmrc.income_tax + single = np.float32 + + def context(pay, savings, dividends): + held = [single(amount) for amount in (pay, savings, dividends)] + inputs, default = one_person( + net_income=float((held[0] + held[1]) + held[2]), + savings=float(held[1]), + dividends=float(held[2]), + ) + inputs = with_marriage_allowance(income_tax, 2026, inputs, default, 0) + return _build(inputs, income_tax), default + + c, default = context(37_401.34, 5_081.08, 7_787.58) + old = (default.savings[:, None], default.dividends[:, None]) + assert not c.old_order_meets_rate_test[0, 0] + assert _rate_test_on_stored(c, *old, np.array([[False]]))[0, 0] + + c, _ = context(32_747.81, 7_271.86, 10_250.33) + moved = (np.array([[single(6_771.86)]]), np.array([[single(5_798.14)]])) + assert not _float32_meets_rate_test(c, *moved)[0, 0] + assert _rate_test_on_stored(c, *moved, np.array([[True]]))[0, 0] + + +def test_search_moves_a_person_whose_first_component_has_no_room(system): + """Savings of £5,500 and dividends of £700.07 in 2026-27, with £3,569.63 of + allowances and no non-savings income. The old order sets the allowances + against savings, which leaves £1,930.37 of savings in the starting rate + band and £700.07 of dividends: £200.07 above the £500 dividend allowance + at 10.75%, £21.51. Setting £200.07 against the dividends and the rest + against savings leaves £2,130.44 of savings in the band and the dividends + within the allowance: nil. + + With no non-savings income the first component has no room, so the two + deductions must add up to £3,569.63, which no float32 rounding of + £3,369.56 does exactly. The search still moves the person.""" + income_tax = system.parameters("2026").gov.hmrc.income_tax + inputs, default = one_person( + savings=5_500, dividends=700.07, allowances=3_569.63, net_income=6_200.07 + ) + for search_all in (False, True): + to_savings, to_dividends = least_tax_deductions( + inputs, income_tax, default.savings, default.dividends, search_all + ) + assert to_dividends[0] == pytest.approx(200.07, abs=1e-3) + assert to_savings[0] + to_dividends[0] == pytest.approx(3_569.63, abs=1e-3) + assert tax_saved(income_tax, inputs, default) == pytest.approx(21.51, abs=0.01) + + +def test_search_moves_a_recipient_with_no_property_income_from_2027(couple_sims): + """Married recipients in 2027-28 whose partners have no income, with no + property income, so the first component has no room. + + Pay £5,000, savings £2,000.37, dividends £20,000.11: net income £27,000.48, + Step 3 income £14,430.48. Section 25(3A) takes £5,000 of the £12,570 + Personal Allowance off pay, and the taxpayer shares the other £7,570 + (s. 25(2)). The old order leaves dividends of £14,430.48, taxed at 10.75% + above the £500 allowance: £1,497.53, less the £252 reduction, £1,245.53. + Setting all £7,570 against the dividends leaves the savings in the starting + rate band and dividends of £12,430.11: £1,282.49, less £252, £1,030.49. + The other rows follow the same way.""" + year = 2027 + rows = [ + (5_000, 2_000.37, 20_000.11, 0), + (5_000.13, 2_000.37, 20_000.11, 0), + (9_000, 300.37, 10_485.76, 0), + (9_000, 3.37, 10_485.76, 0), + ] + sim = run_couples(couple_sims(year, ["LONDON"] * len(rows)), year, rows) + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([1_030.49, 1_030.50, 437.69, 437.69], abs=0.01) + + +def explicit_couples(year, rows, reform=None): + """A Simulation of married recipients whose partners have no income, one + couple a row. Each row maps person variables to the recipient's amounts, + with ``region`` for the household. ``reform`` is a structural reform to + build the simulation with.""" + people, benunits, households = {}, {}, {} + for i, row in enumerate(rows): + amounts = dict(row) + region = amounts.pop("region", "LONDON") + people[f"r{i}"] = {"age": {year: 40}, "uses_property_allowance": {year: False}} + people[f"r{i}"] |= {name: {year: value} for name, value in amounts.items()} + people[f"p{i}"] = {"age": {year: 40}} + benunits[f"b{i}"] = {"members": [f"r{i}", f"p{i}"], "is_married": {year: True}} + households[f"h{i}"] = { + "members": [f"r{i}", f"p{i}"], + "region": {year: region}, + } + return Simulation( + situation={"people": people, "benunits": benunits, "households": households}, + reform=reform, + ) + + +# Whole-pence recipients in 2026-27, each a penny over a limit of the rate test +# under every share, so the law gives none of them the reduction. The model +# sums net income and allowances in float32, which leaves them 0.2p to 0.5p +# over the limit, not a whole penny; the test must still fail them. +PENNY_OVER = [ + dict( + employment_income=46_951.24, + private_pension_income=1_366.33, + property_income=646.47, + savings_interest_income=166.71, + dividend_income=1_305.97, + ), + dict( + employment_income=46_699.33, + private_pension_income=1_000.33, + property_income=1_000.33, + savings_interest_income=200.08, + dividend_income=1_570.02, + ), + dict( + employment_income=53_997.58, + savings_interest_income=24.33, + dividend_income=1_588.58, + other_deductions=5_316.15, + ), + dict( + employment_income=48_944.49, + savings_interest_income=6_399.67, + dividend_income=320.74, + other_deductions=4_894.89, + ), + dict( + employment_income=44_718.49, + savings_interest_income=491.15, + dividend_income=667.15, + other_deductions=1_056.48, + region="SCOTLAND", + ), + dict( + employment_income=51_279.33, + savings_interest_income=50.97, + dividend_income=942.58, + gift_aid=1_561.52, + ), + dict( + employment_income=10_964.47, + savings_interest_income=2_969.9, + dividend_income=42_854.58, + trading_loss=6_518.94, + ), +] + + +def test_rate_test_fails_a_recipient_a_penny_over_a_limit(): + """Pay £53,997.58, savings £24.33, dividends £1,588.58 and other deductions + of £5,316.15 (the third row): allowances of £17,886.15 leave £37,724.34, so + the old order leaves dividends above the basic rate limit, and setting the + £1,588.58 against the dividends leaves pay of £37,700.01, a penny over. The + second row's tax, worked by hand without the reduction, is £7,391.05.""" + year = 2026 + sim = explicit_couples(year, PENNY_OVER) + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert not meets.any() and not reduction.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax[1] == pytest.approx(7_391.05, abs=0.01) + + +# Whole-pence recipients in 2026-27 whose pay, savings and dividends come to +# £50,270. After the £12,570 Personal Allowance the old order leaves income at +# the £37,700 basic rate limit, so the law gives the reduction (s. 10(2)). The +# model holds the amounts in float32, and the float32 test reads income +# £0.0039 over the limit. The last row has a penny more of dividends, so the +# law denies it. +AT_THE_LIMIT_IN_FLOAT64 = [ + dict( + employment_income=37_401.34, + savings_interest_income=5_081.08, + dividend_income=7_787.58, + ), + dict( + employment_income=38_696.41, + savings_interest_income=3_144.24, + dividend_income=8_429.35, + ), + dict( + employment_income=37_401.34, + savings_interest_income=5_081.08, + dividend_income=7_787.59, + ), +] + + +def test_rate_test_reads_the_old_order_in_float64_too(): + """The first row's tax, worked by hand: £24,831.34 of pay at 20%, £4,081.08 + of savings above the £1,000 savings allowance at 20%, and £7,287.58 of + dividends above the £500 dividend allowance at 10.75%: £6,565.90, or + £6,313.90 after the £252 reduction. ``main`` denies it. In the last row the + penny over the limit cuts the savings allowance to £500 (s. 12B), and the + tax is £6,665.90.""" + year = 2026 + sim = explicit_couples(year, AT_THE_LIMIT_IN_FLOAT64) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert meets.tolist() == [True, True, False] + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax[[0, 2]] == pytest.approx([6_313.90, 6_665.90], abs=0.01) + + +def test_rate_test_lets_income_reach_the_basic_rate_limit(): + """Pay £6,214.61, savings £367.26, dividends £47,955.74 and other deductions + of £4,267.61 in 2026-27: allowances of £16,837.61 leave £37,700.00, the + basic rate limit. Setting all £16,837.61 against pay and dividends, none + against savings, leaves the savings in the starting rate band and dividends + of £37,332.74, all the income at the limit: 10.75% above the £500 + allowance, £3,959.52, + less the £252 reduction, £3,707.52. The old order leaves the dividends at + £37,700, £3,999 less £252: £3,747.""" + year = 2026 + sim = explicit_couples( + year, + [ + dict( + employment_income=6_214.61, + savings_interest_income=367.26, + dividend_income=47_955.74, + other_deductions=4_267.61, + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + tax = sim.calculate("income_tax", year).astype(float)[0] + assert tax == pytest.approx(3_707.52, abs=0.01) + + +def test_search_keeps_the_rounding_that_meets_the_rate_test_at_a_gift_aid_limit(): + """Pay £188.28, property income £121.87, savings £175.31, dividends + £52,212.12, other deductions £1,218.28 and a Gift Aid gift of £967.44 in + 2028-29. Allowances of £13,788.28 leave £38,909.30, the basic rate limit + raised by the £1,209.30 grossed-up gift (s. 414(2)(b)). Setting £310.15 + against pay and property income and £13,478.13 against dividends leaves + savings in the starting rate band and dividends of £38,733.99, all the + income at the limit: 10.75% above the £500 allowance, £4,110.15, less the + £252 reduction, £3,858.15. + + The model sums net income to £52,697.58203125, 0.2p over the amounts + entered, and holds the grossed-up gift as £1,209.2999267578125. So it + holds Step 3 income 0.18p over the limit, and no share meets the rate + test in float64. The float32 value nearest the deduction from dividends, + £13,478.12890625, leaves the first component a fraction of a penny below + nil. The float32 amounts it leaves sum to £38,909.30078125, the float32 + value of the limit, so it meets the test in float32 alone + (``_rate_test_on_stored``). The next float32 value up fails the test in + both arithmetics. A rounding that passes only in float32 ranks last + where the exact share fails in float64, but not the nearest rounding, so + ``_stored_share`` keeps it. The two tests below pin each step on the + amounts the model holds, so a change in the parameters that moves the + float32 tie fails them rather than this test's expected tax alone.""" + year = 2028 + sim = explicit_couples( + year, + [ + dict( + employment_income=188.28, + property_income=121.87, + savings_interest_income=175.31, + dividend_income=52_212.12, + other_deductions=1_218.28, + gift_aid=967.44, + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + tax = sim.calculate("income_tax", year).astype(float)[0] + assert tax == pytest.approx(3_858.15, abs=0.01) + + +def gift_aid_limit_context(system): + """The search's context for the recipient of the test above, from the + amounts the model holds for them: net income as the model sums it, the + grossed-up gift as the model holds it, and the rest in float32.""" + income_tax = system.parameters("2028").gov.hmrc.income_tax + single = np.float32 + inputs, default = one_person( + net_income=52_697.58203125, + savings=float(single(175.31)), + dividends=float(single(52_212.12)), + property=float(single(121.87)), + allowances=float(single(13_788.28)), + gift=1_209.2999267578125, + ) + inputs = with_marriage_allowance(income_tax, 2028, inputs, default, 0) + inputs["marriage_allowance_reduction"] = np.array([252.0]) + return _build(inputs, income_tax) + + +def test_rate_test_on_stored_counts_a_moved_share_that_meets_the_test_only_in_float32( + system, +): + """The Gift Aid recipient above. The model holds Step 3 income of + £38,909.3017578125 against a limit of £38,909.29992675781, so the share + that sets £13,478.12890625 against dividends fails the rate test in + float64. In float32 the amounts it leaves sum to the float32 value of the + limit, and the test passes. ``_rate_test_on_stored`` counts a moved share + that meets the test in either arithmetic, so it counts this one.""" + c = gift_aid_limit_context(system) + over = c.step_3_income[0, 0] - c.bands[1][0, 0] + assert over == pytest.approx(0.0018, abs=1e-4) + moved = (np.array([[0.0]]), np.array([[13_478.12890625]])) + assert not _meets_rate_test(c, *_share_left(c, *moved))[0, 0] + assert _float32_meets_rate_test(c, *moved)[0, 0] + assert _rate_test_on_stored(c, *moved, np.array([[True]]))[0, 0] + + +def test_stored_share_keeps_the_nearest_rounding_that_meets_the_test_only_in_float32( + system, +): + """The Gift Aid recipient above. The share with the least ``f`` sets + £13,478.1293 against dividends and fails the rate test in float64, as + every share does. The float32 value nearest that deduction meets the + test in float32 alone. A rounding that passes only in float32 ranks last + where the exact share fails, unless it is the nearest, so + ``_stored_share`` keeps this one and counts the £252 reduction: an ``f`` + of £3,858.15. The other rounding fails the test and leaves £4,110.15.""" + c = gift_aid_limit_context(system) + _, s, v = _least_tax_share(c) + exact = (c.savings - s, c.dividends - v) + assert not _meets_rate_test(c, *_share_left(c, *exact))[0, 0] + stored = _stored_share(c, s, v) + assert stored.to_savings[0, 0] == 0 + assert stored.to_dividends[0, 0] == np.float32(exact[1][0, 0]) + assert stored.to_dividends[0, 0] == 13_478.12890625 + held = (stored.to_savings, stored.to_dividends) + assert not _meets_rate_test(c, *_share_left(c, *held))[0, 0] + assert _float32_meets_rate_test(c, *held)[0, 0] + assert stored.tax[0, 0] == pytest.approx(3_858.15, abs=0.01) + + +def test_rate_test_fails_a_rounding_that_sets_more_than_the_shared_allowances(): + """Two 2026-27 recipients whose Gift Aid gifts raise the basic rate + limit above their pay. The first: pay £157,600, savings £140,500.3125, + other deductions £139,000, a trading loss of £1,000.30 and a gift of + £96,000, which lifts the limit to £157,700 (s. 414(2)(b)). The Personal + Allowance tapers to nil, so the shared amount is £139,000 plus £1,000.30 + of loss relief, and every share leaves income of £158,100.0125: at least + 1.25p of savings above the £500 savings nil rate band that starts at + £157,600. So no share meets the rate test, and the tax is £31,520.005. + Rounding the deduction from savings up to float32 would set 1.25p more + than the shared amount against savings and leave them at the band. The + second does the same with a step of 3.125p: pay £307,600, savings + £301,500.03125, other deductions £300,000, a loss of £1,000.001 and a gift + of £216,000 leave 3.03p of savings above the band and tax of £61,520.01. + In both, the rounding that would pass is not the float32 value nearest + the deduction, so it ranks last.""" + year = 2026 + common = dict(uses_trading_allowance=False) + sim = explicit_couples( + year, + [ + dict( + employment_income=157_600, + savings_interest_income=140_500.3125, + other_deductions=139_000, + trading_loss=1_000.30, + gift_aid=96_000, + **common, + ), + dict( + employment_income=307_600, + savings_interest_income=301_500.03125, + other_deductions=300_000, + trading_loss=1_000.001, + gift_aid=216_000, + **common, + ), + ], + ) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert not meets.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([31_520.005, 61_520.01], abs=0.01) + + +def test_search_keeps_a_rounding_within_half_a_penny_of_the_shared_allowances(): + """A 2027-28 recipient with no non-savings income: savings of £4,025.26, + dividends of £52,610.04 and other deductions of £6,365.30. Allowances of + £18,935.30 leave £37,700, the basic rate limit, so the law gives the + reduction (s. 10(2)). The savings sit in the starting rate band at 0%, so + the whole £18,935.30 goes against the dividends and leaves £33,674.74 of + them: 10.75% above the £500 allowance, £3,566.28, or £3,314.28 after the + £252 reduction. With no property income the two deductions must take the + whole shared amount, and float32 cannot hold the deduction from dividends. + The nearest value, £18,935.298828125, leaves income over the limit; + ``main`` stores it and charges £3,566.29. The next value up, + £18,935.30078125, sets 0.08p more than the shared amount, within + ``OVER_DEDUCTION``, and meets the rate test, so the search keeps it.""" + year = 2027 + sim = explicit_couples( + year, + [ + dict( + savings_interest_income=4_025.26, + dividend_income=52_610.04, + other_deductions=6_365.30, + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + to_dividends = sim.calculate("received_allowances_dividend_income", year)[0] + assert to_dividends == 18_935.30078125 + tax = sim.calculate("income_tax", year).astype(float)[0] + assert tax == pytest.approx(3_314.28, abs=0.01) + + +def test_search_ranks_last_a_rounding_more_than_half_a_penny_beyond_the_allowances( + system, +): + """Amounts as float32 holds them for a 2026-27 taxpayer with no Marriage + Allowance at stake and no non-savings income: savings of £3,000, dividends + of £400,000, other deductions of £139,000 and a trading loss of + £1,000.30. The savings sit in the starting rate band at 0%, so the whole + £140,000.30 of deductions goes against the dividends. Float32 holds that + deduction as £140,000.296875, 0.3p under, or £140,000.3125, 1.25p over. + The value above leaves 1.56p less of the dividends to tax at 39.35%, which + saves more than ``TOLERANCE``. It sets more than ``OVER_DEDUCTION`` beyond + the deductions and is not the nearest value, so it ranks last and the + search stores the value below.""" + income_tax = system.parameters("2026").gov.hmrc.income_tax + single = np.float32 + relief = single(1_000.30) + net_income = single(403_000) - relief + inputs, default = one_person( + net_income=float(net_income), + savings=3_000.0, + dividends=400_000.0, + allowances=139_000.0, + relief=float(relief), + before_relief=403_000.0, + ) + for search_all in (False, True): + to_savings, to_dividends = least_tax_deductions( + inputs, income_tax, default.savings, default.dividends, search_all + ) + assert to_savings[0] == 0 + assert to_dividends[0] == 140_000.296875 + + +def test_search_keeps_the_nearest_rounding_where_stored_savings_sit_over_the_band(): + """2026-27 recipients whose £96,000 Gift Aid gifts raise the basic rate + limit to £157,700 (s. 414(2)(b)), above their pay of £157,600. The first + has savings of £140,500.01, other deductions of £139,000 and a trading + loss of £1,000.01. The Personal Allowance tapers to nil, and setting the + whole £140,000.01 of shared deductions against savings leaves £500 of + them, at the top of the savings nil rate band. So the share meets the rate + test, and the tax is £157,600 × 20% = £31,520, or £31,268 after the £252 + reduction. The model holds the savings in float32 as £140,500.015625, so + the exact share leaves them 0.56p over the band. Only the float32 value + nearest the deduction from savings, 0.56p over the shared amount, puts + them back on it, and that rounding keeps its rank. The second and third + are the first with a loss of £1,000.3055 and savings of £140,500.3125 and + £140,500.305. The law leaves the second 0.7p of savings over the band and + denies it the reduction, but gives the third the reduction. Both hold the + same float32 amounts, so the model gives both the third's tax.""" + year = 2026 + common = dict( + employment_income=157_600, + other_deductions=139_000, + gift_aid=96_000, + uses_trading_allowance=False, + ) + sim = explicit_couples( + year, + [ + dict(savings_interest_income=140_500.01, trading_loss=1_000.01, **common), + dict( + savings_interest_income=140_500.3125, + trading_loss=1_000.3055, + **common, + ), + dict( + savings_interest_income=140_500.305, + trading_loss=1_000.3055, + **common, + ), + ], + ) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert meets.all() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax[1] == tax[2] + assert tax == pytest.approx([31_268, 31_268, 31_268], abs=0.01) + + +def test_search_ranks_last_a_rounding_that_passes_only_in_float32(): + """A 2027-28 recipient with pay of £8,999.58, savings of £43,451.58, + dividends of £3,696.72 and other deductions of £5,877.87. Allowances of + £18,447.87 leave £37,700.01, a penny over the basic rate limit. The pay + takes £8,999.58 of them first (s. 25(3A)), and the other £9,448.29 falls on + savings and dividends, so a penny of them stays above the limit under + every share and the law gives no reduction. Setting the £9,448.29 against + savings leaves £34,003.29 of them: £28,503.29 above the £5,000 starting + rate band and the £500 savings allowance at the model's 24%, £6,840.79. + Of the dividends, £3,196.70 above the £500 allowance bear 10.75% and the + last penny 35.75%: £343.65. That comes to £7,184.44. The model holds + income 0.3p over the limit. The float32 value a step above the deduction + from savings passes the float32 test alone. It is not the nearest value, + so it ranks last, and the model denies the reduction, as ``main`` does.""" + year = 2027 + sim = explicit_couples( + year, + [ + dict( + employment_income=8_999.58, + savings_interest_income=43_451.58, + dividend_income=3_696.72, + other_deductions=5_877.87, + ) + ], + ) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert not meets.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([7_184.44], abs=0.01) + + +@pytest.mark.parametrize( + "year, row, expected", + [ + ( + 2027, + dict( + employment_income=19_149.97, + savings_interest_income=218_144.28, + dividend_income=0.29, + other_deductions=197_220.30, + gift_aid=1_899.40, + ), + 7_925.75, + ), + ( + 2028, + dict( + employment_income=43_314.39, + savings_interest_income=212_622.26, + dividend_income=1_182.93, + other_deductions=219_419.59, + ), + 7_145.51, + ), + ], +) +def test_search_keeps_the_nearest_rounding_where_net_income_rounds_up( + year, row, expected +): + """Scottish recipients whose other deductions exceed their pay, so from + 2027-28 the deductions take all the pay first (s. 25(3A)) and the rest + falls on savings. The savings basic rate is the model's 24% from 2027-28, + which adds Budget 2026's announced rise to the 22% Parliament has set. In + 2028-29: savings of £212,622.26 less £176,105.20 + leave £36,517.06, and with £1,182.93 of dividends income comes to + £37,699.99, a penny inside the £37,700 basic rate limit. The tax is + £30,517.06 × 24% on savings above the £5,000 starting rate band and the + £1,000 savings allowance, plus £682.93 × 10.75% on dividends above the + £500 dividend allowance: £7,397.51, or £7,145.51 after the £252 + reduction. In 2027-28 a £1,899.40 gift raises the limit to £40,074.25 and + income comes to £40,074.24: £34,073.95 × 24% = £8,177.75, or £7,925.75. + The model sums net income in float32 0.69p high in 2027-28 and 1.375p + high in 2028-29, which puts income at the limit and shrinks the shared + amount. Only the nearest float32 rounding of the deduction from savings + meets the rate test, and against the model's shared amount it sets 0.7p + more. As the nearest rounding, it keeps its rank.""" + sim = explicit_couples(year, [dict(row, region="SCOTLAND")]) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert meets.all() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([expected], abs=0.01) + + +def test_search_keeps_the_nearest_rounding_that_puts_income_on_the_limit(): + """A 2026-27 recipient with no non-savings income: savings £3,000.01, + dividends £175,700, other deductions £140,000 and a trading loss of + £1,000.01. The Personal Allowance tapers to nil, and the whole £141,000.01 + of shared deductions goes against the dividends, since the savings sit in + the starting rate band for savings at 0%. That leaves Step 3 income of + £37,700, at the basic rate limit, so the share meets the rate test. The + tax is £34,199.99 × 10.75% on dividends above the £500 dividend + allowance, £3,676.50, or £3,424.50 after the £252 reduction. The nearest + float32 value of the deduction from dividends sets 0.56p more than the + model's shared amount, and the rounding down leaves income over the + limit. The nearest rounding keeps its rank, so the model gives the + reduction.""" + year = 2026 + sim = explicit_couples( + year, + [ + dict( + savings_interest_income=3_000.01, + dividend_income=175_700, + other_deductions=140_000, + trading_loss=1_000.01, + uses_trading_allowance=False, + ) + ], + ) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert meets.all() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([3_424.50], abs=0.01) + + +def test_search_stores_the_nearest_rounding_of_a_deduction_off_the_float32_grid(): + """Two 2026-27 recipients with no non-savings income and shared deductions + off the float32 grid. The first: savings £3,000, dividends £400,000, other + deductions £300,000 and a trading loss of £1,000.02. The whole £301,000.02 + goes against the dividends, and float32 storage rounds it to + £301,000.03125, 1.1p more. The model keeps that nearest value rather than + the rounding down, which would tax 2p more of the dividends at 35.75%. + The law's tax is £34,200 × 10.75% plus £64,299.98 × 35.75%: £26,663.74. + + The second has other deductions of £99,000 and a loss of £1,000.30. The + nearest float32 value of the £100,000.30 is £100,000.296875, 0.3p under. + The value above, £100,000.3046875, sets 0.47p more than the deductions, + within ``OVER_DEDUCTION``, and would save 0.3p of tax. With dividends over + £65,536, the search stores the nearest value for a recipient, as ``main`` + does. The law's tax is £34,200 × 10.75%, £87,440 × 35.75% and £177,859.70 + × 39.35%: £104,924.09.""" + year = 2026 + common = dict( + savings_interest_income=3_000, + dividend_income=400_000, + uses_trading_allowance=False, + ) + sim = explicit_couples( + year, + [ + dict(other_deductions=300_000, trading_loss=1_000.02, **common), + dict(other_deductions=99_000, trading_loss=1_000.30, **common), + ], + ) + to_dividends = sim.calculate("received_allowances_dividend_income", year)[0::2] + assert to_dividends.tolist() == [301_000.03125, 100_000.296875] + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([26_663.74, 104_924.09], abs=0.01) + + +@pytest.mark.parametrize( + "amounts, low_precision", + [ + # Every amount under £65,536. + (dict(net_income=65_535.99), False), + (dict(net_income=50_770.0, relief=14_765.99), False), + (dict(net_income=40_000.0, gift=27_835.5), False), + # Net income, and net income before the trade loss relief. + (dict(net_income=65_536.0), True), + (dict(net_income=50_770.0, relief=26_859.03), True), + # The allowances, savings, dividends and an exclusion from earned income. + (dict(net_income=40_000.0, allowances=65_536.0), True), + (dict(net_income=40_000.0, savings=65_536.0), True), + (dict(net_income=40_000.0, dividends=65_536.0), True), + (dict(net_income=40_000.0, earned_exclusions=65_536.0), True), + # A gift that raises the basic rate limit to £65,536. + (dict(net_income=40_000.0, gift=27_836.0), True), + ], +) +def test_rate_test_keeps_room_where_an_amount_reaches_65_536( + system, amounts, low_precision +): + """From £65,536 float32 holds amounts in steps of 0.78p or more, and can + hold income entered a penny over a limit at the limit. So where any amount + the rate test reads reaches £65,536, the test's limits sit below the + statutory ones by the room ``main``'s search keeps: £0.001 plus a millionth + of the limit or of the income after the deductions, whichever is more. + Under £65,536 the limits are the statutory ones (plus ``TIE``).""" + assert LOW_PRECISION == 65_536 + income_tax = system.parameters("2026").gov.hmrc.income_tax + inputs, default = one_person(**amounts) + inputs = with_marriage_allowance(income_tax, 2026, inputs, default, 0) + c = _build(inputs, income_tax) + assert c.low_precision[0, 0] == low_precision + limit = 37_700 + inputs["gift"][0] + room = RATE_TEST_ROOM + RATE_TEST_ROOM_FRACTION * max( + limit, abs(c.step_3_income[0, 0]) + ) + expected = limit - room if low_precision else limit + TIE + assert c.rate_test_limits.non_savings[0, 0] == pytest.approx(expected, abs=1e-9) + assert c.rate_test_limits.basic_rate[0, 0] == pytest.approx(expected, abs=1e-9) + + +def test_rate_test_denies_a_penny_over_the_limit_that_float32_cannot_hold(): + """A 2026-27 recipient with pay of £190,529.99, savings of £9,000.02, + dividends of £1,570, a trading loss of £50,275 and a Gift Aid gift of + £80,044 (#2913, review of 6788c8e39). The loss fits the cap of s. 24A, a + quarter of £201,100.01. The gift grosses up to £100,055, so adjusted net + income is £50,770.01, the Personal Allowance stays £12,570, and the basic + rate limit rises to £137,755 (ss. 58 and 414(2)(b)). Income after the + allowance is £138,255.01, a penny over that limit plus the £500 savings + allowance, so no share meets the rate test. The least tax leaves £500 of + savings and £500 of dividends at their nil rates and £137,255.01 of pay at + 20%: £27,451.00. + + Float32 holds pay of £190,529.99 as £190,529.984375, and the model sums + net income to £150,825, a penny under. So the model holds income of + £138,255 after the allowance, the limit plus the savings allowance, and a + search that let income reach the limit gave this recipient the £252 + reduction. Pay of £190,529.98 stores as the same float32 value. For that + pay the law gives the reduction and charges £27,299. The model cannot + tell the two apart, and denies both, as ``main`` does.""" + year = 2026 + common = dict( + savings_interest_income=9_000.02, + dividend_income=1_570, + trading_loss=50_275, + gift_aid=80_044, + ) + sim = explicit_couples( + year, + [ + dict(employment_income=190_529.99, **common), + dict(employment_income=190_529.98, **common), + ], + ) + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert not meets.any() and not reduction.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([27_451.00, 27_451.00], abs=0.005) + + +# Whole-pence recipients in 2026-27, each a penny over a limit of the rate test +# under every share, with an amount of £65,536 or more. Float32 hides the penny +# and the model holds each at the limit. The law gives none of them the +# reduction, and neither does ``main``. Each row ends with the law's tax, to +# the penny. +PENNY_OVER_AT_LOW_PRECISION = [ + # Net income of £201,100.07 less £162,900.06 of other deductions leaves + # £38,200.01, a penny over £37,700 plus the £500 savings allowance: £500 of + # savings and of dividends at their nil rates, £37,200.01 of pay at 20%. + ( + dict( + employment_income=200_000.07, + savings_interest_income=500, + dividend_income=600, + other_deductions=162_900.06, + ), + 7_440.00, + ), + # As above, 2p over: £37,200.02 of pay at 20%. + ( + dict( + employment_income=300_000.14, + savings_interest_income=500, + dividend_income=600, + other_deductions=262_900.12, + ), + 7_440.00, + ), + # Pay over £131,072, and net income under it after a £30,000 trade loss: + # £120,825.01. A £56,044 gift raises the limit to £107,755, and income + # after the Personal Allowance is £108,255.01: £107,255.01 of pay at 20%. + ( + dict( + employment_income=140_254.99, + savings_interest_income=9_000.02, + dividend_income=1_570, + trading_loss=30_000, + gift_aid=56_044, + ), + 21_451.00, + ), + # Every amount under £65,536 but their sum: pay, savings and dividends + # come to £77,629.04, and a £26,859.03 trade loss leaves net income of + # £50,770.01, which the model holds as £50,770. Income after the Personal + # Allowance is £38,200.01: £37,200.01 of pay at 20%. + ( + dict( + employment_income=53_360.11, + savings_interest_income=23_554.02, + dividend_income=714.91, + trading_loss=26_859.03, + ), + 7_440.00, + ), + # A £78,284.20 gift raises the basic rate limit to £135,555.25. Net income + # of £148,625.26 less the Personal Allowance is £136,055.26, a penny over + # the limit plus £500: £135,055.26 of pay at 20%, £27,011.05. + ( + dict( + employment_income=140_426.63, + savings_interest_income=7_683.60, + dividend_income=515.03, + gift_aid=78_284.20, + ), + 27_011.05, + ), + # A Scottish taxpayer. A £70,206.24 gift raises the Scottish higher rate + # threshold to £118,849.80 and the intermediate rate threshold to + # £104,713.80. Pay less the Personal Allowance and a £2,721.20 trade loss + # is £118,849.81, so a penny bears the higher rate: £3,967 at 19%, + # £100,746.80 at 20%, £14,136 at 21% and 1p at 42%, £23,871.65. The savings + # and dividends fall within their nil rates. + ( + dict( + employment_income=134_141.01, + savings_interest_income=465.02, + dividend_income=26.60, + trading_loss=2_721.20, + gift_aid=70_206.24, + region="SCOTLAND", + ), + 23_871.65, + ), + # Dividends that the £66,323.33 of deductions cannot clear. The deductions + # take all £21,095.11 of pay and £45,228.22 of the dividends, and leave + # £37,700.01 of them: £37,200 above the £500 allowance at 10.75% and a + # penny at 35.75%, £3,999.00. The old order's deductions, as float32 holds + # them, leave the dividends 0.39p over the limit, and the float32 value a + # step above them leaves the dividends at it. A search that moved the + # share by that step, for the 0.1p of tax it saves, won the reduction. + ( + dict( + employment_income=21_095.11, + dividend_income=82_928.23, + other_deductions=20_605.56, + trading_loss=33_147.77, + uses_trading_allowance=False, + ), + 3_999.00, + ), + # A penny of savings over the savings nil rate band under every share + # (pay × 20% plus 1p × 40%). A £56,111.20 gift raises the limit to + # £107,839, above the pay. + ( + dict( + employment_income=107_739.11, + savings_interest_income=46_884.52, + other_deductions=33_687.92, + trading_loss=126.59, + gift_aid=56_111.20, + uses_trading_allowance=False, + ), + 21_547.83, + ), +] + + +def test_rate_test_denies_a_penny_over_a_limit_at_low_precision(): + """Each recipient above sits a penny (the second, 2p) over a limit, and + the model holds each at the limit. With an amount of £65,536 or more, the + rate test keeps room below each limit, and the search decides and stores + the share as ``main`` does, so the model gives none of them the reduction + and charges the law's tax.""" + year = 2026 + rows = [row for row, _ in PENNY_OVER_AT_LOW_PRECISION] + sim = explicit_couples(year, rows) + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert not meets.any() and not reduction.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + law_tax = [amount for _, amount in PENNY_OVER_AT_LOW_PRECISION] + assert tax == pytest.approx(law_tax, abs=0.01) + + +def test_rate_test_denies_a_penny_over_a_limit_at_low_precision_from_2027(): + """A 2027-28 recipient with pay of £13,251.08, property income of + £43,099.65, savings of £31,948.22, dividends of £540.48 and other + deductions of £38,069.42. Net income is £88,839.43, and allowances of + £50,639.42 leave £38,200.01, a penny over the basic rate limit plus the + £500 savings allowance, so no share meets the rate test. The pay takes + £13,251.08 of the allowances first (s. 25(3A)). The least tax leaves £500 + of savings and £500 of dividends at their nil rates and £37,200.01 of + property income at the model's 24%: £8,928.00. The model sums net income + to £88,839.421875, 0.8p under, and holds income at the limit plus £500. + Net income is over £65,536, so the rate test keeps room and the model + denies the reduction, as ``main`` does.""" + year = 2027 + sim = explicit_couples( + year, + [ + dict( + employment_income=13_251.08, + property_income=43_099.65, + savings_interest_income=31_948.22, + dividend_income=540.48, + other_deductions=38_069.42, + ) + ], + ) + assert not sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + assert sim.calculate("marriage_allowance_tax_reduction", year)[0] == 0 + tax = sim.calculate("income_tax", year).astype(float)[0] + assert tax == pytest.approx(8_928.00, abs=0.01) + + +def test_search_stores_the_nearest_rounding_at_low_precision(): + """A 2027-28 recipient with pay of £53,841.65, savings of £141,016.82, + dividends of £17,661.55, other deductions of £174,818.25 and a trading + loss of £1.77. The Personal Allowance tapers to nil, and Step 3 income is + £37,700, the basic rate limit, so the law gives the reduction and charges + £4,962.09. The deductions take all the pay first (s. 25(3A)), and with no + property income the other £120,978.37 falls on savings and dividends. The + least tax sets it against savings. The model holds that amount midway + between two float32 values. The nearest, + £120,978.359375, leaves income 0.39p over the limit. The other sets 0.39p + more than the shared amount and puts income on the limit. Savings and + deductions sit over £65,536, so the rate test keeps room, the search + stores the nearest value and the model denies the reduction, as ``main`` + does: £14,038.46 of savings above + the starting rate band and the savings allowance at 24%, £3,369.23, and + £17,161.55 of dividends above their allowance at 10.75%, £1,844.87.""" + year = 2027 + sim = explicit_couples( + year, + [ + dict( + employment_income=53_841.65, + savings_interest_income=141_016.82, + dividend_income=17_661.55, + other_deductions=174_818.25, + trading_loss=1.77, + uses_trading_allowance=False, + ) + ], + ) + assert not sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + to_savings = sim.calculate("received_allowances_savings_income", year)[0] + assert to_savings == 120_978.359375 + tax = sim.calculate("income_tax", year).astype(float)[0] + assert tax == pytest.approx(5_214.10, abs=0.01) + + +def test_search_finds_the_share_at_the_limit_less_the_room(): + """A Scottish recipient in 2027-28 with pay of £41,941.71, property income + of £33,570.74, savings of £21,500, other deductions of £33,208.09 and a + trading loss of £19,142.37. Net income of £77,870.08 less allowances of + £45,778.09 leaves £32,091.99. The deductions take all the pay first + (s. 25(3A)), and the other £22,978.75 falls on property income and + savings. Savings above the £1,000 savings allowance bear the model's 24%, + and property income under the Scottish higher rate threshold of £31,092 + bears 21% at most. So the least tax leaves as much property income as the + rate test allows. Net income is over £65,536, so the test keeps 3.9p of + room under the threshold. The share that leaves £31,091.96 of property + income and £1,000.03 of savings lies on the line the search adds at the + limit less the room, and it sets £20,499.96 against savings. Without that + line the search sets £20,000 against savings, which leaves £500 more of + savings at 24% and £500 less of property income at 21%, and the tax is + £15 more. The test pins the share and not the tax, since the model + projects the Scottish starter and basic rate thresholds for 2027-28.""" + year = 2027 + sim = explicit_couples( + year, + [ + dict( + employment_income=41_941.71, + property_income=33_570.74, + savings_interest_income=21_500, + other_deductions=33_208.09, + trading_loss=19_142.37, + uses_trading_allowance=False, + region="SCOTLAND", + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + to_savings = sim.calculate("received_allowances_savings_income", year)[0] + assert to_savings == 20_499.96484375 + property_left = sim.calculate("non_savings_income_after_allowances", year)[0] + assert property_left == pytest.approx(31_092 - 0.0387, abs=0.01) + + +# Whole-pence recipients in 2026-27, each a penny over a limit of the rate test +# under every share (the last three, a quarter of a penny). The penny hides +# behind an amount the rate test does not read: every amount the test reads +# sits under £65,536, and the model holds each recipient at the limit. The law +# gives none of them the reduction, and neither does ``main``. Each row ends +# with the law's tax, to the penny. +PENNY_HIDDEN_BEHIND_THE_INCOME_TESTED = [ + # Pay less expenses of £40,200.01. With £9,000 of savings and £1,570 of + # dividends, the Personal Allowance leaves £38,200.01, a penny over £37,700 + # plus the £500 savings allowance: £500 of savings and of dividends at + # their nil rates and £37,200.01 at 20%. Float32 holds pay of £148,609.46 + # and of £148,609.45 as one value. + ( + dict( + employment_income=148_609.46, + employment_expenses=108_409.45, + savings_interest_income=9_000, + dividend_income=1_570, + ), + 7_440.00, + ), + ( + dict( + employment_income=300_000.01, + employment_expenses=259_800, + savings_interest_income=9_000, + dividend_income=1_570, + ), + 7_440.00, + ), + # Pay less expenses of £50,000.01, with £500 of savings and £270 of + # dividends at their nil rates: £37,430.01 at 20%. + ( + dict( + employment_income=300_000.01, + employment_expenses=250_000, + savings_interest_income=500, + dividend_income=270, + ), + 7_486.00, + ), + # An £800 gift raises the limit to £38,700, and £41,200.01 of pay less + # expenses leaves £39,200.01: £38,200.01 at 20%. + ( + dict( + employment_income=300_000.01, + employment_expenses=258_800, + savings_interest_income=9_000, + dividend_income=1_570, + gift_aid=800, + ), + 7_640.00, + ), + # Trading profits less capital allowances of £40,200.01. + ( + dict( + self_employment_income=224_417.13, + capital_allowances=184_217.12, + savings_interest_income=9_000, + dividend_income=1_570, + ), + 7_440.00, + ), + # Dividends less deficiency relief of £1,570.01, with pay of £40,200. + ( + dict( + employment_income=40_200, + savings_interest_income=9_000, + dividend_income=201_570.01, + deficiency_relief=200_000, + ), + 7_440.00, + ), + # A £9,995.19 gift grosses up to £12,493.9875 and raises the limit to + # £50,193.9875. The Personal Allowance leaves £50,693.99, a quarter of a + # penny over that limit plus £500: £49,693.99 at 20%. Every amount sits + # under £65,536, and float32 sums net income to £63,263.988, 0.2p under. + ( + dict( + employment_income=52_693.99, + savings_interest_income=9_000, + dividend_income=1_570, + gift_aid=9_995.19, + ), + 9_938.80, + ), + # An £8,609.19 gift raises the limit to £48,461.4875, and the allowance + # leaves £48,961.49, a quarter of a penny over it plus £500. + ( + dict( + employment_income=50_961.49, + savings_interest_income=9_000, + dividend_income=1_570, + gift_aid=8_609.19, + ), + 9_592.30, + ), + # A £100.03 gift raises the limit to £37,825.0375. Pay of £120,000 less + # expenses of £79,675.02 leaves £40,324.98, and the allowance leaves + # £38,325.04, a quarter of a penny over that limit plus £500: £37,325.04 + # at 20%. Float32 holds the pay exactly and the expenses 0.34p over, so + # the bound comes from an amount the model takes off. + ( + dict( + employment_income=120_000, + employment_expenses=79_675.02, + savings_interest_income=9_000.06, + dividend_income=1_570, + gift_aid=100.03, + ), + 7_465.01, + ), +] + +# 2026-27 recipients the model holds at the limit, for whom the amounts behind +# the income tested give no bound under a penny, so the rate test keeps room +# and the model denies the reduction, as ``main`` does. Each has £9,000 of +# savings and £1,570 of dividends and is charged £7,440. +KEEPS_ROOM_AT_THE_LIMIT = [ + # Pay of £148,609.45 less expenses of £108,409.45 is £40,200, at the limit, + # so the law gives the reduction. Float32 holds that pay as it holds + # £148,609.46, a penny over. + dict(employment_income=148_609.45, employment_expenses=108_409.45), + # Dividends of £201,570 less deficiency relief of £200,000, at the limit. + # Both are £131,072 or more, where float32 can store two entries a penny + # apart as one, so the entries get no bound. + dict(employment_income=40_200, dividend_income=201_570, deficiency_relief=200_000), +] + +# 2026-27 recipients entered at the limit, with amounts behind the income +# tested that float32 holds to well under a penny. The law gives each the +# reduction, and so does the model; ``main`` denies each. Each row ends with +# the law's tax. +REACHES_THE_LIMIT = [ + # Pay less expenses of £40,200: 37,700 x 20% less £252. + (dict(employment_income=45_200, employment_expenses=5_000), 7_288.00), + # Float32 holds £100,000.37 0.28p under and £59,800.37 0.11p over, so + # taxable pay 0.39p under £40,200. The bound is 0.39p. + (dict(employment_income=100_000.37, employment_expenses=59_800.37), 7_288.00), + # Pension contributions of £5,000 under net pay. + ( + dict(employment_income=45_200, employee_pension_contributions_reported=5_000), + 7_288.00, + ), + # A trading loss of £5,000 against pay of £45,200. + ( + dict( + employment_income=45_200, trading_loss=5_000, uses_trading_allowance=False + ), + 7_288.00, + ), + # An £800 gift raises the limit to £38,700: 38,700 x 20% less £252. + (dict(employment_income=41_200, gift_aid=800), 7_488.00), +] + + +def test_rate_test_reads_the_amounts_behind_the_income_it_tests(): + """Float32 can hide a penny behind amounts the rate test never reads. Pay + of £148,609.46 less £108,409.45 of expenses leaves £40,200.01, but float32 + holds that pay as £148,609.453125 and the model holds £40,200.00. The + amounts the test reads all sit under £65,536, so ``_low_precision`` sees + nothing. ``_can_hide_an_excess`` bounds the float32 error from the inputs + up, and the test keeps room wherever the bound reaches the least amount + the entries can sit over a limit: a penny, or a quarter of a penny at a + limit a Gift Aid gift leaves between two pence. The model then denies the + reduction to each recipient entered over a limit, and charges the law's + tax. It also denies it to recipients it cannot tell from those, as + ``main`` does. It still gives the reduction at the limit where the bound + stays under a penny.""" + year = 2026 + over = [row for row, _ in PENNY_HIDDEN_BEHIND_THE_INCOME_TESTED] + common = dict(savings_interest_income=9_000, dividend_income=1_570) + kept = [dict(common) | row for row in KEEPS_ROOM_AT_THE_LIMIT] + reaches = [dict(common, **row) for row, _ in REACHES_THE_LIMIT] + sim = explicit_couples(year, over + kept + reaches) + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + tax = sim.calculate("income_tax", year).astype(float)[0::2] + denied = len(over) + len(kept) + assert not meets[:denied].any() and not reduction[:denied].any() + law_tax = [amount for _, amount in PENNY_HIDDEN_BEHIND_THE_INCOME_TESTED] + assert tax[: len(over)] == pytest.approx(law_tax, abs=0.01) + assert tax[len(over) : denied] == pytest.approx(7_440.00, abs=0.01) + assert meets[denied:].all() + assert reduction[denied:] == pytest.approx(252) + assert tax[denied:] == pytest.approx( + [amount for _, amount in REACHES_THE_LIMIT], abs=0.01 + ) + # The search and the model's answer read the same flag. + period = periods.period(year) + income_tax = sim.tax_benefit_system.parameters(str(year)).gov.hmrc.income_tax + person = sim.populations["person"] + flagged = _inputs(person, period, income_tax)["can_hide_an_excess"][0::2] + assert flagged[:denied].all() and not flagged[denied:].any() + assert _context(person, period, sim.tax_benefit_system.parameters).low_precision[ + 0::2, 0 + ].tolist() == [True] * denied + [False] * len(reaches) + + +def test_held_error_follows_the_formulas_behind_the_rate_test(): + """``_held_error`` checks each amount the model holds against the float64 + sum of the amounts its formula reads (``SUMS``), so its bound stays small + only while those sums match the formulas. The first person below enters + every input the sums read, each in whole pence under £65,536, and the + bound on net income and on the allowances stays under a penny. The second + has pay of £131,072 or more, which gives no bound. The third has a company + car, whose benefit the function does not follow, the fourth a trading loss + over the £50,000 cap of s. 24A, and the fifth pension contributions over + the earnings that limit their relief.""" + year = 2026 + busy = dict( + employment_income=21_003.17, + employment_expenses=412.39, + statutory_sick_pay=318.44, + self_employment_income=9_871.03, + capital_allowances=611.27, + loss_relief=205.11, + private_pension_income=4_102.59, + property_income=3_950.41, + savings_interest_income=2_210.83, + other_investment_income=97.21, + dividend_income=1_733.69, + deficiency_relief=12.07, + trading_loss=1_044.63, + other_deductions=310.19, + covenanted_payments=55.43, + charitable_investment_gifts=120.77, + employee_pension_contributions_reported=1_250.33, + personal_pension_contributions=480.21, + gift_aid=163.24, + uses_trading_allowance=False, + ) + sim = explicit_couples( + year, + [ + busy, + dict(employment_income=131_072), + dict(employment_income=30_000, company_car_list_price=20_000), + dict( + employment_income=60_000, + trading_loss=50_000.01, + uses_trading_allowance=False, + ), + dict(employment_income=45_200, personal_pension_contributions=45_200.01), + ], + ) + period = periods.period(year) + income_tax = sim.tax_benefit_system.parameters(str(year)).gov.hmrc.income_tax + person = sim.populations["person"] + known = {} + net_income = _held_error(person, period, income_tax, "net_income", known)[0::2] + allowances = _held_error(person, period, income_tax, "allowances", known)[0::2] + assert sim.calculate("net_income", year)[0] == pytest.approx(41_001.90, abs=0.01) + assert sim.calculate("allowances", year)[0] == pytest.approx(14_786.93, abs=0.01) + assert net_income[0] < 0.01 and allowances[0] < 0.01 + # Each input's bound is its distance from whole pence. + pay = np.float64(np.float32(21_003.17)) + assert known["employment_income"][0] == pytest.approx(abs(pay - 21_003.17)) + assert np.isinf(net_income[1:4]).all() and np.isinf(allowances[4]) + limits = marriage_allowance_rate_limits(person, period, income_tax) + flagged = _can_hide_an_excess(person, period, income_tax, limits, np.zeros(10)) + assert flagged[0::2].tolist() == [False, True, True, True, True] + + +def test_rate_test_reads_an_amount_supplied_in_place_of_its_formula_as_an_entry(): + """Taxable pay of £40,200 supplied as an input, with £9,000 of savings and + £1,570 of dividends in 2026-27, sits at the limit. The model reads the + supplied amount in place of the formula, so it is an entry. + ``_held_error`` bounds it by its distance from whole pence, nil here, and + reads none of the pay, benefits and expenses the formula would. The rate + test keeps no room, and one share meets it: £2,500 of the £12,570 + Personal Allowance against pay, £8,500 against savings and £1,570 against + dividends. That leaves £37,700 of pay at 20%, £7,540, and £500 of savings + at the savings nil rate: £7,288 after the £252 reduction (ITA 2007 + ss. 10, 12B, 25 and 55B). ``main`` denies the reduction and charges + £7,440.""" + year = 2026 + sim = explicit_couples( + year, + [ + dict( + taxable_employment_income=40_200, + savings_interest_income=9_000, + dividend_income=1_570, + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0] + assert reduction == pytest.approx(252) + assert sim.calculate("income_tax", year)[0] == pytest.approx(7_288.00, abs=0.01) + period = periods.period(year) + income_tax = sim.tax_benefit_system.parameters(str(year)).gov.hmrc.income_tax + known = {} + bound = _held_error( + sim.populations["person"], period, income_tax, "net_income", known + ) + assert bound[0] == pytest.approx(0, abs=1e-9) + assert known["taxable_employment_income"][0] == pytest.approx(0, abs=1e-9) + # The function stopped at the supplied amount. + assert "employment_income" not in known + + +def held_errors(sim, year, name): + """``_held_error`` for ``name``, and the bounds it worked out on the way.""" + period = periods.period(year) + income_tax = sim.tax_benefit_system.parameters(str(year)).gov.hmrc.income_tax + known = {} + bound = _held_error(sim.populations["person"], period, income_tax, name, known) + return bound, known + + +def can_hide_an_excess(sim, year): + period = periods.period(year) + income_tax = sim.tax_benefit_system.parameters(str(year)).gov.hmrc.income_tax + return _inputs(sim.populations["person"], period, income_tax)["can_hide_an_excess"] + + +def test_rate_test_reads_pay_set_as_an_input_as_an_entry(): + """A married recipient with pay of £40,200, savings of £9,000 and dividends + of £1,570 in 2026-27, whose partner has no income. The constructor moves a + situation's pay into employment_income_before_lsr, and the model adds + employment_income up from it. ``Simulation.set_input`` stores the pay in + employment_income itself, as a YAML test does, with nothing behind it. + Either way the pay is an entry in whole pence, so the rate test keeps no + room. One share meets the test: £2,500 of the £12,570 Personal Allowance + against pay, £8,500 against savings and £1,570 against dividends. It + leaves £37,700 of pay at 20%, £7,540, and £500 of savings at the savings + nil rate: £7,288 after the £252 reduction (ITA 2007 ss. 10, 12B, 25 and + 55B). ``_held_error`` reads pay set as an input as it stands, and none of + the amounts employment_income adds up.""" + year = 2026 + others = dict(savings_interest_income=9_000, dividend_income=1_570) + built = explicit_couples(year, [dict(employment_income=40_200, **others)]) + set_after = explicit_couples(year, [others]) + set_after.set_input("employment_income", year, [40_200, 0]) + for sim, adds_pay_up in ((built, True), (set_after, False)): + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0] + assert reduction == pytest.approx(252) + tax = sim.calculate("income_tax", year)[0] + assert tax == pytest.approx(7_288.00, abs=0.01) + bound, known = held_errors(sim, year, "net_income") + assert bound[0] == pytest.approx(0, abs=1e-9) + assert ("employment_income_before_lsr" in known) == adds_pay_up + assert not can_hide_an_excess(sim, year).any() + + +def test_rate_test_keeps_room_for_pay_set_as_an_input_from_131_072(): + """Pay set through ``Simulation.set_input`` is an entry, and an entry of + £131,072 or more gets no bound: float32 holds pay of £148,609.45 and of + £148,609.46 both as £148,609.453125. Less £108,409.45 of expenses the + model holds £40,200 of taxable pay for both, at the limit, with £9,000 of + savings, £1,570 of dividends and net income of £50,770, under £65,536. The + law gives the first recipient the reduction and denies the second, who is + a penny over. The test keeps room for both, so the model denies both, as + ``main`` does, and charges £500 of savings and £500 of dividends at their + nil rates and £37,200 at 20%: £7,440 (£37,200.01 and £7,440.002 for the + second by the law).""" + year = 2026 + row = dict( + employment_expenses=108_409.45, + savings_interest_income=9_000, + dividend_income=1_570, + ) + sim = explicit_couples(year, [row, row]) + sim.set_input("employment_income", year, [148_609.45, 0, 148_609.46, 0]) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + assert not meets.any() and not reduction.any() + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([7_440.00, 7_440.00], abs=0.01) + # No amount the rate test reads reaches LOW_PRECISION. + assert (sim.calculate("net_income", year) < LOW_PRECISION).all() + bound, known = held_errors(sim, year, "net_income") + assert np.isinf(bound[0::2]).all() + assert np.isinf(known["employment_income"][0::2]).all() + assert "employment_income_before_lsr" not in known + assert can_hide_an_excess(sim, year)[0::2].all() + + +def test_rate_test_keeps_room_where_a_reform_replaces_a_formula(system): + """A reform takes £10,000 off pay in working out taxable pay. Pay of + £50,200 then leaves £40,200, at the limit, with £9,000 of savings and + £1,570 of dividends in 2026-27. ``_held_error`` checks taxable pay against + the pay, benefits and expenses the model's own formula reads (``SUMS``), + which come to £50,200, so £10,000 stays unexplained and the bound is far + over a penny. The rate test keeps room, and the model denies the reduction + and charges £7,440: £37,200 at 20%, with £500 of savings and £500 of + dividends at their nil rates. The reformed law gives the reduction and + £7,288, so the £152 is the cost of the room.""" + model = type(system.variables["taxable_employment_income"]) + + class taxable_employment_income(model): + def formula(person, period, parameters): + return np.maximum(0, person("employment_income", period) - 10_000) + + class take_10_000_off_pay(Reform): + def apply(self): + self.update_variable(taxable_employment_income) + + year = 2026 + sim = explicit_couples( + year, + [ + dict( + employment_income=50_200, + savings_interest_income=9_000, + dividend_income=1_570, + ) + ], + reform=take_10_000_off_pay, + ) + assert sim.calculate("taxable_employment_income", year)[0] == 40_200 + assert not sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + assert sim.calculate("marriage_allowance_tax_reduction", year)[0] == 0 + assert sim.calculate("income_tax", year)[0] == pytest.approx(7_440.00, abs=0.01) + bound, _ = held_errors(sim, year, "net_income") + assert bound[0] == pytest.approx(10_000) + assert can_hide_an_excess(sim, year)[0] + + +def test_rate_test_bounds_the_amounts_behind_a_supplied_net_income(): + """Net income of £50,770 supplied as an input is an entry, and + ``_held_error`` stops there. The rate test also reads savings, dividends, + property income, the trade loss relief and pension relief on their own, so + ``_can_hide_an_excess`` bounds each one the walk did not reach. Both + recipients have pay of £40,200 and £9,000 of savings in 2026-27, and + dividends of £1,570, at the limit. The first has them as £201,570 less + £200,000 of deficiency relief. Each of those is £131,072 or more and gets + no bound, so the test keeps room, and the model denies the reduction and + charges £7,440, as it does without the supplied net income + (``KEEPS_ROOM_AT_THE_LIMIT``). The second gets the reduction: £7,288. The + law gives both the reduction and £7,288, so the first's £152 is the cost + of the room.""" + year = 2026 + common = dict( + net_income=50_770, employment_income=40_200, savings_interest_income=9_000 + ) + sim = explicit_couples( + year, + [ + dict(common, dividend_income=201_570, deficiency_relief=200_000), + dict(common, dividend_income=1_570), + ], + ) + meets = sim.calculate("meets_marriage_allowance_income_conditions", year)[0::2] + assert meets.tolist() == [False, True] + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0::2] + assert reduction == pytest.approx([0, 252]) + tax = sim.calculate("income_tax", year).astype(float)[0::2] + assert tax == pytest.approx([7_440.00, 7_288.00], abs=0.01) + bound, known = held_errors(sim, year, "net_income") + assert bound[0::2] == pytest.approx([0, 0], abs=1e-9) + assert "taxable_dividend_income" not in known + assert can_hide_an_excess(sim, year)[0::2].tolist() == [True, False] + + +def test_rate_test_reads_a_gross_gift_supplied_as_an_input_as_an_entry(): + """A grossed-up gift of £1,000 supplied as an input raises the basic rate + limit to £38,700 (s. 414(2)(b)), as an £800 gift does + (``REACHES_THE_LIMIT``). With pay of £41,200, £9,000 of savings and £1,570 + of dividends in 2026-27, the Personal Allowance leaves £39,200, the limit + plus the £500 savings allowance. The one share that meets the test leaves + £38,700 of pay at 20%, £7,740: £7,488 after the £252 reduction. The gross + gift is an entry in whole pence, so the rate test keeps no room, though + no gift stands behind it.""" + year = 2026 + sim = explicit_couples( + year, + [ + dict( + employment_income=41_200, + savings_interest_income=9_000, + dividend_income=1_570, + gift_aid_grossed_up=1_000, + ) + ], + ) + assert sim.calculate("meets_marriage_allowance_income_conditions", year)[0] + reduction = sim.calculate("marriage_allowance_tax_reduction", year)[0] + assert reduction == pytest.approx(252) + assert sim.calculate("income_tax", year)[0] == pytest.approx(7_488.00, abs=0.01) + assert not can_hide_an_excess(sim, year).any() + + +def test_held_error_gives_a_value_the_model_stored_no_bound(): + """A scenario stores an amount it calculated with ``set_model_value``, + which the engine keeps like an input. It is not an entry: for a trading + allowance deduction of £500 stored that way, ``_held_error`` follows the + variable's own rule, and a formula it does not list gets no bound. The + same £500 set through ``Simulation.set_input`` is an entry in whole + pence, with a nil bound.""" + year = 2026 + sim = explicit_couples(year, [dict(employment_income=40_200)]) + name = "trading_allowance_deduction" + set_model_value(sim, name, year, np.array([500.0, 0.0])) + bound, _ = held_errors(sim, year, name) + assert np.isinf(bound[0]) and bound[1] == 0 + sim.set_input(name, year, np.array([500.0, 0.0])) + bound, _ = held_errors(sim, year, name) + assert bound.tolist() == [0, 0] diff --git a/policyengine_uk/tests/test_marriage_allowance_least_liability_properties.py b/policyengine_uk/tests/test_marriage_allowance_least_liability_properties.py new file mode 100644 index 0000000000..1a5b7220a2 --- /dev/null +++ b/policyengine_uk/tests/test_marriage_allowance_least_liability_properties.py @@ -0,0 +1,460 @@ +"""The least liability under the Marriage Allowance rate test, against the law. + +A married recipient in London whose partner has no income. The oracle works +the law itself (ITA 2007 ss. 10-13A, 12B, 16, 23, 25(2)/(3A), 55B), reading only +rate and allowance values from the parameter tree. The model holds incomes in +float32 and sums them in float32, so the income it tests can sit either side +of the amounts drawn. For a recipient within reach of the rate test, whose +incomes come to about £50,770, the gap is under a penny. It grows with income: +the strategies draw up to £80,000 a component, and a draw of £263,789.16 in +all holds net income 3.5p lower. So the oracle applies the rate test twice: +strictly, with the basic rate limit a penny lower, and loosely, with it a +penny higher. At the model's own share (check 1), both tests allow half a +penny at the savings nil rate band and for dividends. Over the other shares +(check 2), the strict test also asks for the savings above the limit to sit a +penny inside the savings nil rate band. For every drawn recipient: + +1. at the model's own share, the model's income tax equals the oracle's Step 5 + tax less the reduction where the model gives it; the model gives it wherever + the strict test passes and Step 5 leaves tax, and nowhere the loose test + fails; +2. the model's income tax is no more than the least liability under the + strict test over every share on a grid, the thin sets where the rate test + holds and the shares at the limit, and no more than the old order's. + +Check 1 bounds the tax from below: it is the law's liability at a share the +model holds. Between the strict and loose tests the law itself lets income +reach each limit (s. 10(2)). + +Check 1 first asks that the model's share is one the law allows: each +deduction within its component. The model works the share from the amounts it +holds, so a deduction can pass its component by the float32 error in those +amounts, which ``feasibility_margin`` bounds. That bound is separate from the +half penny the rate test allows, and reaches a few pence for the largest +draws. + +From £65,536 the model holds a recipient's rate test to room below each limit +(``LOW_PRECISION`` in ``income_tax_allocation``). The only deduction here is +the Personal Allowance, so a recipient with that much income has Step 3 income +over £52,966 and fails the rate test under either margin. The model also keeps +the room where its bound on the float32 error in a recipient's amounts reaches +a penny (``_can_hide_an_excess``). A recipient here has at most four amounts, +set through ``Simulation.set_input``, which the bound reads as entries. +Under £65,536 in all, their float32 errors come to under 0.39p and the three +float32 sums to under 0.59p, so the bound stays under a penny. The room never +decides a recipient these strategies draw. + +Both checks leave out a recipient whose float32 net income sits on the other +side of an allowance step from the amounts entered. The Personal Allowance +falls a pound at a time (s. 35(2) and (3)), and the savings allowance steps +down at the basic and higher rate limits (s. 12B), so the model then works a +different allowance from the law's. + +Check 2's strict test stops a penny inside the band because the search can +miss a share that meets the rate test only with savings at the top of the +band. Float32 can hold the savings and dividends a fraction of a penny over +the amounts entered. The share the search scores then leaves the savings that +much over the band, and it fails the test. A float32 rounding of that share's +deductions would put the savings back on the band, but the search rounds only +the share it scores lowest, which is then another. ``main`` misses the same +shares. So check 2 requires the reduction only where a share leaves the +savings a penny inside the band, as it does only where one leaves pay a penny +under the limit. + +In the first three explicit examples one share qualifies, and it leaves pay at +the basic rate limit, where only the loose test passes, so neither check +requires the reduction. In the fourth, the qualifying shares leave pay from 2p +under the limit up to it, and savings from the top of the band down to 2p +inside it. The strict test passes the one that leaves pay a penny under the +limit and savings a penny inside the band, so check 2 fails a model that +denies the reduction, which ``main`` does. In the last three, one share +qualifies: it sets nothing against the £37,500 of pay and leaves £500 of +savings, the whole band. The model holds their savings and dividends 0.024p, +0.024p and 0.049p over the £13,070 entered, and it denies all three the +reduction, as ``main`` does. The strict test fails that share, so check 2 +does not require the reduction. + +The last two examples, in 2027-28, pin the feasibility bound. In the first, +pay takes £8,542.77 of the Personal Allowance and savings the other £4,027.23. +The model sums net income of £76,085.96 to £76,085.953125, reads pay 0.83p low +from it, and sets £4,027.2383 against savings. In the second, with pay of +£11,743.36, savings of £6,860.08 and dividends of £23,171.51, the model sets +£826.6348 against savings, 0.52p under the £826.64 left. Neither has property +income, so the oracle reads each difference as a deduction from property +income that is not there. Each is more than the half penny the rate test +allows, and within the float32 error ``feasibility_margin`` bounds. +""" + +import numpy as np +import pytest +from hypothesis import HealthCheck, example, given, settings +from hypothesis import strategies as st + +from policyengine_uk import Simulation + +K = 8 +SIMS = {} +# The rate test's margin either side of each limit (see the module docstring). +PENNY = 0.01 + + +@pytest.fixture(scope="module", autouse=True) +def release_simulations(): + yield + SIMS.clear() + + +def sim_for(year): + if year not in SIMS: + people, benunits, households = {}, {}, {} + for i in range(K): + people[f"r{i}"] = {"age": {year: 40}} + people[f"d{i}"] = {"age": {year: 40}} + benunits[f"b{i}"] = { + "members": [f"r{i}", f"d{i}"], + "is_married": {year: True}, + } + households[f"h{i}"] = { + "members": [f"r{i}", f"d{i}"], + "region": {year: "LONDON"}, + } + SIMS[year] = Simulation( + situation={"people": people, "benunits": benunits, "households": households} + ) + return SIMS[year] + + +def scale_tax(thresholds, rates, income): + tax = np.zeros_like(income) + bounds = list(thresholds) + [np.inf] + for i, rate in enumerate(rates): + tax += rate * np.clip(income - bounds[i], 0, bounds[i + 1] - bounds[i]) + return tax + + +def net_income(pay, prop, sav, div): + # The amounts are whole pence, and so is their sum. Rounding it keeps + # float64 addition from carrying a sum at a limit over it. + return np.round(pay + prop + sav + div, 2) + + +def float32_step(amount): + """The gap between ``amount`` and the next float32 value.""" + return np.spacing(np.abs(amount).astype(np.float32)).astype(float) + + +def feasibility_margin(pay, prop, sav, div, to_s, to_v): + """A bound on how far float32 can carry the model's share past a component + of the amounts drawn. The model holds each component within half a float32 + step of the amount drawn. It sums up to four of them into net income (three + additions), then takes savings and dividends back off for the non-savings + income the old order reads (two subtractions): each can round by half a + step of net income. The old order then takes that from the allowances, at + most a step of them. A moved share stores each deduction rounded down or + up, at most a step of the deduction. 1e-6 covers float64 rounding.""" + n = pay + prop + sav + div + components = sum(float32_step(amount) for amount in (pay, prop, sav, div)) / 2 + net_income_and_old_order = 5 * float32_step(n) / 2 + allowances = float32_step(np.minimum(n, 12_570.0)) + stored = float32_step(to_s) + float32_step(to_v) + return components + net_income_and_old_order + allowances + stored + 1e-6 + + +def banded(lo, hi, B, H, r): + """Tax at the basic, higher and additional rates ``r`` on income from lo to hi.""" + + def seg(a, b): + return np.maximum(0, np.minimum(hi, b) - np.maximum(lo, a)) + + return r[0] * seg(-np.inf, B) + r[1] * seg(B, H) + r[2] * seg(H, np.inf) + + +class Law: + def __init__(self, year): + p = ( + sim_for(year) + .tax_benefit_system.parameters(f"{year}-06-01") + .gov.hmrc.income_tax + ) + rates, allow = p.rates, p.allowances + self.year = year + self.other_first = year >= 2027 # FA 2026 s. 6(8) + self.uk = (list(rates.uk.thresholds), list(rates.uk.rates)) + self.B, self.H = rates.uk.thresholds[1], rates.uk.thresholds[2] + self.div = (list(rates.dividends.thresholds), list(rates.dividends.rates)) + self.sav = (rates.savings.basic, rates.savings.higher, rates.savings.additional) + self.prop = ( + rates.property.basic, + rates.property.higher, + rates.property.additional, + ) + self.L = rates.savings_starter_rate.allowance + self.psa = allow.personal_savings_allowance + self.DA = allow.dividend_allowance + self.PA0 = allow.personal_allowance.amount + # s. 55B(3)-(5): basic rate of 10% of the s. 35(1) amount, rounded up to £10. + self.ma = rates.uk.rates[0] * np.ceil(0.1 * self.PA0 / 10) * 10 + + def pa(self, n): + # s. 35(2) takes half the excess off; s. 35(3) rounds what remains up to + # a whole pound. 1e-6 keeps float64 rounding from adding a pound. + tapered = self.PA0 - np.maximum(0, n - 100_000) / 2 + return np.maximum(0, np.ceil(tapered - 1e-6)) + + def savings_allowance(self, n): + # s. 12B: by the Step 3 income above the basic and higher rate limits. + T = n - np.minimum(self.pa(n), n) + psa = self.psa + return np.where( + T > self.H, psa.additional, np.where(T > self.B, psa.higher, psa.basic) + ) + + def liability( + self, + pay, + prop, + sav, + div, + to_s, + to_v, + margin=0.0, + e=1e-6, + inside=0.0, + feasibility=None, + ): + """Step 5 tax, rate-test result with every limit moved by ``margin``, + and feasibility, for each share (arrays broadcast). ``e`` is what the + rate test allows at the savings nil rate band and for dividends: float64 + rounding on the grid's shares, half a penny at the model's. + ``feasibility`` is how far a deduction may pass its component, ``e`` + unless given: for the model's share, the float32 error in the amounts + the model works the share from (``feasibility_margin``). ``inside`` is + how far under the top of the savings nil rate band the savings above + the limit must sit.""" + n = net_income(pay, prop, sav, div) + d = np.minimum(self.pa(n), n) + g = e if feasibility is None else feasibility + if self.other_first: # s. 25(3A): other income first + to_other = np.minimum(pay, d) + to_p = d - to_other - to_s - to_v + O, P = pay - to_other, prop - to_p + feasible = (to_p >= -g) & (to_p <= prop + g) + else: + to_ns = d - to_s - to_v + O, P = pay + prop - to_ns, 0 * pay + feasible = (to_ns >= -g) & (to_ns <= pay + prop + g) + feasible &= (to_s >= -g) & (to_s <= sav + g) & (to_v >= -g) & (to_v <= div + g) + S, V = sav - to_s, div - to_v + X = O + P + B, H = self.B, self.H + psa = self.savings_allowance(n) + tax = scale_tax(*self.uk, O) + banded(O, X, B, H, self.prop) + Z = np.minimum(X + S, np.maximum(X, self.L) + psa) # ss. 12, 12A + tax += banded(Z, X + S, B, H, self.sav) + top_nil = X + S + np.minimum(self.DA, V) # s. 13A + tax += scale_tax(*self.div, X + S + V) - scale_tax(*self.div, top_nil) + limit = B + margin + band_top = np.maximum(X, self.L) + psa # ss. 12, 12A + meets = ( + (X <= limit) + & ~((X + S > limit) & (X + S > band_top - inside + e)) + & ~((V > e) & (X + S + V > limit)) + ) + return tax, meets, feasible + + def least(self, pay, prop, sav, div, to_s, to_v, margin): + """The least liability over the shares given, with the rate test's limits + moved by ``margin``.""" + tax, meets, ok = self.liability( + pay, prop, sav, div, to_s, to_v, margin, inside=max(0.0, -margin) + ) + liability = np.maximum(0, tax - self.ma * meets) + return np.where(ok, liability, np.inf), liability + + +def money(hi): + return st.integers(0, int(hi * 100)).map(lambda c: c / 100) + + +amount = st.one_of( + st.just(0.0), + money(20_000), + money(80_000), + st.sampled_from([500.0, 1_000.0, 1_570.0, 5_000.0, 9_000.0, 12_570.0]), +) +offset = st.one_of( + st.sampled_from([0.0, -0.01, -0.02, -0.05, 0.01, -0.5]), + money(600).map(lambda v: v - 300), +) +independent = st.tuples(st.just("free"), amount, amount, amount, amount) +near_limit = st.tuples(st.just("near"), offset, amount, amount, amount) +rows = st.lists(st.one_of(independent, near_limit), min_size=K, max_size=K) + + +def incomes(year, row): + kind, a, sav, div, prop = row + prop = prop if year >= 2027 else 0.0 + if kind == "near": # Step 3 income near the basic rate limit plus £500 + pay = max(0.0, round(37_700 + 12_570 + 500 + a - sav - div - prop, 2)) + else: + pay = a + return pay, prop, sav, div + + +def check(year, drawn): + law = Law(year) + data = np.array([incomes(year, row) for row in drawn], dtype=float) + pay, prop, sav, div = data.T + sim = sim_for(year) + for name, values in zip( + [ + "employment_income", + "property_income", + "savings_interest_income", + "dividend_income", + ], + [pay, prop, sav, div], + ): + full = np.zeros(2 * K) + full[0::2] = values + sim.set_input(name, year, full) + sim.set_input("uses_property_allowance", year, np.zeros(2 * K, dtype=bool)) + sim.reset_calculations() + + def calc(v): + return np.asarray(sim.calculate(v, year), dtype=float)[0::2] + + model_tax = calc("income_tax") + granted = calc("marriage_allowance_tax_reduction") > 0 + m_s, m_v = ( + calc("received_allowances_savings_income"), + calc("received_allowances_dividend_income"), + ) + n = net_income(pay, prop, sav, div) + tol = 0.02 + 1e-6 * n + # The Personal Allowance falls a pound at a time (s. 35(2) and (3)), and the + # savings allowance steps down at the basic and higher rate limits (s. 12B). + # The model sums net income in float32, which leaves it off the amounts + # entered: by under a penny for incomes near the rate test's limits, and by + # several pence for the largest draws. That can put it on the other side of + # a step. The model then works a different allowance from the law's, so the + # checks leave that recipient out. + held = calc("net_income") + assert np.all(np.abs(held - n) <= tol) + steady = (law.pa(held) == law.pa(n)) & ( + law.savings_allowance(held) == law.savings_allowance(n) + ) + assert np.allclose( + calc("allowances")[steady], np.minimum(law.pa(n), 1e12)[steady], atol=0.02 + ) + # 1. At the model's own share: the Step 5 tax, and the reduction wherever + # the strict test passes and there is tax to reduce, and nowhere the loose + # test fails. marriage_allowance_tax_reduction never exceeds the tax left, + # so a person with no Step 5 tax gets none. + # The share must be one the law allows, within the float32 error in the + # amounts the model works it from. The rate test keeps its half penny. + within = feasibility_margin(pay, prop, sav, div, m_s, m_v) + step_5, strict, ok = law.liability( + pay, prop, sav, div, m_s, m_v, -PENNY, 0.005, feasibility=within + ) + _, loose, _ = law.liability(pay, prop, sav, div, m_s, m_v, PENNY, 0.005) + assert np.all(ok | ~steady), ( + "model share infeasible", + [data[i].tolist() for i in np.flatnonzero(~ok & steady)], + ) + at_share = np.maximum(0, step_5 - law.ma * granted) + bad = steady & ( + (np.abs(model_tax - at_share) > tol) + | (strict & ~granted & (step_5 > tol)) + | (granted & ~loose) + ) + assert not bad.any(), [ + (data[i].tolist(), model_tax[i], at_share[i], granted[i], strict[i], loose[i]) + for i in np.flatnonzero(bad) + ] + # 2. No share on the grid, the thin sets or the old order leaves less liability. + d = np.minimum(law.pa(n), n) + shared = d - np.minimum(pay, d) if law.other_first else d + f = np.linspace(0, 1, 41) + psa = law.savings_allowance(n) + s_opts = np.column_stack( + [ + np.minimum(sav, shared)[:, None] * f, + np.maximum(0, sav - psa), + np.maximum(0, sav - psa - 0.01), + np.minimum(sav, shared), + ] + ) + v_opts = np.column_stack( + [np.minimum(div, shared)[:, None] * f, div, np.minimum(div, shared)] + ) + to_s = np.repeat(s_opts, v_opts.shape[1], axis=1) + to_v = np.tile(v_opts, s_opts.shape[1]) + # Old order (non-savings, savings, dividends), and the shares leaving + # non-savings income at the basic rate limit, and a penny under it, with no + # dividends. + first = (pay + prop) if not law.other_first else prop + old_s = np.minimum(sav, np.maximum(0, shared - first)) + old_v = np.minimum(div, np.maximum(0, shared - first - old_s)) + corner_v = np.minimum(div, shared) + corners = [ + np.clip( + shared + - corner_v + - np.maximum( + 0, + first + - np.where(law.other_first, limit - np.maximum(0, pay - d), limit), + ), + 0, + sav, + ) + for limit in (law.B, law.B - PENNY) + ] + to_s = np.column_stack([to_s, old_s, *corners]) + to_v = np.column_stack([to_v, old_v, corner_v, corner_v]) + + def col(a): + return a[:, None] + + shares = (col(pay), col(prop), col(sav), col(div), to_s, to_v) + strict_options, strict_liability = law.least(*shares, -PENNY) + strict_least = strict_options.min(axis=1) + old = strict_liability[:, -3] + # The grid holds the least liability only approximately, so it bounds the + # model's tax from above. Check 1 bounds it from below: the model's tax is + # the law's liability at a share the model can hold, with the reduction only + # where the loose test passes. + worse = steady & ((model_tax > strict_least + tol) | (model_tax > old + tol)) + assert not worse.any(), [ + dict( + incomes=data[i].tolist(), + model=model_tax[i], + strict_least=strict_least[i], + old=old[i], + ) + for i in np.flatnonzero(worse) + ] + + +SETTINGS = settings( + max_examples=40, + deadline=None, + derandomize=True, + suppress_health_check=list(HealthCheck), +) + + +@SETTINGS +@given(st.sampled_from([2026, 2027]), rows) +@example(2026, [("free", 40_200.0, 9_000.0, 1_570.0, 0.0)] * K) +@example(2026, [("free", 50_000.0, 500.0, 270.0, 0.0)] * K) +@example(2026, [("free", 50_269.99, 500.0, 0.01, 0.0)] * K) +@example(2026, [("free", 40_199.98, 9_000.0, 1_570.0, 0.0)] * K) +@example(2026, [("free", 37_500.0, 11_000.03, 2_069.97, 0.0)] * K) +@example(2026, [("free", 37_500.0, 11_001.07, 2_068.93, 0.0)] * K) +@example(2026, [("free", 37_500.0, 11_001.98, 2_068.02, 0.0)] * K) +@example(2027, [("free", 8_542.77, 58_071.67, 9_471.52, 0.0)] * K) +@example(2027, [("free", 11_743.36, 6_860.08, 23_171.51, 0.0)] * K) +def test_model_tax_is_the_least_liability_the_law_allows(year, drawn): + check(year, drawn) diff --git a/policyengine_uk/utils/income_tax_allocation.py b/policyengine_uk/utils/income_tax_allocation.py index 538dfd94e8..41014b0f1a 100644 --- a/policyengine_uk/utils/income_tax_allocation.py +++ b/policyengine_uk/utils/income_tax_allocation.py @@ -110,10 +110,157 @@ each side of those edges ``f`` differs from the Step 5 tax by a constant, so the least ``f`` again lies where two lines cross. -The model stores its variables in float32, whose rounding can carry income at -a limit over it. So the search counts a share as meeting the rate test only -with room below each limit, under 4p on £38,000, and sets savings at the -savings allowance, or dividends at nil, exactly where the test reads them. +The rate test lets income reach each limit (s. 10(2)): income at a limit +passes, and income above it fails. A share that leaves non-savings income at +the limit can be the only one that meets the test, so the search keeps no room +below the limit for a person whose amounts float32 holds closely enough (see +below). The test reads the income the model holds. The model holds incomes in +float32 and sums net income and allowances in float32, so the income it holds +can sit either side of the sum of the amounts entered. A household entered at +a limit can then sit just above it and fail, as at the model's other limits. + +One entered a penny over a limit can also sit on it, where the amounts are +large enough or many enough. Each amount the model holds, and each float32 +sum, sits within half a float32 step of the true amount. Under £65,536 the +step is 0.39p or less, and it takes more than five roundings the same way to +hide a penny. From £65,536 the step is 0.78p, and three can. Pay of +£53,360.11, savings of £23,554.02 and dividends of £714.91 come to £77,629.04, +and after a trade loss of £26,859.03 the model holds net income of £50,770.00, +a penny under the £50,770.01 entered. From £131,072 the step is 1.56p, and two +whole-pence amounts can store as one: the model holds pay of £190,529.99 and +of £190,529.98 both as £190,529.984375. + +So the test in float64 keeps room below each limit, as ``main``'s search does +for everyone, unless the model can show that float32 hid less than the entries +can sit over a limit. The room is ``RATE_TEST_ROOM`` plus +``RATE_TEST_ROOM_FRACTION`` of the limits and of the income after the +deductions, 3.9p at £37,700 and 13.9p at £138,255. Two rules keep it +(``c.low_precision``): + +- An amount the test reads reaches £65,536 (``LOW_PRECISION``; + ``_low_precision`` lists the amounts). +- A bound on the float32 error in net income, the allowances and the gift + reaches the least amount the entries can sit over a limit + (``_can_hide_an_excess``). That amount is a penny for entries in whole + pence. A Gift Aid gift raises the limits by five quarters of the gift + (s. 414(2)(b)), which can leave a limit between two pence, and the least + excess is then a quarter, a half or three quarters of a penny. + +The second rule reads the amounts behind the income the test reads. +``_held_error`` works the bound from the entries up. An entry under £131,072 +sits within 0.39p of the whole-pence amount entered, so its distance from +whole pence is its error; from £131,072 the function gives no bound. For each +sum the model makes on the way to net income and the allowances, it adds the +distance between the amount held and the float64 sum of the parts as held, +which is the float32 rounding in that sum. So pay of £148,609.46 less +£108,409.45 of expenses, which the model holds as £40,200.00, gets no bound, +though no amount the test reads reaches £65,536. An amount the function +cannot follow gets no bound either: a formula it does not list, such as the +State Pension or a company car benefit, a relief the law caps, or a formula a +reform replaces, whose amount stops matching the parts listed here. The +function assumes entries in whole pence. An entry in fractions of a penny +reads as an error and adds to the bound. + +An entry is an amount with no formula, or any amount supplied as an input for +the year: in a situation, a dataset or a YAML test, or through +``Simulation.set_input``. The model reads a supplied amount in place of the +variable's formula, so ``_held_error`` bounds it as an entry and reads none of +the amounts the formula would. Pay set through ``Simulation.set_input`` sits +in employment_income itself, with nothing in the amounts that variable adds +up. It gets the same bound as pay the constructor moves into +employment_income_before_lsr. A value the model stored itself +(``set_model_value``) is not an entry and keeps its variable's rule. Where +``_held_error`` stops at a supplied net income or supplied allowances, +``_can_hide_an_excess`` bounds on their own the other amounts the test reads: +savings, dividends, property income, the trade loss relief and pension +relief. A grossed-up gift supplied as an input is an entry too. + +For a recipient the test holds to room, the search does what ``main``'s does +(``_moves_with_room``). It counts on the reduction only where the exact share +meets the test with the room, and it stores the float32 values nearest that +share's deductions. So the search never moves such a recipient to win the +reduction at a limit itself. A recipient entered at a limit is then denied +the reduction where ``main`` denies it: the model cannot tell that recipient +from one entered over the limit. Both pay amounts above, with savings of +£9,000.02, dividends of £1,570, a trade loss of £50,275 and a Gift Aid gift of +£80,044, leave the model £138,255 of income after the allowances, the raised +limit plus the £500 savings allowance. The law gives the reduction for +£190,529.98 and not for £190,529.99, and the model denies both. + +The model has always applied the test in float32, to non-savings income, +savings and dividends each held in float32. Float32 rounding there can land +income entered over a limit back on it, where a float64 sum of the same +float32 amounts sits over; and a float64 sum can sit at a limit where float32 +rounding of the limit or of the amounts carries them apart. So a share meets +the test where it does in either arithmetic (``_rate_test_on_stored``): in +float32 as before, which for the old order is ``_old_order_meets_rate_test`` +and for a moved share ``_float32_meets_rate_test``, or in float64 from the +amounts the model holds (``_meets_rate_test``), with the room for a person the +test holds to it. The float64 test adds the grossed-up gift, as the model holds +it in float32, to the basic rate limit. +The vertex search scores each candidate in float64 only, so it can miss a +share that passes only in float32, as at a limit a Gift Aid gift raises. It +rounds only the share with the least ``f``, so it can also miss a share that +meets the test only once its deductions are rounded to float32. ``main`` +misses both kinds too. The float32 test rounds each limit to float32 after it +adds the gift, where ``main`` rounds the limit first. The two agree where the +statutory limit is a float32 amount, as every limit up to 2028-29 is. The +model then never denies the reduction where ``main``'s float32 test gave it +for the same share. From 2029-30 the uprated Scottish higher rate threshold is +not a float32 amount, so for a Scottish taxpayer with a Gift Aid gift the two +limits can sit a float32 step apart. The model grants the reduction over a +limit only where float32 rounding of the amounts it holds carries income onto +the limit, as the float32 test always could. For a recipient the test holds to +room, the search decides and stores as ``main``'s does, so the float32 test +reads the share ``main`` stores. + +The model stores the deductions from savings and dividends in float32 too. The +search finds the least ``f`` in float64, then ``_stored_share`` rounds that +share's deductions to float32, down or up, and takes the rounding with the +least ``f``. Among roundings within ``TOLERANCE`` of each other it prefers one +that leaves the first component within its bounds, but a rounding outside them +still wins where it leaves less ``f``. A first component can have no room: a +person may have no non-savings income, or from 2027-28 no property income. The +two deductions then have to add up to the shared amount exactly, which float32 +rarely allows, and at a limit only one rounding may meet the rate test. The +model then sets up to a float32 step more than the shared amount against +savings and dividends, which leaves that much non-savings income beyond the +first component, or it leaves up to a step of the shared amount unset. + +A rounding that sets more than half a penny beyond the shared amount +(``OVER_DEDUCTION``) ranks below every other, since the rate test would then +read that much of the savings or dividends as non-savings income the person +does not have. So does a rounding that meets the rate test only in float32, +for a recipient. Otherwise the search could pick a rounding for the slack in +the float32 test alone, and give the reduction to a recipient entered a penny +over a limit, whom ``main`` denies. And for a recipient the test holds to +room, every rounding but the nearest ranks last, so the model stores the +nearest, as ``main`` does. + +None of these rules applies to the rounding nearest the exact deductions, +which storing them in float32 gives and the model has always held. It sits up +to half a float32 step of each deduction beyond the shared amount: 0.39p for a +deduction from £65,536 to £131,072, 0.78p up to £262,144, and twice as much in +each range above. Storing the savings or dividends that take the deduction, +which are at least as large, can move them by as much. So a household entered +at a limit can hold the same amounts as one entered just over it, and the +model gives both the same answer. Where the nearest rounding meets the test +only in float32, the slack grows with the float32 step of the income, as for +the old order. A recipient entered a few pence over a limit can then get the +reduction, as on ``main``. Any other rounding that sets more than half a penny +beyond the shared amount ranks last, and the nearest never does, so the model +stores no other rounding more than half a penny beyond it. + +The search prices the stored share and the old order each with +``_rate_test_on_stored``, and moves a person only where the stored share leaves +less ``f``. +``share_meets_marriage_allowance_rate_test`` gives the model's answer from the +stored deductions through the same function, so the model applies the +reduction wherever the search counted on it. For a person the test does not +hold to room it applies the reduction nowhere else. A recipient it holds to +room can also get it where the search did not count on it, as on ``main``: +where the search moves the share for the Step 5 tax alone and the float32 test +passes at a limit. The search keeps the order non-savings, savings, dividends unless another share lowers ``f``, so it never leaves more liability than that order. @@ -170,20 +317,107 @@ marriage_allowance_rate_limits, meets_marriage_allowance_rate_test, ) +from policyengine_uk.utils.supplied_inputs import is_supplied_input # A share replaces the order non-savings, savings, dividends only where it -# saves more than this (in pounds), so rounding never moves a deduction. +# saves more than this (in pounds). So float32 rounding moves a deduction only +# where one float32 step of it saves more, and then by that step. The step +# saves tax at the person's rates. For a Marriage Allowance recipient it can +# also carry the share across a limit of the rate test and win the reduction. TOLERANCE = 1e-3 # The search treats shares whose tax differs by less than this (in pounds) # as equally good. TIE = 1e-6 # Bound on people times candidates evaluated at once, to bound memory. CHUNK = 2_000_000 -# The search counts a share as meeting the Marriage Allowance rate test only -# with this much room below each limit: pounds, plus a fraction of the amounts -# involved, several times the float32 rounding of the variables the test reads. +# A float32 rounding of a share can set more than the shared allowances against +# savings and dividends, by up to a float32 step of each deduction (over a +# penny once a deduction passes £131,072). ``_stored_share`` ranks a rounding +# that sets more than this (in pounds) beyond them below every other, unless it +# is the rounding to the nearest float32 values, which storing the deductions +# gives. +OVER_DEDUCTION = 0.005 +# From this amount (in pounds, 2**16) the float32 step is 1/128 of a pound, +# 0.78p, or more. Each amount the model holds, and each float32 sum of them, +# can then sit 0.39p or more from the amount entered, so three such roundings +# can move income a whole penny, and a person entered a penny over a limit of +# the Marriage Allowance rate test can hold income at the limit. Where any +# amount the test reads reaches this, the search counts a share as meeting the +# test only with the room below (``_low_precision``). The amounts behind +# those, such as pay before expenses, fall to ``_can_hide_an_excess``. +LOW_PRECISION = 65_536.0 +# The room below each limit of the rate test for a person with such an amount: +# pounds, plus a fraction of the limits and of the income after the deductions. RATE_TEST_ROOM = 1e-3 RATE_TEST_ROOM_FRACTION = 1e-6 +# From this amount (in pounds, 2**17) the float32 step is 1/64 of a pound, +# 1.56p, and two whole-pence amounts can store as one. Below it a float32 +# amount sits within 0.39p of the whole-pence amount entered, so rounding the +# amount held to whole pence recovers the entry. +INDISTINCT = 131_072.0 +# ``_held_error`` rounds an input to whole pence to recover the amount entered. +PENNY = 0.01 +# The variables below are an amount less other amounts, floored at nil where +# the third item says so, plus the amounts in the last item. Unless the amount +# is supplied as an input, when it is an entry, ``_held_error`` checks the +# amount the model holds against that sum, so a formula that stops matching +# its entry here leaves an error no rounding explains, and the person keeps +# the room. +SUMS = { + "net_income": ( + ("net_income_before_trade_loss_relief",), + ("trade_loss_relief_against_general_income",), + True, + (), + ), + "taxable_employment_income": ( + ("employment_income", "employment_benefits", "benefits_in_kind"), + ("employment_deductions",), + True, + (), + ), + "taxable_property_income": ( + ("property_income",), + ("deductible_property_finance_costs", "property_allowance_deduction"), + True, + ("taxable_rent_a_room_income",), + ), + "taxable_savings_interest_income": ( + ("savings_interest_income", "other_investment_income"), + (), + True, + (), + ), + "taxable_dividend_income": ( + ("dividend_income",), + ("deficiency_relief",), + True, + (), + ), + "taxable_miscellaneous_income": ( + ("miscellaneous_income_before_trading_allowance",), + ("miscellaneous_income_trading_allowance_deduction",), + False, + (), + ), +} +# The Personal Allowance is a whole number of pounds (``np.ceil`` in its +# formula), which float32 holds exactly. Float32 rounding of adjusted net +# income could move it by a pound only in the taper, above £100,000 of net +# income, where ``_low_precision`` keeps the room anyway. +EXACT = ("personal_allowance",) +# The amounts the rate test reads beside net income and the allowances, which +# hold them as parts: savings, dividends and property income in net income +# with the trade loss relief taken off it, and pension relief in the +# allowances. ``_can_hide_an_excess`` bounds each on its own where the walk +# from net income and the allowances does not reach it. +READ_WITH_NET_INCOME = ( + "taxable_savings_interest_income", + "taxable_dividend_income", + "taxable_property_income", + "trade_loss_relief_against_general_income", + "pension_contributions_relief", +) # The variables on earned_taxable_income_exclusions that the allocation sets # rather than reads. ALLOCATED_EXCLUSIONS = ( @@ -231,6 +465,9 @@ def values(variable): ] savings = values("taxable_savings_interest_income") limits = marriage_allowance_rate_limits(person, period, income_tax) + earned_exclusions = sum( + (values(variable) for variable in exclusions), np.zeros_like(savings) + ) return dict( savings=savings, dividends=values("taxable_dividend_income"), @@ -239,9 +476,7 @@ def values(variable): property=values("taxable_property_income"), allowances=values("allowances"), pension=values("pension_contributions_relief"), - earned_exclusions=sum( - (values(variable) for variable in exclusions), np.zeros_like(savings) - ), + earned_exclusions=earned_exclusions, gift=values("gift_aid_grossed_up"), scottish=np.asarray(person("pays_scottish_income_tax", period)), savings_allowance=values("savings_allowance"), @@ -251,6 +486,9 @@ def values(variable): np.asarray(limits.savings_allowance, dtype=np.float64), savings.shape ), old_order_meets_rate_test=_old_order_meets_rate_test(person, period, limits), + can_hide_an_excess=_can_hide_an_excess( + person, period, income_tax, limits, earned_exclusions + ), ) @@ -339,26 +577,248 @@ def _build(inputs, income_tax): # that passes no recipients. zero = np.zeros_like(savings) c.marriage_allowance_reduction = columns.get("marriage_allowance_reduction", zero) - c.old_order_meets_rate_test = columns.get("old_order_meets_rate_test", zero > 0) c.rate_test_savings_allowance = columns.get("rate_test_savings_allowance", zero) + # The model's own answer to the test for the old order, which it gives a + # person the search leaves there (``_old_order_meets_rate_test``). + c.old_order_meets_rate_test = columns.get("old_order_meets_rate_test", zero > 0) non_savings_limit = columns.get("rate_test_non_savings_limit", zero) basic_rate_limit = c.bands[1] # The rate test reads all the non-savings income, so it adds what s. 25(3A) # leaves below the first component. c.non_savings_below_first = c.other_left if c.other_first else zero + # The statutory limit on non-savings income. + c.rate_test_non_savings_limit = non_savings_limit + c.low_precision = _low_precision( + columns, non_savings_limit, basic_rate_limit + ) | columns.get("can_hide_an_excess", zero > 0) + # The limits the test applies. Income can reach a limit (s. 10(2)), and a + # share that leaves non-savings income at one can be the only share that + # meets the test. TIE covers float64 rounding of a share the search puts + # at a limit. For a person with an amount at or above ``LOW_PRECISION``, + # or whose float32 error the model cannot bound under the least excess + # (``_can_hide_an_excess``), the test keeps room below each limit, as + # ``main``'s search does for everyone: income held at a limit may have + # been entered over it. room = RATE_TEST_ROOM + RATE_TEST_ROOM_FRACTION * np.maximum( np.maximum(np.abs(non_savings_limit), np.abs(basic_rate_limit)), np.abs(c.non_savings_below_first + c.step_3_income), ) c.rate_test_limits = SimpleNamespace( - non_savings=non_savings_limit - room, - basic_rate=basic_rate_limit - room, - starting=c.starting_limit - room, + non_savings=np.where( + c.low_precision, non_savings_limit - room, non_savings_limit + TIE + ), + basic_rate=np.where( + c.low_precision, basic_rate_limit - room, basic_rate_limit + TIE + ), + starting=np.where(c.low_precision, c.starting_limit - room, c.starting_limit), savings_allowance=c.rate_test_savings_allowance + TIE, ) return c +def _low_precision(columns, non_savings_limit, basic_rate_limit): + """People with an amount the Marriage Allowance rate test reads at or + above ``LOW_PRECISION``: net income, net income before the trade loss + relief, that relief, the allowances, savings, dividends, an exclusion from + earned income, the grossed-up gift or a limit the gift raises. Net income + before the relief counts because the relief can take net income below + ``LOW_PRECISION`` where the float32 sum it comes off sits above. These are + the amounts the test reads, not the amounts behind them: pay and the + expenses the model takes off it can each sit above ``LOW_PRECISION`` and + leave net income below it. ``_can_hide_an_excess`` covers those.""" + relief = columns["relief"] + largest = np.maximum.reduce( + [ + np.abs(amount) + np.zeros_like(relief) + for amount in ( + columns["net_income"], + columns["net_income"] + relief, + relief, + columns["allowances"], + columns["savings"], + columns["dividends"], + columns["earned_exclusions"], + columns["gift"], + non_savings_limit, + basic_rate_limit, + ) + ] + ) + return largest >= LOW_PRECISION + + +def _whole_pence(amount): + return np.round(amount / PENNY) * PENNY + + +def _entry_error(amount): + """How far an entry the model holds as ``amount`` can sit from the amount + entered: its distance from whole pence under ``INDISTINCT``, and no bound + from there.""" + return np.where( + np.abs(amount) < INDISTINCT, np.abs(amount - _whole_pence(amount)), np.inf + ) + + +def _listed(person, period, variables): + """The variables an ``adds`` or ``subtracts`` attribute names, which can + be a list or the path of a parameter that holds one.""" + if isinstance(variables, str): + node = person.simulation.tax_benefit_system.parameters(period.start) + for part in variables.split("."): + node = getattr(node, part) + variables = node + return tuple(variables or ()) + + +def _held_error(person, period, income_tax, name, known): + """A bound, in pounds, on how far the float32 amount the model holds for + ``name`` can sit from the amount the entries give, or infinity where the + function cannot give one. + + - An entry: the distance from the amount held to the nearest whole penny, + which is the amount entered where the entry is in whole pence and under + ``INDISTINCT``. From ``INDISTINCT`` two entries can store as one, so the + function gives no bound. An entry is a variable with no formula, or any + amount supplied as an input for the year (``is_supplied_input``): from + a situation, a dataset or ``Simulation.set_input``. The model reads a + supplied amount in place of the variable's formula or sum, so the + function applies this rule before the rules below and reads none of the + amounts that formula or sum would. A value the model stored itself + (``set_model_value``) is not supplied. + - A sum of other variables (``adds`` and ``subtracts``, or an entry in + ``SUMS``): the bounds of its parts, plus the distance from the amount + held to the float64 sum of the parts as held. That distance is the + float32 rounding inside the sum. It is also large where the model did + not work the amount from those parts, as for a formula a reform + replaces. + - The trade loss relief: the bound of the loss, where the relief is the + whole loss. Pension relief: the bound of the contributions, where the + relief is all of them. A capped relief gets no bound. + - A variable in ``EXACT``: nil. + - Any other formula: nil where the amount is nil, and no bound elsewhere, + as for the State Pension or a company car benefit. + + ``known`` holds the bounds already worked out, by variable.""" + if name in known: + return known[name] + + def values(variable): + return np.asarray(person(variable, period), dtype=np.float64) + + def error(variable): + return _held_error(person, period, income_tax, variable, known) + + def unless_nil(amount, bound): + return np.where(amount == 0, 0.0, bound) + + variable = person.simulation.tax_benefit_system.variables[name] + held = values(name) + if is_supplied_input(person, name, period, held): + # The model reads this amount as supplied and never works out its + # parts, so neither does this function: a part can be a formula the + # tax calculation would not otherwise run. + known[name] = _entry_error(held) + return known[name] + plus, minus, floored, after = (), (), False, () + if name == "net_income_before_trade_loss_relief": + plus = tuple(income_tax.adjusted_net_income_components) + elif name == "taxable_self_employment_income": + plus, floored = ("self_employment_income",), True + minus = tuple(income_tax.bases.taxable_self_employment_income_deductions) + elif name in SUMS: + plus, minus, floored, after = SUMS[name] + elif variable.adds or variable.subtracts: + plus, minus = ( + _listed(person, period, listed) + for listed in (variable.adds, variable.subtracts) + ) + if name in EXACT: + bound = np.zeros_like(held) + elif name == "trade_loss_relief_against_general_income": + loss = values("trading_loss") + bound = unless_nil(held, np.where(held == loss, _entry_error(loss), np.inf)) + elif name == "pension_contributions_relief": + contributions = values("pension_contributions") + bound = unless_nil( + held, + np.where(held == contributions, error("pension_contributions"), np.inf), + ) + elif plus or minus: + total = sum(values(part) for part in plus) - sum( + (values(part) for part in minus), np.zeros_like(held) + ) + if floored: + total = np.maximum(0, total) + total = total + sum((values(part) for part in after), np.zeros_like(held)) + bound = np.abs(held - total) + sum(error(part) for part in plus + minus + after) + elif variable.is_input_variable(): + bound = _entry_error(held) + else: + bound = unless_nil(held, np.inf) + known[name] = bound + return bound + + +def _can_hide_an_excess(person, period, income_tax, limits, earned_exclusions): + """People for whom float32 can have moved the amounts the Marriage + Allowance rate test reads by as much as the least amount the entries can + sit over a limit. For them income the model holds at a limit may have been + entered over it, so the test keeps room below each limit. + + The test compares sums and differences of net income, the allowances, + savings and dividends with the limits. ``_held_error`` bounds how far the + model's net income and allowances sit from the entries. Savings, + dividends, property income and the trade loss relief are parts of net + income, and pension relief is part of the allowances, so their rounding + counts there. Where ``_held_error`` does not reach one of them, the + function adds its bound (``READ_WITH_NET_INCOME``). That happens where it + stops at an amount supplied as an input above them, or a reform takes + them out of the parts of net income. A Gift Aid gift raises the limits by + its gross amount (s. 414(2)(b)), five quarters of the gift, which float32 + also rounds. A gross amount supplied as an input is an entry: its bound + is its distance from whole pence. + + Amounts entered in whole pence sit at least a penny over a whole-pence + limit they exceed. A gift whose gross amount is not in whole pence leaves + the limit between two pence, and income can then sit a quarter, a half or + three quarters of a penny over it. So the function asks for the bound to + stay below that least excess. An exclusion from earned income, which only + a reform gives, gets no bound.""" + known = {} + bound = sum( + _held_error(person, period, income_tax, name, known) + for name in ("net_income", "allowances") + ) + # The walk reaches the other amounts the test reads, unless it stopped at + # a supplied amount above them or a reform took them out of the parts of + # net income. Then each gets its own bound. + for name in READ_WITH_NET_INCOME: + if name not in known: + bound = bound + _held_error(person, period, income_tax, name, known) + held_gross = np.asarray(person("gift_aid_grossed_up", period), dtype=np.float64) + if is_supplied_input(person, "gift_aid_grossed_up", period, held_gross): + # A gross gift supplied in place of its formula is an entry. + gross = _whole_pence(held_gross) + bound = bound + _entry_error(held_gross) + else: + gift = np.asarray(person("gift_aid", period), dtype=np.float64) + gross = _whole_pence(gift) / (1 - income_tax.rates.uk.rates[0]) + bound = bound + np.where( + np.abs(gift) < INDISTINCT, np.abs(held_gross - gross), np.inf + ) + bound = bound + np.where(earned_exclusions == 0, 0.0, np.inf) + least_excess = PENNY + for limit in (limits.non_savings, limits.basic_rate): + pence = (np.asarray(limit, dtype=np.float64) - held_gross + gross) / PENNY + # The distance from the limit to the next whole penny above it. + least_excess = np.minimum( + least_excess, (np.floor(pence + 1e-6) + 1 - pence) * PENNY + ) + # TIE keeps the comparison clear of the slack the float64 test allows. + return bound > least_excess - 10 * TIE + + def _non_savings_tax(c, income): return np.where(c.scottish, _tax(income, c.scotland), _tax(income, c.uk)) @@ -444,8 +904,10 @@ def step_5_tax(c, x, s, v): def _meets_rate_test(c, x, s, v): - """Whether the share meets the Marriage Allowance rate test with room - below each limit. ``TIE`` covers float64 rounding of a share that leaves + """Whether the share meets the Marriage Allowance rate test. Income at a + limit meets it (s. 10(2)), except for a person the test holds to room + (``c.low_precision``), whose limits carry it (``c.rate_test_limits``). + ``TIE`` covers float64 rounding of a share that leaves income at a limit, savings at the savings allowance or no dividends.""" return meets_marriage_allowance_rate_test( x + c.non_savings_below_first, @@ -524,13 +986,13 @@ def _candidates(c): # else repeats the line x = 0, so the search finds them the same share # whoever else it searches. recipient = c.marriage_allowance_reduction > 0 - vertical += [ - np.where( - recipient, - c.rate_test_limits.non_savings - c.non_savings_below_first, - zero, - ) - ] + # With room for a person the test holds below the limit. + limit = np.where( + c.low_precision, + c.rate_test_limits.non_savings, + c.rate_test_non_savings_limit, + ) + vertical.append(np.where(recipient, limit - c.non_savings_below_first, zero)) xs = np.concatenate([np.broadcast_to(v, zero.shape) for v in vertical], axis=1) ws = np.concatenate([np.broadcast_to(h, zero.shape) for h in horizontal], axis=1) ds = np.concatenate([np.broadcast_to(d, zero.shape) for d in diagonal], axis=1) @@ -746,6 +1208,198 @@ def _exact_where_the_rate_test_reads(c, s, v): return s, np.where(recipient & (v <= TIE), 0, v) +def _share_left(c, to_savings, to_dividends): + """The first component, savings and dividends left where the model holds + ``to_savings`` and ``to_dividends`` as the deductions from savings and + dividends, and sets the rest of the deductions against the first + component.""" + s = np.maximum(0, c.savings - to_savings) + v = np.maximum(0, c.dividends - to_dividends) + return np.maximum(0, c.step_3_income - s - v), s, v + + +def _float32_meets_rate_test(c, to_savings, to_dividends): + """Whether the share the model holds as ``to_savings`` and + ``to_dividends`` meets the rate test in the arithmetic the model has + always applied: on non-savings income left, savings and dividends each + held in float32, against the limits rounded to float32.""" + single = np.float32 + x, _, _ = _share_left(c, to_savings, to_dividends) + savings = np.maximum( + single(0), + c.savings.astype(single) - np.asarray(to_savings).astype(single), + ) + dividends = np.maximum( + single(0), + c.dividends.astype(single) - np.asarray(to_dividends).astype(single), + ) + return meets_marriage_allowance_rate_test( + (x + c.non_savings_below_first).astype(single), + savings, + dividends, + SimpleNamespace( + non_savings=np.asarray(c.rate_test_non_savings_limit).astype(single), + basic_rate=np.asarray(c.bands[1]).astype(single), + starting=c.starting_limit, + savings_allowance=np.asarray(c.rate_test_savings_allowance).astype(single), + ), + ) + + +def _rate_test_on_stored(c, to_savings, to_dividends, moved): + """Whether the share the model holds as ``to_savings`` and + ``to_dividends`` meets the rate test, in float64 from the amounts held + (``_meets_rate_test``, with room for a person the test holds to it, + ``c.low_precision``) or in float32 as the model has always applied it: + ``_float32_meets_rate_test`` where ``moved``, and the old order's own + answer (``_old_order_meets_rate_test``) elsewhere.""" + single = np.where( + moved, + _float32_meets_rate_test(c, to_savings, to_dividends), + c.old_order_meets_rate_test, + ) + return _meets_rate_test(c, *_share_left(c, to_savings, to_dividends)) | single + + +def _with_room(c): + """Marriage Allowance recipients whom the rate test holds to room below + each limit (``c.low_precision``). For them the search decides and stores + as ``main``'s does (``_moves_with_room``).""" + return c.low_precision & (c.marriage_allowance_reduction > 0) + + +def _moves_with_room(c, x, s, v, to_savings, to_dividends): + """Whether the search moves a recipient the rate test holds to room below + each limit (``_with_room``) from the old order's deductions + (``to_savings`` and ``to_dividends``) to the share that leaves ``x``, + ``s`` and ``v``. The search decides as ``main``'s does. It counts on the + reduction for the share only where the exact share meets the test with the + room, and for the old order only where the old order meets it in float32 + (``_old_order_meets_rate_test``). It prices both in float64, and reads the + first component the old order leaves without a floor at nil. So the search + never moves such a person to win the reduction at a limit itself, where + the amounts held may hide a penny.""" + old_s = c.savings - to_savings + old_v = c.dividends - to_dividends + old_order_tax = step_5_tax(c, c.step_3_income - old_s - old_v, old_s, old_v) - ( + c.marriage_allowance_reduction * c.old_order_meets_rate_test + ) + return tax_after_marriage_allowance(c, x, s, v) < old_order_tax - TOLERANCE + + +def _float32(values, toward): + """``values`` rounded to float32 towards ``toward`` (-inf or +inf), as + float64. A float32 amount stays as it is.""" + nearest = values.astype(np.float32) + back = nearest.astype(np.float64) + past = back < values if toward > 0 else back > values + stepped = np.nextafter(nearest, np.float32(toward)) + return np.where(past, stepped, nearest).astype(np.float64) + + +def _stored_share(c, s, v): + """The share the model can hold in place of the one that leaves ``s`` of + savings and ``v`` of dividends. The model stores the deductions from + savings and dividends in float32, so the function rounds each down or up + to float32 and takes the pair with the least ``f`` under the rate test the + model applies to a stored share (``_rate_test_on_stored``). Among pairs + within ``TOLERANCE`` of each other, it prefers one that leaves the first + component within its bounds. A pair can leave the first component outside + its bounds by less than a float32 step of the deductions, and the model + then reads it as ``_share_left`` does, at nil where it would fall below + nil. Such a pair still wins where it gives less ``f``, for example where + only its rounding meets the rate test. Three kinds of pair rank below + every other: one that sets more than ``OVER_DEDUCTION`` beyond the shared + allowances; for a recipient, one that meets the rate test only in float32; + and, for a recipient the rate test holds to room (``_with_room``), every + pair. None of that applies to the pair nearest the exact deductions, + which storing the deductions in float32 gives, as the model always has. + So a recipient the test holds to room gets the nearest pair. Deductions + that are float32 amounts already stay as they are. Returns the + deductions, the first component, savings and dividends they leave, and + their ``f``.""" + s, v = _exact_where_the_rate_test_reads(c, s, v) + exact_savings = np.clip(c.savings - s, 0, c.savings) + exact_dividends = np.clip(c.dividends - v, 0, c.dividends) + recipient = c.marriage_allowance_reduction > 0 + with_room = _with_room(c) + nearest_savings = exact_savings.astype(np.float32).astype(np.float64) + nearest_dividends = exact_dividends.astype(np.float32).astype(np.float64) + highest = np.minimum(c.first, c.step_3_income) + best = None + for toward_savings in (-np.inf, np.inf): + for toward_dividends in (-np.inf, np.inf): + to_savings = _float32(exact_savings, toward_savings) + to_dividends = _float32(exact_dividends, toward_dividends) + first = ( + c.step_3_income + - (c.savings - to_savings) + - (c.dividends - to_dividends) + ) + share = _share_left(c, to_savings, to_dividends) + meets = _rate_test_on_stored(c, to_savings, to_dividends, True) + tax = step_5_tax(c, *share) - c.marriage_allowance_reduction * meets + outside = (first < -TIE) | (first > highest + TIE) + nearest = (to_savings == nearest_savings) & ( + to_dividends == nearest_dividends + ) + float32_only = meets & ~_meets_rate_test(c, *share) + option = SimpleNamespace( + last=( + (first > highest + OVER_DEDUCTION) + | (float32_only & recipient) + | with_room + ) + & ~nearest, + outside=outside, + tax=tax, + to_savings=to_savings, + to_dividends=to_dividends, + x=share[0], + s=share[1], + v=share[2], + ) + if best is None: + best = option + continue + level = np.abs(option.tax - best.tax) <= TOLERANCE + lower = (option.tax < best.tax - TOLERANCE) | ( + level + & ( + (best.outside & ~option.outside) + | ((best.outside == option.outside) & (option.tax < best.tax - TIE)) + ) + ) + better = (best.last & ~option.last) | ((best.last == option.last) & lower) + best = SimpleNamespace( + **{ + name: np.where(better, values, vars(best)[name]) + for name, values in vars(option).items() + } + ) + return best + + +def share_meets_marriage_allowance_rate_test(person, period, parameters): + """Whether the share the model holds meets the Marriage Allowance rate test + (ITA 2007 s. 55B(2)(b) and (ba)). The function reads the deductions from + savings and dividends as the model stores them and applies + ``_rate_test_on_stored``, as the search does to the share it stores and to + the old order, so the model gives the answer the search counted on.""" + c = _context(person, period, parameters) + default = default_order_deductions(person, period) + + def stored(variable): + return np.asarray(person(variable, period), dtype=np.float64) + + to_savings = stored("received_allowances_savings_income") + to_dividends = stored("received_allowances_dividend_income") + moved = (to_savings != default.savings) | (to_dividends != default.dividends) + return _rate_test_on_stored( + c, to_savings[:, None], to_dividends[:, None], moved[:, None] + )[:, 0] + + def allocate_deductions(person, period, parameters): """Deductions against savings and dividend income in the share that gives the least Income Tax after the Marriage Allowance (ITA 2007 s. 25(2) and @@ -766,39 +1420,53 @@ def least_tax_deductions( inputs, income_tax, to_savings, to_dividends, search_all=False ): """The deductions from savings and dividends: the old order's - (``to_savings`` and ``to_dividends``) unless another share saves more than - TOLERANCE. The search runs only on people the tests in - ``_old_order_is_least`` leave and on Marriage Allowance recipients whom - another share could win the reduction, or on everyone with - ``search_all``.""" + (``to_savings`` and ``to_dividends``) unless another share, as the model + stores it, saves more than TOLERANCE. The search runs only on people the + tests in ``_old_order_is_least`` leave and on Marriage Allowance + recipients whom another share could win the reduction, or on everyone + with ``search_all``.""" to_savings = np.asarray(to_savings, dtype=np.float64) to_dividends = np.asarray(to_dividends, dtype=np.float64) if search_all: search = np.ones(len(to_savings), dtype=bool) else: everyone = _build(inputs, income_tax) + old_order = to_savings[:, None], to_dividends[:, None] search = ( - ~_old_order_is_least(everyone, to_savings[:, None], to_dividends[:, None]) + ~_old_order_is_least(everyone, *old_order) | _marriage_allowance_in_reach(everyone) )[:, 0] if not search.any(): return to_savings, to_dividends c = _build({name: values[search] for name, values in inputs.items()}, income_tax) x, s, v = _least_tax_share(c) - default_s = c.savings - to_savings[search][:, None] - default_v = c.dividends - to_dividends[search][:, None] - default_x = c.step_3_income - default_s - default_v - # The old order takes the rate test's own answer, not the search's with - # room, so a person it already qualifies never moves for the reduction. - old_order_tax = step_5_tax(c, default_x, default_s, default_v) - ( - c.marriage_allowance_reduction * c.old_order_meets_rate_test + # Both sides as the model holds them: the share's deductions rounded to + # float32, and the old order's as given, each with the model's own answer + # to the rate test (``_rate_test_on_stored``, which + # ``share_meets_marriage_allowance_rate_test`` applies). So the model gives + # the reduction wherever this comparison counts it. + stored = _stored_share(c, s, v) + old_savings = to_savings[search][:, None] + old_dividends = to_dividends[search][:, None] + old_order = _share_left(c, old_savings, old_dividends) + old_order_tax = step_5_tax(c, *old_order) - c.marriage_allowance_reduction * ( + _rate_test_on_stored(c, old_savings, old_dividends, False) ) - lower = (tax_after_marriage_allowance(c, x, s, v) < old_order_tax - TOLERANCE)[:, 0] - s, v = _exact_where_the_rate_test_reads(c, s, v) + lower = stored.tax < old_order_tax - TOLERANCE + # A recipient the rate test holds to room below each limit moves as on + # ``main``, to the nearest rounding of the exact share. + lower = np.where( + _with_room(c), + _moves_with_room(c, x, s, v, old_savings, old_dividends), + lower, + ) + # A stored share with the old order's deductions is the old order. + same = (stored.to_savings == old_savings) & (stored.to_dividends == old_dividends) + lower = (lower & ~same)[:, 0] moved = np.flatnonzero(search)[lower] to_savings, to_dividends = to_savings.copy(), to_dividends.copy() - to_savings[moved] = (c.savings - s)[lower, 0] - to_dividends[moved] = (c.dividends - v)[lower, 0] + to_savings[moved] = stored.to_savings[lower, 0] + to_dividends[moved] = stored.to_dividends[lower, 0] return to_savings, to_dividends @@ -818,16 +1486,25 @@ def _old_order_non_savings_left(person, period): def _old_order_meets_rate_test(person, period, limits): """Whether the old order meets the Marriage Allowance rate test, in the - arithmetic ``meets_marriage_allowance_income_conditions`` applies where the - share keeps that order.""" + arithmetic the model has always applied to that order: in float32, on net + income less savings, dividends and allowances, against ``limits`` rounded + to float32. Where the statutory limit is a float32 amount, that is the + float32 sum of the limit and the gift, as on ``main``. The Scottish higher + rate threshold is not one from 2029-30, and with a gift the rounded limit + can then sit a float32 step from that sum.""" default = default_order_deductions(person, period) + savings = person("taxable_savings_interest_income", period) + dividends = person("taxable_dividend_income", period) return meets_marriage_allowance_rate_test( _old_order_non_savings_left(person, period), - np.maximum( - 0, person("taxable_savings_interest_income", period) - default.savings + np.maximum(0, savings - default.savings), + np.maximum(0, dividends - default.dividends), + SimpleNamespace( + non_savings=np.asarray(limits.non_savings).astype(np.float32), + basic_rate=np.asarray(limits.basic_rate).astype(np.float32), + starting=limits.starting, + savings_allowance=limits.savings_allowance, ), - np.maximum(0, person("taxable_dividend_income", period) - default.dividends), - limits, ) diff --git a/policyengine_uk/utils/marriage_allowance.py b/policyengine_uk/utils/marriage_allowance.py index 99b51f586b..75c6d90250 100644 --- a/policyengine_uk/utils/marriage_allowance.py +++ b/policyengine_uk/utils/marriage_allowance.py @@ -1,9 +1,10 @@ """The rate test for the Marriage Allowance (ITA 2007 s. 55B(2)(b) and (ba)). The test reads the income the reliefs and allowances leave in each component, -so it depends on how s. 25(2) shares them. -``meets_marriage_allowance_income_conditions`` applies it to the share the -model settles on, and ``income_tax_allocation`` to each share it compares. +so it depends on how s. 25(2) shares them. ``income_tax_allocation`` applies +it to each share it compares and, for +``meets_marriage_allowance_income_conditions``, to the share the model +settles on. """ from types import SimpleNamespace @@ -21,7 +22,8 @@ def marriage_allowance_rate_limits(person, period, income_tax): schedule. """ rates = income_tax.rates - gross_gift = person("gift_aid_grossed_up", period) + # In float64, so a limit raised by a gift keeps the gift's pence. + gross_gift = np.asarray(person("gift_aid_grossed_up", period), dtype=np.float64) # The Scottish higher rate follows the starter, basic and intermediate # rates; in 2017-18 the Scottish scale had only the basic rate below it. # Before 2017-18 the test holds a Scottish taxpayer to the basic rate diff --git a/policyengine_uk/utils/supplied_inputs.py b/policyengine_uk/utils/supplied_inputs.py index a5d5f05ab6..81e8f15680 100644 --- a/policyengine_uk/utils/supplied_inputs.py +++ b/policyengine_uk/utils/supplied_inputs.py @@ -43,6 +43,16 @@ ``supplied_input_arrays`` answers for every variable. It includes values ``set_model_value`` stored, which ``is_model_value`` tells apart. + +``is_supplied_input`` also answers for every variable. It takes a value the +engine has already returned and says whether that value is the input supplied +for its period. It needs no entry in ``SUPPLIED_INPUT_VARIABLES``, because it +cannot read a record whose value is gone as supplied. It reads the stored +input as ``supplied_input`` does. policyengine-core, from the version +pyproject.toml requires, stores every value the engine fills in (a formula +result, or a carried-over, uprated or default value) as derived, and +``stored_supplied_value`` never reads a derived value. And the stored input +must equal the value the engine returned. """ from functools import cache @@ -294,6 +304,33 @@ def supplied_input( period, or cached after a calculation, is not an input, including one cached over the input.""" _check_registered(variable_name) + return _supplied_value(population, variable_name, period) + + +def is_supplied_input(population, variable_name: str, period: Period, value) -> bool: + """Whether ``value``, which the engine returned for ``variable_name`` and + ``period``, is the input supplied for that period (from a situation, a + dataset or ``Simulation.set_input``), on this branch or the nearest one it + was made from. + + It is not where the engine worked the value out: from a formula, or by + carrying over or uprating another period's input. Nor is a value the model + stored itself (``set_model_value``), or the default a neutralized variable + reads as. With no record of inputs (policyengine-core's legacy mode), no + value counts as supplied. + + This answers for any variable (see the module docstring). A caller that + walks variables named by parameters, which a reform can change, cannot + list them in ``SUPPLIED_INPUT_VARIABLES``.""" + variable = population.simulation.tax_benefit_system.get_variable(variable_name) + if variable.is_neutralized: + return False + supplied = _supplied_value(population, variable_name, period) + return supplied is not None and np.array_equal(supplied, value) + + +def _supplied_value(population, variable_name: str, period: Period): + """``supplied_input``, for any variable.""" simulation = population.simulation input_keys = getattr(simulation, "_user_input_keys", None) or () holder = population.get_holder(variable_name) diff --git a/policyengine_uk/variables/gov/hmrc/income_tax/allowances/meets_marriage_allowance_income_conditions.py b/policyengine_uk/variables/gov/hmrc/income_tax/allowances/meets_marriage_allowance_income_conditions.py index c2114c7381..cd7cdadf95 100644 --- a/policyengine_uk/variables/gov/hmrc/income_tax/allowances/meets_marriage_allowance_income_conditions.py +++ b/policyengine_uk/variables/gov/hmrc/income_tax/allowances/meets_marriage_allowance_income_conditions.py @@ -1,7 +1,6 @@ from policyengine_uk.model_api import * -from policyengine_uk.utils.marriage_allowance import ( - marriage_allowance_rate_limits, - meets_marriage_allowance_rate_test, +from policyengine_uk.utils.income_tax_allocation import ( + share_meets_marriage_allowance_rate_test, ) @@ -31,27 +30,23 @@ def liable_only_at_marriage_allowance_rates(person, period, parameters): marriage_allowance_tax_reduction takes the allowance off the tax, and the test reads income after allowances and the savings, dividend and Gift Aid amounts directly. + + The test reads the share of reliefs and allowances the model holds + (received_allowances_savings_income and + received_allowances_dividend_income) and counts the share as meeting the + test where it does in either arithmetic: in float32, as the model always + has, or in float64 from the amounts the model holds. The search that sets + the share (s. 25(2); policyengine_uk.utils.income_tax_allocation) counts + the share it stores the same way, so this gives the answer the search + counted on. Income at a limit meets the test, since Parliament lets income + reach each limit (s. 10(2)). Float32 can hold income entered over a limit + at the limit. So the float64 test keeps room below each limit where an + amount it reads reaches £65,536, or where the model cannot bound the + float32 error in the amounts behind it under the least amount the entries + can sit over a limit: a penny, or less at a limit a Gift Aid gift leaves + between two pence. """ - # Earnings, pensions, property and other non-savings income after the - # allowances set against it (s. 25(2)): the income below savings and - # dividends (s. 16). Savings and dividends left after the allowances the - # share sets against them. - return meets_marriage_allowance_rate_test( - person("non_savings_income_after_allowances", period), - max_( - 0, - person("taxable_savings_interest_income", period) - - person("received_allowances_savings_income", period), - ), - max_( - 0, - person("taxable_dividend_income", period) - - person("received_allowances_dividend_income", period), - ), - marriage_allowance_rate_limits( - person, period, parameters(period).gov.hmrc.income_tax - ), - ) + return share_meets_marriage_allowance_rate_test(person, period, parameters) class meets_marriage_allowance_income_conditions(Variable): diff --git a/policyengine_uk/variables/gov/hmrc/income_tax/bracketized_earned_income/non_savings_income_after_allowances.py b/policyengine_uk/variables/gov/hmrc/income_tax/bracketized_earned_income/non_savings_income_after_allowances.py index c42df88337..0c596cc299 100644 --- a/policyengine_uk/variables/gov/hmrc/income_tax/bracketized_earned_income/non_savings_income_after_allowances.py +++ b/policyengine_uk/variables/gov/hmrc/income_tax/bracketized_earned_income/non_savings_income_after_allowances.py @@ -18,9 +18,9 @@ class non_savings_income_after_allowances(Variable): "the grossed-up Gift Aid gift (s. 414(2)(b)). The allocation computes " "it from its own share, without the float32 rounding of a sum that " "adds the relief back. property_income_tax, " - "taxable_property_income_after_allowances, earned_taxable_income, the " - "Marriage Allowance rate test and, for Scottish taxpayers, tax_band " - "read it, and non_savings_taxable_income reads it through " + "taxable_property_income_after_allowances, earned_taxable_income and, " + "for Scottish taxpayers, tax_band read it, and " + "non_savings_taxable_income reads it through " "taxable_property_income_after_allowances and earned_taxable_income." ) definition_period = YEAR