Skip to content

Report each QRF output's largest single record in the US tail-concentration evidence - #1268

Merged
MaxGhenis merged 14 commits into
mainfrom
us-qrf-tail-top1-share
Nov 12, 2026
Merged

MaxGhenis merged 14 commits into
mainfrom
us-qrf-tail-top1-share

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1259, #1251 and #1239. This PR adds no threshold and changes no gate verdict.

The US release's QRF tail gate fails a sparse column of 500 or more carriers whose top 100 records carry more than 75% of its weighted absolute mass, unless the run's register waives it. That test cannot see one record carrying a fifth of a column. In #1259's seed-0 rerun of the #1251 chain, one tax unit of weight 644.7 drew a $218.9M Schedule E line-32 value. At initial weights it carried 21.7% of the column's weighted absolute mass, while the top 100 of the column's 16,001 carriers carried 45.1%, under the gate's 75%.

This PR adds the missing number to the release's tail evidence. For each numeric QRF output it reports the largest single record's share of weighted absolute mass (the top-1 share) and that record's weight, beside the top-100 share.

What it adds

  • microcosm.build.gates.single_record_concentration(values, weights, *, top_k=100) returns carriers, top_1_share, top_1_weight and top_k_share for one column.
    • It uses the gate's own population: finite value and weight, positive |value| × weight.
    • It skips no column as thin.
    • For a column with no mass, or a total past the float64 range, it returns None for all three measurements and still reports carriers.
  • qrf_tail_concentration.json gains surface.single_record_concentration: {top_k, columns: {column: measurements}, unmeasured: {column: reason}}.
    • It covers every numeric QRF output, dense and thin ones included. It does not measure absent or boolean outputs.
    • The gate's details keep main's keys; a builder property pins them.
  • The release dry run (--dry-run-gates-report) adds top_1_share_at_base_weights and top_1_weight_at_base_weights to each tail row. No class or verdict reads them.
  • The post-export probe compares the block with a reference release that recorded it: the measured columns, each column's four fields, the unmeasured outputs and each one's reason. Against a release that predates this PR, such as Route A, it compares none.
  • Docs and receipts. The README (release gate evidence) and changelog.d/us-qrf-tail-single-record-share.added.md describe the block. experiments/1268-us-qrf-tail-top1-share/ holds the scripts and logs behind the Route A measurements.

The gate path keeps main's behavior

  • _qrf_tail_concentration_gate builds the gate's inputs and runs tail_concentration_gate with main's statements, and it resolves weights only for sparse outputs, as main does. The measurements run after the gate returns.
  • A sparse output's measurement reuses the values and weights the gate checked, and a test pins that the frame resolves the weights of a sparse output's entity once. For a dense output, the gate loop stores the entity's name and none of the column's values. After the gate, the measurement reads the column from the frame again with the loop's two conversion statements. The gate never weighs a dense output, so the measurement resolves that entity's weights itself unless the loop already resolved them for a sparse output.
  • When reading a dense column, resolving its weights or measuring an output raises any Exception, the release tool lists that output under unmeasured with the error's type and text, and the run goes on. For an error that cannot print itself, the tool records the type's name with unprintable error, and unprintable error alone when the type cannot give its name. No Exception from a measurement can therefore fail a run the gate would pass.
  • A sparse output whose weights do not resolve still raises the frame's resolution error before the gate runs, as on main. One whose weights do not align with its values still raises the gate's own error, which names the column. Tests pin both cases. A third test pins that, when the gate raises, the frame has resolved only the sparse output's weights, so no measurement ran first.
  • On 346 frames, main's release tool (27e305d) and the head's raise the same error or return the same gate result, and their surfaces match apart from the new block. The frames cover NaN and infinite values, weights that do not resolve or do not align, all-zero mass, ties, masses near 2**53, overflow, registers, and dense and sparse outputs on one entity.

Invariants

A Hypothesis property pins each one (300 examples for the function, 40 to 200 through the release tool and the dry run), with example tests beside it. Where an invariant says "exactly", the tests compare float64 values with no tolerance.

  1. Bounds. A measured column has 0 < top_1_share ≤ top_k_share ≤ 1, exactly.
  2. No rounding step lifts a share past 1. The gate and the measurement both sum the tail in the order np.partition leaves it and the total in record order. When the records outside the tail hold less mass than one rounding step, that ratio can pass 1: [2**53 + 2, 3, 1, 0.5] at top_k = 3 gives the gate 1.0000000000000002 or 1.0000000000000004, on every platform. top_k_share equals min(ratio, 1) under every partition arrangement numpy may return; a property draws the arrangement.
  3. One record. With top_k = 1, top_k_share equals top_1_share, and both equal the largest mass over the total mass, exactly.
  4. Monotone in top_k, to rounding. A larger top_k never lowers top_k_share by more than float rounding. It can lower it by one step: [0.5, 1, 2**53, 0.5, 0.5] gives 1.0 at top_k = 3 and 0.9999999999999998 at top_k = 4 on macOS arm64. The measurement accepts that drop, an intended violation of exact monotonicity, so that top_k_share stays the gate's own number (6). With at most top_k carriers, top_k_share equals 1 exactly.
  5. Pigeonhole floors. top_1_share ≥ 1/carriers and top_k_share ≥ min(top_k, carriers)/carriers, to float rounding.
  6. Differential, against the gate. On every column tail_concentration_gate checks whose total mass is finite, top_k_share equals min(top_share, 1) for the gate's top_share, exactly and under every partition arrangement, and carriers equals its carrier count. On a thin column, carriers equals the gate's thin count.
  7. Differential, against exact arithmetic. Both shares match ratios of the same float masses summed as Python Fractions, to 1e-12 relative.
  8. Scale-free, while no product or sum overflows or underflows. Scaling every weight by a power of two (2**-20 to 2**20 in the property) leaves the carrier count and both shares unchanged and scales top_1_weight by the same factor, exactly. Other positive factors (1e-3 to 1e3 in the property) leave the carrier count unchanged and both shares unchanged to 1e-9 relative. Near a tie for the largest mass, they can move top_1_weight to another record: [1, 10] at weights [1, 0.1] gives a top_1_weight of 1, and three times those weights give 0.30000000000000004, the second record's weight. Scaling every value by a power of two, or negating it, changes nothing.
  9. Total. Any float64 input gives a JSON-safe report with no NaN or infinity, and numpy warns about nothing; a property turns every warning into a failure. Inputs include NaN, ±inf, zero or negative weights, and products or sums past the float64 range. The report holds all three measurements or None for all three, and None for a column with no carriers.
  10. Order-free. Permuting records leaves carriers and top_1_weight unchanged, and the shares unchanged to float rounding. Among records tied for the largest mass, the function reports the largest weight.
  11. Monotone in the largest record, to rounding. Raising the largest record's value never lowers top_1_share by more than float rounding (1e-12 relative for lifts of up to 1,000 times in the property). It can lower it by one float step: the masses [2**53, 2**51 + 1] give a share of 0.8, and [2**53 + 2, 2**51 + 1] give 0.7999999999999999.
  12. Reporting only. For random geometries (0 to 1,200 carriers in each of three columns, random weights, signs and spreads), the release tool's gate returns the same verdict, failures and details as the shared gate does on the sparse columns alone.
    • Whatever values the block holds, and when the surface lacks the block, the dry run reports the same status, failure lines, at-risk lines, summary and details.
    • Rows differ only in the two new fields.
  13. A reason for every error. For every error a property draws (200 examples), _unmeasured_reason returns a reason and raises no Exception. The property draws errors whose __str__, __format__ or type name raise, and it pins which of the three reasons each one gets.

Mutation checks

At 3ccae30, the function's tests killed fourteen mutants of it: tie handling, no cap, a cap one rounding step too loose, carriers < top_k for ≤, no ≤ top_k shortcut, a tail one record short, the tail summed in sorted order, no finite-total check, zero-mass carriers, unfiltered weights, signed mass, median for max, the top-1 mass over the tail, and overflow left to warn (experiments/1268-us-qrf-tail-top1-share/README.md lists them). Since that commit the function has changed in its docstring and one comment only, and its test module has gained two example tests while no other test's code changed. Every property in test_single_record_concentration.py runs without Hypothesis's example database, so no saved example carries a kill from one mutant to the next.

A second pass at bbfd188 ran 35 mutants: those fourteen, eight more of the function, eight of the probe's comparison and five of the release tool's measurement path. The tests killed 30. Each of the other five now fails a test at the head:

  • one counts a near-tie as a tie for top_1_weight;
  • one makes the probe skip a measurement that it lacks and the release recorded;
  • one makes the probe report a difference when its own surface lacks the block;
  • one runs the measurements before the gate;
  • and one resolves a sparse output's weights a second time.

The gate's own share can still pass 1

The gate itself can report top_share above 1, by up to two rounding steps in the cases we found (1.0000000000000004). That needs a column whose records outside the top top_k hold less mass than the rounding error of the column's sum. The release checks only columns with 500 or more carriers at top_k = 100, so 400 or more carriers would have to hold almost none of the column. Such a column exceeds the 0.75 threshold whether its share reads 1 or a step above, so the overshoot never changes a verdict. This PR records the overshoot in a test and leaves the gate alone.

Certified Route A release: which columns exceed 5% and 10%

We measured the certified release populace-us-2024-0fb05b6-4b57d15a287c-20260930T150401Z.

  • Inputs. The published H5 (sha256 417d23ae…, as the release manifest certifies) and its calibration sidecar. Anyone can download both from the ungated policyengine/populace-us repository; the run read no restricted data.

  • Method. experiments/1268-us-qrf-tail-top1-share/route_a_top1.py runs _qrf_tail_concentration_gate as of this branch's commit a6644d6 on the H5's QRF columns, using the post-export probe's loader. It runs once at the release's calibrated weights, then at the sidecar's initial_household_weight.

  • Fidelity. At calibrated weights, that commit's gate matches the release's recorded qrf_tail_concentration.json (sha256 83119434…) on every field the script compares:

    • the same verdict and surface buckets;
    • the gate's details after a JSON round trip, including all 33 top-100 shares and carrier counts;
    • and no difference from the probe's compare_qrf_tail.

    The sidecar's final weights equal the H5's, so its initial weights align by row.

  • The head against that commit. single_record_concentration and tail_concentration_gate have the same code at the head as at a6644d6, docstrings and comments aside. The release tool's tail functions have changed since. They now measure after the gate and read a dense column from the frame again when they measure it. They list any Exception from a measurement as unmeasured, where a6644d6 listed only a dense output's weight-resolution ValueError.

    • On 62 frames, the tool at 3ccae30 (which has a6644d6's _qrf_tail_concentration_gate) and at 52ad0d1 (which measures after the gate) give the same gate result and surface JSON.
    • On 346 frames, the tool at e42137a (which has 52ad0d1's tail functions) and at 8794c3f (which has the head's) give the same errors, gate result, warnings and surface JSON on 345. On the other frame an error's __str__ raises: the run failed at e42137a, and at 8794c3f the output goes under unmeasured.

    Main's source manifest has since added eight QRF outputs, and the Route A H5 carries none of them, so the 68 measured columns and their numbers stand.

At the release's final weights, 17 of the 64 QRF outputs with a share have a top-1 share above 5%, and 7 have one above 10% (the first seven rows below; the surface measures 68 outputs, and four have no mass):

Column Gate class Carriers Top-1 Its weight Largest record (|value| × weight) Top-100 Top-1 at initial weights
long_term_capital_gains_on_collectibles checked, waived 1,138 48.1% 1,757.8 $6,761,367 × 1,757.8 = $11.89B 93.3% 13.9%
investment_income_elected_form_4952 thin 394 44.1% 1,989.2 $6,715,619 × 1,989.2 = $13.36B 98.2% 23.7%
farm_income thin 445 43.5% 4,673.2 $3,804,933 × 4,673.2 = $17.78B 96.8% 22.1%
casualty_loss thin 138 18.7% 2,418.8 $618,387 × 2,418.8 = $1.50B 99.8% 7.5%
sstb_w2_wages_from_qualified_business thin 480 18.4% 3,675.7 $9,294,781 × 3,675.7 = $34.16B 94.7% 20.4%
investment_interest_expense dense 64,778 17.5% 2,120.1 $3,681,182 × 2,120.1 = $7.80B 65.0% 7.6%
bond_assets checked, waived 8,887 14.9% 5,381.4 $20,590,000 × 5,381.4 = $110.80B 77.3% 4.4%
estate_income checked, waived 2,567 8.5% 176.1 $27,594,578 × 176.1 = $4.86B 79.2% 21.6%
w2_wages_from_qualified_business checked, waived 1,563 7.8% 3,675.7 $9,294,781 × 3,675.7 = $34.16B 77.3% 10.4%
sstb_self_employment_income_before_lsr checked 11,696 6.6% 2,353.0 $10,162,242 × 2,353.0 = $23.91B 52.4% 2.9%
unadjusted_basis_qualified_property checked 24,729 6.6% 1,701.6 $119,257,711 × 1,701.6 = $202.93B 47.2% 3.6%
qualified_bdc_income checked 5,276 6.6% 2,509.2 $2,747 × 2,509.2 = $0.01B 69.0% 4.2%
charitable_non_cash_donations checked 45,215 6.6% 973.7 $7,753,602 × 973.7 = $7.55B 65.3% 7.4%
unrecaptured_section_1250_gain checked 6,310 6.3% 2,353.0 $1,349,338 × 2,353.0 = $3.18B 58.9% 16.7%
sstb_unadjusted_basis_qualified_property checked 5,378 6.1% 2,353.0 $23,399,794 × 2,353.0 = $55.06B 72.3% 4.2%
tax_exempt_interest_income checked 16,664 5.5% 1,115.9 $3,078,364 × 1,115.9 = $3.44B 51.5% 2.6%
farm_rent_income checked 2,063 5.1% 3,896.9 |−$288,692| × 3,896.9 = $1.12B 74.8% 1.7%
  • Above 10%. The tail gate never grades five of the seven, four thin columns and one dense one:

    • investment_income_elected_form_4952 and farm_income, each above 43%;
    • casualty_loss;
    • sstb_w2_wages_from_qualified_business;
    • and the dense investment_interest_expense, at 17.5%.

    The gate checks the other two, collectibles gains and bond_assets, and the run's register (build.qrf_tail_register in the release manifest) waives both.

  • Above 5% and passing the gate. Eight of the 12 checked columns above 5% pass the gate unwaived. Their top-100 shares run from 47.2% to 74.8%.

  • Calibration moves these shares. At the same household total (138.1M), calibration lowers the median household weight from 378 to 118 and raises the 99th percentile from 1,021 to 3,731. It raises no household's weight more than fivefold (largest final/initial ratio 5.0).

    • Collectibles gains rose from 13.9% to 48.1%, and seven of the 17 columns sat under 5% at initial weights. For example, bond_assets rose from 4.4% to 14.9%.
    • Others fell. estate_income dropped from 21.6% to 8.5%, unrecaptured_section_1250_gain from 16.7% to 6.3%, and domestic_production_ald from 21.2% to 3.4%.
  • At initial weights, 13 columns exceed 5% and 8 exceed 10%. Three exceed 5% at initial weights only: domestic_production_ald, stock_assets (5.3%) and partnership_income (5.3%).

  • Shared largest records. One household holds three columns' largest records: one of its members holds the largest SSTB self-employment and SSTB UBIA records, and that person's tax unit holds the largest unrecaptured section 1250 gain. Another person holds the largest record of both W-2 wage columns. The other 12 columns above 5% have their largest records in 12 different households.

  • Non-amount outputs. Four numeric QRF outputs hold codes, a wage rate or a year (cps_race, detailed_occupation_recode, hourly_wage, first_home_mortgage_origination_year). The surface measures them like the rest, but their shares carry no dollar meaning; all sit below 0.3%.

  • Zero mass. Four outputs have no mass in this release (s_corp_income and the three second-home mortgage columns). The surface reports them with no share.

  • What this release predates. Route A predates Bound each PUF QRF draw by the population slice its recipient stands for #1259's draw bound and Carry Schedule E line 32 and its passive subset on the US PUF donor #1251's line-32 donor columns. The bound acts on raw draws at initial weights; the table measures the finished release frame.

experiments/1268-us-qrf-tail-top1-share/route_a_top1.md lists all 68 rows.

Decision

This PR leaves open whether the top-1 share becomes a failing gate. That methodology choice covers:

  • per-column thresholds or one threshold with reviewed exceptions;
  • the weights it grades at, final or initial;
  • and whether thin and dense columns join it.

Merge with main

Two merges bring in main up to 27e305d. The first (f4d8593) took in 817 commits, up to 6255b4a, and conflicted in one file, test_us_post_export_probe.py, where both sides appended tests; it keeps both. The second (7ca50ad) took in #1571 with no conflict. #1571 changed 13 functions in the release tool and two in the probe tool, and this branch edits none of them. Across both merges, main and this branch changed no function in common in the release tool, the probe tool, gates.py or the dry run.

Tests

  • At 676015c, in one process on Python 3.14, test_single_record_concentration.py, test_us_post_export_probe.py, test_us_release_gate_dry_run.py and test_us_fiscal_refresh_builder.py give 501 passed and 8 skipped (six need the policyengine-us extra, and two need the Chronicle facts feed). The commit after it, 2eb4eb8, changes two docstrings and one README paragraph and no code.
  • ruff check and ruff format --check pass on every changed Python file.
  • CI on 2eb4eb8: all twelve checks pass (lint, wheels, select-countries, installed-emitter, engine-free, engine-us and engine-uk on Python 3.13 and 3.14, integration-uk and ci).

Review

  • A review of the gate path at bbfd188 removed this PR's additions from _qrf_tail_concentration_gate and found main's syntax tree. It ran main's release tool and the branch's on the 346 frames and found one difference: when a dense output's weight resolution raised an error that cannot print itself, main passed and the branch failed. It also found that the loop held every dense output's values through the gate's run. Commit 8794c3f fixes both. The review found no change to a dataset value, a threshold, a gate verdict or the set of files a release publishes. qrf_tail_concentration.json gains the block, so the sha256 the manifests bind for that file changes.
  • A review of the prose at bbfd188 checked each sentence of this PR's README text, changelog fragment and new docstrings against the code. It found four claims the code did not support. Float rounding broke three of them, about scaling and the largest record, and the fourth, a README sentence, left out the gate's 500-carrier floor and its register. Commit e42137a rewrites them and adds two example tests that pin the rounding cases.
  • A mutation pass at bbfd188 ran the 35 mutants and found the five behaviors no test pinned. Commits e42137a and 676015c add their tests.
  • A review of the fix commits at 676015c ran git merge-tree on the second merge's two parents and found the merge's own tree. It built 400 more frames on a stub of the frame, compared the head with main on each one and found no difference outside the new block. It ran 13 mutants of 8794c3f's two changes: 12 fail a test, and the other, a reread without fillna, changes no measurement. It found four nits. Commit 2eb4eb8 applies three, and the fourth stays open: when one case of the _unmeasured_reason test fails, pytest prints its own error in place of the two reasons the test compares.

Evidence

experiments/1268-us-qrf-tail-top1-share/ holds the Route A script with its table and printed summary. It also holds three scripts with their logs (the calibration statistics, the minimized cap example and the monotonicity search), the shared-record script, and a README that says how to rerun each one. The Route A JSON and the shared-record script's output name records, so they stay outside the repository. The mutation runs and the 62-, 346- and 400-frame comparisons have no log in the repository.

axiom: n/a: release diagnostics; no policy rule changes.

🤖 Generated with Claude Code

MaxGhenis and others added 14 commits October 13, 2026 14:11
The release's tail-concentration gate tests the top 100 records' share of
a column's weighted absolute mass (fails above 0.75). One record can carry
a fifth of a column and pass: in the #1251 rebuild one tax unit of weight
644.7 drew a $218.9M Schedule E line-32 value and carried 21.7% of
partnership_income at initial weights, while the column's top-100 share
was 0.399.

microcosm.build.gates.single_record_concentration measures, for one
column, the largest single record's share of weighted absolute mass, that
record's weight, the top-k share and the carrier count, on the gate's own
population. It returns measurements only: no verdict, no threshold.

The US release tool's tail surface now carries it for every numeric QRF
output, dense and thin ones included, as
surface.single_record_concentration in qrf_tail_concentration.json. The
gate's details are unchanged. The dry run's rows carry the share and the
weight at base weights, and the post-export probe compares the block with
a release that recorded it.

Tests, docs and the changelog entry follow on this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… unchanged

single_record_concentration gets Hypothesis properties for the share chain
(0 < top-1 <= top-k <= 1), the pigeonhole floors, scale-freedom in the
weights (exact for powers of two), sign and value-scale invariance,
totality on any float64 input (zero or non-finite mass is reported as
None, never divided by), order-freedom, monotonicity, and two
differentials: against tail_concentration_gate's share and counts, and
against exact rational arithmetic. Example tests pin the #1251 line-32
shape, ties, and two float cases: the gate's own top-k sum can pass the
total by one rounding step (the measurement caps at 1), and a column with
at most top_k carriers reports exactly 1. All ten mutants of the function
are killed.

The release tool once again resolves weights only where main did for
sparse outputs. A dense output whose entity's weights do not resolve is
listed under single_record_concentration.unmeasured with the reason, so
the reporting-only measurement cannot fail a run the gate would pass. A
property checks the gate's verdict, failures and details equal the shared
gate on the sparse columns alone for random geometries. The dry run's
verdicts, lines and details are unchanged by any values in the block, or
by its absence; the probe compares the block with a release that has it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its weights

README and a changelog entry describe surface.single_record_concentration:
reported for every numeric QRF output, never graded, no share for a column
without mass. The motivating figures are now paired with the weights they
were measured at: in the #1251 rebuild the weight-644.7 unit carried 21.7%
of line 32's weighted absolute mass at initial weights while the top 100
tax units carried 44.9%; at the certified Route A final weights the two
were 4.4% and 39.9%. The first commit's message paired 21.7% with 0.399.
The worked example is re-tuned to the initial-weight top-100 share.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both sides appended tests at the end of test_us_post_export_probe.py; the
resolution keeps main's commit-at-load test and this branch's single-record
probe tests. same_edit.py (206ce71 vs 4a36edc) flags no shared count edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's engine-free jobs failed test_a_rounding_step_never_lifts_the_top_k_share_past_1.
The test asserted that the gate's share passes 1 on an input where that
depends on the order np.partition leaves the tail in, which numpy leaves
undefined. On CI's x86-64 runners numpy 2.4 partitions float64 through
x86-simd-sort, which sorts a small array outright; the sorted tail sums to
the record-order total, a share of exactly 1.0. macOS arm64 uses
introselect, whose order sums 2 higher, a share of 1.0000000000000002. The
exact share rounds to 1, so the measurement's capped 1.0 was right on both
platforms and the test was wrong.

- Replace the example with [2**53 + 2, 3, 1, 0.5] at top_k = 3, the
  smallest gate-checked input that passes 1 under every legal tail order
  (1.0000000000000002 or 1.0000000000000004), and keep CI's input as a test
  that walks all 5,040 of its tail orders.
- Add a numpy stand-in that returns a chosen legal partition, and
  properties that draw the arrangement: no arrangement lifts top_k_share
  past 1, it equals min(the gate's share, 1), top_k = 1 gives the largest
  mass over the total, and a larger top_k lowers the share by at most
  rounding. The totality property now fails on any numpy warning. The
  module docstring states eleven invariants; all 14 mutants of the
  measurement die.
- Quote one measurement of the #1251 line-32 draw everywhere: #1259's
  seed-0 rerun at initial weights, 21.7% top-1 and 45.1% top-100 of
  16,001 carriers (docstring, README, changelog, probe fixture, example
  test).
- State that top_k_share is the gate's share capped at 1, why the gate's
  share can pass 1, and which bounds hold exactly and which to rounding.
- Docstrings and comments only elsewhere: the release tool's rule for
  dense outputs whose weights do not resolve, and active voice in the
  prose this PR adds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes the four open nits from review r1 of this PR.

- N1: _qrf_tail_concentration_gate keeps main's loop and gate call,
  apart from recording each dense output, and a new
  _qrf_single_record_measurements measures every numeric output after
  the gate returns. It turns any Exception, from resolving a dense
  output's weights or from measuring, into an `unmeasured` entry, so
  the measurements cannot fail a run the gate would pass. Before, only
  a ValueError from resolving weights landed there.
- N2: a sparse output with misaligned weights fails with the gate's own
  message again ("casualty_loss: values and weights must align, ..."),
  because no measurement runs before the gate.
- N4: a dense output that sorts before a sparse output of the same
  entity no longer resolves that entity's weights first; the sparse
  output's error fails the run after one resolution, as on main.
- N8: the post-export probe compares `unmeasured` reasons. An output
  unmeasured in both runs for different reasons, or in one run only,
  is a difference.

Each nit has a test that fails at 3ccae30 and passes here. On 62
valid frames (the builder fixture, an unresolvable dense entity and 60
random frames), the old and new code give the same gate result and the
same surface JSON, key order included.

The invariant list now labels top_k_share's one-step drop at a larger
top_k as intended: it keeps top_k_share the gate's own share, capped
at 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The branch sat 817 commits behind main (base 4a36edc). One file
conflicted: test_us_post_export_probe.py, where both sides appended tests
at the end of the file. The merge keeps both: the branch's QRF tail
comparison tests, then main's timed-scorer test (#1556).

Main changed none of the functions this branch edits
(_qrf_tail_concentration_gate, tail_concentration_gate, compare_qrf_tail)
and left gates.py and release_gate_dry_run.py untouched since the base, so
every other file merged without a hand edit.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A review at 52ad0d1 mutated compare_qrf_tail and found six mutants the
probe tests let through: either set comparison (measured columns,
unmeasured columns) made one-sided, and carriers or top_k_share dropped
from the compared fields.

The tests now also cover a column only the probe measured, an output only
the probe left unmeasured, and a moved carriers count and top_k_share.
Tests only; compare_qrf_tail does not change.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Main gained #1571 (US builds publish through the emitter) after this
branch's last merge. Git merged every file without a conflict. #1571
changes 13 functions in the release tool and 2 in the probe tool, none of
them a function this branch edits, and it leaves gates.py and
release_gate_dry_run.py untouched.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review of #1268 found sentences in the repository that the code does not
support. This commit rewords them and adds two example tests. It changes
what no function returns.

- gates.py and the test module's invariant 8 said that scaling every weight
  by a positive factor scales top_1_weight by it "up to float rounding", and
  that scaling every value changes nothing. Both hold exactly for a power of
  two only. Any other factor keeps both shares to float rounding (a property
  pins 1e-9 relative for weight factors from 1e-3 to 1e3) and can move
  top_1_weight to another record whose mass ties the largest to within
  rounding: [1, 10] at weights [1, 0.1] reports weight 1, and at three times
  those weights it reports 0.30000000000000004. A new example test pins that
  move, on the weights and on the values, and labels it intended.
- Invariant 11 said that raising the largest record's value never lowers
  top_1_share. It can lower it by one float step, because the total is a
  rounded sum: [2**53, 2**51 + 1] gives 0.8 and [2**53 + 2, 2**51 + 1] gives
  0.7999999999999999. The property already allowed 1e-12 relative; the
  docstring now says so, and a new example test pins the one-step drop and
  labels it intended.
- The README said the tail gate fails a sparse column whose top 100 records
  carry more than 75% of its mass. The gate checks only a column with 500 or
  more carriers, and the run's register waives a column it lists with a
  reason. The sentence now says both.
- "Can pass 1 by a rounding step" becomes "by a rounding step or two" in
  gates.py and in a test comment: the module's own example records
  1.0000000000000004.
- The gates.py docstring keeps the 33% collectibles fact and no longer
  concludes that a threshold needs a justification per column, a question
  this pull request leaves open.
- The changelog fragment says when top_k_share equals the gate's capped
  share: on a column the gate checks whose total mass is finite.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Two review findings on the release tool's single-record measurements
(#1268).

- G-1. The handler in _qrf_single_record_measurements built its reason as
  f"{type(error).__name__}: {error}". An error whose __str__ raises made
  that line raise in its turn, so a diagnostic could fail a run the gate
  had passed. _unmeasured_reason now builds the reason: the type's name
  and the text; when formatting raises an Exception, the type's name with
  "unprintable error"; when reading the type's name raises too,
  "unprintable error" alone. No Exception raised while formatting leaves
  it, so the docstrings' "turns every Exception into an unmeasured entry"
  now holds as written.
  Tests: an error whose __str__ raises goes through the gate and lands
  under unmeasured (before this commit that call raised "RuntimeError:
  str failed"); four exact reasons; and a Hypothesis property over errors
  whose __str__, __format__ or type name raise.
- G-2. The gate loop kept every dense output's float64 array until after
  the gate call, where main frees each at the next iteration. The loop
  now keeps only a dense output's entity, and the measurement reads the
  column from the frame again inside its try, with the loop's two
  conversion statements. Frame.table returns the stored table, so the
  reread copies no table.
  Tests: a frame that fails after the gate leaves the gate's verdict
  alone and lists the dense output under unmeasured; a dense column
  stored as text measures as the loop read it; and a dense float64 column
  equals the shared measurement with no tolerance.

What stays the same: the statements of _qrf_tail_concentration_gate that
main (27e305d) has are still main's, by syntax tree with this pull
request's four additions stripped. On 346 differential frames the tool
gives the same gate, errors and surface as main's, the new block aside.
Against the commit before this one, the whole surface JSON is the same on
345 of them; the one that differs is the frame whose error's __str__
raises, which raised before and now lists the output under unmeasured.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The pull request's body cited two folders on one machine for its
evidence. This puts the evidence a reader can open and rerun under
experiments/1268-us-qrf-tail-top1-share/.

- README.md: what the scripts measure, the release and commit they
  measured (populace-us-2024-0fb05b6-4b57d15a287c-20260930T150401Z at
  a6644d6, with the H5's and the recorded tail evidence's sha256), how
  to rerun each script, and the results the body quotes that no other
  file here holds: the calibration weight statistics, which columns share
  a largest record, the input whose gate share passes 1, the one-step
  drop in top_k_share, and the fourteen-mutant run at 3ccae30.
- route_a_top1.py with its table (route_a_top1.md) and printed summary
  (route_a_top1.log); calibration_weight_stats.py, minimize.py and
  monotone_k.py with their logs; shared_largest_records.py.
- Left out because they name records: route_a_top1.json and the printed
  output of shared_largest_records.py. The README states the shared-record
  finding without any identifier. Left out because it patches a checkout
  in place: the mutation script.

The scripts differ from the ones that ran in form only: ruff format, three
noqa comments and one unused import removed. Each parses to the same
syntax tree as its original, that import aside. The committed logs are
the original runs' output, byte for byte. Reruns on 2026-11-11 printed
minimize.log, monotone_k.log and calibration_weight_stats.log again
without a difference.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A mutation review at bbfd188 ran 35 mutants of this pull request's
code; five passed every test (its findings M-1 to M-5). Commit e42137a
closed M-1. These tests close the other four. Tests only: no source file
changes.

- M-2, mutant P7: compare_qrf_tail flags a field that is None on one
  side only when the release's side holds the None.
  test_the_probe_compares_single_record_measurements gains the mirrored
  case. The probe's column has the release's carriers and no share (what
  the measurement reports when a column's mass sums past the float
  range), and the comparison lists top_1_share, top_1_weight and
  top_k_share as "None vs" the release's number. Under P7 it lists
  nothing.
- M-3, mutant P8: a probe result without the single-record block,
  against a release that has one, adds a difference.
  test_a_probe_surface_without_the_block_compares_none pins that the
  comparison skips the block when the probe's side lacks it, as an
  existing test does for the release's side. A probe run takes its
  surface from its builder's gate function, and the release tool's always
  writes the block, so the test builds the result by hand.
- M-4, mutant R-a: the measurement call moves above the gate call.
  test_no_measurement_runs_before_the_gate_raises gives the gate a sparse
  household output whose weights do not align and a dense person output.
  The gate raises its own message, and the frame has resolved the
  household's weights only. Under R-a the frame has resolved the person's
  too. Commit 8794c3f's
  test_a_dense_output_the_frame_cannot_give_again_is_unmeasured_not_fatal
  already fails under R-a, on a run the gate passes. The new test pins
  the run where the gate raises: of the 36 builder tests that name the
  tail gate or its measurements, it alone fails when the measurement runs
  in a finally clause after the gate raises.
- M-5, mutant R-e: a sparse output's measurement resolves its entity's
  weights again instead of reusing the gate loop's.
  test_any_error_measuring_a_dense_output_lands_under_unmeasured now ends
  by asserting, in its three cases, that the frame resolved the
  household's weights and then the person's, each once. Under R-e the
  frame resolves the household's twice.

Checked in one process that calls pytest once per mutant, each mutant
applied to this head's code (R-a's text follows commit 8794c3f's
rename to dense_entities): every one fails the test named for it above,
and N8, the mutant behind M-1, fails
test_a_factor_that_is_not_a_power_of_two_can_move_the_largest_record.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Wording only, in two docstrings and one README paragraph; no statement of
any function changes.

- gates.py, single_record_concentration: the sentence on a factor that is
  not a power of two now opens "Under the same condition". Such a factor
  leaves both shares unchanged to float rounding only while no product or
  sum overflows or underflows: [1, 1.4] reports a top_1_share of
  0.5833333333333334 at weights [1, 1] and 0.6 at weights
  [1e-323, 1e-323].
- tools/build_us_fiscal_refresh_release.py,
  _qrf_single_record_measurements: the docstring now says a dense column's
  converted values stay in local names, and that the function stores them
  in no mapping and returns none of them. It no longer says the next dense
  output's values replace them: the last dense output's values stay bound
  until the function returns, and a later reread that raises rebinds
  nothing.
- experiments/1268-us-qrf-tail-top1-share/README.md, "Mutation run": the
  last paragraph now says the pull request's description reports a later
  mutation pass and that the folder holds no log of either run. It no
  longer names a session or a file outside the repository.

For gates.py and the release tool, the syntax tree with every docstring
stripped equals the one at 676015c, and no comment token differs.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 12, 2026 17:50
@MaxGhenis
MaxGhenis merged commit c93419c into main Nov 12, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Landed by the Microcosm hub as c93419c (merge of 2eb4eb8 into main 90e4d94; main's tree equals the merge the hub tested).

  • Checks on 2eb4eb8: 12 passed.
  • Declare list edits on reform-coverage probes instead of pinning whole lists #1197 landed after this head's CI started and changes one other function in the release tool, so the hub ran this pull request's test files on the merge with main first: 548 passed, 9 skipped.
  • Reviews: the gate path against main, the prose sentence by sentence, a mutation pass and the fix commits each had an independent review; the last approved 676015c, and 2eb4eb8 changes docstrings and a README only.
  • Left open: one test assertion a reviewer would tighten (test_a_single_record_unmeasured_reason_never_raises).

@MaxGhenis
MaxGhenis deleted the us-qrf-tail-top1-share branch November 12, 2026 17:51
MaxGhenis added a commit that referenced this pull request Nov 12, 2026
Main gained #1268 (each QRF output's largest single record in the tail
evidence). Git merged every file without a conflict. Both sides change
tools/build_us_fiscal_refresh_release.py and test_us_fiscal_refresh_builder.py,
in different functions: by syntax tree, no function or test that main
changed since 90e4d94 is one this branch changes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant