Repository navigation
Conversation
…ansfer impute_us_org_wages and the ACS transfer's in-memory pattern fits used to fit every target and then predict, so every target's forests stayed referenced until prediction ended (and, in the ACS transfer, while the next pattern fitted). Both now call RegimeGatedQRF.start_chain and fit_draw_next once per target through helpers that return only the draw and the next chain state, so each target's forests and sign classifier are freed before the next fit. The checkpointed ACS path already fitted per target but its loop kept the previous step result referenced during the next fit; it now uses the same helper. Fitting and drawing use separate random streams and the same conditioning in both orders, so imputed values are byte-identical. New tests pin output digests recorded from main's code, compare against fitting every target first on the callers' own inputs, and check with weak references (garbage collector off) that no earlier target's fitted objects are alive when the next fit starts. Fit/predict QRF test doubles gain the chain methods through a shared adapter. acs_transfer's new bytes move the attested seed-kernel digests; pins regenerated with tools/repin_us_spec_engine_contracts.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for the one-target-at-a-time QRF change: - QRF draws are bitwise reproducible only within one platform, and these fixtures' outputs differ between macOS arm64 and Linux x86_64 even on main. The frozen digests are now kept per operating system and machine, recorded from main's code on macOS arm64 natively and on Linux x86_64 under Docker emulation (Python 3.13 and 3.14 agree on each); other platforms skip them, and the in-process comparisons run everywhere. - Add an in-process comparison of the checkpointed ACS path, cold and resumed, against fitting every target first. - _ProducerDtypeQRF, patched into the ACS transfer by a multi-line setattr, gets the chain adapter. - The legacy staging tool's --max-targets-per-fit help no longer says it bounds retained targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 2 fixes: - The frozen digests are checked only when the installed numpy, pandas, scipy, scikit-learn and quantile-forest match the versions they were recorded with, as the calibration tests' frozen bytes are; otherwise the tests skip and the in-process comparisons remain. - The CPS-ORG fixture builds its people, frame and donor itself (copies of the shared ORG test helpers), so edits there for other tests cannot move its bytes. - table_bytes_digest includes a categorical column's categories and order. - New test: both ACS paths refuse a chain step that reports a regime other than the donors' (the per-target regime check had no test). - The in-memory ACS path refuses a QRF without start_chain/fit_draw_next with the same message as the checkpointed path; a comment about test doubles now covers both; the changed lines follow ruff format, as the file did on main. acs_transfer's new bytes move the seed-kernel pins again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…check The lifetime check now also requires every tree of each earlier forest, the numpy array that owns each quantile leaf store's values, and the sign classifier's binning and trees to be freed before the next target's fit, so code that kept only part of a forest alive would fail it. A new test shows the check notices a kept leaf array or tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 3 additions: - A family whose two exported immigration columns share one joint model target now has an in-process comparison against fitting every target first, in memory and with a target bank. The existing banked-versus- in-memory tests compare the chain with itself since both paths use it. - Both ACS paths refuse a QRF without start_chain/fit_draw_next, tested. - The lifetime check's docstring names the parts it cannot see: scikit- learn's compiled Tree node storage and the sign classifier trees' node arrays, which cannot be weakly referenced and are freed with their trees. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lifetime check now also weakly references each sign-classifier tree's node array, and its docstring names the parts it cannot see (scikit-learn's compiled Tree storage and quantile-forest's leaf store object). The table digest's docstring says which columns it hashes byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Say what a change that moves the outputs records; say which digested tables have string columns; rename a test whose name said the batch split bounds retained forests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The quantile leaf array holds each leaf's training-row positions, not values; the digest is byte-exact for NumPy bool, integer and float columns; the re-recording note names what to rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both sides moved the US spec digests: main through microcosm#1585's us/gates.json row, this branch through acs_transfer's attested bytes. Keep both explanatory comments and regenerate the digests and the coverage evidence for the merged code with tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
reviewed
Nov 18, 2026
juaristi22
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed at 19d3d4b4a. The chaining is unchanged, and the outputs match main byte for byte. Three things before merge, and two for the follow-ups.
Chaining
Each chained target conditions on the same values, in the same order, as before:
fit_draw_nextfits target k on the predictors plus the donors' observed targets 1…k−1. It draws it on the recipients' predictors plus their raw draws of 1…k−1, through the same_draw_target_with_rngand the same split fit and draw streams asfitthenpredict.- Both callers pass each step's
raw_drawback unchanged.org_wages.pymasks inactive workers, andacs_transfer.pydecodes the joint codecs, only after the loop. - The chains cover the same targets as before: each availability pattern's
model_targets, split byDEFAULT_ACS_TRANSFER_MAX_TARGETS_PER_FIT = 8. - So later stages that use these columns as predictors see identical inputs.
Before merge
- Add a changelog fragment. Each of the last 25 PRs merged to main adds one.
- Say in the PR why these two callers move first. From the code, they are the ones the chain API takes as they are, with the most forests alive at once: up to 16 targets in the in-memory ACS path and 2 in the checkpointed one, against 1 now. For CPS-ORG, the PR's own estimate is about 60 MB off the peak, because
hourly_wage's forest is fitted first either way; #1682 targets that peak. - Update "Not verified". CI's engine-free jobs run on Linux x86_64 with the locked versions, so they checked the Linux digests recorded under emulation, and passed.
If #1682 merges first, re-run tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py: both PRs move the same US seed-protocol pins.
For the follow-ups
- UK SPI income holds about 120 targets' forests until it returns, by the PR's counts. It can't move as it stands: its stage-1 model declares
gate_level_columns(spi_age_band,main_source,has_pay,has_trade), andstart_chainrefuses them.fit_draw_nextalready draws from caller-owned uniforms, so the gate levels, notpredict_from_uniforms, block SPI.spi_housing_shell.pyandlcfs_consumption.pydeclare none. fit_draw_nexttrustsraw_prior_draws. It checks their names, order, index, dtype and finiteness, but not that they are the chain's own raw draws. Before more callers move, consider recording a digest of each step'sraw_drawin the chain state and refusing priors that don't match. A post-processed prior would then fail loudly instead of changing the conditioning.
Checked
- 321 of 324 tests on the touched surface pass locally (macOS arm64, Python 3.14); the 3 skips need the US engine. They include all 21 in
test_us_qrf_target_release.py, where main's macOS digests ran and matched. mainhasn't changedacs_transfer.py,org_wages.pyorqrf.pysince the digests were recorded at3b75cf151.
This was referenced Nov 19, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Thanks for the review. The three items before merge are done, in aa26137 and the PR description:
If #1682 merges first, this branch will re-run |
Brings in #1682 (the QRF leaf array filled in place). Both PRs move the US seed-map and seed-protocol digests, so the four spec-pin files conflicted; they are regenerated with tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py, keeping both PRs' comments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The CPS-ORG wage imputation (
impute_us_org_wages) and the ACS input transfer (transfer_acs_inputs) now fit and draw one QRF target at a time. Each target's forests and sign classifier are freed before the next target is fitted. Imputed values are byte-identical tomain.org_wages.py).QRF.fitoverhourly_wageandis_paid_hourly, then onepredict. Both targets' forests stayed referenced until the function returned.RegimeGatedQRF.start_chain, then onefit_draw_nextper target, through_fit_draw_next_org_target. That helper returns only the target name, its draw and the next chain state, so the step result that holds the forests is dropped before the next fit._fit_family_patterns).max_targets_per_fit).fitran, so up to 16 targets' forests could be alive at once._fit_family_patterns_banked, used when a target bank is passed)._fit_draw_pattern_target. It keeps the existing target-name, weight-kind, regime and finite-value checks._require_chain_api)._verify_fitted_target_regimesis removed. Regimes are now checked per step, by the_verify_chain_target_regimethe checkpointed path already used.qrf.pyis not changed.tools/_legacy/build_us_acs_multispine_base.py: the--max-targets-per-fithelp now says the option sets how many targets are chained together. It no longer limits how many targets' forests are in memory.Why these two callers first
gate_level_columns, whichstart_chainrefuses, soqrf.pyneeded no change. That refusal is what keeps the UK SPI income imputation, whose stage-1 model declares gate level columns, from moving the same way.hourly_wage's forest is the largest item and is fitted first in either order (see "Expected memory change"). Fill the QRF leaf array in place and pin quantile-forest 1.4.2 #1682, which fills the leaf array in place, targets that peak.Why the draws are unchanged
RegimeGatedQRF.fitandstart_chainboth split the model seed withSeedSequence(seed).spawn(2)into a fit generator and a draw generator.randomper tree, and only while fitting, so reordering doesn't shift it.Tests
New file
packages/microcosm-build/tests/engine_free/us/test_us_qrf_target_release.py. Helpers are intest_support/microcosm_build/qrf_chain_doubles.pyand fixtures intest_support/microcosm_build/qrf_per_target.py.Frozen bytes from
main. SHA-256 digests of every output table, recorded by runningmain's code (3b75cf1) on small fixtures.hourly_wageis positive-only;is_paid_hourlyhas zero and positive donors.docs/graph-qrf-cross-platform.md), and these fixtures' outputs differ between macOS arm64 and Linux x86_64 even onmain. So the digests are kept per platform:Same-run reference.
FitThenPredictQRFis the realRegimeGatedQRFwhose chain methods fit every target and then predict:main's computation, run on the callers' own inputs. It runs on every platform. Exactly equal frames, imputed-input records and fit records with it for:Lifetimes.
tracked_target_fitswrapsRegimeGatedQRF._fit_target.mainthis fails for CPS-ORG, the in-memory ACS path and the checkpointed ACS path. With the change it passes for all three, and each target is fitted once per availability pattern.Treenode storage, which can't be weakly referenced and is freed with its tree.Other new tests.
Test doubles. The
fit/predictstand-ins for the QRF in the ORG and ACS tests now inheritFitPredictChainDouble. That includes_ProducerDtypeQRFintest_support/microcosm_build/us_multispine_pool.py, which is patched in by a multi-linesetattr. The adapter providesstart_chainandfit_draw_nexton top of the stand-in's ownfitandpredict, so the stand-ins' recorded calls and outputs are unchanged.The local-area staging tool.
tools/_legacy/build_us_acs_multispine_base.pyreaches the QRF only throughtransfer_acs_inputswithout a target bank: three calls inbuild_optional_acs_multispine, one of them with a person predictor extension. That is the path the tests above cover, extension included.Spec pins
acs_transfer.pyis one of the attested QRF seed-kernel modules, so its new bytes move the US seed map and seed protocol digests and the specs that contain them:EXPECTED_HASHESininventory_coverage.py;docs/evidence/spec-engine/us-f0-coverage.json;spec_sha256.These were regenerated with
tools/repin_us_spec_engine_contracts.py, which now passes--check.tools/spec_engine_coverage.pypasses (42401/42401 fields, 41/41 inventory checks). Re-deriving the pins withmain'sacs_transfer.pybytes gives backmain's values, so nothing else moved. No draw, random-number rule or QRF target changes.Expected memory change
Not measured: the project's rules allow no full-size fits. The estimates follow from which objects are alive at the same time.
fit_draw_nextdraws inside the same call. That is one float64 table of recipient rows × predictors, about 160 MB for 3.3 million people and 6 predictors. Later patterns already held the previous pattern's table onmain.hourly_wage's single uncapped forest (100 trees on about 119,000 donor rows; the measured spike is +30–35 GB), and it is fitted first in both orders. What changes:is_paid_hourly's forests are no longer alive alongside it. That target's positive forest has pure single-node trees whose one leaf holds the whole bootstrap sample: roughly 100 trees × ~70,000 entries × 8 bytes ≈ 60 MB, plus a small classifier.hourly_wage's forest is freed beforeis_paid_hourlyis fitted and drawn, not when the function returns, so the large allocation is held for a shorter time.Behaviour differences
These arise only outside the callers' real configurations:
POPULACE_FIT_N_JOBSset to an invalid value, a chain whose targets are all zero now fails when it starts. Before, it returned zeros, because the setting was read only when a forest was grown.Follow-ups (unchanged here)
Other production callers still fit several targets in one
fitand then predict, so all those targets' forests are alive together at least until the fit ends. They are left as they are. Target counts come from the constants; the UK SPI counts were counted by hand.us_runtime/:puf_support.pyimpute_us_puf_tax_detail_support: 52 targets. It usespredict(release_models=True), but all 52 targets' forests exist when the fit ends. This is only the non-checkpointed path; the checkpointed base stage already usespuf_qrf_chain.scf_education_debt.py, the composition model: 10 targets, withrelease_models=True.retirement_contributions.py(5 targets);retirement_distributions.py(4);scf_wealth.pyfinancial assets (3);prior_year_income.py,other_health_insurance.py,child_support.pyandscf_auto_loans.py(2 each).sipp_financial_assets.pyfits one target per fit in a loop, but the previous model stays bound while the next one fits, so two exist briefly.uk_runtime/. These draw withpredict_from_uniforms, which has no option to release models.spi_income.pyimpute_uk_spi_income_support: a stage-1 model (about 21 targets), up to 5 reference fits (about 12 each) and a stage-2 model (38 targets), all held until the function returns.spi_housing_shell.py(12 targets).lcfs_consumption.pyredraw_zero_road_fuel(2 targets).Checks
spec_sha256test, with the US engine installed.uv run ruff check .,python3 tools/ci_test_plan.py verify,tools/repin_us_spec_engine_contracts.py --checkandtools/spec_engine_coverage.pypass.main'sorg_wages.pyandacs_transfer.pyside by side with the branch over many option sets and seeds, and found no byte difference. They also counted live trees, regressors and classifiers with the garbage collector off, and found none surviving.changelog.d/qrf-fit-draw-per-target.changed.md.🤖 Generated with Claude Code