Repository navigation
Score single-class folds and refuse weightless balanced classes in microcosm-fit - #1617
Merged
Merged
Conversation
out_of_fold_proba left each learner's rows at zero in a fold whose training rows hold a single class, since CategoricalClassifier fits no learner there; categorical_scores now refuses such rows. Those rows now take the model's probabilities, certainty of that class, which the model and the training folds' shares already gave. A learner no fold fitted still stays out of the result. _class_balanced_weights divided by each class's total weight and returned NaN weights for a class with none. It now refuses that class, or one whose total overflows, by name. The cross-fitted gate never reached it with such weights: it falls back to the in-sample gate (in_sample_weightless_fold) first. Valid input is unchanged: two differential workloads (772 arrays) match main and microcosm#1543 bit for bit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm.build.spec_engine.seeds attests microcosm.fit.qrf's source bytes. Repinned with tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py (41/41 inventory checks, 42,394/42,394 fields, --check clean), then python -m tools.graph_parity_repin fit.qrf, which writes only when the local direct call's bytes are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1543 merged main again (085672f), which moved the attested spec-engine hashes. The four hash-only conflicts took #1543's side, then tools/repin_us_spec_engine_contracts.py, tools/spec_engine_coverage.py and the repin again (--check clean) repinned them for this branch's qrf.py, and ruff format restored the double quotes the repin tool drops. No microcosm-fit source or fit.qrf parity pin changes in the merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An in-session pre-review found that a fit does reach _class_balanced_weights with a class whose weight total overflows to infinity. Each weight is finite, so the fit's weight checks take it, and _folds_carry_every_class counts inf as weight. Main completes that fit (cross_fitted_class_rates). 94ff4a7 made it raise, and its message, the changelog and the PR body said no fit reached the refusal. The brief says not to change any fit whose classes all carry weight, so the helper now refuses only a total that is not positive: the test _folds_carry_every_class runs on the same totals first, so no fit meets the refusal. Guards pin an overflowing class in the helper and in a full fit against the previous helper, bit for bit; both fail on b90f098. Tests: - The grouped-folds property checks that the rarest class sits in every fold. It draws households at the can_deal_folds bound, with boundary examples. A dealing by the commonest class and a random dealing now fail it; both passed before. - The out_of_fold_proba property gains examples for the boosted-only and spline-only models on a single-class fold, and for folds that each hold one class. Wording: - The whole gate falls back, not "other folds". - The reach claim holds for targets with two classes or more. - Every array holds a distribution in every row a fold scores. - The scorer's zero-row hint no longer names a cause out_of_fold_proba removed. Repinned the attested fit.qrf hashes: tools/repin_us_spec_engine_contracts.py, tools/spec_engine_coverage.py (41/41 inventory checks), the repin again, --check clean, then python -m tools.graph_parity_repin fit.qrf. ruff format restores the double quotes the repin tool drops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-1 Subfleet review (job 20261114-143433-fit-fold-r1) found the
code fixes correct and valid input bit-identical, and flagged wording that
overstates guarantees for weights whose totals overflow float64, which the
fit accepts:
- out_of_fold_proba's docstring promised a distribution in every row of
every array a fold scores, and "as the training folds' shares do"; a
fold whose training weights overflow gives NaN marginal rows (and
sometimes NaN boosted and blend rows), unchanged from main. The
docstring now says only what the fill guarantees: in a single-class fold
each learner's rows equal the model's, so none is left at zero.
- _class_balanced_weights' docstring, the changelog and the test module
said an overflowing class's rows weigh zero. When every class's total
overflows, every row's weight is NaN and the fit fails in scikit-learn
("Input sample_weight contains NaN."), on main as here; each now says so.
Docstrings, comments and the changelog only: qrf.py, held_out.py and the
test module are AST-equal to f3b489a with docstrings stripped.
Repinned the attested fit.qrf hashes: repin, coverage (41/41), repin,
--check clean, then graph_parity_repin fit.qrf; ruff format restores the
double quotes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arners The round-2 Subfleet review (job 20261115-223630-fit-fold-r2) raised no P0 or P1. Its one P2: the docstring said the model, the learners and the constant differ only where a learner was fitted. In a single-class fold whose training weights overflow float64, no learner is fitted, yet the constant (training shares) is NaN while the model and the filled learners give the class with certainty. The clause now covers the model and the learners only, which holds by construction. Docstring only; held_out.py's bytes are attested nowhere, so no repin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
changed the base branch from
fix/fit-input-validation-bug-parity
to
main
November 17, 2026 00:15
…-degenerate-classes #1543 landed on main as a merge commit, so this branch's copies of its commits are already in main's history. The only conflicts were the attested spec-engine hashes in four pin files. They took main's side, and #1617's line in inventory_coverage.py's repin comment was restored. Then tools/repin_us_spec_engine_contracts.py, tools/spec_engine_coverage.py (42,401/42,401 fields, 41/41 inventory checks), the repin again and --check (all exit 0), and python -m tools.graph_parity_repin fit.qrf. The fit.qrf node keys are unchanged from 708d67e, because qrf.py is unchanged. ruff format restores the double quotes. Main's other change to microcosm-fit since #1617's base is #1543's docstring edit to categorical_scores, which merged cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on microcosm#1543, which landed on main as
e0f728f. This branch has merged main (336b80c40) and been repinned. The independent review has finished (round 2 raised no P0 or P1).microcosm#1543 made
categorical_scoresrefuse a probability row that is not a distribution. While verifying it, that session found two fold-level cases it left alone because they were outside its brief. This PR fixes both.What changes
out_of_fold_proba: a fold whose training rows hold a single classCategoricalClassifier.fitfits no learner there, so each learner's rows (boosted,spline_logit) in that fold stay at zero. #1543's scorer refuses them ("row(s) of proba do not sum to 1"); main scored them at the 1e-4 log-loss floor.qrf._class_balanced_weights: a class whose rows carry no weightlabels=[0,0,1,1], weights=[1,1,0,0]gives NaN weights and a RuntimeWarning.ValueErrornaming the class and its total. A class whose total overflows to infinity still counts as carrying weight and is balanced as before (its rows weigh zero). If every class's total overflows, every row's weight is NaN and the fit fails in scikit-learn, also as before.Why fill instead of dropping or refusing
stage_health.py,_occupation_redraw_gate:if len(learners) == 2). Dropping a learner in a degenerate fold would switch that check off without a word.Who can reach these
can_deal_folds. In a major group with two sub-major groups or more, every class sits in every fold, so no training fold holds one class. In a major group with one sub-major group, every training fold holds that class, but no fold fits a learner, so nothing is filled and the output is unchanged.can_deal_foldsrequires n_R2 ≥ K._cross_fit_gatefirst runs_folds_carry_every_class, which tests the same per-class totals with the same predicate (not total > 0). Any fold whose training rows carry no weight for a class sends the whole gate to the in-sample gate (in_sample_weightless_fold). That check came before the balanced gate (UK SPI stage 1 keeps 4–17% of the tape's donors; replace the removed QRF tail gate with a donor-fidelity check #1276). The refusal guards the helper for any other caller.cross_fitted_class_rates. The brief says not to change any fit whose classes all carry weight, so the helper now balances that class as before, and a guard pins the fit bit for bit. That such a class's rows silently weigh zero is pre-existing and out of scope here. So is the case where every class's total overflows: every row's weight is then NaN and the fold gate's fit fails in scikit-learn (Input sample_weight contains NaN.), on main and on this branch alike.Valid input is unchanged
A main-vs-branch differential runs two fixed workloads through each tree's
microcosm.fit:out_of_fold_probaand the scorer._class_balanced_weightson 300 inputs where every class carries weight, and on 60 inputs with 1e308 weights (52 with an overflowing class total, 19 of those with every class overflowing);RegimeGatedQRFwithcross_fitted_balanced: weighted, grouped, the weightless-fold fallback, and three fits with an overflowing class total (two complete; one raises on every tree, and its error message is compared);out_of_fold_probawith row and household folds, everyboosted_weightand the no-learner cases.Every array matches bit for bit against both main
5658fc015and #1543's head18924d618: 33,492 values in the first workload and 28,871 in the second.After merging main, the branch (
ea5857e16) was compared again with main336b80c40, which includes #1543: both workloads match bit for bit (33,492 and 28,871 values). Main's only change tomicrocosm-fitsource since18924d618is #1543's docstring edit tocategorical_scores(AST-equal once docstrings are stripped).Tests
The new module is
packages/microcosm-fit/tests/engine_free/shared/test_fold_degenerate_classes.py, with 12 tests. Against both main and #1543's head, the 5 that check the fixes fail and the 7 guards pass. On this PR's earlier headb90f098fb, which also refused an overflowing total, the two overflow guards and the balanced-weights property fail.event()labels and explicit examples for the edge cases):out_of_fold_probareturns is a distribution in every row. Against an embedded copy of the old function, only learner rows that were all zeros move, and they equal the model's. Examples cover the boosted-only and spline-only models on a single-class fold, and folds that each hold one class.can_deal_foldsbound. Swapping in a dealing by the commonest class, or a random one, fails the property.Attested hashes
microcosm.build.spec_engine.seedsattestsmicrocosm.fit.qrf's source bytes. They were repinned with these tools,--checkclean:tools/repin_us_spec_engine_contracts.pytools/spec_engine_coverage.py(41/41 inventory checks)python -m tools.graph_parity_repin fit.qrfruff formatrestores the double quotes the repin tool drops. They were repinned again after the round-1 review's docstring fixes, and again after merging main, whose only conflicts were these pin files (--checkclean; 42,401/42,401 fields; thefit.qrfnode keys are unchanged). Atea5857e16, 392 pin, parity, seed andmicrocosm-fittests pass locally.Review
In-session pre-review (supporting evidence only): found the overflow reach (fixed in
f3b489a1b), a grouped-folds property that passed broken dealings (fixed), and wording slips (fixed).Round 1, Subfleet hard tier (job
20261114-143433-fit-fold-r1, headf3b489a1b): REQUEST_CHANGES. Both code fixes are correct and valid input is bit-identical. The reviewer reran both differentials, the old-tree tests, the repin--checkand a fuzz ofstratified_folds. What it flagged:out_of_fold_proba's docstring promised a distribution in every row of every array, which weights whose fold total overflows float64 break. The sentence now says only what the fill guarantees: in a single-class fold each learner's rows equal the model's, so none is left at zero.qrfdocstring, changelog and this body said an overflowing class's rows weigh zero, but when every class overflows every row's weight is NaN. Each now says so, as before on main.94ff4a721's message says no fit reached the refusal, which was false for the overflow half that commit also refused.f3b489a1bcorrects it. History is left as is, with no force-push.708d67e49, change docstrings, comments and the changelog only:qrf.py,held_out.pyand the test module are AST-equal tof3b489a1bonce docstrings are stripped.Round 2, Subfleet hard tier (job
20261115-223630-fit-fold-r2, head708d67e49, a full review): REQUEST_CHANGES, with no P0 or P1, so the review has finished under Review by class.--check, coverage and the parity tests.out_of_fold_proba's docstring said the model, the learners and the constant differ only where a learner was fitted. In a single-class fold whose training weights overflow, the constant is NaN while the model and filled learners are certain. The clause now covers the model and the learners only (docstring only;held_out.pyis not attested, so no repin).94ff4a721's message, as above; left in history and disclosed.Merge
microcosm-fitcode of this PR's: it brings in main and regenerates the attested hashes. The differential above shows valid input still bit-identical, so the finished review stands.🤖 Generated with Claude Code