Repository navigation
Name the US calibration arms by builder flag in pinned and generated text - #1641
Merged
Merged
Conversation
…text microcosm#1540 left five pieces of wording because changing them moves hash pins, a test-pinned string or a generated contract. This changes them and regenerates every pin with the repo's tools. - solve.py: calibrate_l0_refit says what it does instead of calling itself the production path; L0RefitResult, calibrate_l0_refit and refit_l0_selection drop "production artifact" and "shipped weights". - ssi_take_up.py: the delivery gate docstring names the arms by flag. The builder fences only --dense-default-dataset, keyed on the flag alone; the L0+refit arm and --exact-k keep hard enforcement. - build_us_fiscal_refresh_release.py: the dense SSI fence text that builds write into manifests names the arms the same way, and its test requires the flag names and refuses the retired labels. - US calibration contract: release modes take the builder's dataset.default.method values (l0_refit, dense_no_l0), and refit_contracts.default_sparse becomes l0_refit. Nothing outside the generator, the schema and the generated YAML reads the names. - us-release-build-rule.md: says which code uses microcosm.build.holdout and that no held-out test compares a US national candidate with the incumbent. Every pin below was written by its tool, none by hand: repin_us_spec_engine_contracts.py (two passes, --check rc 0), generate_us_bundle_from_constants.py (--check rc 0), spec_engine_coverage.py (two passes, --check rc 0; 42395/42395 fields, 41/41 inventory checks), graph_parity_repin.py calibrate (pins.json only; direct.csv unchanged). ruff format changed only the quotes the repinner writes. No flag, default, calibration, draw, RNG rule or dataset changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The comments beside EXPECTED_HASHES, the loader golden and the multispine spec_sha256 list each change that moved them. This adds microcosm#1641: the solve and ssi_take_up docstrings and the calibration contract's renamed modes. Comments only; the repinner, coverage and generator --check runs pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1387 changed acs_transfer.py, another attested module, so the seed, loader, multispine and coverage-report pins conflicted. Both PRs' comments stay beside each pin; the repinner, bundle generator, coverage tool and graph-parity repin rewrote every value, and the three --check runs exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
solve.py's module docstring, L0RefitResult.effective_sample_size and L0RefitResult.options still said the refit weights "ship" or called them "production"; they now say what calibrate_l0_refit returns, and three test_solve.py comments follow. The build rule's holdout paragraph now scopes "only the UK runtime runs the rotated folds" to the packages and tools, names the ACS local-area experiment that also uses them, and says support_mix also orders ACS households with the hash helpers. The contract generator's release_modes comment no longer says the kernel follows from _fiscal_release_cli_defaults: the kernel and default mode are set by hand to match that parse. solve.py's bytes moved again, so the repinner, bundle generator, coverage tool and graph-parity repin rewrote the seed, loader, multispine, coverage-report and calibrate pins; the three --check runs exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1614 and #1543 moved the same pins as this PR, so the seed, loader, multispine, pointer-inventory and coverage-report pins conflicted. Main's side was taken in each pin file and both PRs' comments kept; the repinner, bundle generator, coverage tool and graph-parity repin then rewrote every value. The three --check runs exit 0 and tools/ci_test_plan.py verify reports ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SSI gate docstring now calls P2 and P3 that arm's two observed full-pool frames: microcosm#567 records two attempts on each. A comment in _optimize_proximal and the last test_solve docstring that said the refit weights ship or are published now say returned weights. The release_modes comment notes that solver.l0.mode keeps the approved binding literal exact_k_projection. solve and ssi_take_up are attested modules, so the repinner, coverage tool and graph-parity repin rewrote the seed, loader, multispine, coverage-report and calibrate pins. The three --check runs exit 0 and tools/ci_test_plan.py verify reports ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1617 moved the seed, loader and multispine pins and the coverage report that this PR also moves. Main's side was taken in each pin file and both PRs' comments kept; the repinner, bundle generator, coverage tool and graph-parity repin then rewrote every value. The three --check runs exit 0 and tools/ci_test_plan.py verify reports ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1650 edited docs/us-release-build-rule.md in other paragraphs (a sentence on Route A's builds and a 16 November 2026 block before section 1); git merged it cleanly around this PR's holdout paragraph. It touches no attested module or spec input, so no pin moves; the three --check runs and tools/ci_test_plan.py verify confirm it on the merged tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Hand-off to the Microcosm hub to land on green CI.
|
juaristi22
added a commit
that referenced
this pull request
Nov 17, 2026
main's 124 commits since 9504278 conflicted only in the six F0 pin files. They were resolved to main's side and re-pinned: repin --legacy-only (no change), the generator with the US engine (no bundle file changed), the full repin, spec_engine_coverage.py, then ruff format on the five Python files it rewrites. Coverage reads 42404/42404 configuration fields and 41/41 inventory checks, and every --check passes. spec_sha256 is now 0088d3f6; the pool test's comment keeps main's three lines (#922, #1543, #1641) and this branch's us/gates.json line. main's microcosm#922 county work merged cleanly into the ACS local tool: county support, its re-applied moves, the resume shortcut's evidence check and do_package's county CSV all sit outside this branch's shadow sites, which keep their call shapes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
microcosm#1540 aligned DESIGN.md, the US release build rule and the US builder's help text with the code and the release manifests. It left four pieces of wording because changing them moves hash pins, a test-pinned string or a generated contract. This PR changes them and regenerates every pin and generated file with the repo's tools. It also corrects what
docs/us-release-build-rule.mdsays about wheremicrocosm.build.holdoutis used. No flag, default, calibration, draw, RNG rule or dataset changes.What the code does
tools/build_us_fiscal_refresh_release.py) has three arms.--exact-krunscalibrate_exact_k_ladder.--dense-default-datasetmakes onecalibrate()call with no L0 term and recordsdataset.default.method: dense_no_l0. With neither flag, it runscalibrate_l0_refitand recordsl0_refit. All three reach the same_enforce_ssi_take_up_deliverycall in_main. That call passesUS_DENSE_SSI_TAKE_UP_ENFORCEMENT_FENCESonly whenargs.dense_default_datasetis set, so a dense run whose selection source first cuts the pool is fenced too.calibrate_l0_refitreturns anL0RefitResultwhose frame and weights are the refit stage's; its summary properties delegate to the refit._fiscal_release_cli_defaultsintools/us_bundle_generation/contracts.pyreads the contract's solver defaults (epochs, learning rate, weight cap, L0 and L2 settings, seed) by parsing the builder with only its required arguments, so neither arm flag is set. The contract'ssolver.kernel: kernel:calibrate_l0_refitandl0_contract.default_mode: fixed_l0_then_refitare set by hand, and they match that parse, which is the L0+refit arm.calibration.schema.jsonand the generatedcalibration.yamlname the release modes. No Python reads a renamed mode or refit-contract key;test_us_spec_bundle.pycountsrefit_contracts' entries, and there are still two. The legacy adapter projects the whole calibration document intocalibration_contract, and the repinner and coverage tool hash it, so the rename moves those pins and nothing else.microcosm.build.holdout's rotated folds run only inuk_runtime/local_rowwise.py; the ACS local-area experimentexperiments/us-acs-local-l2-basis-20260928/prepare.pyalso uses them. US package and tool code imports only the module's keyed-hash helpers.tools/us_acs_local_cd_surface.pyandtools/us_acs_local_county_surface.pyhold district and county SOI target blocks out of the ACS local-area solve, andscore_cd_holdoutandscore_county_holdoutmark their resultsreport_only.build_us_acs_local_release.py'scd_holdoutgate is alwayspassed: True.us_runtime/support_mix.pyuses the helpers to split targets and to order ACS households for the support-mix bake-off. The national builder imports neither helper. Its three*_holdout.jsonwriters turn a failed measurement into an "unavailable" report, and no gate reads the reports.Changes
calibrate_l0_refit(solve.py) now says what it does: a "generate big, then prune" solve whose result is the refit stage's frame and weights. It no longer calls itself "the production … path". The rest of the module's wording about the refit stage follows:L0RefitResultno longer calls the refit weights "the production artifact", itseffective_sample_sizeandoptionsdocstrings say "the returned vector" and "refit options" where they said "the shipped vector" and "production options", the module docstring's L2 note sayscalibrate_l0_refitreturns the refit stage's weights where it said they "ship",calibrate_l0_refitandrefit_l0_selectionsay "returned weights" where they said "shipped weights", and a comment in_optimize_proximalsays the returned weights, not "the shipped dataset", agree with the reportedn_nonzero. The module's pins move either way. The comments and docstrings intest_solve.pythat said the refit weights ship or are published take the same wording.us_ssi_take_up_delivery_gatedocstring (us_runtime/ssi_take_up.py) names the arms by flag where it said "the sparse certified default" and "the dense full-pool arm". The builder passes fences on--dense-default-datasetonly, keyed on the flag alone. The L0+refit arm (neither flag) and--exact-kpass none. It calls P2 and P3 "that arm's two observed full-pool frames" where it said "either observed frame"._US_DENSE_SSI_FENCE_ADJUDICATION, which builds write into their manifests, now opens "Fenced for the --dense-default-dataset arm", calls P2 and P3 "either observed full-pool frame", and ends "The L0+refit and --exact-k arms pass no fences and keep hard enforcement." Its recompute figures are unchanged.test_us_fiscal_refresh_builder.pynow requires the opening and closing phrases and refuses "diagnostic arm" and "sparse certified". The fixture fence strings intest_us_ssi_take_up.pyfollow the new wording.dataset.default.methodvalues the builder writes:default_sparse_l0_refit→l0_refitanddense_diagnostic→dense_no_l0.refit_contracts.default_sparse→refit_contracts.l0_refit, with the matchingconstreference.exact_k_or_full_pool_refitkeeps its name. The edits are intools/us_bundle_generation/contracts.pyand the hand-authoredcalibration.schema.json, and the generator wrotecalibration.yaml.solver.kernelstayskernel:calibrate_l0_refit, andl0_contract.default_modestaysfixed_l0_then_refit. Both are set by hand to match the no-flag parse above, and a comment besiderelease_modesnow says so. It also notes thatsolver.l0.modekeeps the approved binding literalexact_k_projection, which neither the hand-set fields nor the parse sets.docs/us-release-build-rule.mdno longer saysmicrocosm.build.holdout"is imported by the UK runtime and UK tools only". It says which code uses which part, as above. It keeps the paragraph's point in a narrower form: no held-out test compares a US national candidate with the incumbent.changelog.d/calibration-arm-wording.fixed.mdrecords the change.Pins and generated files
The values below are against main at
17fecf5e. While this PR was open, microcosm#1619 (for #1387), #1614, #1543 and #1617 changed attested modules or the same pins, so the seed, loader, multispine and pointer-inventory pins and the coverage report conflicted. Each time I merged main, took main's side in the pin files, kept both PRs' comments beside each pin, and re-ran the whole sequence below on the merged tree after the last edit to an attested module; the tools wrote every value.Every value was written by its tool, none by hand, on Python 3.14 (arm64/darwin) with numpy 2.4.6, pandas 3.0.3, scipy 1.17.1 and torch 2.12.0, the versions
pins.jsonrecords. The sequence follows the onetools/repin_us_spec_engine_contracts.pydocuments, with a second repinner pass, a second coverage pass and the graph-parity repin.tools/repin_us_spec_engine_contracts.py --legacy-only, thentools/generate_us_bundle_from_constants.py(rewrotecalibration.yaml;spec_sha2564e2b6c19→372858d0), then the repinner twice:inventory_coverage.pyEXPECTED_HASHES:seed_map6474719c→d756dc67,seed_protocold3a0baf8→2c57cbfb. Both moved becausesolve.pyandssi_take_up.pyare in_DIRECT_KERNEL_MODULES.field_usage.py_PINS["calibration"]: still 320 pointers, digest49a153b0→5336f77a(the renamed keys).tools/spec_engine_coverage.pyEXPECTED_POINTER_INVENTORY_SHA256:7765ab26→65c75e48.test_spec_engine_calibration_semantics.pyCANONICAL_LEGACY_PROJECTION_SHA256:b289d8a5→025ed103.test_spec_engine_loader.pygolden:d4fc68aa→cb9742d5.test_us_multispine_pool_tool.pyspec_sha256:4e2b6c19→372858d0.test_spec_engine_coverage_tool.py: the first pass moved its count and the second restored it, so it is unchanged.tools/spec_engine_coverage.py, run twice: ten digest lines indocs/evidence/spec-engine/us-f0-coverage.json.tools/graph_parity_repin.py calibrate:pins.jsonimplementation_hash07e4330f→3df328c2,node_key2972c813→98df2301(arm64/darwin/py3.14).direct.csv,graph.jsonandinputs.csvare unchanged.generate_us_bundle_from_constants.py --check,repin_us_spec_engine_contracts.py --checkandspec_engine_coverage.py --checkall exit 0, andtools/ci_test_plan.py verifyreports ok. Coverage reports 42401/42401 configuration fields and 41/41 inventory checks.The comments that record why each pin moved, beside
EXPECTED_HASHES, the loader golden and the multispinespec_sha256, now cite this PR. They are comments, so the three--checkruns above still pass with them.ruff formatchanged only the quotes the repinner writes. It also rewrapped two unrelated statements inssi_take_up.py; I reverted those so that the module's bytes match its pins.What this PR leaves as written
changelog.d/dense-takeup-enforcement.changed.mdstill says "Dense diagnostic arm" and "the sparse certified default", andexperiments/build_j_recert/issue368_verdict.mdandPROGRESS_BUILDJ.mdstill say "dense diagnostic arm". They record what was true when they were written, so this PR leaves them.--selection-source-h5and--selection-source-manifesthelp (tools/build_us_fiscal_refresh_release.py, around lines 1668 and 1679), three builder comments (around lines 16514, 16575 and 16882),us_runtime/warm_start_selection.pyaround line 579,docs/us-esi-employer-premiums.mdanddocs/informed-l0-warm-start-design.md. microcosm#1540's table shows Builds I, J and M–P ran the dense arm on a selected 57,240-household cut, so the selection-source help is misleading: it implies an L0 arm, while those builds ran--dense-default-dataseton a selected cut. None of it was on Align DESIGN.md and US builder help with how releases calibrate #1540's list, and this PR leaves it for a separate change.axiom: n/a: wording in docs, docstrings, test strings, the builder's manifest fence text and a generated contract, with regenerated pins
🤖 Generated with Claude Code