Repository navigation
US: one copy of each shared helper (#1585 items 4, 5 and 8.3) - #1601
Conversation
…pers Before item 4 of #1585 moves the US seeded BLAKE2b draws onto stochastic_assignment.stable_identity_uniforms, this test copies each of the eleven loops as main has them at 90e4d94, with its salt written out, and holds each module's helper to its copy bit for bit: the same float64 bits in the same container (an array of the same shape, a Series on the same index, or a Python float). The inputs cover integer, float and string keys, Python and NumPy integer seeds, empty input and the composite salt of the SSI disability draw. It also checks that the private names other modules import still reach the helpers, and holds the shared helper to the copied loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…em 4)
Eleven US modules carried copies of the seeded BLAKE2b loop that
microcosm.build.stochastic_assignment.stable_identity_uniforms implements.
Each now builds its keys as before and passes them to the shared helper,
with the salt its copy used:
- take_up: the program variable (also reached by medicaid_take_up,
snap_state_take_up and acs_local_take_up);
- snap_take_up: "snap_take_up";
- snap_discretionary_exemption: "snap_abawd_discretionary_exemption";
- ssi_take_up: "takes_up_ssi_if_eligible", one key per call;
- wic_claim: "would_claim_wic";
- pregnancy: "pregnancy";
- immigration: the caller's salt (it used np.fromiter; the values are the
same);
- source_runtime (the ACA-family operations): the caller's salt, on the raw
tax_unit_id values or the positional index, returned as a Series on the
frame's index;
- acs_local_ssi_disability: "acs_local_ssi_disability_criteria:{stream}";
- acs_local_ssi_medicaid_take_up: the stream alone, one key per call;
- acs_local_vehicles_head_start: "acs_local_head_start_take_up".
The private helper names stay, because other modules and test support
import them. The two one-key helpers still return a Python float. The shared
helper formats int(seed), where take_up, pregnancy and the SNAP exemption
formatted the seed as given; the bytes are the same for every integer seed,
and every caller passes one. hashlib stays only where a module still takes a
sha256 digest.
test_us_keyed_draw_equivalence.py holds every helper to its frozen copy bit
for bit. The seed-protocol anchors and the US F0 pins move in the next
commits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The US keyed draws are now computed by stochastic_assignment.stable_identity_uniforms, so the seed protocol's legacy_v1_direct_draws kernel attests that module's source as well as the callers'. Without it, an edit to the helper would change the US draws without moving the seed-protocol hash. The cost: UK edits to the module (six since August) will also move the US F0 pins. The audited-source test accepts stable_identity_uniforms as a stochastic anchor, since nine sites no longer contain "hashlib.", and requires a site whose source calls the helper to attest the helper's module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The eleven copies shared one loop, so the test freezes that loop once, as main had it at 90e4d94. It holds stable_identity_uniforms to it, and each module's draw to it on the module's own keys and salt, in the container the module returned. The modules' key-building code is unchanged, so it needs no frozen copy. 703 lines become 175. The test passes on main's untouched modules (64 passed) and on this branch (64 passed), with the country engines hidden as on CI's engine-free job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The US keyed draws now call stochastic_assignment.stable_identity_uniforms, and that module joins the seed protocol's attested kernel modules. Both change attested source bytes, so the seed-protocol and seed-map digests move, with the spec hash that binds them. The new module adds one field to the seed protocol's inventory. No draw changes. - seed_protocol e1f88676… -> 63a04284…, seed_map bcb012ed… -> af9d9678… - US spec_sha256 3dabec5c… -> 007cd898… (pool tool pin) - loader golden d5c303a4… -> 5e2053da… - pointer inventory f6832e8d… -> 81bc649e…; resolved_seed_protocol pointers 827 -> 828 (525ed862… -> c358fa57…) - configuration fields 42394 -> 42395, resolved bindings 9785 -> 9786, compiler_semantic 27673 -> 27674, no_generation0_effect 3614 -> 3615 - docs/evidence/spec-engine/us-f0-coverage.json regenerated The documented sequence: repin --legacy-only (no change), the bundle generator (no YAML change), repin, spec_engine_coverage, then repin once more for the coverage tool's counts. All three --check modes pass, and the coverage reads 42395/42395 fields and 41/41 inventory checks. Tests: loader 10, field usage 14, seeds 11, engine-free spec bundle 4, coverage tool 7, engine-contract spec bundle 22, the pool tool's pin test 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The release tool's US_POPULATION_AGGREGATE_VARIABLES and _signed_component are now compatibility_recertification's POPULATION_AGGREGATE_VARIABLES and signed_component. Each pair was identical, and a test held them equal. - One known-period walker, compatibility_recertification's engine_known_periods, replaces the two copies, which differed. Its defaults are the release tool's: it walks live branches, _target_materialization_branches and baseline, and raises on a variable an engine does not define. The recertification tool passes walk_branches=False, skip_unknown=True, as its copy read the root alone and skipped such a variable. The walker stays in that module, so PRODUCER_SOURCE_FILES is unchanged. - support_mix.hash_uniform is holdout.hash_holdout_uniform, and support_mix.kish_ess calls calibrate.solve.effective_sample_size, imported when called because solve imports torch. The values are identical; only the ValueError text differs, and nothing matches on it. - tools/us_acs_local_cd_surface.py's kish_ess also calls effective_sample_size. That adds validation: it now refuses negative, NaN or infinite weights, which its own copy scored. Every valid vector scores as before. The parity test holds the release tool's names to the shared ones and checks each tool's walker mode; the existing walker tests in test_us_post_export_scoring.py now run against the shared walker. Tests: the parity file 4, batched target materialization 37 (engine-free) and 4 (engine workflow), post-export scoring 48, head-to-head scorer 54, county surface 44, sparse district 27 (1 feed-dependent skip), district properties 12, support mix 25, bake-off tool 27, recertification 105 and its engine scoring 2, solve 82, and the builder's two signed-component tests 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The US keyed draws now import microcosm.build.stochastic_assignment, so the transport gate bindings reach one more first-party module through us_runtime: 245 rather than 244. The battery's binding-identity test lists it among the modules added after its base. The identity digests every module's source at runtime and pins no value, so nothing else moves. Tests: transport evidence kernels 166, binding identity 32, income evidence kernels 516 (1 skipped: the NZ feed audit needs NZ_CHRONICLE_ARTIFACT). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm.calibrate.solve.solve_constraint_system runs the Adam solve that calibrate runs once it has compiled a frame and a target set, for a caller that already holds the compiled system, with no L0 gates. A SciPy matrix becomes the torch operator calibrate builds. Any other operator is passed to the module-level _optimize as given, so an operator the caller built keeps its float32 arithmetic. calibrate's option checks for mass, epochs, the weight ratio, L2 and the mass parametrization move, unchanged and in the same order with the same messages, into four private helpers that calibrate and the new solve both call. The initial weights go through the frame's Weights, as calibrate's do. tools/bakeoff_us_support_mix.py now calls the public solve with its own operator instead of _optimize. The solve checks its inputs as calibrate does and seeds torch's RNG with 0 on entry; a solve without L0 gates draws nothing from it, so the weights and trajectory are unchanged. uk_runtime/firm_generation.py also imports _optimize; that is left for a UK follow-up. The test holds the public solve to: - the direct _optimize call the bake-off made, and the default solve, bit for bit; - calibrate on the problem calibrate compiled, trajectory included; - passing a caller's operator to the module-level _optimize as the same object. Tests: every calibrate engine-free file (22, the new one 10), the bake-off tool 27, and the engine-free transport and UK files that read calibrate.solve (transport compile 11, transport graph 24, UK full calibration graph 15, UK full gates 24, UK local doctrine 9, UK target compile 52, UK UC matrix diagnostics 24, UK size evaluation 15). The calibrate parity case and the US F0 pins move in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The seed protocol and the calibrate.adam@1 parity kernel both hash microcosm.calibrate.solve, so the public solve_constraint_system and the checks it now shares with calibrate move their digests. No solve changes: the parity re-pin refuses if the kernel's direct output bytes move, and direct.csv is unchanged. - seed_protocol 63a04284… -> 0770f952…, seed_map af9d9678… -> 135522c0… - US spec_sha256 007cd898… -> 12530524… (pool tool pin) - loader golden 5e2053da… -> 09b8d791… - the calibrate H1 parity case on arm64/darwin/py3.14 (tools/graph_parity_repin.py calibrate, under the pinned dependency versions): implementation_hash f6554bfe… -> aa55d948…, node_key b6020102… -> 6aa80290… - docs/evidence/spec-engine/us-f0-coverage.json regenerated The documented sequence, with the second re-pin pass for the coverage tool's counts; all three --check modes pass, with 42395/42395 fields and 41/41 inventory checks. Tests: graph parity pins 11, loader 10, field usage 14, seeds 11, engine-free spec bundle 4, coverage tool 7, engine-contract spec bundle 22, the pool tool's pin test 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The seed ledger recorded the TANF take-up site's literal salt as takes_up_tanf, but take_up._stable_unit_draws has always salted that draw with the program variable, takes_up_tanf_if_eligible (us/spec/take_up.yaml). The ledger now records takes_up_tanf_if_eligible. No draw changes. A new test holds each take-up site's ledger salt to the variable its draw is salted with (TANF, EITC, Medicaid, SNAP by state and SSI), and requires every seeded program in the take-up contract to have such a site. It fails on the old ledger and passes on this one. This commit carries its own re-pin, so it can be dropped on its own: - seed_protocol 0770f952… -> 15c30c15…, seed_map 135522c0… -> 208fa485… - US spec_sha256 12530524… -> 9e5979da… (pool tool pin) - loader golden 09b8d791… -> 1d373cec… - us-f0-coverage.json regenerated; the seed sites' observed digest moves a0366cd5… -> 67302e3a… All three --check modes pass, with 42395/42395 fields and 41/41 inventory checks. Tests: seeds 12, loader 10, field usage 14, engine-free spec bundle 4, coverage tool 7, engine-contract spec bundle 22, take-up semantics 16, the pool tool's pin test 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review of c78edd0. I checked each removed copy against its shared helper (salt, key, seed formatting, dtype, return container, error paths), the walker's two modes, the public solve against Nit
Checked and fine
CI at head: lint, wheels, installed-emitter, select-countries, integration-uk and engine-uk (3.13 and 3.14) pass. engine-free and engine-us (3.13 and 3.14) are still running, so the darwin-recorded H1 parity re-pin and the engine-backed spec bundle checks are not confirmed on CI yet. Verdict: the refactor keeps every draw, walker result, ESS and solve bit for bit on valid input, as #1585 asks; the three nits are a changelog line and two tests. 🤖 Generated with Claude Code |
Vahid's review of c78edd0, nits 1 and 2: - cd_surface.kish_ess is also read by the county surface and by the ACS local release's weight-origin summaries of the design weights, not only the calibrated ones, so the changelog fragment names them. Nothing that runs today reaches the refusal: the design weights pass through Weights first, and an empty cell still scores 0.0. - A test holds cd_surface.kish_ess to the refusal of a negative, NaN and infinite weight, and to 0.0 for an empty vector. - A test runs the bake-off's own _SparseOperator through solve_constraint_system with the arm's options and holds it bit for bit to the direct _optimize call the arm made before, so a change to the class or to the solve's shape check is caught. Tests: district ESS gate 52, bake-off tool 28. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vahid's review of c78edd0, nit 3: the ledger test now builds the generic seeder's sites from seeded_take_up_programs(), one per program, instead of two literals, and its failure names the site and the variable. On the old ledger it fails with ('tanf_take_up_assignment', 'takes_up_tanf_if_eligible'). This belongs to the TANF commit, so the two revert together. Tests: seeds 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Vahid. All three are in:
🤖 Generated with Claude Code |
Main (52aea14) moved the same pins this branch moves: #1598 changes medicaid_take_up, an attested module. The five pin files conflicted. Both sides' reason comments are kept, and every value is recomputed on the merged tree: - the documented sequence (repin --legacy-only, the bundle generator, repin, spec_engine_coverage, repin again), with all three --check modes passing and 42395/42395 fields and 41/41 inventory checks; - tools/graph_parity_repin.py calibrate, with the case's direct output unchanged. Merged values, against main's: - seed_protocol 15e6cad1… -> a696ad74…, seed_map 0c19b903… -> 270cf649… - US spec_sha256 98d58365… -> bd526d8c… (pool tool pin) - loader golden 99f9b82e… -> 1ff3b42c… - calibrate parity implementation_hash 0c669ad2… -> 018974ad… The release tool and test_spec_engine_seeds.py (#1047) merged without conflict. The venv is resynced for main's microcosm-emitter-interface 0.15.0. Tests on the merged tree: seeds 13, loader 10, field usage 14, engine-free spec bundle 4, graph parity pins 11, keyed draws 64, recertification parity 4, post-export scoring 48, batched target materialization 37, the builder's signed-component tests 2, head-to-head scorer 54, coverage tool 7, engine-contract spec bundle 22, the pool tool's pin test 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Round 2 at 6ace386. I checked f3eb245 and 6e78f90 against the round 1 nits, ran each new test against the old code, and checked the merge of Round 1 items
Round 1's open CI point is closed: engine-free and engine-us (3.13 and 3.14) pass at this head, so the H1 parity re-pin is confirmed on CI. The calibrate case is No new findings. Checked and fine
CI at head: all 12 checks pass. Verdict: all three nits are fixed, the new tests fail on the old code, and the outputs stay bit for bit the same. 🤖 Generated with Claude Code |
…1603) Main re-pinned the US spec-engine files this branch also pins. The six conflicts were all generated counts and digests; each took main's side and the five-step repin recomputed them on the merged tree (repin_us_spec_engine_contracts.py --legacy-only, generate_us_bundle_from_constants.py, repin_us_spec_engine_contracts.py twice per #1530, spec_engine_coverage.py). - Configuration fields: 42,401 (main's 42,395 plus this branch's six pointers for the two declared legacy resources); 41/41 inventory checks. - US bundle spec_sha256: d3a3c6c6fbcf4bd7cae760215b909214e5593d606da74aa8f78e6c347e3b41b5. - The three --check modes and tools/ci_test_plan.py verify exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I checked the eleven US draws against the loops they replace, the release and recertification tools against their old copies of the shared rules, and calibrate's moved checks, and found nothing to change. |
Main gained #1612, #1596, #1603, #1601 and #1597 since the 68bbff7 this branch last merged (45 commits, 174 files). #1601 ("US: one copy of each shared helper") edited microcosm.calibrate.solve, an attested kernel module, and regenerated the shared pins, so the five pin files conflicted. No code or test file conflicted. Each conflict block takes main's text, with this branch's #1096 comment sentence kept after main's, and the tools then wrote the values for the merged tree: - docs/evidence/spec-engine/us-f0-coverage.json: main's five blocks (eight digest lines), then rewritten by tools/spec_engine_coverage.py (two map, two protocol, two implementation and two spec digests; 42395/42395 fields, 41/41 checks). - spec_engine/inventory_coverage.py: main's #1585 comment and digests, then the #1096 sentence after it; tools/repin_us_spec_engine_contracts.py wrote seed_map 8d0542f7 and seed_protocol 3ab23e82 (main: 270cf649, a696ad74; this branch before: cd5c2f2c, 75891f4d). - test_spec_engine_loader.py: main's digest line under the #1096 comment line; the repinner wrote the golden fac42090 (main: 1ff3b42c; branch: a7b6a302). - test_us_multispine_pool_tool.py: the #1096 comment line and main's digest line; the repinner wrote spec_sha256 fa75bd68 (main: bd526d8c; branch: b768e100). - parity/kernels/calibrate/pins.json: main's line, then tools/graph_parity_repin.py calibrate wrote implementation_hash 07e4330f and node_key 2972c813 (main: 018974ad, 30ad70b1; branch: 1b3e2a71, f9472a28). direct.csv, graph.json and inputs.csv did not move and equal main's, so the default path's bytes are main's. The repinner and the coverage tool ran twice each, then both passed --check; the repinner's first pass moved test_spec_engine_coverage_tool.py's count and its second restored main's. tools/ci_test_plan.py verify passes with main's 968 test files. ruff format changed only the quotes the repinner writes. solve.py merged without conflict. #1601 and this branch both changed calibrate: #1601 moved its option checks into _check_mass, _check_epochs_and_weight_ratio, _checked_l2_options and _check_l2_anchor_shape, and this branch rewrote the mass_parametrization paragraph of its docstring. The merged function has #1601's code and this branch's docstring. Every other definition is one side's alone: the merged file is main's plus exactly this branch's edit. #1601's new solve_constraint_system calls _optimize, which this branch changed, with the signature it still has; under mass_parametrization="softmax" it now runs the exact per-step projection. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Main gained 47 commits since the 68bbff7 this branch last merged: #1612 (a test of the build-folder publisher), #1596 (the US bundle generator's tombstone renewal), #1603 (UK local-area releases enter build folders through the registry), #1601 (US: one copy of each shared helper, #1585) and #1597 (UK stock cells, #1524). Four files conflicted, all of them pin files. No conflicted hunk takes a whole-file side: each takes main's text, keeps this branch's comment sentences, and leaves the value to the tools. - spec_engine/inventory_coverage.py, one hunk at seed_map and seed_protocol: main's seven comment lines byte for byte (#1585), then this branch's two sentences (#1444 and #1453) on three new lines. - tests/engine_free/shared/test_spec_engine_loader.py, one hunk: the golden value. This branch's two comment lines had merged cleanly. - tests/engine_workflow/us/test_us_multispine_pool_tool.py, one hunk: this branch's two comment lines, whose first is main's line, and the spec_sha256. - docs/evidence/spec-engine/us-f0-coverage.json, five hunks, digests only. One conflict git did not report, in tests/engine_free/us/test_us_spine_blindness.py. #1601 made us_runtime/support_mix.py re-export microcosm.build.holdout's hash_holdout_uniform as hash_uniform. This branch's test_out_of_runtime_implementations_are_exactly_the_runtime_re_exports requires the registry of out-of-runtime implementations to name every such re-export, so it fails on the merged tree though each parent passes alone. The registry gains ("support_mix.py", "microcosm.build.holdout"), so the runtime scans read holdout.py. The pool tool's import graph does not reach support_mix.py (only the support-mix bake-off tool imports it), so the pool test now requires the graph to reach exactly the re-exports in the new _POOL_BUILD_RE_EXPORTS (local_jurisdictions.py), each of which must be registered. The pinned graph size stays 84. main's support_mix.py is unchanged. Both sides changed five definitions, none of them in its logic. Three hold a pin (EXPECTED_HASHES and the two pinned tests above): with the digests masked, the merged AST equals main's and this branch's. The other two are unions. BINDING_MODULES_ADDED_AFTER_BASE in test_transport_evidence_kernels.py holds main's stochastic_assignment and this branch's two modules, 26 names. The imports of tools/build_us_fiscal_refresh_release.py hold main's recert and this branch's rederive_us_local_jurisdiction_flags. Every other definition in the files both sides changed was changed by one side, and the merged AST equals that side's. In tools/build_us_fiscal_refresh_release.py main re-bound US_POPULATION_AGGREGATE_VARIABLES, _signed_component and _engine_known_periods to the shared rules in microcosm.data.compatibility_recertification, and this branch changed _main and added _with_local_jurisdiction_flags. In test_us_fiscal_refresh_builder.py each side added tests. Pins on the merged tree, each written by its tool: - tools/repin_us_spec_engine_contracts.py: seed_map a402868b and seed_protocol 27fd0f86 in inventory_coverage.py; the golden 7a2c31e2 in shared/test_spec_engine_loader.py; the US spec_sha256 3ec2d79d in engine_workflow test_us_multispine_pool_tool.py. Its covered_item_count in engine_contract test_spec_engine_coverage_tool.py ends at main's 41. - tools/spec_engine_coverage.py: us-f0-coverage.json (the seed map and protocol, the seed-stream implementation digest and spec_sha256, each twice); 42395/42395 configuration fields and 41/41 inventory checks. - Unmoved by the merge, as the coverage tool observes them again: full_checkpoint 47d4474d and the stacked pool's code digest 3c5e14be, this branch's values on lines main did not touch. The calibrate parity fixture (direct.csv, graph.json, inputs.csv, pins.json) is byte-identical to main's; this branch changes nothing under microcosm.calibrate. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Main brings 495 commits, #1601's shared keyed-draw helper among them. Eight files conflicted, all of them pins the spec-engine tools write: the inventory-coverage digests, the field-usage counts and pointer pins, the coverage tool's pointer-inventory digest, the loader and pool-tool spec_sha256 goldens, the field-usage and coverage-tool count tests, and the coverage evidence report. Each hunk takes main's text so the files parse, and the inventory-coverage, loader and pool-tool comments keep this branch's microcosm#776 lines. The next commit recomputes the values with the tools. #1601 moves immigration._stable_person_draws onto stochastic_assignment.stable_identity_uniforms, the same blake2b(seed:salt:key) / 2**64 draw with an integer seed, so the arrival-year draw this branch adds keeps its values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's medicaid_take_up value ranks and #1601's shared draw helper moved the same pins this branch's entry clock moves, so neither side's value holds for the merge. The tools wrote every value, in the documented order: the release input coverage manifest and the legacy-only repin changed nothing, the bundle generator (policyengine-us 2.2.1) rewrote no file and reports spec_sha256 c02312b5, then the repinner, the coverage report, the repinner again (covered_item_count back to 41) and the coverage report again. - seed map e476e135, seed protocol e7faba84 (54 sites); the source manifest keeps this branch's ee208024. - Field ledger 42,452 fields (32,647 authored, 9,805 resolved); the resolved seed protocol claim covers 844 pointers; pointer inventory 59df84d5. - Loader golden e8679fec; the pool tool's spec_binding c02312b5. The repinner, coverage and pool engine-contract checks pass, and ci_test_plan verify is ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #1585 (items 4, 5 and 8.3).
Summary
The US pipeline carried its own copies of four shared helpers. This PR replaces each copy with a call to the shared function: the keyed BLAKE2b draw, the population-aggregate list with its known-period walker, the SOI signed-component rule, and the Kish ESS and hash draw in the support-mix bake-off. It also adds a public solve for a prebuilt constraint system, so the bake-off no longer calls the private
_optimize.Every draw, ESS, walker result and solve stays bit for bit the same on valid input. There is one intended behaviour change: the ACS local-area district surface's ESS now refuses negative, NaN or infinite weights (see below). No dataset is rebuilt.
Changes
Item 4: one keyed-draw function
Eleven US modules each had a private copy of
blake2b(f"{seed}:{salt}:{key}", digest_size=8), read big-endian and divided by 2^64. Each module still builds its keys as before, and now passes them tostochastic_assignment.stable_identity_uniformswith the salt its copy used:take_up(also reached bymedicaid_take_up,snap_state_take_upandacs_local_take_up)snap_take_upsnap_take_upsnap_discretionary_exemptionsnap_abawd_discretionary_exemptionssi_take_up(one key per call)takes_up_ssi_if_eligiblewic_claimwould_claim_wicpregnancypregnancyimmigrationimmigration:ead_workers,immigration:ead_students)source_runtime(the ACA-family operations)tax_unit_idvalues or the positional indexacs_local_ssi_disabilityacs_local_ssi_disability_criteria:{stream}acs_local_ssi_medicaid_take_up(one key per call)acs_local_vehicles_head_startacs_local_head_start_take_upsource_runtimestill returns a Series on the frame's index.int(seed), wheretake_up,pregnancyand the SNAP exemption formatted the seed as given. Both give the same bytes for every integer seed, Python or NumPy, and every caller passes one: atype=intcommand-line flag, an integer constant, orint(...).The seed protocol's
legacy_v1_direct_drawskernel now attestsmicrocosm.build.stochastic_assignmentas well as the modules that call it. Without that, an edit to the helper could change the US draws without moving the seed-protocol hash. The audited-source test acceptsstable_identity_uniformsas a stochastic anchor, since nine sites no longer containhashlib.. It also requires a site that calls the helper to attest the helper's module.Item 5: one copy of each release rule
The aggregate list and the signed-component rule. The release tool's
US_POPULATION_AGGREGATE_VARIABLESand_signed_componentare nowcompatibility_recertification.POPULATION_AGGREGATE_VARIABLESandsigned_component. The two pairs were already identical, and a test held each pair equal.The known-period walker. The two copies were not identical:
_target_materialization_branchesandbaseline, and raised on a variable the engine does not define.One walker,
compatibility_recertification.engine_known_periods, now has explicitwalk_branchesandskip_unknownoptions, with defaults matching the release tool. The recertification tool passeswalk_branches=False, skip_unknown=True, so both tools behave exactly as before. The walker stays in that module, soPRODUCER_SOURCE_FILESis unchanged.The support-mix helpers.
support_mix.hash_uniformis nowholdout.hash_holdout_uniform.support_mix.kish_esscallscalibrate.solve.effective_sample_size. The import sits inside the function, becausesolve.pyimports torch at module level.ValueErrordiffers (for example "holdout key must be…" for "hash key must be…"), and no caller or test matches on it.The district surface's ESS.
tools/us_acs_local_cd_surface.py'skish_essalso callseffective_sample_size. See the behaviour change below.Item 8.3: a public solve for a prebuilt system
microcosm.calibrate.solve.solve_constraint_system(matrix, targets, initial_weights, ...)is the Adam solve thatcalibrateruns once it has compiled a frame and a target set.calibrate's own checks, which move unchanged, in the same order and with the same messages, into four private helpers that both functions call. The initial weights go through the frame'sWeights, ascalibrate's do: one-dimensional, finite, non-negative and not all zero.calibratebuilds. Any other operator is passed to_optimizeunchanged, so an operator the caller built for speed keeps its float32 arithmetic._optimize, which tests monkeypatch.tools/bakeoff_us_support_mix.pynow calls it with the operator it builds itself, instead of calling_optimize.uk_runtime/firm_generation.pyalso imports_optimize; that is left for a UK follow-up.The TANF ledger salt (a separate commit)
The seed ledger recorded the TANF take-up site's salt as
takes_up_tanf. The draw has always been salted with the program variabletakes_up_tanf_if_eligible. The last commit corrects the ledger and moves its pins. It adds a test that each take-up site's ledger salt is the variable its draw uses, and that every seeded program in the take-up contract has such a site. No draw changes. The commit can be dropped without touching the others.How each piece is proven identical
test_us_keyed_draw_equivalence.pyfreezes as main had it at 90e4d94.stable_identity_uniformsto that loop on integer, float, string, NumPy and empty keys, with Python and NumPy integer seeds.source_runtime.test_us_post_export_scoring.pynow run against the shared walker, including its reference traversal over random engine graphs._optimizecall the bake-off made, and as the default free-mass solve.calibrateon the problemcalibratecompiled, trajectory included._optimizeas the same object._SparseOperator, run through it with the arm's options, gives the same bits as the direct_optimizecall the arm made before.Behaviour changes
District surface ESS validation.
tools/us_acs_local_cd_surface.py's ESS now refuses negative, NaN or infinite weights with aValueError. Its own copy scored them without a check:Every valid weight vector scores as before, because the arithmetic is unchanged:
weights**2andnp.squaregive the same bits, and a zero denominator still gives 0, so an empty cell still scores 0.0. Its callers are the district and county surfaces and the ACS local release's weight-origin summaries of the design and calibrated weights. The design weights pass throughWeightsfirst, so no input that runs today reaches the refusal. A test holds it to the refusal of a negative, NaN and infinite weight.The bake-off's solve. The solve now checks its inputs as
calibratedoes, and seeds torch's RNG with 0 on entry. A solve without L0 gates draws nothing from the RNG, so the weights and trajectory are unchanged.Pins that moved
All are hashes or field counts; no dataset value moves. Main's value is given first and this branch's last. Each re-pin commit lists its own step.
inventory_coverage.EXPECTED_HASHES):seed_protocole1f88676…→15c30c15…,seed_mapbcb012ed…→208fa485….spec_sha256(the pool tool's pin):3dabec5c…→9e5979da…. The bundle's YAML files are unchanged.d5c303a4…→1d373cec….compiler_semantic27,673 → 27,674;no_generation0_effect3,614 → 3,615;f6832e8d…→81bc649e….calibrateH1 parity case, re-recorded on arm64/darwin/py3.14 under the pinned dependency versions:implementation_hashf6554bfe…→aa55d948…,node_keyb6020102…→6aa80290…. Itsdirect.csvoutput is unchanged, which the re-pin tool enforces.docs/evidence/spec-engine/us-f0-coverage.jsonis regenerated: 42,395 of 42,395 fields and 41 of 41 inventory checks. The observed digest of the 53 seed sites moves with the TANF salt (a0366cd5…→67302e3a…).Why they moved:
stochastic_assignment.solve.py, which both the seed protocol andcalibrate.adam@1hash.Each re-pin used the documented sequence:
repin_us_spec_engine_contracts.py --legacy-only, the bundle generator, the re-pin tool,spec_engine_coverage.py, and a second re-pin pass for the coverage tool's counts. All three--checkmodes pass. The parity case usedtools/graph_parity_repin.py calibrate.The transport gate bindings now also reach
stochastic_assignmentthroughus_runtime. Their binding identity digests each reached module's source at runtime and pins nothing, so only the battery test's module count changes, from 244 to 245.Tests
Each file ran in its own process, counted from JUnit. Engine-free files ran with the country engines hidden, in
sys.modulesand inimportlib.metadata, as on CI's engine-free job. Every listed file passes; each skip needs a pinned feed or cached artifact this machine doesn't have.stochastic_assignment21, release target parity 37 (2 skips).test_solve.py82, and the builder's two signed-component tests 2.calibrate.solve: transport compile 11, transport graph 24, UK full calibration graph 15, UK full gates 24, UK local doctrine 9, UK target compile 52, UK UC matrix diagnostics 24, UK size evaluation 15.tools/ci_test_plan.py verifyandverify-workflowpass, andruff checkpasses on every changed file.ruff formatran only on files whose copy on main is clean. Five edited files that are not ruff-format clean on main (ssi_take_up.py,wic_claim.py,support_mix.py, the district surface tool and the bake-off tool) keep exactly their existing format diffs.This branch is on 90e4d94. Main has since moved to 89e12fb; none of its 24 commits touches an attested module or a pin file, and the branch merges cleanly with it.
Overlap with open pull requests
_SOURCESlist oftest_us_compatibility_recertification_parity.py. This PR also edits that docstring, so whichever merges second resolves a docstring conflict. Bind SOI income tax after credits to Form 1040 line 22 #1539's other hunks in the release tool, the recertification tool andcompatibility_recertification.pysit away from this PR's.immigration.py) and Generate complete ACS WIC participation and reject missing release inputs #1156 (wic_claim.py) re-pin the same pin lines. Whichever merges second re-runs the re-pin sequence. Neither changes the draw functions this PR edits.compatibility_recertification.pynearrequire_parent_publication, and Align DESIGN.md and US builder help with how releases calibrate #1540 changessolve.pyinsidecalibrate_l0_refit's docstring. Both are away from this PR's edits.Two questions for Max
microcosm.build.stochastic_assignmentstay in_DIRECT_KERNEL_MODULES? With it, the US seed-protocol hash attests the code that now computes the US draws. The cost is that UK edits to that module, of which there have been six since August, will also move the US F0 pins.docs/us-release-build-rule.md? Its trigger, a change "large enough to change how the dataset is built", is not defined. This is US: use the shared Microcosm modules where no output changes #1585's open question 1.Requested reviewer: Max, who owns
us_runtime,seeds.pyand the release tool. Possibly also DTrim99, for theacs_local_*modules.🤖 Generated with Claude Code