Skip to content

UK geography step 2: identity-keyed atomic-area assignment on rebuilt supports (#931) - #932

Merged
juaristi22 merged 6 commits into
mainfrom
uk-geography-step2
Sep 30, 2026
Merged

juaristi22 merged 6 commits into
mainfrom
uk-geography-step2

Conversation

@juaristi22

@juaristi22 juaristi22 commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Base main (re-based 2026-09-29 onto main 7683c09 after #901 merged; the old stack on uk-full-build-graph-registration is gone). Step 2 of #931 (item a). Five commits: ed0b13c (the shared atomic-geography operators from #893, unchanged), af89d22 (#893's UK-side atomic-area files, verbatim), 955435b (the identity-keyed assignment: supports, identity kernel, node swap, CLI, export, measurement), 9b2914f (the graph guide), 1cc92e3 (the #971 ladder-vintage refusal fixed here: the full-UK composite vintage the ladder tool writes is accepted beside the plain one, anything else refused; two tests). Publisher-file registration in Chronicle: PolicyEngine/chronicle#269.

What this does

The UK full build assigns geography after expansion with the shared atomic-geography operators, keyed by a stable post-clone household identity, under a single-stage law (one draw of the atomic area by census household count within the household's FRS region). The three support artifacts are rebuilt from published ONS / NRS / NISRA lookups at the FY2024-25 vintages with a committed provenance register. The previous sequential ladder draw stays behind --geography-assignment legacy for measurement builds; a release candidate refuses it.

The three substantive commits, in the order they should be read (hashes from the first stack; the re-based commits above carry the same content):

  1. 3cc384c4 supports and provenance. tools/build_uk_atomic_area_supports.py joins the sixteen publisher files the OA ladder already used, re-targeted at LAD April 2023 codes and region per OA from the ONS OA → Parish/NCP → LAD → Region (December 2024) V2 file, Ward 2024 and ITL 2025 for every nation (PCON July 2024, Scotland and NI products unchanged), and assembles one support per nation through assemble_uk_atomic_area_support with a declarative relation table (exact / best_fit / official_tabulation). Each column's source is <chronicle_package_id>@sha256:<publisher sha256>, so the assignment definition and every keyed draw move iff the publisher bytes move. Built against the pinned ladder bed3f13d: 239,023 areas; zero moves in LSOA / MSOA / LA / constituency / region / population / households; ward moves for 37,426 areas and ITL3 for 87,985, by design. Registered in uk/uk_atomic_area_supports.provenance.json (country-package resource), three sources.yaml entries, twelve geography_vintage_ref records, one contract-only kernel id in the shared allow-list (US precedent load_us_puma_ladder; implementation digest unchanged), UK spec digest re-pinned cbdbda2b… → b6e8bddc…. geography.yaml stays phase: legacy with a status note; the F3 bundle diff is item (f) of UK graph: move geography assignment ahead of enrichment (geography-first reordering) #931.
  2. fef75c3f identity kernel, node swap, CLI, export. UKGeographyIdentityKernel (uk.full.identity@1) keys every household with household_draw_key from the spine's explicit lineage (source household id, SPI support channel and clone index, CGT clone and donor flags) and the pool clone index. append_uk_population_nodes composes identity → atomic_geography_nodes(prefix="uk.full.geography", emit_validation_artifact=True) → the UK distribution gate; the legacy nodes are kept verbatim under the flag. CLI: --geography-assignment, --atomic-support-{ew,scotland,ni} with --atomic-support-sha256-* pins (required with --release-candidate), bindings.geography in the request and in the size-checkpoint identity (atomic mode only). Export drops the five nation-native aliases and ships the identity key and the three assignment columns; the export-surface allowlist gains the nine names.
  3. 5ccf1eaa measurement. tools/measure_uk_atomic_assignment.py runs the production preparation through both geography gates for {legacy, keyed} × {K=1, K=15} and records expectations, realized support, z-scores, breach tables, gate verdicts, identity stability and cost. Receipts: experiments/931-uk-atomic-assignment-receipts.md; committed 1 % evidence docs/evidence/uk-931/atomic-assignment-cells.json (per-area counts below 3 suppressed, no unit records).

Measurement (spine-r, 100 %, design weights, no solve)

  • Constituency-grain expectations are identical between the two laws (max difference 0). Local-authority expectations move for 25 of 361 LAs by more than 2 % and 3 by more than 5 % (City of London +16 %), because the keyed law allocates LA mass by census households rather than by population within the drawn constituency.
  • Every cell passes both gates; region mix and London share (0.1264) are bitwise-identical across laws; max |z| 3.52 with at most 0.83 % of areas beyond 3.
  • Identity stability holds (production columns equal in-process re-draws on the same, reversed and clone-0 rows) and K=1 draws are nested in K=15 draws.
  • At K=15: 0 constituency breaches and exactly {E06000053} LA breaches under both laws. City of London clears the design-weight floors under both laws; its reviewed exclusion is a calibration-time breach and is unchanged here.
  • Cost at K=15: assign_atomic 852 s and the UK distribution gate 213 s cold, so the pre-registered 15-minute budget for the geography nodes is exceeded by 3.7 minutes (see flags); peak RSS 10.4 GB.
  • Three pre-registered readings were mis-specified (3 % max relative error, ±10 % band on minima, row-exact reproduction of First calibrated rowwise UK local candidate (#495 increment 6) #762 R1 on a different spine) and are recorded as such in the receipts, not as defects.

Shared touches and design decisions to review

  • Typed-artifact numeric scope. The shared geography.gate@1 result is a platform_bitwise typed artifact (amendment 17) and the UK chain is declared bitwise; a UK gate consuming it would be refused, and declaring it platform_bitwise would cascade through every typed edge to export. The verdict is therefore consumed by UKPoolCheckpointKernel (uk.full.pool@1, FILTER, platform_bitwise, no typed outputs), which refuses unless the shared gate passed and orders it before every preflight, solve and export. @MaxGhenis: the shared atomic kernels' arithmetic is exact (sha256-u53 keyed draws, integer inverse CDF); either they can honestly declare bitwise, or the UK post-assignment chain should adopt the US pattern. Open question, not decided here.
  • assign_atomic wall time. 852 s at 792,690 rows (per-row iloc loop); validate_geography takes 20 s. Vectorising the shared kernel is flagged for the shared lane.
  • One allow-list line in spec_engine/resolver.py (load_uk_atomic_area_support, contract-only).
  • Pre-existing Register one UK full-build graph with all-geography calibration #901 gap (F8), not fixed here: the full-build export-surface gate compares the graph frame's in-memory column names (household_clone_index, constituency_code, ward_code, itl*_code, …) with the eFRS reference and the allowlist's export names (clone_index, constituency_code_oa, …); on a native run the release-blocking gate would flag them. Needs its own ruling.
  • The shared ITL loader keeps the first ITL3 row per LAD; the ONS ITL 2025 file lists North Ayrshire under Arran and Cumbrae first, so it carries TLM20 as the ladder did (recorded in the provenance notes; loader follow-up).

Out of scope (tracked on #931)

Enrichment reorder (b), origin budget and the two-cap question (c), rosters and area support from the supports and ladder retirement (d), the LAD 2025 recode translation (e), the F3 geography.yaml bundle diff (f), any calibration A/B, K / seed / solve doctrine.

Verification

  • uv run pytest on the affected files: 24 (supports tool and provenance) + 225 (population, identity, target, admission, preparation, CLI, terminal, adapter, lineage, ladder, calibration graph) + 4 (measurement) green; 152 spec/country-spec green after the digest re-pin; 269 gate-register/spec tests green after the allowlist change.
  • tools/ci_test_groups.py --verify ok (new files in the uk / spine-uk groups); uv run ruff check . clean.
  • tools/build_uk_full.py --dry-run on spine-r with the pinned supports: 24 nodes, uk.full.geography.gate precedes uk.full.pool.
  • Every uk.full.* node key moves (new kernels, new SourceRefs); nothing committed pins them.

🤖 Generated with Claude Code

@juaristi22
juaristi22 force-pushed the uk-full-build-graph-registration branch from 2da4f42 to 688dd60 Compare September 25, 2026 23:28
juaristi22 added a commit that referenced this pull request Sep 25, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 25, 2026
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 25, 2026
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 26, 2026
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-full-build-graph-registration branch from acbe928 to 7d27320 Compare September 28, 2026 09:44
juaristi22 added a commit that referenced this pull request Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the
invented three-system support helpers (parts, payloads) that the lineage test imported
from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 28, 2026
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Re-stacked on #901 b2538d1 (main 9e5b0ce with #998's test layout): the new tests live
in tests/engine_free/uk/ (test_uk_atomic_area_supports_provenance, _supports_tool,
_assignment_measurement, _identity_graph; none imports the country engine), the toy
ladder-matched supports helper is test_support/microcosm_build/uk_atomic_support_fixtures.py,
and every sibling-test import is a test_support import. Where #901 moved a helper into
test_support, this commit's change follows it: build() and the support sources in
uk_full_target_graph.py, arguments() plus SUPPORT_ARGUMENTS/RELEASE_PINS in
uk_full_build_cli.py, _write_staging_h5/_build_args/_dense_argv plus _SUPPORT_FLAGS/
_SUPPORT_PINS/_toy_support_flags in uk_rowwise_candidate.py, and SOURCE_VINTAGE/
source_node/graph_and_registry in uk_full_population_graph.py. The test bodies follow
#901's engine/engine_free split (the full-graph compile of test_uk_country_adapter and
the two engine-backed target-graph tests under engine/uk; the rest under engine_free/uk).
Tool and evidence paths come from test_support.paths.paths_for, not __file__.

Verified on this base: 682 engine-free + 8 engine/uk tests passed; ci_test_groups --verify ok;
the gate-register pins in microcosm-data test_contract.py hold their #932 values.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 28, 2026
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-full-build-graph-registration branch 2 times, most recently from eac7643 to b6861cc Compare September 28, 2026 13:22
juaristi22 added a commit that referenced this pull request Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the
invented three-system support helpers (parts, payloads) that the lineage test imported
from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 28, 2026
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Re-stacked on #901 b2538d1 (main 9e5b0ce with #998's test layout): the new tests live
in tests/engine_free/uk/ (test_uk_atomic_area_supports_provenance, _supports_tool,
_assignment_measurement, _identity_graph; none imports the country engine), the toy
ladder-matched supports helper is test_support/microcosm_build/uk_atomic_support_fixtures.py,
and every sibling-test import is a test_support import. Where #901 moved a helper into
test_support, this commit's change follows it: build() and the support sources in
uk_full_target_graph.py, arguments() plus SUPPORT_ARGUMENTS/RELEASE_PINS in
uk_full_build_cli.py, _write_staging_h5/_build_args/_dense_argv plus _SUPPORT_FLAGS/
_SUPPORT_PINS/_toy_support_flags in uk_rowwise_candidate.py, and SOURCE_VINTAGE/
source_node/graph_and_registry in uk_full_population_graph.py. The test bodies follow
the two engine-backed target-graph tests under engine/uk; the rest under engine_free/uk).
Tool and evidence paths come from test_support.paths.paths_for, not __file__.

Verified on this base: 682 engine-free + 8 engine/uk tests passed; ci_test_groups --verify ok;
the gate-register pins in microcosm-data test_contract.py hold their #932 values.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 28, 2026
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Posting here on María's call: the licensed 10 % smoke run for #901 surfaced this, and it will be addressed on this PR rather than on #901. It is a main-side defect from #971, so it also applies to the ladder law this PR keeps beside the atomic assignment and to tools/build_uk_rowwise_candidate.py on main.

Observed (2026-09-28, pre-merge smoke run for #901, licensed inputs). Both the graph driver and
tools/build_uk_rowwise_candidate.py on main 937aca4 refuse the shipped ladder
(build/uk/uk_oa_ladder_2021.npz, every copy on disk, built by tools/build_uk_oa_ladder_artifact.py):

Node 'uk.full.locations' kernel 'uk.full.locations@1' failed: UK OA ladder local authority vintage
'ew:2023_april_lad;scotland:2019_council_area;ni:2014_lgd' is not '2023_april_lad', the vintage of
the names resource that resolves local_authority; regenerate local_authority_names.json for the new
roster first.

Cause. #971 (46a9ba5, "refuse a foreign ladder vintage") compares
ladder.layer_vintages["local_authority"] with UK_LOCAL_AUTHORITY_VINTAGE = "2023_april_lad" in
uk_runtime/geography_ladder.py::draw_uk_ladder_locations. The artifact tool writes one composite
vintage per full-UK layer (_uk_layer_metadata: ew:…;scotland:…;ni:…), so no ladder it produces can
pass; the tests pass because every fixture ladder (test_uk_geography_ladder, uk_ladder_rowwise_clone,
uk_rowwise_candidate helpers) carries the plain EW string. Rebuilding the ladder does not help: the tool
writes the same composite. Main's dense line has therefore been unrunnable on a real ladder since
2026-09-23; the national line does not touch the ladder, which is why nothing surfaced.

Proposed fix (small, main-side, independent of #901). Pin the composite the LAD23 frame implies:
UK_LOCAL_AUTHORITY_LADDER_VINTAGE = "ew:2023_april_lad;scotland:2019_council_area;ni:2014_lgd" beside
the names-resource constant, compare the ladder against it (the April 2023 LAD list is exactly EW LADs +
2019 Scottish council areas + 2014 NI LGDs, which is what the names resource enumerates), keep the
message, and add one test with a composite-vintage fixture ladder (and one refusing a different composite).
Alternatively parse the ew: component; the pinned composite is stricter and matches the tool's policy.

Measurement device used meanwhile. An uncommitted local patch accepting the composite vintage in the two throwaway worktrees the smoke run used; nothing on any branch.

@juaristi22
juaristi22 force-pushed the uk-full-build-graph-registration branch from b6861cc to c9ce92b Compare September 28, 2026 17:34
juaristi22 added a commit that referenced this pull request Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the
invented three-system support helpers (parts, payloads) that the lineage test imported
from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22
juaristi22 marked this pull request as ready for review September 29, 2026 11:52
juaristi22 and others added 5 commits September 29, 2026 14:36
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are
the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's
validation artifact and emit_validation_artifact, which the UK node swap consumes);
they are blob-identical to #893's current head a402dfc, so #893 rebases over this
commit without conflict. Country-agnostic: their imports (keyed_uniform,
load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in
tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1).

Verified: 37 tests passed; ci_test_groups --verify ok.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and
atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and
test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and
docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56
(Max's b1626e1 and aaf3174, as amended on that line through 1cebe35).
The modules stay module-local imports: nothing on #932 reaches them through
uk_runtime/__init__.py, so main's eager package init is untouched.

Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the
invented three-system support helpers (parts, payloads) that the lineage test imported
from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ased #901: supports, identity kernel, node swap, CLI, export, measurement

Source commits (old #932 head, tag uk-932-pre-main-rebase): 3cc384c (rebuild the
UK atomic-area supports from published lookups with committed provenance),
fef75c3 (assign UK geography on the shared atomic operators, keyed by household
lineage), 5ccf1ea (measure the assignment before and after, with committed 1%
evidence), 0cd18d5 (name the step-2 evidence after #931 and cite chronicle#269).
Applied with `git apply -3` over the re-based #901 head 7eaf6ff; nine files
conflicted with main or the re-based series and were merged by hand.

What changed against those commits:
- Release-role driver: the geography flags (--geography-assignment, default
  atomic; --atomic-support-{ew,scotland,ni}; --atomic-support-sha256-*) sit
  beside main's --ladder pins; the cross-flag rules moved from parse_args into
  rowwise_cli.validate_cli_args next to the ladder/pin rules (atomic requires
  the three supports, legacy takes none, a release candidate refuses legacy
  and requires the three support digest pins) and the national role refuses
  the support flags by name in its refusal table. The supports enter `pins` as
  flat roles named by their graph source, so the Logbook input-pins digest,
  the candidate identity digest and uk_size_checkpoint_identity (geography
  binding in atomic mode only) all bind them; the support admission runs after
  the spine admission. bindings["geography"] reaches the export descriptor and
  the rowwise manifest projection (geography.assignment).
- #971 interplay: main's export carries the engine local_authority enum
  resolved from local_authority_code, so the atomic path composes a UK-side
  node uk.full.geography.local_authority (resolve_local_authority_engine_keys,
  fail-closed on the April 2023 roster) between the shared derive and the gates.
  The country adapter's raw-source projection check now scopes to the FRS rows
  and admits the three support rows by role and loader.
- The UK spec digest pin in test_spec_engine_country_bundles is dropped (main
  removed it). Gate-register re-pin: uk_export_surface's parameters moved the
  full-manifest policy/manifest/fingerprint digests and the release_cut part
  digests; the live values are pasted into microcosm-data contract.py and its
  test. The release-input coverage manifest is unchanged.
- tools/build_uk_atomic_area_supports.py carries its own _pinned_sources and
  _verify_pinned_source (main's US PUMA tool no longer has them). The synthetic
  rowwise staging spine gains household_support_channel, which the identity
  kernel keys on. docs/uk-full-build-graph.md does not exist on the re-based
  line, so #932's hunk to it is not carried.

Re-stacked on #901 b2538d1 (main 9e5b0ce with #998's test layout): the new tests live
in tests/engine_free/uk/ (test_uk_atomic_area_supports_provenance, _supports_tool,
_assignment_measurement, _identity_graph; none imports the country engine), the toy
ladder-matched supports helper is test_support/microcosm_build/uk_atomic_support_fixtures.py,
and every sibling-test import is a test_support import. Where #901 moved a helper into
test_support, this commit's change follows it: build() and the support sources in
uk_full_target_graph.py, arguments() plus SUPPORT_ARGUMENTS/RELEASE_PINS in
uk_full_build_cli.py, _write_staging_h5/_build_args/_dense_argv plus _SUPPORT_FLAGS/
_SUPPORT_PINS/_toy_support_flags in uk_rowwise_candidate.py, and SOURCE_VINTAGE/
source_node/graph_and_registry in uk_full_population_graph.py. The test bodies follow
the two engine-backed target-graph tests under engine/uk; the rest under engine_free/uk).
Tool and evidence paths come from test_support.paths.paths_for, not __file__.

Verified on this base: 682 engine-free + 8 engine/uk tests passed; ci_test_groups --verify ok;
the gate-register pins in microcosm-data test_contract.py hold their #932 values.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… guide

The re-based guide (688dd60) predates the #932 re-stack; this carries #932's geography section onto it: the three support flags and their pins, the identity-keyed assignment and derivation nodes, the local-authority resolve node added for #971's engine key, the shared and UK gates, the legacy measurement mode, and the node table rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…usal)

microcosm#971 compares the ladder's local_authority layer vintage with the
names resource's '2023_april_lad'. tools/build_uk_oa_ladder_artifact.py
writes one composite vintage per full-UK layer
('ew:2023_april_lad;scotland:2019_council_area;ni:2014_lgd'), so every real
ladder was refused and the dense line has been unrunnable on a real ladder
since 2026-09-23 (found by the licensed #901 smoke run; reported on #932).

The April 2023 LAD list is exactly the England and Wales LADs plus the 2019
Scottish council areas plus the 2014 Northern Ireland LGDs, which is what the
names resource enumerates, so the composite is pinned beside the names
constant (UK_LOCAL_AUTHORITY_LADDER_VINTAGE) and the refusal accepts the
plain vintage of an England-and-Wales-only ladder or that composite, and
nothing else. Two tests: the composite is accepted, a foreign composite is
refused; the existing plain-vintage refusal stands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
test_sample_admission_carries_the_transferred_gate_posture (added on #901
by the smoke-run fix) composes the population nodes with the defaults; on
this branch the default assignment law is the atomic one, which requires
the assignment definition at composition, so the test refused before it
reached the admission it checks (both engine-free CI jobs). The admission
under test sits on the sample node, before any assignment: the test now
composes under the legacy law, which needs no definition.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Automated review pass (Claude Code, high effort) — round 1 at 94f843f2

Verdict: nothing blocking. The code looks sound, CI is 10/10 green at this head, and the targeted suites pass locally. Three should-fixes: the support digests are registered but never enforced; the evidence file's suppression can be undone from its own z-scores; the #971 fix should ship on its own. Plus a few nits and questions.

Since b5b1739c. git range-diff against this head shows five patch-identical commits on newer main, plus 94f843f2. That commit sets geography_assignment="legacy" in test_sample_admission_carries_the_transferred_gate_posture. The fix is right: the admission under test sits on the sample node, before any assignment. It does mean the #901 smoke-run test no longer runs under the default atomic law. The other spine.gates.transferred tests (test_uk_graph_evidence.py:112, test_uk_full_build_cli.py:564) check the spine graph, not the population composition. As far as I can find, nothing now checks the sample admission under the default law. A sibling case that passes a synthetic assignment definition would be cheap. No src file cited below changed between the two heads, so the line references stand. Main's changes in that range are US-only.

What I checked. The branch sits on main 7683c097, after the #901 merge. By line count the diff is roughly 45.7k data/spec, of which 44.9k is docs/evidence/uk-931/atomic-assignment-cells.json, plus 2.3k src, 3.7k tests, 1.5k tools and 0.4k docs. I compared the #893 port blob by blob:

  • atomic_geography.py, graph_atomic_geography.py and test_atomic_geography.py are identical to Consolidate the US survey graph build and enrichment pipeline #893 at 2300e56b and at its current head a402dfcc.
  • The five UK and doc files are identical at af89d224.
  • The two tests differ only by the relocation described in the commit message.

The identity key, the single-stage law, the CLI cross-flag rules, the size-checkpoint binding and the export drop all read correctly. The band-donor lineage cannot collide: donors are drawn without replacement and carry their source's clone flag, so a donor copy of a CGT clone gets a distinct path. Suites I ran at b5b1739c (the code is patch-identical here), all green: shared test_atomic_geography; the UK atomic support, lineage, identity-graph, provenance, supports-tool and measurement tests; test_uk_geography_ladder, test_uk_full_build_cli, test_uk_full_population_graph, test_uk_full_build_preparation, test_uk_country_adapter, test_uk_full_target_graph and test_uk_rowwise_candidate; and test_country_spec plus test_spec_engine_country_bundles.


1. Should-fix: the registered support digests are never checked against the supports a build actually uses.

  • sources.yaml pins each support's sha256 and byte_size, and so does uk_atomic_area_supports.provenance.json.
  • But load_uk_atomic_area_support is contract-only (spec_engine/resolver.py:172).
  • validate_uk_country_source_projection (country_adapter.py:62-77) checks only role and loader.
  • _prepare_geography (full_build_cli.py:464) compares the supplied bytes only with the digest supplied on the command line (--atomic-support-sha256-*, via _pin, :447).
  • Nothing in src reads the provenance register or the support rows' sha256.

So a release candidate with a rebuilt support passes as long as the caller pins its own digest, and the register is not the gate the PR body implies. Relatedly, the refusal of legacy under a release candidate and the support-pin requirement both live only in rowwise_cli.validate_cli_args (:481-528). build_uk_country_graph(release_candidate=True, config=<legacy>) is not refused, and bindings.geography.assignment is written to the manifest but no certifier reads it. Suggestion: under --release-candidate, require each supplied support digest to equal the sources.yaml row. Also have the release-cut certifier refuse a manifest whose geography.assignment is not atomic or whose support_pins differ from the register.

2. Should-fix: the evidence file's suppression can be undone from its own z-scores. atomic-assignment-cells.json suppresses rows < 3 as "<3" but publishes expected_rows and z for the same cells. Method:

  • Take the scale factor from any zero-row cell in the same region and level: f = -expected / z / sqrt(expected).
  • Then round(expected + z·sqrt(expected)·f) recovers every published count of 3–5 exactly: 157 of 157 across the four cells, 0 wrong.
  • Applied to the suppressed cells, the estimates cluster within 0.17 of an integer. Legacy K=1 splits 313 ones and 161 twos; keyed K=1 splits 365 and 130.

The practical risk is low, since these are counts of synthetic geography draws over a 1 % sample. But the file states that "per-area counts below the minimum are suppressed", and as written that is not true. Either blank z wherever rows is suppressed, or drop the suppression claim and give the reason it isn't needed.

3. Should-fix: split 1cc92e3f (the #971 composite-vintage refusal) into its own PR against main. The fix matches the proposal in the 28 Sep comment:

  • It pins UK_LOCAL_AUTHORITY_LADDER_VINTAGE beside the names constant (local_authority_input.py:51).
  • geography_ladder.py:442 accepts only the plain value or that composite.
  • It adds a test with a composite-vintage fixture and one that refuses a different composite (ew:2025_april_lad;…).
  • The string matches what tools/build_uk_oa_ladder_artifact.py:456 writes.

However, main's dense line and tools/build_uk_rowwise_candidate.py have been unrunnable on a real ladder since 23 Sep. Tying a four-file fix to a 53k-line PR keeps main broken until this merges.

4. Question / nit: the numeric-scope workaround and its docstring. The shared assign_atomic, derive and support_import kernels are declared PLATFORM_BITWISE too (graph_atomic_geography.py:61,130,152), not just the gate. Their columns (atomic_area_code, constituency_code, local_authority_code, …) reach uk.full.geography_gate, the target nodes and calibration through population slices. _input_numerics (microcosm-graph/…/executor.py:852) already resolves those reads to platform scope. UKPoolCheckpointKernel therefore avoids only the typed-artifact refusal (artifact_edges.py:104). Its docstring says the scope stops there "instead of laundering into the bitwise UK target and calibration chain", which overstates it; I'd reword it. This strengthens the open question to @MaxGhenis: if the shared kernels can honestly declare bitwise, the pool FILTER workaround goes away.

5. Nit: _normalise_ew_lad_lookup now accepts any LAD??CD column (geography_sources.py:1507). The new load_ew_oa_lad_region_lookup also takes the first such column. Two probes:

  • A frame with LAD25CD = E08000038/39 (the Barnsley/Sheffield recode) is silently relabelled lad23_code.
  • A frame with both LAD22CD and LAD23CD fails with AttributeError: 'DataFrame' object has no attribute 'str'.

The names-resource resolve fails closed later, so this is not a correctness hole. Still, since the provenance notes treat the 2025 recode as a separate follow-up, I'd pin the accepted names to {LAD23CD, LAD24CD}.

6. Nit: the export allowlist admits the five nation-native aliases that export drops. terminal_gates.py and gates.json add output_area_code, data_zone_code, intermediate_zone_code, super_data_zone_code and district_electoral_area_code. But graph_terminal._tables drops exactly those (UK_NATIVE_ALIAS_COLUMNS), so they never reach an artifact. If they are there for the F8 in-memory comparison, a comment should say so. Otherwise, remove them and keep the gate-battery digest re-pin to the four shipped names.

7. Question: the law change in Max's module. 955435ba changes uk_atomic_assignment_definition in the ported atomic_area_support.py from #893's two-stage law to the single-stage law, and edits both ported docs. The commit message's "#893 rebases over this commit without conflict" holds only for the three shared operators. #893 will conflict on atomic_area_support.py, docs/geography-assignment.md and docs/uk-atomic-household-lineage.md. Has Max signed off on the change to his module (the 2026-09-15 ruling the comment cites)?

8. Nit: the PR body and the merge order with #1057.

The merge order needs settling. The #971 fix is still not on main, so there's one more reason to split it out (finding 3).

Not findings, noted: the 15-minute node budget overrun (assign_atomic 852 s from the per-row iloc loop), the F8 export-surface gap and the North Ayrshire ITL3 first-row rule are already flagged in the body.

@juaristi22
juaristi22 merged commit 827dc3c into main Sep 30, 2026
10 checks passed
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Posting on María's call. Thanks for the round. Her decision on merge order: this PR merges first, as it stands at 94f843f2; #1057 rebases over it and carries every fix below in its next round, one commit per finding, so this PR takes no further CI round. I'll link the #1057 commits from here once they are pushed.

1 (register digests never enforced). Agreed. In #1057: under --release-candidate each supplied support digest must equal its sources.yaml row, not only the pin on the command line, and the release-cut certifier refuses a manifest whose geography.assignment is not atomic or whose support_pins differ from the register.

2 (evidence suppression undone from z). Agreed, the claim is false as written. In #1057 the harness suppresses z wherever it suppresses rows, and the committed file is re-suppressed in place with the same rule (same cells, no regeneration; the sha in the receipts moves and is re-recorded).

3 (split the #971 fix). Merging this PR now puts the fix on main directly, so no separate PR.

4 (numeric scope). Agreed the docstring overstates it: the shared kernels' columns already reach the UK target and calibration chain at platform scope through _input_numerics, so the pool FILTER only avoids the typed-artifact refusal. Reworded in #1057. The question to Max (can the shared atomic kernels declare bitwise?) stays open as the body states.

5 (LAD??CD). In #1057 both helpers accept exactly LAD23CD or LAD24CD, refuse a frame carrying more than one, and the two probes become tests.

6 (alias names in the allowlist). They are there for the export-surface gate, which in the full-build battery reads the graph frame's in-memory columns (full_gates.py, parity_evidence.candidate_columns) before graph_terminal._tables drops the five aliases; without the allowlist entries the gate flags them as unreviewed extras. That is the F8 gap the body records. #1057 adds the comment saying so and keeps the names, so the gate-register digests do not move again.

7 (law change in Max's module). The single-stage law is María's ruling of 2026-09-15 for step 2 (the approved plan of record). Max has not reviewed the edit to his module, and you are right that the commit message's "no conflict" holds only for the three shared operators. @MaxGhenis: uk_runtime/atomic_area_support.py, docs/geography-assignment.md and docs/uk-atomic-household-lineage.md will conflict when #893 rebases; this PR's versions carry the single-stage law. Happy to help with that rebase.

8 (body, merge order, stale comment). 94f843f2 is the CI fix: the sample-admission test composes under the legacy law because the admission it checks sits on the sample node, before any assignment. The sibling case under the default atomic law, the stale "dispatched to the seam" comment in test_uk_full_build_cli.py and the full_build_cli.py conflict all land in #1057's rebase.

juaristi22 added a commit that referenced this pull request Sep 30, 2026
…932 review 1)

`sources.yaml` and the provenance register pin each atomic-area
support's sha256 and byte size, but nothing compared the supports a
build used with them: `_prepare_geography` checked the bytes only
against the digest pinned on the command line, and no release check
read the manifest's `geography.assignment`. A rebuilt support therefore
passed as a release candidate as long as the caller pinned its own
digest.

`country_adapter.uk_atomic_support_register` reads the register from
the UK spec by system. Under `--release-candidate` each supplied support
pin must equal its register row, and
`tools/preflight_uk_local_release_candidate.py` refuses a manifest whose
`geography.assignment` is not the atomic law or whose `support_pins`
differ from the register (a pre-#932 manifest with no binding is
refused like a pre-role candidate). The synthetic dense fixtures bind
synthetic pins (`SYNTHETIC_SUPPORT_PINS`, the `RELEASE_PINS` digests)
and the preflight and assembler suites register those; one preflight
test reads the committed register and shows the synthetic pins refused
on all three systems.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 30, 2026
…view 2)

`atomic-assignment-cells.json` suppressed `rows < 3` as `<3` but
published `expected_rows` and `z` for the same cells, from which the
review recovered every suppressed count exactly. The harness now blanks
`z` wherever it suppresses `rows` (`_suppressed_area`), the purpose
text says so, and the committed file is re-suppressed in place under
that rule (985 z-scores blanked, no cell regenerated; sha256
32f494df… → 7d120d52…, recorded in the receipts).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 30, 2026
…review 5)

`_normalise_ew_lad_lookup` and `load_ew_oa_lad_region_lookup` accepted
any `LAD??CD` column, so a `LAD25CD` frame (the Barnsley/Sheffield
recode, a separate follow-up) was silently relabelled `lad23_code`, and
a frame with two such columns failed with an AttributeError. Both now
accept exactly `LAD23CD` or `LAD24CD` (the December 2024 OA lookup
carries the April 2023 set under the latter), ignore other names and
refuse a frame carrying both accepted names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 30, 2026
…review 4, 6, 8)

The pool checkpoint's docstring said the platform scope "stops here";
it only avoids the typed-artifact refusal, since the shared assignment
kernels' columns already reach the UK chain at platform scope through
population slices (reworded; the question to Max stays open). The five
nation-native alias names in the export allowlist are there because the
full-build export-surface gate reads the graph frame's in-memory columns
before `graph_terminal._tables` drops them (the F8 gap on #932); the
allowlist now says so, and the names stay so the gate-register digests
do not move again. The CLI test's "dispatched to the seam" comment is
replaced, and the sample-admission test gains its sibling under the
default atomic law.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 30, 2026
…parison as evidence

Records the review-date clock as a deviation from the seam (the dense
convention, a node parameter), the occupied-output refusal, the interrupt
disposition, the re-issued completion marker, the scratch-relative
resolver path and the registered-support check in the graph doc and the
#623 runbook; extends the changelog fragment; corrects the receipts'
test counts (R2, R3), notes the committed comparison in R5 and adds R6
(round 1 and the #932 folds); commits R5's seam-vs-graph comparison
under docs/evidence/uk-901-national-ab (aggregates and field-level
differences only, machine paths omitted).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Posting on María's call, as promised above: the round-1 items landed on #1057 (head 63771aff6, rebased over this PR as merged), one commit each.

  • 1 (register digests): a4308eb. country_adapter.uk_atomic_support_register reads the sources.yaml rows by system; --release-candidate requires each supplied support pin to equal its row, and tools/preflight_uk_local_release_candidate.py refuses a manifest whose geography.assignment is not the atomic law on those pins (a pre-UK geography step 2: identity-keyed atomic-area assignment on rebuilt supports (#931) #932 manifest with no binding is refused like a pre-role candidate).
  • 2 (evidence suppression): 11f4014. The harness blanks z wherever it suppresses rows; the committed file is re-suppressed in place under that rule (985 z-scores, no cell regenerated; sha 32f494df… → 7d120d52… in the receipts).
  • 4 (scope docstring): edb478f, reworded as agreed above.
  • 5 (LAD??CD): c7057e9. LAD23CD or LAD24CD only; other names ignored, two accepted names refused; your two probes are the tests.
  • 6 (alias names): edb478f. The names stay, with the comment explaining the in-memory export-surface gate (F8); the gate-register digests do not move.
  • 8 (stale comment, sibling test): edb478f. The CLI test comment is replaced and the sample-admission test has its sibling under the default atomic law.

Findings 3 and 7 were answered above and need no code.

juaristi22 added a commit that referenced this pull request Sep 30, 2026
…amily-folded weight fences, top-up lineage receipt

- cgt_support_split writes cgt_support_copy_index (0 on roots and unsplit
  households, k on copy k) beside its flag and count; the geography identity
  kernel keys a support copy on it as the cgt_support_split branch, between
  the SPI support channel and the incidence clone, in place of the retired
  donor flag, and refuses a flag that disagrees with the index; the identity
  tool reads the stored index and holds it to the id scheme.
- uk_weight_ratio and uk_weight_ess evaluate on family-folded weights (each
  root with its copies summed, on each side of the clone, keyed on the
  explicit lineage) and report the row-level summary beside them, so the
  certified June bound reads weight dispersion rather than the number of
  copies.
- The student-loans top-up receipt records the taken rows and mass by
  household support channel, on support copies and on incidence clones.
- Manifest, operation schema, graph cells, rowwise lineage, coverage
  manifest, H2 fixture and gate-battery digests regenerated from the tree
  rebased onto main 827dc3c (#932); stale headroom text fixed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 12, 2026
…931)

The dense line no longer samples, expands, keys or places households
of its own. Its targets, measures and problem run on the spine's
version as it stands, its pool checkpoint keeps every row of that
version, and its geographic clone count K, its sampling rung and seed
and its geography binding are the spine's. It still refuses, before it
pins or reads any input, every spine whose roster is not
geography-first (spine_posture), so no dense build runs until the
spine declares its area stages; the national line still refuses every
clone-first spine.

Why. María's ruling of 10 Nov (decision 4) holds the dense and
national lines back until (c) re-bases them, and C-21 re-bases the
dense line in three commits, refusal first: the roster refusal
(commit 2), the four localisation families (commit 3) and this one,
the last. K, sampling and the geography binding now come from the
spine, so no dense node draws them a second time and the run records
the spine's K and rung.

- graph_population: uk.full.sample, uk.full.normalize, uk.full.expand
  (with uk.full.expand.owned), uk.full.identity, uk.full.locations,
  uk.full.geography_mapping, uk.full.geography.local_authority and
  uk.full.geography_gate go, with their eight kernels, the
  population-receipt, location-draw and geography-gate artifact types,
  append_uk_population_nodes, uk_pool_validation_inputs,
  GEOGRAPHY_ASSIGNMENTS and UK_GEOGRAPHY_IDENTITY_INPUTS. The UK
  registry stops registering the four shared atomic-geography kernels
  (NZ's transport registers its own), so the uk.full.geography.*
  support, assign, derive and gate nodes go too. uk.full.pool@1 keeps
  its id: a keep-all FILTER on uk.full.spine_checkpoint (--input-h5)
  or the spine's endpoint (--spine-request), with no typed edge,
  declared bitwise (platform-bitwise before, for the shared gate's
  edge) and hashing its own module and national_frame. The dense
  geography gate's ladder and installed-engine local-authority checks
  and the pool's refusal of a failed shared validation leave with
  their nodes; the spine's geography gates take them over once they
  are registered (commit 11), and until then the roster refusal holds
  every dense build.
- graph_targets: append_uk_target_nodes takes population with no
  default. The measures lose the geography-gate edge and check, and
  take the spine's transferred-gate admission, which the sample node
  held, wherever the graph declares spine.gates.transferred; they
  check it before they read anything.
- graph_build: UKFullBuildConfig loses sample_fraction, sample_seed,
  geography_assignment, source_vintage, constituency_vintage and
  source_lineage_modulus; n_clones defaults to 1, and
  effective_sample_fraction is source_sample_fraction, the spine's
  rung. uk_full_graph drops spine_weight_kind and
  atomic_geography_definition and refuses a composed spine whose
  geographic_expand node declares another K, naming both;
  build_uk_country_graph drops atomic_geography_definition.
  uk.full.bound_spine@1 also hashes geographic_clones.
- calibration_run: _assert_spine_sidecar_binds_frame holds the
  sidecar's geographic_clone_count (one where a sidecar predates it)
  to the frame's, naming both; spine_provenance_from_sidecar adds K,
  geography, sampling and release_posture.
- full_build_cli and rowwise_cli: prepare_full_build reads K,
  sampling, release posture and the geography binding from the
  authenticated sidecar, or from the spine request's --n-clones,
  --sample-fraction and --sample-seed, and sets the run's n_clones,
  sample_fraction and, for a sampled spine, sample_seed before the
  parameters, identity digest, size-checkpoint identity, staging rung
  and Logbook rungs record them. A spine whose sampling fraction is
  not a scale-ladder rung is refused, naming the rungs only. The
  attempt's build id is minted with the spine's rung, read ahead of
  preparation (unauthenticated, for the label) off the bound spine's
  sidecar or the request's --sample-fraction through the spine's own
  parser, and f100 where that read fails; preparation refuses an
  attempt whose build id carries another rung than the spine it
  binds, naming both rung tokens, and a refusal before the spine is
  bound records the build id's rung on its Logbook row. The request
  bindings gain a spine block, and geography is the spine's binding
  with K beside it. microcosm-build-uk no longer takes
  --geography-assignment, --atomic-support-ew,
  --atomic-support-scotland, --atomic-support-ni,
  --atomic-support-sha256-ew, --atomic-support-sha256-scotland,
  --atomic-support-sha256-ni or --source-lineage-modulus; the parser
  refuses each as an unrecognized argument on either role. The dense
  role refuses, by name and before any input is read, a
  --sample-fraction other than 1.0, any --sample-seed,
  --candidate-clone-counts with --input-h5, and --n-clones with
  --candidate-clone-counts. --n-clones only asserts the spine's K: the
  role no longer fills it from the doctrine, and preparation refuses
  one that differs, naming both, and on a spine request pointing at
  the request's own --n-clones. --candidate-clone-counts plans the
  spine request once per count, as its --n-clones.
  --expected-constituency-vintage is still taken and recorded, and its
  help now says no dense node checks the ladder against it. Release
  candidates: a sampled spine is refused at preparation, where its
  rung is read, on either route; the doctrine-K check and the
  support-pin requirement leave validation for the spine's K pin and
  binding check (commit 5); the ladder, Ledger and input pins, target
  rule, epochs, measure exclusions, local target levels and signing
  key are refused as before.
- graph_terminal: the rowwise manifest reads sampling,
  identity.sampling, the rung and the release verdict's full-rung leg
  from the request's spine block, refusing a descriptor without it,
  and its ladder gate from the export descriptor's geography check,
  at phase post_calibration; the weight-kind chain names
  geographic_expand.
- national_sampling: the dense family sampler (sample_uk_spine_frame,
  uk_spine_source_family_units) and its re-exports go; the root's
  sampler, the rung tokens and the default seed stay.
- size_checkpoint: the identity loses source_lineage_modulus and the
  atomic geography block.
- household_lineage: the reader's three lineage refusals open with the
  household table and its clone paths, not "Geography identity", and
  UK_OPTIONAL_LINEAGE_COLUMNS, which only the retired identity kernel
  read, goes.
- Texts: rowwise_dataset's clone refusals point at the dense graph on
  a geography-first spine and promise no later group;
  tools/build_uk_rowwise_dataset.py says it serves spines without
  clone indices; the docstrings of geographic_identity,
  household_lineage (owed by commit 3), geographic_clones (which now
  names the national gates among its hashers) and diagnostics (whose
  support-family key comment names the lineage columns, not the
  identity kernel's inputs) stop naming the dense identity node or
  expansion as a writer or reader; tools/measure_uk_atomic_assignment.py
  refuses its graph route, and its command before it reads any input,
  with a pointer to the spine's atomic_area_assign stage; the Orrery
  roster loses its geography group and the retired nodes. Three older
  unreleased fragments that this change makes stale (the #932 wiring,
  the national sampler's retirement, commit 3's) each gain a clause
  saying what retires later.

Defaults taken, each listed for María with its options: the
configuration's K defaults to 1; a composed spine of another K is
refused at composition, and a bound spine's K is read at preparation;
a sidecar without geographic_clone_count reads as K = 1; the run's
arguments are rewritten to the spine's K, fraction and seed; the
spine binding's shape, with the manifest failing closed without it;
--candidate-clone-counts replaces a request's own --n-clones, and
--n-clones beside it is refused; the measures take the admission only
where the graph declares the transferred gate; the pool is bitwise;
the manifest's ladder gate is the descriptor's check at phase
post_calibration; the weight-chain label is geographic_expand; the
Orrery geography group goes; the identity node's tests retire with it
and the lineage reader's own tests move to test_uk_household_lineage.py;
the local area support gate's row, effective-sample-size and source
floors are kept K-sensitive constants; the build id's rung is read
ahead and held to the spine's at preparation; a sampling fraction off
the ladder is refused; --expected-constituency-vintage stays a
recorded assertion.

Left for later:
- The geographic_identity stage's notes in sources.yaml, and so
  source_stages.json and the H2 fixture, still name uk.full.identity:
  commit 8, which regenerates H2 with the geography declarations.
- On a spine request the geography binding carries K alone: commit 8
  composes the request route's block with the sidecar writer's helper,
  or commit 5 refuses --release-candidate with --spine-request; commit
  5's binding check also derives the stand-in sidecar's geography
  values from the stand-in's own definition.
- The spine's mapping-integrity, ladder and LA-engine-domain gates
  that take over the retired dense checks are declared
  release-blocking when commit 11 registers them.
- --expected-constituency-vintage: commit 5's binding check, or a
  ruling (hold it to the ladder, retire it, or keep it recorded).
- tools/measure_uk_atomic_assignment.py's manifest readers, past the
  refusal, still name retired nodes: commit 9 re-points the tool.
- refuse_dense_clone_id_overflow keeps its name, though only the
  row-wise lane calls it, as do two row-wise driver test names;
  test_uk_national_graph.py's "group (c)" text goes with commit 12.
- docs/uk-full-build-graph.md, docs/geography-assignment.md,
  docs/uk-atomic-household-lineage.md and the docstrings of
  tools/build_uk_brma_area_layer.py, tools/build_uk_rfdr_area_layer.py,
  tools/uk_private_rent_pool.py and tools/uk_private_rent_receipt.py:
  (f).
- Until commit 5, validation holds no release candidate to the
  doctrine's K or the conditioned register; every dense build stops
  at the roster refusal first.
- The PR body's sentence that dense multi-clone builds return
  "until (c)" is reworded at this commit's push.

What moves. Keys and records only; no dataset value, and none at
K = 1. No spine stage's code changes: household_lineage,
geographic_clones and geographic_identity change in docstrings,
comments and refusal texts (household_lineage also loses a constant
nothing reads), and national_sampling loses only the dense sampler,
which no stage calls. Through the engine slot, the H2 legacy oracle's
final frame, stage records and stage evidence are identical at
8030471 and here. The dense line refuses every spine until commit
11, and a national build on a spine without clone indices computes
the same dataset.
- Kernels: the UK registries register 84 (96 before: the eight dense
  population kernels and the four shared atomic-geography ones go),
  and 64 change hash. 39 are of the spine registry's 58: uk.create@1,
  the 35 stages that hash household_lineage (geographic_identity,
  which also hashes its own module, among them), and
  cgt_support_split, was_wealth and was_lisa, which hash
  geographic_clones alone. 25 are uk.full kernels that hash an edited
  module: uk.full.target_compilation@1, uk.full.target_selection@1,
  uk.full.measures@1, uk.full.problem@1, uk.full.pool@1,
  uk.full.dense@1, uk.full.calibrated@1, the five size kernels
  (uk.full.size_checkpoint_import@1, uk.full.size_search@1,
  uk.full.size_draw@1, uk.full.size_refit@1 and uk.full.selected@1),
  uk.full.rotated-holdout@1, uk.full-gates.preflight@1,
  uk.full-gates@1, uk.full.certification-readiness@1,
  uk.full-export.prepare@1, uk.full-export.prepare-inspection@1,
  uk.full-export.package@1, uk.full-export.readback@1,
  uk.full.bound_spine@1, uk.full.national_targets@1,
  uk.full.national_problem@1, uk.full.national-gates@1 and
  uk.full.national-readback@1. uk.spine-gates@1, uk.claim@1,
  uk.identity@1 and 17 stages keep theirs. No kernel hashes
  full_build_cli or rowwise_cli, and diagnostics only the national
  gates and problem.
- Node keys, on composed graphs with one stand-in content key per
  source: all 84 nodes of the declared spine with both gate batteries
  re-key (a spine build's root also hashes national_sampling through
  its sampler). The dense graph on that spine loses 13 nodes
  (uk.full.sample, uk.full.normalize, uk.full.expand,
  uk.full.expand.owned, uk.full.identity, uk.full.geography.support.0,
  .1 and .2, uk.full.geography.assign, uk.full.geography.derive,
  uk.full.geography.gate, uk.full.geography.local_authority and
  uk.full.geography_gate), and its other ten re-key, uk.full.measures
  also through its spine-gate parameters and uk.full.gates.calibrated
  through its terminal_code. On a stand-in bound spine the same 13 go
  and all eleven left re-key; the national graph's six nodes re-key.
- No UK kernel is platform-bitwise now, so no dense node key carries
  the platform; on the declared spine 16 of the parent's 23 dense
  nodes did.
- Of the gate phase codes only the full battery's terminal code moves.
- Records: a dense run's input pins lose the three atomic-area
  supports, its parameters lose source_lineage_modulus, and its
  size-checkpoint identity loses both that and the geography block, so
  its input-pins digest, candidate identity digest and size-checkpoint
  identity change, and a size checkpoint cut before this change does
  not resume. The bound spine's provenance, which the national build
  record, the national calibration diagnostics' build block and the
  dense manifest carry, gains K, geography, sampling and
  release_posture. The rowwise manifest's sampling is the spine
  binding's (fraction, seed, sampled, rung token), and its ladder gate
  the descriptor's passed and failures. A dense build id on a sampled
  spine now carries the spine's rung token where it carried f100.
- Unchanged: the NZ toy transport registry's 45 kernel hashes, the six
  US kernels and the US post-transfer graph's eight node keys, the spec
  engine's compiled am, be, nz, us and uk bundles and its kernel
  registry digest, and the packaged country specs. No declaration,
  generated file, gates.json or schema changes, and H2 is not
  regenerated.

Tests (invented data):
- Removed with the nodes they ran: test_uk_full_population_graph.py
  and test_uk_atomic_identity_graph.py. Their tests of live code move:
  the lineage reader's branch paths at one and two clones, residential
  arms and refusals to the new test_uk_household_lineage.py, which
  runs uk_household_clone_paths itself, and the row-wise lane's clone
  and source-lineage refusals, with their new texts, to
  test_uk_rowwise_dataset.py.
- test_support/microcosm_build/uk_full_population_graph.py is a
  clone-first, geography-first spine stand-in: the toy households at
  K clones in the copy-id table's geographic_support field, keyed by
  household_draw_key and placed on the toy supports with the shared
  atomic operators, with a BRMA and a fuel-relief flag. The CLI,
  target-graph and row-wise stand-ins bind it.
- New: each retired flag refused on both roles; the dense role's
  spine-bound refusals on both routes, --n-clones with
  --candidate-clone-counts among them; K, sampling and seed taken from
  the spine on both routes (a seven-clone spine without --n-clones
  records seven in the parameters, the identity digest, the
  configuration, the bindings and a written manifest; a sampled spine
  at a non-default seed gives the run that seed); a disagreeing
  --n-clones on both routes; a sidecar whose K is not its frame's;
  engine blocks held to the spine's K on both routes;
  --candidate-clone-counts planning a spine request once per count,
  with or without the request's own --n-clones; a release candidate on
  a sampled spine on both routes; a sampling fraction off the ladder;
  the build id's rung read ahead, carried by a sampled run's id and
  staging run, held to the spine's at preparation and recorded on an
  early refusal's Logbook row; a composed spine of another K; the
  measures' admission and its order; the pool on the spine's version
  and its keep-all mask and mass policy; the manifest's spine-bound
  sampling and ladder gate; the dense allow-list pin (what a
  geography-first spine adds to the export surface is in
  UK_ALLOWED_EXTRA_EXPORT_COLUMNS); the local area support gate's
  three floors in the K-sensitive constants table; the measurement
  tool's refusal, its command's included; geographic_identity's keys
  held to household_draw_key; the root's sampler in
  test_uk_national_sampling.py.
- Reworded: the row-wise lane's id-rule reasons and refusal tests in
  test_uk_copy_id_table.py, and test_uk_graph.py's binding-table
  reasons, kernel census and the pool's hashed modules.

Checks: through the engine slot, test_uk_full_target_graph,
test_uk_full_graph_admission and test_uk_country_adapter (10 passed);
the H2 legacy oracle identity of a fresh 8030471 snapshot against
this tree (equal: 56 stages in the same order, the three tables
identical, the weights bitwise equal, the mass log, metadata and stage
records equal, and 48 stage-evidence digests and 56 evidence documents
equal); the H2 fixture tool's --check (uk_spine regenerates
byte-identically); and test_acceptance_h_parity.py (passed). In
memory, five mutants of this change's binding (the request's K
ignored, the identity digest taken before binding, the spine's seed
dropped, the request route's --n-clones check skipped, the rung check
dropped) each fail the preparation tests. Engine-free, a batch of the 158
files that import an edited module, test_uk_graph.py in full among
them: 5,109 passed, 23 skipped and 2 failed, both environment-only:
test_uk_cgt_projection's engine-unavailable test fails wherever the
uk extra is installed, as here, and CI's engine-free job installs
none; test_uk_orrery_reconstruction's Orrery check fails on this
machine only. The coverage manifest's --check, ci_test_plan.py
verify, ruff check and ruff format --check on the edited Python
files, and git diff --check. An engine-free probe that reads code
only compared this tree with its parent: every UK kernel's
implementation hash and the edited modules it names, the gate phase
codes, the node keys of the declared spine, of the dense graph on it
and on a stand-in bound spine, and of the national graph on the
stand-in, with one stand-in content key per source, the dense graph's
platform-scoped keys under two invented platforms, and the NZ, US,
spec-engine and packaged-spec identities above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 16, 2026
…931)

The dense line no longer samples, expands, keys or places households
of its own. Its targets, measures and problem run on the spine's
version as it stands, its pool checkpoint keeps every row of that
version, and its geographic clone count K, its sampling rung and seed
and its geography binding are the spine's. It still refuses, before it
pins or reads any input, every spine whose roster is not
geography-first (spine_posture), so no dense build runs until the
spine declares its area stages; the national line still refuses every
clone-first spine.

Why. María's ruling of 10 Nov (decision 4) holds the dense and
national lines back until (c) re-bases them, and C-21 re-bases the
dense line in three commits, refusal first: the roster refusal
(commit 2), the four localisation families (commit 3) and this one,
the last. K, sampling and the geography binding now come from the
spine, so no dense node draws them a second time and the run records
the spine's K and rung.

- graph_population: uk.full.sample, uk.full.normalize, uk.full.expand
  (with uk.full.expand.owned), uk.full.identity, uk.full.locations,
  uk.full.geography_mapping, uk.full.geography.local_authority and
  uk.full.geography_gate go, with their eight kernels, the
  population-receipt, location-draw and geography-gate artifact types,
  append_uk_population_nodes, uk_pool_validation_inputs,
  GEOGRAPHY_ASSIGNMENTS and UK_GEOGRAPHY_IDENTITY_INPUTS. The UK
  registry stops registering the four shared atomic-geography kernels
  (NZ's transport registers its own), so the uk.full.geography.*
  support, assign, derive and gate nodes go too. uk.full.pool@1 keeps
  its id: a keep-all FILTER on uk.full.spine_checkpoint (--input-h5)
  or the spine's endpoint (--spine-request), with no typed edge,
  declared bitwise (platform-bitwise before, for the shared gate's
  edge) and hashing its own module and national_frame. The dense
  geography gate's ladder and installed-engine local-authority checks
  and the pool's refusal of a failed shared validation leave with
  their nodes; the spine's geography gates take them over once they
  are registered (commit 11), and until then the roster refusal holds
  every dense build.
- graph_targets: append_uk_target_nodes takes population with no
  default. The measures lose the geography-gate edge and check, and
  take the spine's transferred-gate admission, which the sample node
  held, wherever the graph declares spine.gates.transferred; they
  check it before they read anything.
- graph_build: UKFullBuildConfig loses sample_fraction, sample_seed,
  geography_assignment, source_vintage, constituency_vintage and
  source_lineage_modulus; n_clones defaults to 1, and
  effective_sample_fraction is source_sample_fraction, the spine's
  rung. uk_full_graph drops spine_weight_kind and
  atomic_geography_definition and refuses a composed spine whose
  geographic_expand node declares another K, naming both;
  build_uk_country_graph drops atomic_geography_definition.
  uk.full.bound_spine@1 also hashes geographic_clones.
- calibration_run: _assert_spine_sidecar_binds_frame holds the
  sidecar's geographic_clone_count (one where a sidecar predates it)
  to the frame's, naming both; spine_provenance_from_sidecar adds K,
  geography, sampling and release_posture.
- full_build_cli and rowwise_cli: prepare_full_build reads K,
  sampling, release posture and the geography binding from the
  authenticated sidecar, or from the spine request's --n-clones,
  --sample-fraction and --sample-seed, and sets the run's n_clones,
  sample_fraction and, for a sampled spine, sample_seed before the
  parameters, identity digest, size-checkpoint identity, staging rung
  and Logbook rungs record them. A spine whose sampling fraction is
  not a scale-ladder rung is refused, naming the rungs only. The
  attempt's build id is minted with the spine's rung, read ahead of
  preparation (unauthenticated, for the label) off the bound spine's
  sidecar or the request's --sample-fraction through the spine's own
  parser, and f100 where that read fails; preparation refuses an
  attempt whose build id carries another rung than the spine it
  binds, naming both rung tokens, and a refusal before the spine is
  bound records the build id's rung on its Logbook row. The request
  bindings gain a spine block, and geography is the spine's binding
  with K beside it. microcosm-build-uk no longer takes
  --geography-assignment, --atomic-support-ew,
  --atomic-support-scotland, --atomic-support-ni,
  --atomic-support-sha256-ew, --atomic-support-sha256-scotland,
  --atomic-support-sha256-ni or --source-lineage-modulus; the parser
  refuses each as an unrecognized argument on either role. The dense
  role refuses, by name and before any input is read, a
  --sample-fraction other than 1.0, any --sample-seed,
  --candidate-clone-counts with --input-h5, and --n-clones with
  --candidate-clone-counts. --n-clones only asserts the spine's K: the
  role no longer fills it from the doctrine, and preparation refuses
  one that differs, naming both, and on a spine request pointing at
  the request's own --n-clones. --candidate-clone-counts plans the
  spine request once per count, as its --n-clones.
  --expected-constituency-vintage is still taken and recorded, and its
  help now says no dense node checks the ladder against it. Release
  candidates: a sampled spine is refused at preparation, where its
  rung is read, on either route; the doctrine-K check and the
  support-pin requirement leave validation for the spine's K pin and
  binding check (commit 5); the ladder, Ledger and input pins, target
  rule, epochs, measure exclusions, local target levels and signing
  key are refused as before.
- graph_terminal: the rowwise manifest reads sampling,
  identity.sampling, the rung and the release verdict's full-rung leg
  from the request's spine block, refusing a descriptor without it,
  and its ladder gate from the export descriptor's geography check,
  at phase post_calibration; the weight-kind chain names
  geographic_expand.
- national_sampling: the dense family sampler (sample_uk_spine_frame,
  uk_spine_source_family_units) and its re-exports go; the root's
  sampler, the rung tokens and the default seed stay.
- size_checkpoint: the identity loses source_lineage_modulus and the
  atomic geography block.
- household_lineage: the reader's three lineage refusals open with the
  household table and its clone paths, not "Geography identity", and
  UK_OPTIONAL_LINEAGE_COLUMNS, which only the retired identity kernel
  read, goes.
- Texts: rowwise_dataset's clone refusals point at the dense graph on
  a geography-first spine and promise no later group;
  tools/build_uk_rowwise_dataset.py says it serves spines without
  clone indices; the docstrings of geographic_identity,
  household_lineage (owed by commit 3), geographic_clones (which now
  names the national gates among its hashers) and diagnostics (whose
  support-family key comment names the lineage columns, not the
  identity kernel's inputs) stop naming the dense identity node or
  expansion as a writer or reader; tools/measure_uk_atomic_assignment.py
  refuses its graph route, and its command before it reads any input,
  with a pointer to the spine's atomic_area_assign stage; the Orrery
  roster loses its geography group and the retired nodes. Three older
  unreleased fragments that this change makes stale (the #932 wiring,
  the national sampler's retirement, commit 3's) each gain a clause
  saying what retires later.

Defaults taken, each listed for María with its options: the
configuration's K defaults to 1; a composed spine of another K is
refused at composition, and a bound spine's K is read at preparation;
a sidecar without geographic_clone_count reads as K = 1; the run's
arguments are rewritten to the spine's K, fraction and seed; the
spine binding's shape, with the manifest failing closed without it;
--candidate-clone-counts replaces a request's own --n-clones, and
--n-clones beside it is refused; the measures take the admission only
where the graph declares the transferred gate; the pool is bitwise;
the manifest's ladder gate is the descriptor's check at phase
post_calibration; the weight-chain label is geographic_expand; the
Orrery geography group goes; the identity node's tests retire with it
and the lineage reader's own tests move to test_uk_household_lineage.py;
the local area support gate's row, effective-sample-size and source
floors are kept K-sensitive constants; the build id's rung is read
ahead and held to the spine's at preparation; a sampling fraction off
the ladder is refused; --expected-constituency-vintage stays a
recorded assertion.

Left for later:
- The geographic_identity stage's notes in sources.yaml, and so
  source_stages.json and the H2 fixture, still name uk.full.identity:
  commit 8, which regenerates H2 with the geography declarations.
- On a spine request the geography binding carries K alone: commit 8
  composes the request route's block with the sidecar writer's helper,
  or commit 5 refuses --release-candidate with --spine-request; commit
  5's binding check also derives the stand-in sidecar's geography
  values from the stand-in's own definition.
- The spine's mapping-integrity, ladder and LA-engine-domain gates
  that take over the retired dense checks are declared
  release-blocking when commit 11 registers them.
- --expected-constituency-vintage: commit 5's binding check, or a
  ruling (hold it to the ladder, retire it, or keep it recorded).
- tools/measure_uk_atomic_assignment.py's manifest readers, past the
  refusal, still name retired nodes: commit 9 re-points the tool.
- refuse_dense_clone_id_overflow keeps its name, though only the
  row-wise lane calls it, as do two row-wise driver test names;
  test_uk_national_graph.py's "group (c)" text goes with commit 12.
- docs/uk-full-build-graph.md, docs/geography-assignment.md,
  docs/uk-atomic-household-lineage.md and the docstrings of
  tools/build_uk_brma_area_layer.py, tools/build_uk_rfdr_area_layer.py,
  tools/uk_private_rent_pool.py and tools/uk_private_rent_receipt.py:
  (f).
- Until commit 5, validation holds no release candidate to the
  doctrine's K or the conditioned register; every dense build stops
  at the roster refusal first.
- The PR body's sentence that dense multi-clone builds return
  "until (c)" is reworded at this commit's push.

What moves. Keys and records only; no dataset value, and none at
K = 1. No spine stage's code changes: household_lineage,
geographic_clones and geographic_identity change in docstrings,
comments and refusal texts (household_lineage also loses a constant
nothing reads), and national_sampling loses only the dense sampler,
which no stage calls. Through the engine slot, the H2 legacy oracle's
final frame, stage records and stage evidence are identical at
8030471 and here. The dense line refuses every spine until commit
11, and a national build on a spine without clone indices computes
the same dataset.
- Kernels: the UK registries register 84 (96 before: the eight dense
  population kernels and the four shared atomic-geography ones go),
  and 64 change hash. 39 are of the spine registry's 58: uk.create@1,
  the 35 stages that hash household_lineage (geographic_identity,
  which also hashes its own module, among them), and
  cgt_support_split, was_wealth and was_lisa, which hash
  geographic_clones alone. 25 are uk.full kernels that hash an edited
  module: uk.full.target_compilation@1, uk.full.target_selection@1,
  uk.full.measures@1, uk.full.problem@1, uk.full.pool@1,
  uk.full.dense@1, uk.full.calibrated@1, the five size kernels
  (uk.full.size_checkpoint_import@1, uk.full.size_search@1,
  uk.full.size_draw@1, uk.full.size_refit@1 and uk.full.selected@1),
  uk.full.rotated-holdout@1, uk.full-gates.preflight@1,
  uk.full-gates@1, uk.full.certification-readiness@1,
  uk.full-export.prepare@1, uk.full-export.prepare-inspection@1,
  uk.full-export.package@1, uk.full-export.readback@1,
  uk.full.bound_spine@1, uk.full.national_targets@1,
  uk.full.national_problem@1, uk.full.national-gates@1 and
  uk.full.national-readback@1. uk.spine-gates@1, uk.claim@1,
  uk.identity@1 and 17 stages keep theirs. No kernel hashes
  full_build_cli or rowwise_cli, and diagnostics only the national
  gates and problem.
- Node keys, on composed graphs with one stand-in content key per
  source: all 84 nodes of the declared spine with both gate batteries
  re-key (a spine build's root also hashes national_sampling through
  its sampler). The dense graph on that spine loses 13 nodes
  (uk.full.sample, uk.full.normalize, uk.full.expand,
  uk.full.expand.owned, uk.full.identity, uk.full.geography.support.0,
  .1 and .2, uk.full.geography.assign, uk.full.geography.derive,
  uk.full.geography.gate, uk.full.geography.local_authority and
  uk.full.geography_gate), and its other ten re-key, uk.full.measures
  also through its spine-gate parameters and uk.full.gates.calibrated
  through its terminal_code. On a stand-in bound spine the same 13 go
  and all eleven left re-key; the national graph's six nodes re-key.
- No UK kernel is platform-bitwise now, so no dense node key carries
  the platform; on the declared spine 16 of the parent's 23 dense
  nodes did.
- Of the gate phase codes only the full battery's terminal code moves.
- Records: a dense run's input pins lose the three atomic-area
  supports, its parameters lose source_lineage_modulus, and its
  size-checkpoint identity loses both that and the geography block, so
  its input-pins digest, candidate identity digest and size-checkpoint
  identity change, and a size checkpoint cut before this change does
  not resume. The bound spine's provenance, which the national build
  record, the national calibration diagnostics' build block and the
  dense manifest carry, gains K, geography, sampling and
  release_posture. The rowwise manifest's sampling is the spine
  binding's (fraction, seed, sampled, rung token), and its ladder gate
  the descriptor's passed and failures. A dense build id on a sampled
  spine now carries the spine's rung token where it carried f100.
- Unchanged: the NZ toy transport registry's 45 kernel hashes, the six
  US kernels and the US post-transfer graph's eight node keys, the spec
  engine's compiled am, be, nz, us and uk bundles and its kernel
  registry digest, and the packaged country specs. No declaration,
  generated file, gates.json or schema changes, and H2 is not
  regenerated.

Tests (invented data):
- Removed with the nodes they ran: test_uk_full_population_graph.py
  and test_uk_atomic_identity_graph.py. Their tests of live code move:
  the lineage reader's branch paths at one and two clones, residential
  arms and refusals to the new test_uk_household_lineage.py, which
  runs uk_household_clone_paths itself, and the row-wise lane's clone
  and source-lineage refusals, with their new texts, to
  test_uk_rowwise_dataset.py.
- test_support/microcosm_build/uk_full_population_graph.py is a
  clone-first, geography-first spine stand-in: the toy households at
  K clones in the copy-id table's geographic_support field, keyed by
  household_draw_key and placed on the toy supports with the shared
  atomic operators, with a BRMA and a fuel-relief flag. The CLI,
  target-graph and row-wise stand-ins bind it.
- New: each retired flag refused on both roles; the dense role's
  spine-bound refusals on both routes, --n-clones with
  --candidate-clone-counts among them; K, sampling and seed taken from
  the spine on both routes (a seven-clone spine without --n-clones
  records seven in the parameters, the identity digest, the
  configuration, the bindings and a written manifest; a sampled spine
  at a non-default seed gives the run that seed); a disagreeing
  --n-clones on both routes; a sidecar whose K is not its frame's;
  engine blocks held to the spine's K on both routes;
  --candidate-clone-counts planning a spine request once per count,
  with or without the request's own --n-clones; a release candidate on
  a sampled spine on both routes; a sampling fraction off the ladder;
  the build id's rung read ahead, carried by a sampled run's id and
  staging run, held to the spine's at preparation and recorded on an
  early refusal's Logbook row; a composed spine of another K; the
  measures' admission and its order; the pool on the spine's version
  and its keep-all mask and mass policy; the manifest's spine-bound
  sampling and ladder gate; the dense allow-list pin (what a
  geography-first spine adds to the export surface is in
  UK_ALLOWED_EXTRA_EXPORT_COLUMNS); the local area support gate's
  three floors in the K-sensitive constants table; the measurement
  tool's refusal, its command's included; geographic_identity's keys
  held to household_draw_key; the root's sampler in
  test_uk_national_sampling.py.
- Reworded: the row-wise lane's id-rule reasons and refusal tests in
  test_uk_copy_id_table.py, and test_uk_graph.py's binding-table
  reasons, kernel census and the pool's hashed modules.

Checks: through the engine slot, test_uk_full_target_graph,
test_uk_full_graph_admission and test_uk_country_adapter (10 passed);
the H2 legacy oracle identity of a fresh 8030471 snapshot against
this tree (equal: 56 stages in the same order, the three tables
identical, the weights bitwise equal, the mass log, metadata and stage
records equal, and 48 stage-evidence digests and 56 evidence documents
equal); the H2 fixture tool's --check (uk_spine regenerates
byte-identically); and test_acceptance_h_parity.py (passed). In
memory, five mutants of this change's binding (the request's K
ignored, the identity digest taken before binding, the spine's seed
dropped, the request route's --n-clones check skipped, the rung check
dropped) each fail the preparation tests. Engine-free, a batch of the 158
files that import an edited module, test_uk_graph.py in full among
them: 5,109 passed, 23 skipped and 2 failed, both environment-only:
test_uk_cgt_projection's engine-unavailable test fails wherever the
uk extra is installed, as here, and CI's engine-free job installs
none; test_uk_orrery_reconstruction's Orrery check fails on this
machine only. The coverage manifest's --check, ci_test_plan.py
verify, ruff check and ruff format --check on the edited Python
files, and git diff --check. An engine-free probe that reads code
only compared this tree with its parent: every UK kernel's
implementation hash and the edited modules it names, the gate phase
codes, the node keys of the declared spine, of the dense graph on it
and on a stand-in bound spine, and of the national graph on the
stand-in, with one stand-in content key per source, the dense graph's
platform-scoped keys under two invented platforms, and the NZ, US,
spec-engine and packaged-spec identities above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 16, 2026
…positions or counts (#931)

Code prose, test text and fifteen #931 changelog fragments still gave
spine stages by their position, or the spine by a stage count, in the
order the text was written against. The geography-first stages and
main's two inserted stages have since moved those positions, so many
of the numbers named the wrong stage. This commit names each stage
instead, mapping every number through the roster its own text was
written against: the 54-stage order for #931's text, and main's
56-stage order for the one fragment that was restated in it. Each name
is checked against the packaged roster, apart from regional_home_values,
which #931 retired. No code behaviour changes.

What changed:
- copy_id_table's field list names the stage that mints each field,
  and drops the positions after the CGT fields, whose names are their
  stages'. graph_kernels' comments name frs_council_tax and
  frs_hmrc_spine_leaves, and spine_council_tax's docstrings name
  frs_council_tax.
- The identity stability tool, the council-tax test fixtures and
  sixteen test files name the stages, among them the nesting test's
  report string. Two test texts cite the rebased commits of the
  keyed tie-breaks and the keyed Bernoulli share (cca9a16 and
  c64c2b3) in place of their pre-rebase hashes.
- The H2 parity test's docstring says that both sides run the root and
  every spine stage, and that the _STAGE_NAMES tuple the driver kept in
  step with the packaged roster by hand is gone, which the test's last
  assertion holds. Its node-count message reads "a CREATE node plus the
  spine's stages", and UK_SPINE_PARITY's comment calls the fixture the
  packaged FRS spine expressed as a graph.
- The fragments name the stages. The copy-id fragment lists by name
  every stage whose values move on copy rows through a draw keyed on
  the copy's own id. The council-tax localisation fragment says
  frs_council_tax's operations now end with the derive, where it
  counted three. The ISA fragment no longer gives its stage two
  numbers. The true-source floors fragment speaks of the sampled
  rung's region floor for private rents.
- Four test comments drop a design reviewer's label.

Five more texts were stale. They change text only and now say what the
code does; the two runtime modules among them, cgt_support and
diagnostics, move keys:
- cgt_support's comment on CGT_SUPPORT_COPY_INDEX_COLUMN, the
  identity tool's support-copy lineage docstring and the row-wise
  candidate fixture's comment said the geography identity kernel keys
  on this lineage (#932). They now say that the copy readers read the
  CGT support copy index, and the support channel as the SPI support
  branch's marker, through the copy lineage
  (household_lineage.UK_COPY_LINEAGE), and that each CGT support copy
  keeps its parent's geography key.
- diagnostics.uk_support_family_weights' docstring said its family key
  is that lineage less the copy flag and index. The key is the
  household lineage columns
  (household_lineage.UK_HOUSEHOLD_LINEAGE_COLUMNS) less the support
  split's copy flag and index and the residential split's arm flag and
  index.
- tools/uk_private_rent_pool.py's second paragraph said that a later
  step of #931 moves BRMAs and private rents into the spine's own
  stages. It now says that the spine's area_brma stage reads each
  Great Britain household's BRMA off its drawn area, and its
  private_rent stage each Great Britain private rent off its BRMA's
  list of rents.

Keys and values: keys only. 68 of the 91 UK kernels hash an edited
module and move: all 65 of the spine registry's, which hash
graph_kernels, the spine gates, which hash it too, and the national
gates and national problem kernels, which hash uk_runtime.diagnostics.
copy_id_table reaches 12 of the 68, the root's CREATE kernel among
them, cgt_support 2 and spine_council_tax 1. The other 23 keep their
hashes, among them every kernel of the dense line. Every node of the
spine graphs re-keys: the spine with its gates (94 nodes, at one clone
and at four), the bundled spine (92) and the dense graph on that spine
(104). On a bound spine the dense graph re-keys only its calibrated
gates (1 of 11 nodes), whose terminal phase code moves, and the
national graph re-keys its problem and the three nodes after it (4 of
6); the bound spine's checkpoint and the national targets keep their
keys. Two gate phase codes move: the spine's transferred code, which
hashes copy_id_table, and the full battery's terminal code, which hashes
uk_runtime.diagnostics. The spine's assembled code and the full
battery's preflight code keep their values. No value moves, and no
stage contract (all 63 equal), country or solve fingerprint, bundle
lock, pinned digest, H2 file or coverage manifest changes. No other
country's key moves: the shared transport registry's 31 kernels, NZ's
fixed-donor graph keys and battery sources, the US kernels and
post-transfer graph, and the spec engine's bundles are unchanged. No
kernel hashes a tool or test module, so the tool, test support, test
and fragment edits move no key.

Checks: ruff check --output-format concise and ruff format --check pass
on the 30 touched Python files, which all compile, and git diff --check
passes on the 45 files. The engine-tier and after-merge files change
only in comments, docstrings and one assert message, so ruff and an AST
parse check them, with no engine-slot run: test_uk_engine_block_correction,
the engine test_uk_graph, test_uk_private_school_pooled_rank,
test_uk_spine_three_systems, the H2 parity test and its support module,
and test_h2_clone_first. These engine-free tests pass (1944 passed):
test_uk_clone_first_nesting, test_uk_spi_band_donors,
test_uk_identity_stability_receipts, test_uk_graph, test_uk_cgt_support,
test_uk_copy_id_table (with its scan of the UK runtime and UK tool
sources), test_uk_frs_spine, test_uk_spi_housing_council_tax,
test_uk_spi_housing_shell, test_uk_spi_spine, test_uk_spine_geography,
test_uk_property_components, test_uk_high_value_home_support,
test_uk_cgt_imputation, test_uk_spine_council_tax, test_uk_diagnostics,
test_uk_e5_identity, test_uk_stochastic_tools, test_uk_receipt_counts,
test_uk_private_rent_receipt_tool, test_uk_stage_health, the row-wise
tests that load the candidate fixture (test_uk_rowwise_candidate,
test_uk_rowwise_national_role, test_uk_rowwise_posture,
test_uk_full_build_cli, test_uk_modal_copy_back_staging,
test_uk_orrery_publication and test_uk_release_assembler), and the
graph package's engine-free test_acceptance_h_parity and
test_acceptance_a_identity. test_us_plan.py's live-tree scan for the
retired data packages passes, and so does tools/ci_test_plan.py verify.
The key probe read kernels, node keys, fingerprints and the four phase
codes on an archive of the parent and on the tree.

Defaults taken, for review:
- F-19: frs_council_tax's own module keeps its two positional mentions;
  the number is right, in the stage's own module. Option: rename them.
- F-20: test and helper names that carry a position stay, since they
  are code, not prose; the nesting test's report string takes the
  name. Option: rename them here.
- F-21: the copy-id fragment lists every stage by name. pip_points is
  in, since it draws on person_id; uc_limited_capability is out, since
  it draws nothing of its own; regional_home_values is marked as
  retired later in #931. Option: by-name ranges, with a clause for the
  stages inside them that draw nothing on copy ids
  (uc_limited_capability, private_rent, council_tax_localise), and the
  second range starting at the stage area_home_values replaced.
- F-22: four test comments drop a design reviewer's label. Three keep
  (microcosm#931); the fourth cited no issue, so it now ends at
  "band.". Option: leave them.
- F-25: the H2 parity test's node-count assert keeps its >= 32
  threshold, which is code and holds; only its message changes.
  Option: tie the threshold to UK_FIXTURE_STAGE_COUNT, a test change
  checked in the engine slot.

Not changed here: the SPI income stage's own stage 1 and stage 2
forests and the cut's stage-1 comparison, which are not spine
positions. Main's two older positional texts (cgt_structure's two
docstrings and the #970 fragment) and the take-up contract's citation
wait for review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants