Skip to content

Pin US county facts beside the main feed for the county rung - #1614

Merged
MaxGhenis merged 10 commits into
mainfrom
lwi-county-facts-pin
Nov 16, 2026
Merged

MaxGhenis merged 10 commits into
mainfrom
lwi-county-facts-pin

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

WIP (draft). Part of #922, the Living Wage Institute County Files.

What this does

The ACS-local county rung needs county-grain Chronicle facts: IRS SOI county TY2022, Census PEP county vintage 2024, and seven ACS 2020-2024 5-year county tables (B01003, B11001, B19001, B19013, B19025, B19051, B19061). This PR pins them as their own file beside the main US feed pin, instead of growing the main feed about 6.4x for every US build.

  • us/chronicle_county_facts.json and us/chronicle_county_facts_scope.json (scope us_county_facts, 10 record-set/period pairs). Built at Chronicle main 3e9d3a0: 180,326 rows on the chronicle.consumer_fact.v5 contract, facts SHA256 67b0998a…. The two scopes share no pair, so the main feed, its scope and the target-parity resources do not move.
  • us_runtime/county_facts.load_us_county_facts first hashes the file and refuses it unless its digest and row count are the pin's. A second pass hashes it again while parsing and refuses non-object rows, other row contracts, non-county rows, rows outside the scope and a file that changed between the passes. It drops the 78 Puerto Rico municipios with a count and keeps only the fields county targets read: about 0.9 GB of rows, loaded in about 5 s.
  • tools/us_acs_local_county_surface.py: a --soi-mode state_county materialize now needs --county-facts and --county-facts-sha256 (it must equal the pin's), plus --feed-sha256. prepare_run loads the county facts once, and refusals are recorded in county_surface.json. The run identity records the county facts receipt; a run without county facts keeps its pre-county bytes. The refresh recipe names a placeholder file and the pinned digest.
  • chronicle_feed.py: one validator (_load_pin) now loads both the main pin and the county pin, and the county pin must be a bare facts file. The main pin's behaviour and messages are unchanged.
  • The US country package declares the two new files: they are added to LEGACY_RESOURCE_PATHS in tools/generate_us_bundle_from_constants.py and country_package.json is regenerated. That moves the US bundle's spec_sha256, so the US spec-engine contracts are repinned with tools/repin_us_spec_engine_contracts.py and tools/spec_engine_coverage.py.
  • Docs: docs/us-acs-local-county-rung.md (county facts input, Kalawao), docs/us-chronicle-feed-repin.md (the county facts pin, where its file lives, and that it moves with the main pin), docs/agent-guide.md.
  • The 922 MB file is never committed: the repository holds only its pin (digest, size, row count, scope). A change that re-pins the main US feed at a Chronicle commit also rebuilds and re-pins the county facts at that commit, so a US build reads one Chronicle base. Today the main pin is at 41b2fb2 and the county pin at 3e9d3a0; the next main re-pin starts at or after 3e9d3a0.

County materialize runs are local: the Modal stage plan cannot pass --county-facts, and its preflight refuses a state_county materialize.

Known gaps, handled in follow-up work on this branch's base

  • Kalawao County (15005) has no IRS SOI county row: the IRS excludes items with fewer than 20 returns for a county. A full state_county run still refuses Hawaii until the rung declares a combined SOI cell {15005, 15009}.
  • (Closed 2026-11-14.) A rebuild of the county facts file at Chronicle 3e9d3a0, from a clean checkout with the documented command, matched the pin byte for byte: SHA256 67b0998a…, 180,326 rows, 922,268,386 bytes. It took 8 minutes; the output folder peaked at 2.89 GB.

Plan

  1. Independent hard review of the whole change. Round 1 (20261114-092140, at e09c1828) ran out of wall time with no findings. Round 2 (20261114-232757-lwi-pin-review-r2, at 5825f5b7, merge included): APPROVE, no P0/P1; four P2s and nine P3s, fixed in ca394849 as listed below.
  2. A scoped re-review of ca394849. Round 3 (20261115-094413-lwi-pin-review-r3, at ca394849): APPROVE, with one nonblocking P2, fixed in 8f71d34a. Round-2 P3-8 (scope completeness) and P3-9 (error order when both resources are invalid) are unchanged; the reviewer found no regression in either. Mark ready once CI is green.
  3. The US feed owner agreed to the separate pin on 11/16 (d2909 lifted; Max's ruling), on four conditions at the landing head: a scoped re-review of the main merge passes and required checks are green; us/chronicle_feed.json, us/chronicle_feed_scope.json and both target-parity resources are byte-equal to main's; the county pin is unchanged (3e9d3a0, facts 67b0998a…, 180,326 rows); and the generated counts and digests are recomputed by the tools. It also asked for a test that the county pin moves with the next main pin (added in c8403535). The Microcosm hub lands it. The scoped re-review of 07a39c95..c8403535 is queued.

Verification so far

  • An in-session review of the first commit (four dimensions, an adversarial verifier each and a completeness critic): 30 findings, 29 confirmed, plus two critic findings, all addressed in the second commit.

  • Before the rebase onto 68bbff774: focused county tests 583 passed; test_country_spec.py 144 passed; ruff check . clean; tools/ci_test_plan.py verify ok.

  • After the rebase, the four-step repin was rerun (repin_us_spec_engine_contracts.py --legacy-only, generate_us_bundle_from_constants.py, repin_us_spec_engine_contracts.py, spec_engine_coverage.py), all exit 0. The US bundle spec_sha256 moves from main's 98d58365… to fdf6e8f4…. The coverage evidence gains exactly six pointers (two legacy rows, three fields each): 42,400/42,400 configuration fields, 41/41 inventory checks.

  • Full local verification on d2e2767f: the three --check modes (repin_us_spec_engine_contracts.py, generate_us_bundle_from_constants.py, spec_engine_coverage.py) exit 0; tools/ci_test_plan.py verify ok; ruff check . clean; pytest tests/engine_free/us tests/engine_free/shared -n 4: 17,149 passed, 156 skipped, 0 failed; test_us_spec_bundle.py, test_spec_engine_coverage_tool.py, test_us_multispine_pool_tool.py: 60 passed. CI on d2e2767f: 12/12 pass.

  • e09c1828 merges main at 52aea1477 (Renew the US bundle generator's migration tombstone to 16 February 2027 (#1585) #1596's tombstone renewal and Test that the build-folder publisher publishes the SCF education-debt fit tables #1612) and adds the doc paragraphs above. The five-step repin on the merged tree (--legacy-only, generate, repin twice per Repinner needs two passes to pin covered_item_count #1530, coverage) changed no file: spec_sha256 stays fdf6e8f4…, coverage 42,400/42,400 and 41/41. The tests were not rerun locally on e09c1828; CI runs them on this head.

  • 5825f5b7 merges main at 085672fa9 (UK stock cells: homes and the land under them in England and Wales (#1524) #1597, US: one copy of each shared helper (#1585 items 4, 5 and 8.3) #1601, Admit a UK local-area release to build folders only once the registry names it #1603), which re-pinned the same US spec-engine files. All six conflicts were generated counts and digests; each took main's side and the five-step repin recomputed them on the merged tree. The US bundle spec_sha256 is now d3a3c6c6…; coverage is 42,401/42,401 configuration fields (main's 42,395 plus this branch's six) and 41/41 inventory checks. The three --check modes, tools/ci_test_plan.py verify and ruff check . exit 0. Focused tests (coverage tool, field usage, multispine pool tool, both spec bundle suites, county facts, county surface, Chronicle feed, test_country_spec.py) at -n 2: 333 passed, 0 failed.

  • ca394849 fixes the round-2 findings. P2-1: the loader's second pass now hashes the whole file before it refuses. A file swapped between the passes is named as changed whatever its rows hold, and a malformed pinned row is always a ValueError, so prepare_run records it in county_surface.json. Before, a swapped row with a text period or a list contract raised an unrecorded AttributeError/TypeError. P2-2, P2-3 and the P3 doc nits: the docs no longer claim the pin holds the file's size, or that nothing reads --feed; they say when refusals are recorded and where the Modal refusal comes from. P2-4: new tests cover the id checks, blank lines, swapped files, any pinned row value (a property test: it loads or raises ValueError), the None guard and the Modal plan. The swap and property tests fail on the old loader. Not changed: the unreachable fallback in attach_surface (the production county rung rewrites that function and takes the fix) and dropping projected rows after the SOI split (the production rung reads them). Focused tests on the fix (county facts, county surface, Chronicle feed, Modal stage plan, county properties) at -n 4: 432 passed, 0 failed; ruff check ., ruff format --check, tools/ci_test_plan.py verify and git diff --check clean. county_facts.py is not byte-attested, so no spec-engine pin moved.

  • d2efc978 merges main at 429d7a0de (Project the softmax parametrization exactly onto the capped simplex each step #1096, the exact softmax cap projection in solve), which moved the shared spec pins. Both conflicts were pin files: us-f0-coverage.json took main's version and was regenerated. In test_us_multispine_pool_tool.py both pin comments are kept and the tools recompute spec_sha256 (fa75bd68… on main, 1e727e1c… here). The five-step repin, the three --check modes, tools/ci_test_plan.py verify and ruff check . exit 0. County focused suite (county facts, county surface, Chronicle feed, Modal stage plan, county properties, release tool) at -n 4: 746 passed, 1 skipped. Spec-engine pin suites at -n 2: 116 passed. National pin suites at -n 2: 481 passed. A repo-wide ruff format --check . flags 53 files, all also on main and none touched here; CI does not run it.

  • 8f71d34a fixes round-3 P2: the county-facts Modal test now runs the check stage's _parse_check on this checkout with the planned state_county argv, and asserts return code 2 and the missing --county-facts/--county-facts-sha256 refusal. With the release tool's parse_function set to None the test fails (return code 0, from --help). test_us_modal_stage_plan_tool.py at -n 4: 284 passed.

  • 07a39c95 merges main at 99814fe43 (Keep ACS partnership_income at zero where SEMP already holds it (#1387) #1619, Add the County Files household measures sidecar and schema manifest tools (#922) #1621, Add the shared gate-battery pieces for US adoption (#1585 item 11, PR 1 of 4) #1609, Align DESIGN.md and US builder help with how releases calibrate #1540, Drop the wall-clock assert from the composed stage-scan test #1629, Add microcosm-withdraw-release-build and refuse withdrawn parent builds #1594). Both conflicts (us-f0-coverage.json, test_us_multispine_pool_tool.py) took main's side and the five-step repin regenerated them: spec_sha256 is 3fd1a384…. The hand-written comment above that pin keeps main's ACS rows carry a donor partnership_income draw ($337.7B weighted) that can stack on SEMP #1387 line and this PR's County-calibrated County Files (3,144 counties, 2026 + 2027): build epic #922 line. The three --check modes, tools/ci_test_plan.py verify, ruff check . and git diff --check exit 0. County focused suite at -n 4: 751 passed, 1 skipped. Spec-engine pin suites at -n 2: 116 passed. National pin suites at -n 2: 481 passed. The PR's own change against main is the same as before the merge apart from spec_sha256 and the new test.

  • c8403535 adds test_the_county_pin_moves_with_the_next_main_pin: the main feed pin may stay at 41b2fb2; at any other commit the county pin must name the same one. A probe with the main pin moved and the county pin left behind fails it; both moved together, or the main pin moved onto 3e9d3a0, pass. County facts suite: 48 passed.

  • At c8403535: the four feed and parity resources are byte-equal to main at 99814fe43 and at 14797c665 (Rename Populace to Microcosm in prose and code #1541); the county pin and its scope file are unchanged from 8f71d34a; git merge-tree with 14797c665 is clean. tools/build_us_fiscal_refresh_release.py imports the same three names from chronicle_feed as at 8f71d34a and reads neither the county pin nor spec_sha256.

🤖 Generated with Claude Code

MaxGhenis and others added 5 commits November 13, 2026 20:32
The ACS-local county rung needs county-grain Chronicle facts: IRS SOI
county TY2022, Census PEP county vintage 2024, and seven ACS 2020-2024
5-year county tables. Folding them into the main US feed would grow it
about 6.5x for every fiscal compile while only county builds read them,
so they get their own pin.

- us/chronicle_county_facts.json and chronicle_county_facts_scope.json
  pin a bare v5 facts file at Chronicle main 3e9d3a069 (scope
  us_county_facts, 10 pairs). The main pin, its scope and the parity
  resources do not change; the two scopes share no pair.
- chronicle_feed.py loads both pins through one validator; the main
  pin's messages are unchanged.
- us_runtime/county_facts.py reads the file in one pass: digest and row
  count against the pin, refusal of any row on another contract, at
  another geography level or outside the scope, Puerto Rico dropped with
  a count, rows projected to the fields the rung reads.
- The county rung takes --county-facts and --county-facts-sha256 (the
  pin's digest), requires --feed-sha256, loads the file once in
  prepare_run and never re-reads --feed for county facts. The run
  identity and refresh recipe record them; a no-county run's identity
  keeps its bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Declare the county pin and its scope as legacy resources in
tools/generate_us_bundle_from_constants.py and regenerate
us/country_package.json, so load_country_spec('us') and the spec-engine
load_bundle('us') load again (test_country_spec failed at 0a6f201b), and
repin the US spec-engine contracts for the new manifest bytes.

Loader: hash the file and check its digest and row count before parsing
any row, so a wrong file is named as such; hash again while parsing and
refuse a file that changed between the passes; refuse a row that is not
a JSON object and a county code with non-ASCII digits; keep fewer fields
per row (the pinned file's kept rows take about 0.9 GB instead of about
1.5 GB, loaded in about 5 s).

County rung: record a county facts file that cannot be read in
county_surface.json like any other refusal; the refresh recipe names a
placeholder county facts file and the pinned digest, never the build
machine's path; the error for missing SOI county rows names the county
facts pin.

Tests: lock the pin's literals; refuse a pin naming a consumer artifact
and malformed pins; match the digest refusal; non-object rows, non-ASCII
digits, the real receipt and a mid-read change; the projected rows give
the same SOI surface as full rows; an off-pin county file and a missing
one are refused before any target; a run without county geography is
refused; a run without county facts adds no metadata key.

Docs: correct the per-table county counts (B19013 has 3,143; B01003
reaches 3,144 across two record sets), the size figures (5.2 KB per ACS
row, a 171.7 MB main feed, 6.4x), the feed build command
(--chronicle-command), the c32c0b0 comparison key and its status, the
raw SOI county sums against their state parents (AGI up to 5% short), what
--feed-sha256 checks, and that county materialize runs are local; the
agent guide no longer describes a county-inclusive main feed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main renewed the US bundle generator tombstone (#1596) and added a
publisher test (#1612). The four-step spec-engine repin on the merged
tree changes nothing: spec_sha256 stays fdf6e8f4 and coverage is
42400/42400 configuration fields, 41/41 inventory checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ther (#922)

The 922 MB county facts file is never committed; the repository holds
only its pin (digest, size, row count, scope), and an operator builds or
copies the file and passes it with --county-facts. A change that re-pins
the main US feed at a Chronicle commit also rebuilds and re-pins the
county facts at that commit, so a US build reads one Chronicle base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1603)

Main re-pinned the US spec-engine files this branch also pins. The six
conflicts were all generated counts and digests; each took main's side and
the five-step repin recomputed them on the merged tree
(repin_us_spec_engine_contracts.py --legacy-only, generate_us_bundle_from_constants.py,
repin_us_spec_engine_contracts.py twice per #1530, spec_engine_coverage.py).

- Configuration fields: 42,401 (main's 42,395 plus this branch's six
  pointers for the two declared legacy resources); 41/41 inventory checks.
- US bundle spec_sha256: d3a3c6c6fbcf4bd7cae760215b909214e5593d606da74aa8f78e6c347e3b41b5.
- The three --check modes and tools/ci_test_plan.py verify exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 3 commits November 15, 2026 09:42
- The loader's second pass now hashes the whole file before it refuses. A row
  it cannot accept stops parsing, not hashing, so a file swapped between the
  two passes is named as changed whatever its rows hold, and a malformed row
  in the pinned bytes is always a ValueError. Before, a swapped row with a text
  period or a list contract raised AttributeError or TypeError, which
  prepare_run did not record in county_surface.json.
- Tests: county ids with the wrong prefix, four or six digits, or letters;
  a text period, a list layout, a list contract and a text geography; blank
  lines are hashed but not counted; every swapped file is named as changed; a
  property test that any pinned row value loads or raises ValueError; prepare_run
  records a run with no county facts and a swapped file; and no Modal plan input
  or option reaches --county-facts.
- Docs: the pin holds the digest and row count, not the size; no county step
  reads --feed and nothing reopens the county facts file after prepare_run;
  refusals are recorded before the staging frame loads (the state admin targets
  come from --feed earlier); the Modal refusal comes from its check; the
  recipe's --feed-sha256 is a placeholder; 6.4x, not 6.5x; ten pairs from nine
  packages; "exceeded" the folder cap; stale feed wording; the changelog names
  --feed-sha256.

Not changed: the unreachable county facts fallback in attach_surface (the
production county rung rewrites that function) and dropping projected rows
after the SOI split (the production rung reads them for the ACS families).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1096's exact softmax cap projection in solve moved the shared spec
pins, so both conflicts were pin files. us-f0-coverage.json takes
main's version and is regenerated by spec_engine_coverage.py. In
test_us_multispine_pool_tool.py both pin comments are kept and the
tools recompute spec_sha256 (fa75bd68 on main, 1e727e1c here).

Repinned with the five-step sequence: repin --legacy-only, bundle
generation, repin twice, coverage. Then:
- repin, bundle and coverage --check: pass
- ci_test_plan.py verify: pass
- ruff check .: pass
- county focused suite (-n 4): 746 passed, 1 skipped
- spec-engine pin suites (-n 2): 116 passed
- national pin suites (-n 2): 481 passed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l's parser (#922)

Round-3 review P2: test_a_release_plan_cannot_pass_the_county_facts
checked only the planned argv, so it still passed with the release
tool's parse_function registration removed. The check stage would then
run the tool with --help and return 0, skipping the county argument
check.

The test now runs the check stage's own _parse_check on this checkout
with the planned state_county argv and asserts return code 2 and the
"--soi-mode state_county needs --county-facts and
--county-facts-sha256" refusal. With parse_function set to None the new
assertion fails (return code 0), so the registration is pinned.

- test_us_modal_stage_plan_tool.py (-n 4): 284 passed
- ruff check and format on the file, ci_test_plan.py verify,
  git diff --check: pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 16, 2026 12:15
Main gained #1619, #1621, #1609, #1540, #1629 and #1594 since 429d7a0.
Two files conflicted. docs/evidence/spec-engine/us-f0-coverage.json and
test_us_multispine_pool_tool.py took main's side, and the five-step US
spec-engine repin then regenerated them: spec_sha256 3fd1a384..., with
the field counts and pointer digests from tools/spec_engine_coverage.py.
The spec_sha256 comment keeps main's #1387 line and this PR's #922 line.

Checks: the three --check modes, ci_test_plan verify, ruff check and git
diff --check pass. Tests: county focused 751 (1 skip), pins 116 and
national 481, all passing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main Chronicle feed may stay at 41b2fb2, the commit it had when the
county facts were pinned on their own. A change that re-pins it anywhere
else must re-pin the county facts to the same commit, as
docs/us-chronicle-feed-repin.md says, so a US build reads one Chronicle
base. Asked for by the feed's owner as a condition of landing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator

Scoped re-review at c8403535: the main merge 07a39c95 and the new test

No blocking findings. All four landing conditions hold at c8403535: this review passes, and all twelve checks are green. Land it with a merge commit (note 1).

The four landing conditions at c8403535

  1. Re-review and required checks. This review covers 07a39c95..c8403535.

    • The merge's only manual edits are its two conflicts (us-f0-coverage.json and test_us_multispine_pool_tool.py). Both are the US bundle spec_sha256 pin, now 3fd1a384…, which the tools reproduce (item 4). Across the merge, this PR's own patch against main changes only in that digest and the comment above it.
    • c8403535 adds only the 22-line test.
    • Required checks: all seven pass (lint, wheels, select-countries, engine-free (3.13), engine-free (3.14), installed-emitter and ci). The engine-us, engine-uk and integration-uk jobs, which are not required, also pass.
  2. Feed and parity resources. us/chronicle_feed.json, us/chronicle_feed_scope.json, us/target_parity_manifest.json and us/target_parity_feed_families.json have the same git blobs as main at 5f18174f1 and at 99814fe43. git diff origin/main c8403535 on the four files is empty.

  3. County pin. us/chronicle_county_facts.json names Chronicle 3e9d3a0694e8…, facts 67b0998aa2ee… and 180,326 rows. Its scope_sha256 (87b12e4f…) is the SHA-256 of the packaged us/chronicle_county_facts_scope.json. Both files have the same blobs in all ten commits of this PR.

  4. Generated counts and digests. On a clean checkout of c8403535 (Python 3.13, uv sync --all-packages --locked --extra us):

    • tools/repin_us_spec_engine_contracts.py --check: exit 0.
    • tools/generate_us_bundle_from_constants.py --check: exit 0, spec_sha256=3fd1a384….
    • tools/spec_engine_coverage.py --check: exit 0, 42,401/42,401 configuration fields and 41/41 inventory checks.
    • tools/ci_test_plan.py verify: verification=ok.

    The working tree was unchanged after all four.

Why GitHub showed BLOCKED

The main ruleset requires seven checks and no approvals. Until 18:58 UTC, two required checks had not passed: engine-free (3.13), which ran for 1 h 39 min, and ci, which runs after every other job. Both then passed, and GitHub now shows the PR as CLEAN. There is no conflict: git merge-tree with main at 5f18174f1 is clean.

Findings

Blocking: none.

Non-blocking, for landing

  1. Land with a merge commit, not a squash. Make the county rung a production county calibration for the County Files #1630 is stacked on lwi-county-facts-pin. I simulated both landings on main at 5f18174f1. After a merge commit, the head of Make the county rung a production county calibration for the County Files #1630 merges into main with no conflict. After a squash, it conflicts in six files: tools/us_acs_local_county_surface.py, test_us_acs_local_county_surface.py, test_us_chronicle_county_facts.py, docs/us-acs-local-county-rung.md, us-f0-coverage.json and test_us_multispine_pool_tool.py. The repository does not delete head branches on merge, so the base of Make the county rung a production county calibration for the County Files #1630 must be changed to main by hand.
  2. Pin race with three open PRs. US gates in us/gates.json, run in shadow beside today's gates (#1585 item 11, PR 2 of 4) #1635, Name the US calibration arms by builder flag in pinned and generated text #1641 and Refuse invalid fit input that the microimpute bug-parity audit reproduced #1543 also regenerate the US spec-engine pins. Each conflicts with this head in us-f0-coverage.json and test_us_multispine_pool_tool.py; US gates in us/gates.json, run in shadow beside today's gates (#1585 item 11, PR 2 of 4) #1635 also in field_usage.py, tools/spec_engine_coverage.py, test_spec_engine_coverage_tool.py and test_spec_engine_field_usage.py, and Name the US calibration arms by builder flag in pinned and generated text #1641 also in tools/spec_engine_coverage.py. Whichever lands second needs a main merge, the repin sequence and a new CI run. This PR is on the county files' critical path, so landing it first keeps that extra cycle off the path.
  3. The new test binds the two open main-feed re-pins. test_the_county_pin_moves_with_the_next_main_pin (test_us_chronicle_county_facts.py:133) accepts the main pin only at 41b2fb2 or at the county pin's commit. Bind national ACS household types and SOI filing statuses (#880) #1610 pins Chronicle 60266a3 and Calibrate ACS local state slices to Census Vintage 2025 age groups #1531 pins e86a83b. GitHub's compare shows both as branch commits that lack 438 commits of 3e9d3a0. After this PR lands, each of them fails the test once it merges main, until it re-pins both files at one commit. Under docs/us-chronicle-feed-repin.md, that commit is at or after 3e9d3a0, so the main feed also takes Chronicle main's newer changes. This is the rule as agreed. I note it so that Bind national ACS household types and SOI filing statuses (#880) #1610 and Calibrate ACS local state slices to Census Vintage 2025 age groups #1531 plan for the county rebuild (about 8 minutes and 2.9 GB of disk, per the PR body).

Non-blocking, code and docs

  1. docs/us-acs-local-county-rung.md:229-231: "The Modal stage plan (tools/modal_us_stage_plan.py) … so its check (check_stage)". check_stage and _parse_check are in tools/modal_us_stage.py. The plan module only declares the tool and its parse function, so a reader who looks for check_stage there does not find it.
  2. docs/us-acs-local-county-rung.md:262-263: the example names the main feed COUNTY_FEED. The file is the main pin's feed, not county data. A name such as MAIN_FEED makes it less likely that someone passes the county file as --feed. This is for clarity only: the parser refuses one file for both flags, and the loader refuses any file that is not the pinned one.
  3. tools/us_acs_local_county_surface.py:875: the attach_surface fallback calls _load_county_facts outside the handler in prepare_run, so a refusal there is not recorded in county_surface.json. do_materialize cannot reach it, because prepare_run either sets county_surface or exits. The PR body defers this to Make the county rung a production county calibration for the County Files #1630, which rewrites the function. I agree.
  4. Optional, at the next re-pin: when both pins have moved together once, remove the 41b2fb2 allowance (_MAIN_PIN_WHEN_THE_COUNTY_PIN_SPLIT, test_us_chronicle_county_facts.py:130), so that the test is plain equality. Until then, the test also accepts a county-only re-pin, which the doc allows.
  5. The PR body still opens with "WIP (draft)". Plan item 2 still says "Mark ready once CI is green", and item 3 says that the scoped re-review "is queued". Update these before landing, so that the record of the four conditions is correct.

What I checked in the code (no change needed)

  • load_us_county_facts. Pass 1 hashes the file and counts non-blank lines. It refuses a digest or a row count that is not the pin's before it parses a row. Pass 2 hashes the same bytes that it parses, continues to hash after the first bad row, and decides only at the end. Thus a file swapped between the passes is always reported as "changed while it was being read". A malformed pinned row always ends as a ValueError, including JSON, Unicode and recursion errors. A missing or unreadable file raises OSError, which prepare_run also records. Puerto Rico rows are dropped only after the contract, scope and county checks, and they are counted as state_72.
  • _load_pin. The main pin keeps the same checks and messages as before. The only change is that the scope digest is computed first. The county pin must have the scope us_county_facts, its own scope digest, and a bare facts file.
  • --county-facts. The parser refuses it without state_county and county geography, without --county-facts-sha256, with a digest that is not the pin's, without --feed-sha256, or when it is the --feed file. It is required for any state_county run whose stages include materialize. The refresh recipe goes back through _parse_args, with placeholder paths and the pinned digest.
  • Modal. The release tool declares no county input or option. The new test runs the real _parse_check on this checkout and gets return code 2 with the county facts refusal.
  • Bundle. The two files are registered as legacy_json resources. The spec binding stays out of the frame metadata and the configured, base and checkpoint identities (test_constants_adapter_equals_live_constants_and_stays_out_of_identities). Thus the new spec_sha256 does not change those identities.

Tests run

At c8403535, on Python 3.13 with -n 4 --dist loadfile: test_us_chronicle_county_facts.py, test_us_acs_local_county_surface.py, test_us_acs_local_county_properties.py, test_us_chronicle_feed.py, test_us_modal_stage_plan_tool.py, and both test_us_spec_bundle.py files. Result: 459 passed, 0 failed, 0 skipped.

🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hand-off to the Microcosm hub (head c84035354).

Review. Finished. Round 3 approved the PR's own change (8f71d34a7). The scoped review of the main merge (07a39c95e) and the coupling test (c84035354) returned APPROVE with no findings at any level. It confirmed the PR's change is unchanged apart from generated digests and the new test. The four feed resources are byte-equal to main 99814fe43, and the county pin is unchanged: Chronicle 3e9d3a0694e8…, facts SHA-256 67b0998aa2ee…, 180,326 rows, scope 87b12e4f…. The national release tool still reads neither the county pin nor spec_sha256. The reviewer's sandbox could not run the --check tools. The author's run of the five-step repin and the --check modes at the merge passed, and the reviewer's source audit found a writer for every generated value.

The feed owner's four conditions. (1) The scoped re-review passed and all 12 checks pass. (2) The four resources are byte-equal to main. (3) The county pin is unchanged. (4) The tools recomputed the generated values. Main (5f18174f1) has touched none of this PR's 21 files since 99814fe43, and the PR is CLEAN and MERGEABLE. The agreement lapses, and the feed owner must be asked again, if tools/build_us_fiscal_refresh_release.py comes to read the county pin or spec_sha256.

Depends on it. microcosm#1630 (the county rung, stacked on this branch; its review is finished) is retargeted to main after this lands.

Follow-ups owed. None on this PR.

@MaxGhenis
MaxGhenis merged commit c7a5f96 into main Nov 16, 2026
12 checks passed
MaxGhenis added a commit that referenced this pull request Nov 16, 2026
Main brought five first-parent merges since 14797c6: #1640 (UK dense
bus floor for 21 families), #1642 (Belgium's ten gate entries in the gate
contract), #1643 (Belgium's Axiom engine pinned at 753ce7d9), #1608
(promotions notify collector; emitter-interface floor 0.16.0) and #1614
(LWI's US county facts pin). Main changed nothing under
packages/microcosm-fit or packages/microcosm-graph.

Two pin files conflicted, both through #1614; each block took main's text
plus this branch's own comment lines, placed after main's sentence:
- docs/evidence/spec-engine/us-f0-coverage.json: two spec_sha256 blocks,
  main's side (the branch has no comment there).
- packages/microcosm-build/tests/engine_workflow/us/test_us_multispine_pool_tool.py:
  main's #922 county-facts comment, then the branch's #1543 line, then
  spec_sha256.
No code or test file other than these two pins conflicted, and no other
file was changed by both sides.

The tools then wrote every value on the merged tree:
- tools/repin_us_spec_engine_contracts.py: the multispine pool
  spec_sha256 9ad067de (tools/generate_us_bundle_from_constants.py --check
  computes the same digest). It left the branch's EXPECTED_HASHES seed_map
  d5898b93 and seed_protocol 206be396 and the loader golden b7c843a1 as
  they were, and main's field-usage and coverage-tool pins as main has them.
- tools/spec_engine_coverage.py: us-f0-coverage.json's two spec_sha256
  lines (42401/42401 configuration fields; 41/41 inventory checks).
- fit.qrf/pins.json keeps the branch's values: the merged fit sources equal
  the branch's and the kernel's implementation hash reproduces, so
  tools/graph_parity_repin.py was not run.
The repinner's quote style was then reformatted with ruff format.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants