Skip to content

Restore release 20261013b's 14 wording amendments for the next board release - #270

Draft
MaxGhenis wants to merge 3 commits into
mainfrom
restore-20261013b-amendments
Draft

MaxGhenis wants to merge 3 commits into
mainfrom
restore-20261013b-amendments

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

WIP (draft). Review round 1 asked for changes: one P1 and five P2s, fixed in fd29edad. Review round 2 found one P1 (a false reason in the docs, on the reference explanations) and five P2s; they are fixed in this push, and a scoped round 3 reviews the fixes. Then the publish call goes to the maintainer. Nothing here changes published text: the committed annotation files and snapshot stay as release dashboard-data-20261111 committed them, and the next board release's driver applies the restoration.

What happened

Release dashboard-data-20261013b (#218) re-judged five cases and amended 14 passages of their text: six on scenario_015 head_medicaid_eligible, three on scenario_015 federal_refundable_credits, two each on scenario_014 and scenario_086 federal_refundable_credits, and one on scenario_045 snap. These are record entries 856 to 869 in us_wording-amendments.json.

Release dashboard-data-20261014 (#221) kept those cases, rebuilt their text from the seed's verdicts and replayed only its own stage's amendments, so it published the old text of all 14. No gate compared a kept case's text with the base's committed files. Releases 20261027 and 20261111 carried that text forward (the Fable 5.2 freeze's committed_amendments docstring notes it).

No other amendment was dropped on a case a later release kept. A record-independent diff of every kept case's text between consecutive releases finds changes only through release 20261006's rewrites, the 14, and new decisions' sentences. docs/wording_amendments.md has the trace. It also names one limit: at release 20261111, 11 row amendments that a re-judge retired have their old wording published again, and whether each is still wrong needs a case-by-case reading.

What this adds

  • policybench/wording_amendments.py:
    • a registry of releases: block sizes, and judge provenance pinned by sha256;
    • which amendments are in force: a row or note amendment until a later release re-judges its case, a reasoning amendment while its decision is in the record;
    • dropped_amendments, over the committed files or a payload. A case note amendment is read in the note's body; a reasoning amendment in the record and in the adjudication sentence;
    • the 14, pinned by content;
    • restore_frames, restore_csv_texts and restore_payload, for the next release's carry, prepare step and check (a).
  • tests/test_wording_amendments.py (38 tests):
    • the working tree's committed files and snapshot publish every amendment in force, except the 14 while the pointer names release 20261111 (a closed list); any later release must publish all of them;
    • the registry must list every release that moves the pointer, appends amendments or commits a judge_provenance.json, and each provenance file must match its pin;
    • restoring the 14 on release 20261111's files, read from git at e61eea4c (so these tests still hold once the next release commits the restored text), reproduces release 20261013b's text for the ten targets byte for byte and changes nothing else: 23 fields in 18 payload cells, each equal to 20261013b's;
    • detection on those same files:
      • a Hypothesis property draws 60 of the 414 droppable row and note amendments (all 414 also pass when run outside the suite);
      • each in-force amendment that later entries rewrite, dropped on its own, is caught (20 can be built);
      • a rebuild of each of the 414 targets in force is caught;
      • reverting the reasoning of each of the 10 decisions the 17 reasoning amendments correct, in the record or only in the sentence, flags all 17;
    • through the Fable 5.2 driver's functions:
      • with the restored base, triage's frames are the restored files and kept_annotation_problems passes;
      • an unrestored carry fails that gate on each of the ten targets;
      • check (a) against the restored base flags the single cell of an export missing one restored row, and all 18 cells of an unrestored one.
  • docs/wording_amendments.md: the trace, the invariant and its limits, and six steps for the next release's driver:
    1. restore in base_annotation_frames;
    2. restore in prepare_inputs, reading from BASE_COMMIT;
    3. compare check (a) with restore_payload(live);
    4. leave the record;
    5. register the release;
    6. give the inherited MOCK boards one explicit seam.

Simulated in a scratch worktree at fd29edad:

  • a next release that commits the restored files and snapshot and registers itself passes;
  • the same release unrestored fails the two invariant tests;
  • a pointer move without registering fails the registry test.

No score, statistic or rank moves. Only the annotation and caseAnnotation text of kept cells changes, and nothing that computes a score reads that text. The reverse change, release 20261014's, moved no incumbent's modelStats or heatmap row.

Proposed release note line

"Restores the corrected wording of 14 annotation passages on five cases, which release dashboard-data-20261014 had reverted to their earlier text. No score or rank changes."

Tests

CI runs the suite. At fd29edad, pytest tests/test_wording_amendments.py -p no:xdist passed 37 locally. This push's changes were checked in memory: every test that takes no pytest fixture passes, and so does the new test, with a negative control that fails it.

🤖 Generated with Claude Code

…release

Release dashboard-data-20261014 rebuilt kept cases' text from the seed's
verdicts and replayed only its own stage's amendments, so it published the
old text of the 14 amendments release 20261013b made on five cases. Releases
20261027 and 20261111 carried that text forward.

policybench/wording_amendments.py says which committed amendments are in
force (no later release re-judged the case) and finds those the committed
files or a payload do not publish. It pins the 14 by content and gives the
next release's driver restore_frames and restore_payload for its carry and
check (a). tests/test_wording_amendments.py holds the committed files and
snapshot to every amendment in force, except the 14 while the pointer names
release 20261111, so a release that drops a committed amendment on a kept
case, or does not restore the 14, fails. No score moves; publishing is the
maintainer's call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policybench-site Ready Ready Preview Nov 18, 2026 4:05pm UTC

Request Review

…e 20261111's files

Review round 1 (Claude Opus 5.5, runs/review-r1) found that a re-judge does
not replace a decision's reasoning, so retiring reasoning entries on a
re-judge left 12 published amendments unguarded. Reasoning entries now stay
in force while the adjudication record holds the decision, and are read in
the record and in the case note's adjudication sentence; case note
amendments are read in the note's body only, which also stops entry 2398's
new text in its case's sentence from masking a drop.

The restoration tests now read release 20261111's files from git at
e61eea4, so they still hold once the next release commits the restored
text; the closed list of releases published without the 14 is pinned; each
judge provenance file is pinned by sha256; restore_csv_texts serves the
driver's prepare step; check (a)'s test now flags a single missing row. The
design note adds the prepare step, reads the record at BASE_COMMIT, and
narrows the scope and invariant sentences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Docs and module docstring: the 9 reference-explanation rewrites stay
  published because every later change to that file (releases 20261010,
  20261013 and 20261027) touched other rows, not because the drivers hold
  the file to the base's bytes. Nothing checks those 9.
- The 17 reasoning amendments correct 10 decisions; the reasoning test now
  requires every amendment in each undone chain to be flagged.
- New test: each in-force amendment that later entries rewrite, dropped on
  its own, is flagged (20 can be built, 12 cannot).
- Docs name the rule's limit: at release 20261111, 11 row amendments a
  re-judge retired have their old wording published again. They also say
  that a replaced or removed decision fails closed.
- Driver steps: step 2 reads the files from BASE_COMMIT; step 4 covers the
  forked docstrings; new step 6 gives the inherited MOCK boards one explicit
  seam instead of a try/except.
- restore_frames requires unique indexes; dropped_by_20261014 refuses a short
  record with a clear message; restore_csv_texts' docstring says what it
  reads; the registry test's docstring says what it pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — c5acac94 Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant