Repository navigation
Conversation
…release Release dashboard-data-20261014 rebuilt kept cases' text from the seed's verdicts and replayed only its own stage's amendments, so it published the old text of the 14 amendments release 20261013b made on five cases. Releases 20261027 and 20261111 carried that text forward. policybench/wording_amendments.py says which committed amendments are in force (no later release re-judged the case) and finds those the committed files or a payload do not publish. It pins the 14 by content and gives the next release's driver restore_frames and restore_payload for its carry and check (a). tests/test_wording_amendments.py holds the committed files and snapshot to every amendment in force, except the 14 while the pointer names release 20261111, so a release that drops a committed amendment on a kept case, or does not restore the 14, fails. No score moves; publishing is the maintainer's call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…e 20261111's files Review round 1 (Claude Opus 5.5, runs/review-r1) found that a re-judge does not replace a decision's reasoning, so retiring reasoning entries on a re-judge left 12 published amendments unguarded. Reasoning entries now stay in force while the adjudication record holds the decision, and are read in the record and in the case note's adjudication sentence; case note amendments are read in the note's body only, which also stops entry 2398's new text in its case's sentence from masking a drop. The restoration tests now read release 20261111's files from git at e61eea4, so they still hold once the next release commits the restored text; the closed list of releases published without the 14 is pinned; each judge provenance file is pinned by sha256; restore_csv_texts serves the driver's prepare step; check (a)'s test now flags a single missing row. The design note adds the prepare step, reads the record at BASE_COMMIT, and narrows the scope and invariant sentences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Docs and module docstring: the 9 reference-explanation rewrites stay published because every later change to that file (releases 20261010, 20261013 and 20261027) touched other rows, not because the drivers hold the file to the base's bytes. Nothing checks those 9. - The 17 reasoning amendments correct 10 decisions; the reasoning test now requires every amendment in each undone chain to be flagged. - New test: each in-force amendment that later entries rewrite, dropped on its own, is flagged (20 can be built, 12 cannot). - Docs name the rule's limit: at release 20261111, 11 row amendments a re-judge retired have their old wording published again. They also say that a replaced or removed decision fails closed. - Driver steps: step 2 reads the files from BASE_COMMIT; step 4 covers the forked docstrings; new step 6 gives the inherited MOCK boards one explicit seam instead of a try/except. - restore_frames requires unique indexes; dropped_by_20261014 refuses a short record with a clear message; restore_csv_texts' docstring says what it reads; the registry test's docstring says what it pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP (draft). Review round 1 asked for changes: one P1 and five P2s, fixed in
fd29edad. Review round 2 found one P1 (a false reason in the docs, on the reference explanations) and five P2s; they are fixed in this push, and a scoped round 3 reviews the fixes. Then the publish call goes to the maintainer. Nothing here changes published text: the committed annotation files and snapshot stay as release dashboard-data-20261111 committed them, and the next board release's driver applies the restoration.What happened
Release dashboard-data-20261013b (#218) re-judged five cases and amended 14 passages of their text: six on
scenario_015head_medicaid_eligible, three onscenario_015federal_refundable_credits, two each onscenario_014andscenario_086federal_refundable_credits, and one onscenario_045snap. These are record entries 856 to 869 inus_wording-amendments.json.Release dashboard-data-20261014 (#221) kept those cases, rebuilt their text from the seed's verdicts and replayed only its own stage's amendments, so it published the old text of all 14. No gate compared a kept case's text with the base's committed files. Releases 20261027 and 20261111 carried that text forward (the Fable 5.2 freeze's
committed_amendmentsdocstring notes it).No other amendment was dropped on a case a later release kept. A record-independent diff of every kept case's text between consecutive releases finds changes only through release 20261006's rewrites, the 14, and new decisions' sentences.
docs/wording_amendments.mdhas the trace. It also names one limit: at release 20261111, 11 row amendments that a re-judge retired have their old wording published again, and whether each is still wrong needs a case-by-case reading.What this adds
policybench/wording_amendments.py:dropped_amendments, over the committed files or a payload. A case note amendment is read in the note's body; a reasoning amendment in the record and in the adjudication sentence;restore_frames,restore_csv_textsandrestore_payload, for the next release's carry, prepare step and check (a).tests/test_wording_amendments.py(38 tests):judge_provenance.json, and each provenance file must match its pin;e61eea4c(so these tests still hold once the next release commits the restored text), reproduces release 20261013b's text for the ten targets byte for byte and changes nothing else: 23 fields in 18 payload cells, each equal to 20261013b's;kept_annotation_problemspasses;docs/wording_amendments.md: the trace, the invariant and its limits, and six steps for the next release's driver:base_annotation_frames;prepare_inputs, reading fromBASE_COMMIT;restore_payload(live);Simulated in a scratch worktree at
fd29edad:No score, statistic or rank moves. Only the
annotationandcaseAnnotationtext of kept cells changes, and nothing that computes a score reads that text. The reverse change, release 20261014's, moved no incumbent'smodelStatsor heatmap row.Proposed release note line
"Restores the corrected wording of 14 annotation passages on five cases, which release dashboard-data-20261014 had reverted to their earlier text. No score or rank changes."
Tests
CI runs the suite. At
fd29edad,pytest tests/test_wording_amendments.py -p no:xdistpassed 37 locally. This push's changes were checked in memory: every test that takes no pytest fixture passes, and so does the new test, with a negative control that fails it.🤖 Generated with Claude Code