Skip to content

Capture sources for ten ruled source conflicts - #10841

Merged
MaxGhenis merged 3 commits into
mainfrom
register-captures-batch-c1
Nov 15, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
register-captures-batch-c1

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Add 33 official source captures for ten source-conflict candidates and the Maine EITC identification-number provision. Later register entries can pin these files after they reach main.

Each capture starts with the official URL and extraction method, then holds unedited text from the declared PDF pages. This PR also adds an extraction-header regression test and a changelog fragment.

Coverage

Capture paths below start at source_conflicts/sources/.

Candidate Captures
us-ar:gambling-loss-deduction-before-2015 Act 155 pages 1 and 9–10; HB1390 DFA impact statement.
us-hi:joint-return-claimable-spouse-exemption N-11 instructions, page 9, 2023–2025.
us-hi:joint-return-one-disabled-spouse-exemptions HAR 18-235-54 pages 100–101; revised TIR 2022-01 page 1; N-11 instructions, page 20, 2022–2025.
us-me:childless-eitc-age-cap Section 5219-S page 2; PL 2019 chapter 527 pages 1–5 and page 5.
us-me:sales-tax-fairness-credit-mixed-joint-return Section 5213-A pages 1–3 and pages 2–3; PL 2017 chapter 474 pages 7–8 and 13.
us-ny:college-tuition-credit-claimed-spouse IT-272 form and instructions, page 1, 2025.
us-ny:real-property-credit-claimable-spouse IT-214 instructions, page 1, 2025, in reading order and content stream order.
us:charitable-non-itemizer-2020-separate-cap CARES Act page 65; 2020 section 62 pages 5–6 and page 5; 2020 Form 1040 instructions, page 30.
us:education-benefits-joint-claimed-spouse 2025 Form 1040 instructions, page 115.
us:recovery-rebate-joint-claimable-spouse Sections 6428 and 6428A, 2020 edition; section 6428B, 2021 edition; Form 1040 recovery rebate worksheets, page 59, 2020–2021.

Section 5219-S page 2 and PL 2021 chapter 479 page 1 carry subsection 7, the Maine EITC identification-number provision. #10527 proposes a candidate for it, us-me:eitc-itin-qualifying-children, and source_conflicts/candidates.yaml doesn't hold that id yet.

Extraction and historical limits

  • Arkansas: pdftotext drops strike-through and underline marks. The Act 155 header identifies that loss and the affected page 9 text. Page 1 supplies the official formatting legend. The official PDF, page 9 strikes 2015, inserts 2017, and underlines the gambling-loss clause in section 21. On page 10, the act also strikes 1995 in section 24 and underlines section 25; the header's general warning covers those marks without naming them. Arkansas lawmakers applied sections 1–24 from tax year 2015 in section 25, PDF page 10. DFA describes HB1390 as a clarification in its impact statement.
  • Federal section 62: the 2020 edition, PDF page 6 supplies section 212(d)'s application date for the repeal of paragraph (a)(22) and subsection (f). Congress applied the repeal to tax years beginning after December 31, 2020; paragraph (a)(22) and subsection (f) therefore still apply to tax year 2020.
  • Maine: section 5213-A, page 1 supplies the credit's 2016 start. The legislature applied subsection 6(C)'s dependent exclusion from tax year 2018 through PL 2017 chapter 474, Part B section 26, page 13. The register entry for this conflict should start in 2018, where the candidate says 2015.
  • Maine: the PL 2019 chapter 527 pages 1–5 capture includes the chapter identifier on page 1, section B-2 on page 4, and section B-3's tax year 2020 application date on page 5.
  • Maine: pdftotext also drops the amendment marks in the three session laws. The header of the PL 2019 chapter 527 pages 1–5 capture warns of that loss and points to PDF page 4; page 5 also underlines subsections 5 and 6. The headers of the PL 2017 chapter 474, PL 2021 chapter 479 and PL 2019 chapter 527 page 5 captures carry no warning. For example, PL 2017 chapter 474, page 8 strikes "or" after paragraph A and underlines the new paragraph C. Consult the PDFs before quoting amended text from these captures.
  • New York: the reading-order capture splits the joint-claim sentence across columns, and pdftotext -layout splits it too. The -raw capture keeps that sentence on consecutive lines of the 2025 IT-214 instructions, page 1. Use the reading-order capture for other passages; -raw joins some neighboring words.
  • Hawaii: the HAR capture reproduces the agency's unofficial compilation as of December 31, 2025. The N-11 tables cover 2022–2025; the line 6a captures cover 2023–2025. The New York captures cover the 2025 editions.

Methodology choices

Retain the candidates' existing readings. These captures supply source text without changing policy, candidate years, register entries, or pins. New York's IT-214 instructions address the taxpayer in the singular and tell joint filers to submit one claim. When registering the conflict, distinguish that wording from an either-spouse inference.

The Arkansas header describes formatting that plain text cannot preserve. A verbatim excerpt check can still match a struck year; consult the official PDF before quoting amended language. The regression test requires the formatting warning, the struck and inserted years, and the section 21 underline range.

Keep the page 5 captures of section 62 and PL 2019 chapter 527 and the pages 2–3 capture of section 5213-A beside the wider captures of the same documents. A register entry can pin either file.

Sources still missing

Two candidates still need documents before registration:

  • Arkansas DFA Subject 208 (2009): the historical URL redirects to the DFA homepage. The indexed agency URL returns HTTP 404. We still lack the earlier guidance; choosing other evidence requires a separate ruling.
  • Maine Revenue Services' site returns HTTP 403 to scripted requests, including one for the 2025 Schedule PTFC/STFC. We still lack the schedule's note on dependents.

Capture hashes and official sources

Capture Official source SHA256
us-ar/act-155-2017-p1.txt PDF b6ad05cde2b560bead541576cbf1716f08828588162c4345db193a0ae4c447b8
us-ar/act-155-2017-p9-10.txt PDF 6c2946ccab817b88cf55fde3108a5e11be7d80309348b627f179665cceb39b66
us-ar/hb1390-dfa-impact-2017.txt PDF 4016f13228fabe5a8139bf21f880e3ac022e4063f0c85b76a3761a9108d035d6
us-hi/har18-235-54-2025-p100-101.txt PDF 38c8ec872582fd42df0b4496346427b635b984f48d6b79eebfbe33762cc47b33
us-hi/n11-instructions-2022-p20.txt PDF dfd6ccda01e4c85c33e238272ed93de4a2cd4d73678f6585b338c93da24b146f
us-hi/n11-instructions-2023-p20.txt PDF 1de090ed1b656d198a04750f0d74c1a79d6f517721e4d4d88b62eebe1f8f9537
us-hi/n11-instructions-2023-p9.txt PDF 625e383143231112be1ba8c523e3fe17daf6802712b0e1dff85633d44b714315
us-hi/n11-instructions-2024-p20.txt PDF 0758efd01332ceee5ee559d6a75cf4cb1a76f005e6b1dc34a00275a023f336ae
us-hi/n11-instructions-2024-p9.txt PDF cbc2a98a2a460f02ff8b2d1a1bdba693c712cdc5c94452603c58ef92d4e50b07
us-hi/n11-instructions-2025-p20.txt PDF 6849f11f73907e523109c7ee1e4da173604c636b502768115e995b8171d4e108
us-hi/n11-instructions-2025-p9.txt PDF b8206c708ced5a331cb33ace8d1890f66f4689984f51471020439b131034a7a3
us-hi/tir2022-01-revised-2022-p1.txt PDF aef9250a44a5fedd00425fad0caf28a8b6dbc0f7f129be0dfe2e45340b5c095a
us-me/5213-a-2026-p1-3.txt PDF 883dfe009c096a78372913583fee41ef828fc62f09a41caf62382240892a2078
us-me/5213-a-2026-p2-3.txt PDF cefe7173aabb21c24f4847105bd0bed6b726ce9efad94ff37463557cd1183bd8
us-me/5219-s-2026-p2.txt PDF c767b8382679c6c3de99fbdbb55ebd44d9fc30c7fbd2b41b7cd6639311e2de7e
us-me/pl-474-2017.txt PDF 30c2de382a4458ecd9643089e924732d8e0fbae2b5b6f8d0fe600717fb2ef7cb
us-me/pl-479-2021-p1.txt PDF 2c2e137905bf49f7af29d33ec7c3f82e2be6181f1f1bde92553c7428a1d880c5
us-me/pl-527-2019-p1-5.txt PDF 18e050eb3dcf35a9b8bd0bd55e392d99f1dd15fefcd8161e2b7ead2dafed509a
us-me/pl-527-2019-p5.txt PDF 96c7b92ad3af066100021db9379124c30e2ca9319c572495980732d3a1c52be5
us-ny/it214-instructions-2025-p1-raw.txt PDF 3a83ed66946afeb4eb65eb0bb016584d5b3ff6f89aa630ee62b6edf7ab161ca6
us-ny/it214-instructions-2025-p1.txt PDF 55d530e053913f5e19e7b2001929810092d2c6f29052feeae13c73bdabf6f39d
us-ny/it272-2025-p1.txt PDF 0675b090a3cb1ee98ee9d9477f0cb1c47ba5ec43e23b2c2f84c32fd1e9e75bff
us-ny/it272-instructions-2025-p1.txt PDF 2c2b73b41aea9cb64c811c42ddcf2b86217ee2358449260fcce8307492843ef2
us/i1040-instructions-2020-p30.txt PDF 7a61a696d444e90bd87edcdeecb1dd903ae9385dbd485f29a7964afafb290b03
us/i1040-instructions-2025-p115.txt PDF b5d56f6049adca83a5683c5903f9e406cc04bf536ef05975a346a1197fd92baa
us/i1040-recovery-rebate-worksheet-2020-p59.txt PDF 9c48310497dd3e4eaeed665593d709e35e2ec26c562e07b84677751f8a83f1b4
us/i1040-recovery-rebate-worksheet-2021-p59.txt PDF 09d90e23b115fffa49f4ac1443b94e7715c8cec0ce42157b49d65e19e925ad0a
us/pl-116-136-2020-p65.txt PDF c77bc95d64ba55bb1ec44bed9eebd521787c8e8dd49e92794c323a90a80b1c1d
us/usc-26-62-2020-p5-6.txt PDF d8b3a4da9e3a3e5cdacf50b294bb9bb7c28f4121d65d478b44f8a7bc0907b92a
us/usc-26-62-2020-p5.txt PDF 34f7ba1e1a5a05f4071418df265ffb69c379eb0064f32e813c5af3e0a48e316b
us/usc-26-6428-2020-p1.txt PDF 35fc7fa7040afca8b92310f6de74f1c9586193ce2125743296a378845882d098
us/usc-26-6428a-2020-p1.txt PDF 8efccc2d07a8e509878cd2f41a496f1255cba689b8b375071fb1938210104230
us/usc-26-6428b-2021-p1.txt PDF dc3c5ff8e30cc0c321f5e63e0f60880043545bfbf5e6f551a31cc496c971dd88

Validation

Pull request run 43752366166 at a35bdaf3b0 passed all 38 jobs on GitHub's merge with main bdaf14f08d.

policyengine_us/tests/code_health/test_source_capture_methods.py holds one test of the Act 155 pages 9–10 capture. It confirms that the capture body still holds "as in effect on January 1, 2015", although the act strikes that year, and that the header holds each of its three notes on the lost marks.

  • Each of the 33 capture bodies matches an extraction of its declared pages from the official PDF, byte for byte.
  • A scan of the 33 captures finds no IP address, bot-challenge marker or local path.
  • git diff --check passes on the test file and the changelog fragment. Among the captures, it flags two lines of us-ny/it272-2025-p1.txt, where the extraction keeps the form's tab characters.

Each header names its source and extraction method, and our notes sit only in those two header lines.

Microsimulation impact

This PR adds source captures, a test of one capture's header and a changelog fragment, and it changes no formula, parameter or numerical expectation.

impact: none: this PR changes no formula, parameter or numerical expectation

axiom: n/a: source captures for the source-conflict register and a capture-header test; no model change

🤖 Generated with Claude Code

Add period-specific statutes, enrolled acts, regulations, guidance, and tax instructions for the shared source-conflict register. No model changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 14, 2026 22:44
Add the official formatting legend and check the extraction header before readers quote amended text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add federal repeal timing, Maine credit and chapter text, and the New York joint-claim sentence in content stream order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 15, 2026 15:34
@MaxGhenis MaxGhenis closed this Nov 15, 2026
@MaxGhenis MaxGhenis reopened this Nov 15, 2026
@MaxGhenis MaxGhenis closed this Nov 15, 2026
@MaxGhenis MaxGhenis reopened this Nov 15, 2026
@MaxGhenis
MaxGhenis merged commit a929f8a into main Nov 15, 2026
85 of 89 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, us#10841 at a35bdaf3b060414e53231eee0cc70547e779007e (squash): Capture sources for ten ruled source conflicts.

  • Reviews:
    • The PR has no GitHub reviews. Two independent review rounds ran outside GitHub, with a fix round between them.
    • The first round, at 857f6d3587, downloaded all 24 cited PDFs and re-extracted each capture's declared pages with its declared options. All 28 capture bodies then in the PR matched byte for byte, and all 21 excerpts the description quoted appeared verbatim in the captures it named. It read every capture against the description's claims about the law, and each claim held.
    • It found that pdftotext drops Act 155's strike-through and underline marks. The pages 9–10 capture printed "January 1, 2015 2017" and showed section 21's new gambling-loss clause as plain text, so a verbatim excerpt check would accept the struck year. It listed four smaller gaps. The section 62 capture stopped before the repeal's application date, and the section 5213-A capture omitted page 1, which holds the credit's 2016 start. The IT-214 capture split the joint-claim sentence across columns, and the PL 2019 chapter 527 capture showed neither its chapter number nor the section B-2 heading.
    • The fix round, which ended at a35bdaf3b0, changed no capture body. It added the lost-marks warning, the struck and inserted years and the section 21 underline range to the Act 155 header, with a test that requires each. It added five captures: the act's page 1 legend, section 62 pages 5–6, section 5213-A pages 1–3, PL 2019 chapter 527 pages 1–5 and a -raw extraction of the IT-214 page.
    • The second round, at a35bdaf3b0, found no source defect. It read both fix commits, reproduced 32 capture bodies byte for byte from the government PDFs and rebuilt the Hawaii rule capture's hash from its header and declared extraction. All 33 listed hashes held, and the dates and extraction limits matched the official sources. It asked for CI at the head merged into current main, since every job had skipped while the PR stayed a draft.
    • It also found no log behind the description's local test counts and mutation results. I removed those claims, and the description now states what the test asserts and leaves its result to CI.
    • I then rendered Act 155 pages 9 and 10 and the amended pages of the three Maine session laws. The Act 155 header names the 2015 and 2017 pair and the section 21 clause; on page 10 the act also strikes 1995 in section 24 and underlines section 25. Three Maine captures lose amendment marks with no header warning, and a fourth warns but points to page 4 alone. The description now lists these limits.
    • I made three more corrections to the description. PL 2021 chapter 479 enacts only subsection 7 of section 5219-S, so I moved it out of the childless EITC row of the coverage table. Apply the Maryland and Maine EITC waivers of the 32(m) SSN requirement #10527, an open draft, proposes the candidate for that subsection, and candidates.yaml doesn't hold it yet. git diff --check flags two lines of the IT-272 form capture, which keeps the form's tab characters.
  • Methodology choices for maintainers:
    • The PR changes no candidate, register entry or pin, and keeps each candidate's reading. All ten candidates stay ruled in candidates.yaml, as on main.
    • Each capture holds plain text, so amendment marks survive only where a header note describes them. The Act 155 body still prints "January 1, 2015 2017", and a verbatim excerpt check accepts the struck year. The test pins the header's three notes; it can't tell whether a register entry quotes struck text.
    • The PR keeps two captures of the IT-214 page. Reading order and -layout both split the joint-claim sentence across columns, and -raw joins some neighboring words, so each file suits different passages.
    • The PR keeps the narrower captures of section 62 (page 5), section 5213-A (pages 2–3) and PL 2019 chapter 527 (page 5) beside the wider ones. No entry pins any of them yet, so a maintainer could still delete the narrower files.
    • The Maine sales tax fairness candidate keeps its 2015 start. The captures show the credit starting in tax year 2016 and the dependent exclusion in 2018.
    • The Hawaii rule text comes from the Department of Taxation's unofficial compilation as of December 31, 2025.
  • Gates checked live at merge:
    • gh pr checks exits 0;
    • the latest Pull request run at the head, 43752366166, passed and tested this head merged into main at bdaf14f08d;
    • MERGEABLE;
    • not a draft;
    • no reviewer's latest review requests changes;
    • the semantic-overlap guard: since bdaf14f08d, main has changed no area this PR touches. Between the PR's merge base 935b24ff4f and main at fac4593393, main changed 1,304 files and none of this PR's 35. Under source_conflicts/, main edited candidates.yaml and added 63 files under sources/, none with a name this PR uses, and it left the ten candidates this PR serves unchanged. Main also added eight allowance lines to test_source_conflicts.py, which reads no capture, and the new test reads the Act 155 capture alone. The branch merges into main at bdaf14f08d without conflict;
    • the merge pins the head.
  • Impact: none: the PR changes no formula, parameter or numerical expectation.
  • Next:
    • Two documents stay uncaptured, so two candidates can't register yet. One URL for Arkansas DFA's 2009 Subject 208 redirects to the DFA homepage, and the other returns 404. Maine Revenue Services' site returns 403 to scripted requests for the 2025 Schedule PTFC/STFC.
    • The register entry for Maine's dependent exclusion should start in 2018, where the candidate says 2015.
    • A follow-up should add the lost-marks warning to pl-474-2017.txt, pl-479-2021-p1.txt and pl-527-2019-p5.txt. It should also name the page 5 underlines in the pl-527-2019-p1-5.txt header and the section 24 and 25 marks in the act-155-2017-p9-10.txt header. It must land before a register entry pins those files, since a header edit changes the sha256.
    • The education credits candidate's instructions URL points to page 98, and its own note says that page covers student loan interest. The capture holds page 115, and the register entry should cite that page.
    • Guard source captures against visitor bot-check tokens #10830, still open, adds a guard against bot-check tokens in captures. The first review ran it over the 28 captures at this PR's first head and found none; the five files added since haven't run through it.
  • Axiom: n/a, since the PR adds source captures and a test and changes no provision.

🤖 Generated with Claude Code

@MaxGhenis
MaxGhenis deleted the register-captures-batch-c1 branch November 15, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant