diff --git a/AGENTS.md b/AGENTS.md index f4102a66..7db04875 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -186,7 +186,7 @@ The library has two layers: `nameparser/config/` (data) and `nameparser/parser.p **Sweep the forms a change can reach: comma shapes, then `name_order`.** No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOTHING before the comma are four paths, and the last two are the misses — #429, #430 and #432 are all the one-word path disagreeing with the full-name path on inputs the full-name path reads correctly, and #613's PR review found `, PhD` and `, Jr.` (an empty surname field, as CSV data writes it) read as given and title by a decision that returned early on an empty head, after two review rounds whose every grid had a name before the comma. Put an empty head and a nickname-only head (`(Bob), PhD`) in every comma grid. Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all. #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. For orders the sweep already exists (`tests/v2/test_cases.py` runs every row under all three) but carries one assertion, R2's family partition, so it checks nothing a new change moves; coverage there has been vacuous before (PR #394's review found the suite passed with `name_order` discarded from grouping). Parse your change's names in each comma shape and each order, and read the ones you did not predict. And read a generated grid's moves by SHAPE, not by corpus membership: #614's first design passed the suite, the five gates and a fingerprint of every corpus name, because its grid moves were sorted into 'corpus name moved' and 'garbage', and the review found the moves that mattered among the 'garbage' -- a bare salutation (`Mr. and Mrs.` lost its connective to the family name) and a name with one trailing word (`Freiherr von Berg X.Y.Z.`). Pull out the head-plus-one-word slice of the grid and a salutation with nothing after it, and read every move there. Two more were the second review's (#614): no grid put TWO words that are both a title and a particle at the head (`Freiherr St van Berg MA`, where an index into the pieces was applied to a merged view of them -- `TITLES ∩ particles_ambiguous` is `{freiherr, st}`, so put both in every head list, each in its bare and its period-marked spelling -- #637's grid had `St` and `Freiherr` alone and missed that `St.` and `Freiherr.` are declined by the bound join and then attached by P6), and the grids compared ambiguity KINDS, so a report telling the caller the wrong field (`Freiherr von Berg Ma`, 'read as a given name' of a word in the family) passed every pin. #701's review added one: put a title/suffix dual (`Ms`, `Sr.`, `Cpl`) SECOND in a title run, behind an ordinary title. A stop that asks only "is this word suffix vocabulary" passes every row that opens with the dual and fails `Rev. Sr. Smith, Mary`; it was found by comparing `T Smith, John` against `T John Smith` over every title word, and a comma-versus-comma-less comparison is the cheap check for any rule that moves a word between the two slots. Compare each report's `detail` against the field its tokens land in -- since #626 `test_a_report_names_the_field_its_word_lands_in` (tests/v2/test_cases.py) does it over every non-locale case row under all three orders (a row carrying its own policy only as declared), for the phrasings its `_FIELD_CLAIMS` lists -- so a row with the shape is the test only where the report's wording is listed: a new emitter's wording goes in that list, and segment's report on a part past the second comma names no field (#629). #626's own first grid then missed an emitter for want of one comma shape (`, Ma Dr.`: an empty head, an ambiguous word, a trailing title, which H1 moves to the family): a trailing title belongs in every empty-head comma grid too. And #620's: an exclusion list in code being replaced is a set of rules, not noise -- #614's merged copy excluded every word the trailing read weighs from its units, the first #620 prototype dropped the list as a symptom of the copy, and its review found the list had been keeping positional counting's rule that a member in front counts as a word (`Freiherr von Berg MA X.Y.Z.` lost its suffix). Before deleting one, mutate each entry out and read what moves. The converse holds for replacing a hand copy WITH the shared predicate: it is a behavior change wherever the two disagreed, and #735's /simplify shipped one as "no reading moves" because the suite and the snapshot were its only check (`St Sir de Mesnil` family-first, a token test replaced by P1's piece test); diff the two predicates' answers over a grid before calling the swap a refactor. -**Then sweep the spellings: a class reached by vocabulary is also reached by SHAPE.** A test that uses only the listed spelling of a word class walks only the vocabulary path, and the shape path is a separate branch that can be missing while every test passes. Roman numerals are the sharpest case: `ii`, `iii` and `iv` are suffix vocabulary; `i` and `v` are vocabulary too but initial-shaped, so `is_suffix_piece` vetoes them and only the numeral fork reads them; and `vi` through `x` are in no list at all, read by the fork's shape test alone (checked 2026-10-04 against `Lexicon.default()`) -- which stops at `x`: `_vocab._ROMAN` matches I to X, so `John Smith XI` reads `XI` as a name word (re-checked 2026-10-07, #614's /simplify, which found a docstring claiming "any length"). And the fork counts a ONE-letter numeral besides, since #718 (rules/S10): it is the suffix only where the name has a word to spare, so `John V` and `John Smith V` take different branches of one fork, a one-case name being counted like capitals whatever the letter -- since #721 a one-case one-letter numeral carries the `initial` tag its capital does, and since #723 every one-case bare letter does (rules/T4), and a bare letter in front of the numeral is initial-shaped as its capital is, so `john v` reads as `JOHN V` does over `test_a_one_case_name_reads_as_its_capitals`'s grid (comma shapes, orders, a maiden clause, a particle chain, a bare letter in front, a trailing title, a non-ASCII letter in front: 315 of its then 1,872 pairs disagreed at 2bfa9533, before #721, and none after). Its first version claimed "every shape" with no bare letter in front and no trailing title in the grid, and #721's reviews found both broken; a grid's claim is the grid -- put a head-plus-numeral row beside every words-to-spare row. So a change that handles `V` and `III` can still miss `VI` — #610 found exactly that, the maiden take's candidate loop admitting a shape-only numeral in one position where the peel reads it in two (`Jane Doe nee Smith VI Prof.` kept maiden `Smith VI` while `V` and `III` worked). #656's sweep found the same class from the other side: C12 takes the word ending the name before a comma as EVIDENCE, and its first draft asked the suffix-word tag, so `John Smith VI, AD` missed the rule `III` reached, and then counted the whole part, so `Smith VI, AD` fired with `VI` as its own name word and `Smith V, AD` with a `V` that S10 reads as the family. A rule that weighs a numeral as a generation must ask the predicate that READS it as one (the trailing fork, `_vocab.reads_as_generation`, and its words-to-spare count), not the vocabulary that lists some of its spellings. The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, `MD/PhD` by its listed chunks and `PU-PH` by the caps shape's joined spelling, #752, each with its joiner glued, spaced and glued to one side, which are three token streams -- and a new chunk reading inherits the old one's exceptions, which #752's first draft did not (`Jack I-V` lost its family to the listed numerals, and the fix's length test counted a chunk's period, so `Jack I.-V.` did too, found by the second review), and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there, and `Mr. and Mrs.` a run P3 joins, carrying a piece tag a one-word vocabulary test never sees -- PR #735's review found C12's hand copy of C13's title test (C8's until its 2026-10-19 split) reading `John Smith Jr., Mr. and Mrs. AD` as all suffix; ask C13's predicates in C13's order, `not is_suffix_piece` then `is_title_piece`, not the token tags, since `is_title_piece` alone reads the duals `Ms` and `Sr.` as titles). When a change touches a word class, test one spelling from each path. The same holds for a separator's SPACING: a comma (or any punctuation) can stand alone, glue to the word before it or glue to the word after, and each is a different token stream. #653's first fix swept `(née, Jones)` and missed `(née , Jones)`, where the kept comma became a word after the marker and M3 read the clause as maiden ', Jones'; its review found it. Put all three spacings in any grid around a separator. And a separator glued to a word hides the word from every vocabulary test, not only the one being changed: the semicolon delimiter's first draft read a glued word's stem in C1's run test alone and put 365 of ClinicalTrials.gov's reversed names right, missing the ones C13, C4 and C6 read off a word's TAGS (`MBBS;`, `MA;`); tagging the stem put 412 right, and reading it again in the comma decision's shape tests, which read a word's text rather than its tags (`MBBS;`), 431 (decisions/C10, 2026-10-24). A rule over a glued spelling reads the stem everywhere the word is read, its tags and every test of its text, and is swept against the space twin -- with a declared word delimiter too, which a substring test finds inside other words ('Stand' under ' and ', the review of the same change). A full stop INSIDE a word has four spellings of its own: initials run together (`J.R.R.`), joined words each closed by a stop (`Lt.Gov.`, `Msc.Ed.`), a joined word or name closed by none (`St.John`, `J.Smith`), and an abbreviation's own ending, closed or not (Italian `Sig.ra`, Latvian `Dr.philol.`). Shape tells the last from none of the others, so a rule over the shape decides which it costs; #838 found the ending only through the snapshot's CLDR row `Dut.ra Prof.ra`, and its first narrowing traded `St.John` for `Sig.ra` while the snapshot listed the `Dr.philol.` move among moves read as intended, until the design-docs review named it. Initials run together are also how some SURNAMES are written (Nepali `B.K.`, `K.C.`), and a caseless script writes its abbreviations, degrees included, as dotted letters (Thai `พ.บ.`): #864 found S3's position reading taking the first from 2.4.0 on, visible only once #866's Nepal candidates gave the corpora a population that writes them, and its first fix moved CLDR's Thai degrees into the name. A rule over dotted letters reads both before it ships. Capitals between two name words have the same second reading: some writers put their own initials there (`Maria Ana MA Silva`), so a rule that reads a word's capitals as a credential reads the initials too; #785 met it only in ctgov's field moves, no grid having held the shape, and Derek chose the initials (rules/S8: a name word behind capitals keeps them a name word where no family comma stands). A predicate over ADJACENT PAIRS of words meets two shapes no grid of one word holds: an overlapping pair and a joining word beside an initial. #752's split-credential test (phase 2e) read `Dr. Ph.` in `Jane Doe Dr. Ph. D.` as the listed `drph`, leaving `D.` a word of its own (found by the maiden take's candidate grid), and a particle beside a surname written short as the listed `ded` (`DE D JUNIOR`, found only in tse-donors' readings), so put a title, a particle and a connective beside each pair spelling, and a pair overlapping another, in the grid. +**Then sweep the spellings: a class reached by vocabulary is also reached by SHAPE.** A test that uses only the listed spelling of a word class walks only the vocabulary path, and the shape path is a separate branch that can be missing while every test passes. Roman numerals are the sharpest case: `ii`, `iii` and `iv` are suffix vocabulary; `i` and `v` are vocabulary too but initial-shaped, so `is_suffix_piece` vetoes them and only the numeral fork reads them; and `vi` through `x` are in no list at all, read by the fork's shape test alone (checked 2026-10-04 against `Lexicon.default()`) -- which stops at `x`: `_vocab._ROMAN` matches I to X, so `John Smith XI` reads `XI` as a name word (re-checked 2026-10-07, #614's /simplify, which found a docstring claiming "any length"). And the fork counts a ONE-letter numeral besides, since #718 (rules/S10): it is the suffix only where the name has a word to spare, so `John V` and `John Smith V` take different branches of one fork, a one-case name being counted like capitals whatever the letter -- since #721 a one-case one-letter numeral carries the `initial` tag its capital does, and since #723 every one-case bare letter does (rules/T4), and a bare letter in front of the numeral is initial-shaped as its capital is, so `john v` reads as `JOHN V` does over `test_a_one_case_name_reads_as_its_capitals`'s grid (comma shapes, orders, a maiden clause, a particle chain, a bare letter in front, a trailing title, a non-ASCII letter in front: 315 of its then 1,872 pairs disagreed at 2bfa9533, before #721, and none after). Its first version claimed "every shape" with no bare letter in front and no trailing title in the grid, and #721's reviews found both broken; a grid's claim is the grid -- put a head-plus-numeral row beside every words-to-spare row. So a change that handles `V` and `III` can still miss `VI` — #610 found exactly that, the maiden take's candidate loop admitting a shape-only numeral in one position where the peel reads it in two (`Jane Doe nee Smith VI Prof.` kept maiden `Smith VI` while `V` and `III` worked). #656's sweep found the same class from the other side: C12 takes the word ending the name before a comma as EVIDENCE, and its first draft asked the suffix-word tag, so `John Smith VI, AD` missed the rule `III` reached, and then counted the whole part, so `Smith VI, AD` fired with `VI` as its own name word and `Smith V, AD` with a `V` that S10 reads as the family. A rule that weighs a numeral as a generation must ask the predicate that READS it as one (the trailing fork, `_vocab.reads_as_generation`, and its words-to-spare count), not the vocabulary that lists some of its spellings. The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, `MD/PhD` by its listed chunks and `PU-PH` by the caps shape's joined spelling, #752, each with its joiner glued, spaced and glued to one side, which are three token streams -- and a new chunk reading inherits the old one's exceptions, which #752's first draft did not (`Jack I-V` lost its family to the listed numerals, and the fix's length test counted a chunk's period, so `Jack I.-V.` did too, found by the second review), and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there, and `Mr. and Mrs.` a run P3 joins, carrying a piece tag a one-word vocabulary test never sees -- PR #735's review found C12's hand copy of C13's title test (C8's until its 2026-10-19 split) reading `John Smith Jr., Mr. and Mrs. AD` as all suffix; ask C13's predicates in C13's order, `not is_suffix_piece` then `is_title_piece`, not the token tags, since `is_title_piece` alone reads the duals `Ms` and `Sr.` as titles). When a change touches a word class, test one spelling from each path. That holds for a word a rule only asks ABOUT, too: #810's first draft asked whether any word after a bracketed credential lacked the suffix TAG, and kept `John Smith (MA) VI`, `(MA) Ph. D.` and `(MA) Dr.` nicknames while `(MA) III` and `(MA) PhD` escaped; asking the role the parse read for the following word answers for every spelling. The same holds for a separator's SPACING: a comma (or any punctuation) can stand alone, glue to the word before it or glue to the word after, and each is a different token stream. #653's first fix swept `(née, Jones)` and missed `(née , Jones)`, where the kept comma became a word after the marker and M3 read the clause as maiden ', Jones'; its review found it. Put all three spacings in any grid around a separator. And a separator glued to a word hides the word from every vocabulary test, not only the one being changed: the semicolon delimiter's first draft read a glued word's stem in C1's run test alone and put 365 of ClinicalTrials.gov's reversed names right, missing the ones C13, C4 and C6 read off a word's TAGS (`MBBS;`, `MA;`); tagging the stem put 412 right, and reading it again in the comma decision's shape tests, which read a word's text rather than its tags (`MBBS;`), 431 (decisions/C10, 2026-10-24). A rule over a glued spelling reads the stem everywhere the word is read, its tags and every test of its text, and is swept against the space twin -- with a declared word delimiter too, which a substring test finds inside other words ('Stand' under ' and ', the review of the same change). A full stop INSIDE a word has four spellings of its own: initials run together (`J.R.R.`), joined words each closed by a stop (`Lt.Gov.`, `Msc.Ed.`), a joined word or name closed by none (`St.John`, `J.Smith`), and an abbreviation's own ending, closed or not (Italian `Sig.ra`, Latvian `Dr.philol.`). Shape tells the last from none of the others, so a rule over the shape decides which it costs; #838 found the ending only through the snapshot's CLDR row `Dut.ra Prof.ra`, and its first narrowing traded `St.John` for `Sig.ra` while the snapshot listed the `Dr.philol.` move among moves read as intended, until the design-docs review named it. Initials run together are also how some SURNAMES are written (Nepali `B.K.`, `K.C.`), and a caseless script writes its abbreviations, degrees included, as dotted letters (Thai `พ.บ.`): #864 found S3's position reading taking the first from 2.4.0 on, visible only once #866's Nepal candidates gave the corpora a population that writes them, and its first fix moved CLDR's Thai degrees into the name. A rule over dotted letters reads both before it ships. Capitals between two name words have the same second reading: some writers put their own initials there (`Maria Ana MA Silva`), so a rule that reads a word's capitals as a credential reads the initials too; #785 met it only in ctgov's field moves, no grid having held the shape, and Derek chose the initials (rules/S8: a name word behind capitals keeps them a name word where no family comma stands). A predicate over ADJACENT PAIRS of words meets two shapes no grid of one word holds: an overlapping pair and a joining word beside an initial. #752's split-credential test (phase 2e) read `Dr. Ph.` in `Jane Doe Dr. Ph. D.` as the listed `drph`, leaving `D.` a word of its own (found by the maiden take's candidate grid), and a particle beside a surname written short as the listed `ded` (`DE D JUNIOR`, found only in tse-donors' readings), so put a title, a particle and a connective beside each pair spelling, and a pair overlapping another, in the grid. **And the encoding: a letter class written `[^\W\d_]` or tested with `isalpha()` is blind to a combining mark.** Python's `\w` and `isalpha()` do not match Mn, so a letter written as base plus accent (NFD) ends a word early; the vocabulary path is immune because `_normalize` NFC-folds, the shape path is not. #652: `Dr. inž. Janez Novak` read `inž.` as the given name when the ž was decomposed, and `Janez Novak, ŽŠČ` reversed, in two families that share the blind spot — the letter-shape regexes (`period_abbreviation`, `_DOTTED`, `_CHUNKED`, `_PAIRED_INITIALS`), now spelled with `_vocab._LETTER` (a letter plus the general combining blocks; `test_regex_sync` compares the two `_MARKS` strings and pins `period_abbreviation` against config's copy), and the caps predicates, which NFC-compose a non-ASCII word before `isalpha()` (`is_caps_word`, behind C-level `isalpha()`/`is_normalized` tests, so already-composed text pays no frame — measured 2026-10-12, `Smith, José` was 193 frames before and after; the first version called it for every non-ASCII word and cost one frame each, and no guard saw it because the reference name is ASCII). A width test must count letters, not code points (`composed_len`): admitting a decomposed word to the caps set put `John Smith, MÁ` on the pair exception's wrong side, and the review's grid found it where the rows (all three letters long) did not — put a two-letter word in every caps grid. The initial shape (`_INITIAL`, `\w\.`, v1-verbatim and public) is not widened but asked of the COMPOSED word, in `is_initial_shaped` and in `_render`'s two copies; and the single-letter connective tests count composed letters, since a decomposed `й` is two code points (#665, where `É. y É. Smith` and `John й Smith` read differently decomposed). Compose the WHOLE predicate, not one half: `is_initial` is a shape ANDed with the #320 script veto, and composing the shape alone made a decomposed `씨.` an initial, since decomposed hangul is conjoining jamo outside every script range (#665's first version; `in_initialless_script` composes too now). Not covered, deliberately: the marks only some scripts write (Indic vowel signs, Arabic and Hebrew points), declined in decisions/nfd-shapes because widening the class would move readings nobody had measured; rules/H2's Accepted clause records the one consequence it names, that a period-marked abbreviation in Bengali or Devanagari never reads as a title by shape. The guard is `test_a_decomposed_name_renders_as_its_composed_twin` over the CLDR corpus plus `test_a_decomposed_abbreviation_reads_as_its_composed_twin` (rows × three orders × EVERYWHERE); the corpus guard reaches a shape only if some corpus name does, which is how the caps shape was missed until review, so when adding a shape rule add a row there. Four of #652's rows and two of #665's pass with every fix reverted (measured) and are parity pins, and the test compares ambiguity kinds and `initials()` too, since #665's `John й Smith` differed in its report; `_DOTTED` has no observable input and is pinned by a direct call. @@ -240,7 +240,7 @@ The 2.0 rewrite lands as underscore-private modules alongside the v1 code. These - **Method organization**, fixed section order in every class: fields + `__post_init__` validation → alternative constructors → dunders (construction/equality → protocol → operators) → properties → public methods by concern (access → editing → comparison → rendering delegates) → private helpers last, except a helper serving exactly one section may sit at that section's head. Sanctioned deviation, facade layer only: `HumanName` and the shim `Constants` organize by v1 concern groups (`# -- render defaults --`, `# -- config / parsing --`, `# -- fields --`, ..., dunders and pickle last) — the classes mirror v1's own surface and die in 3.0; the canonical order still binds every core type. - **Validation is eager and fail-loud**: every `raise` states the offending value, the expected form, and the fix. Exception taxonomy: wrong type — including wrong element type inside a collection, bare `str` where an iterable of strings is expected, or a `Mapping` where a plain iterable is expected — raises `TypeError`; well-typed but unacceptable values raise `ValueError`; failed enum lookups stay `ValueError` for any input (stdlib `EnumType` precedent). **When the message hands the reader code to paste, that code has to survive a type checker, and then RUN** — nameparser ships `py.typed`, and a placeholder can type-check and still raise: the 2.4 release sweep's first fix to the `capitalization_exceptions` hint offered `('key', 'Value')`, which passes mypy and fails #459's mask check, so `test_add_capitalization_exceptions_raises_pointing_at_replace` now executes the offered code. #337's segmenterless warning offered `Policy(segment_scripts=())`, an `arg-type` error, because these fields are annotated with what they STORE rather than everything the constructor accepts. Prefer the `frozenset()` / `()` spellings in messages and docstrings, and pin the offered spelling in a test — the warning tests matched on `ja_segmenter` and never checked the actionable half of the message. **A warning emitted in `Parser.__post_init__` needs `parser_for` to re-emit it from its own frame** (the `catch_warnings(record=True)` block at its return): `__post_init__`'s `stacklevel` is sized for direct `Parser(...)` construction, and through `parser_for`'s extra frame the default one-line rendering attributes the warning to the library's own `return Parser(...)` — the exact call the message tells the user to change becomes invisible. No single stacklevel serves both entry points; a new construction warning gets the re-emission for free, but a new CONSTRUCTION SITE for `Parser` inside this package needs its own re-emission or its callers get library-attributed warnings (#337 review). - **Guard, hint, and emit for the WHOLE family, and parametrize the test over it**: a check added to one member of a set belongs on all of it, and the test must sweep the family, not one example. This session shipped `_reject_str_and_mapping` on `Policy` but not `PolicyPatch`, the bytes decode hint on three of five config entry points, and a regex-sync roster missing four of its copies — each a separate follow-up bug that a `{class} × {field} × {bad-value}` parametrization would have caught and a per-example test hid. When you find you're guarding member N, grep for the other members first. -- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel (read once in `_group` for a main segment since #614, its picks reported by `_assign` once the roles they took are known, except a pick the particle chain takes into the name, which the chain reports and the read then gives up), the delimiter escape's follow-up in `classify` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge_pieces(pieces, ptags, k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all ambiguous particles on "Do Van Jr." (`Dr.` when that was written, before #367 made a plain title transparent and put the shape out of the loop's reach entirely), where the particle stayed a lone leading name piece — the GIVEN name under the default order, the family name under `FAMILY_FIRST` — and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`, or a run of particles group merged into the piece ahead of S2's read, which the chain then claims nothing past, #625) before recording. Structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is not emitted on the `FAMILY_COMMA` path's WHOLLY-FAMILY read -- the comma fixed which piece is the family -- and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". A tail segment is the same case from the other side: assign reads it as suffixes, a maiden marker there included since #601, and since #603 a title word past the second comma as a title (rules/C2, so `Freiherr` below reads title) (`Jane Doe, PhD, Jr nee van Ma` reads suffix `PhD, Jr nee van Ma`, the marker an ordinary word, rules/M2), so group makes no particle chain there at all (#636; its two emitters had been handed no report list there), after either comma — `John Smith, Jr., Freiherr von Richthofen` leaves `von` unchained and reports only `comma-structure` (rules/C2, 2026-09-28). Read that scope narrowly: the comma settles nothing about a particle trailing the given name, so P6's attachment decides that fork on the same path and reports it (#405; in `post_rules` until #613, in `_assign` since, right after the walk that reads the given part), in the kind naming the reading it OVERRODE, which is the reading assign made and not the word's vocabulary: `SUFFIX_OR_NAME` where assign had read the run as a post-nominal (`vd`, `mc`), else `PARTICLE_OR_GIVEN` where the run holds an ambiguous particle (`van`, and `do`, which is in the suffix vocabulary too but in its AMBIGUOUS half, so no credential reading was overridden), else silence. The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct -- all but the FIELD a report names, which a rule in post_rules can still change (H1's move behind a title, P1's family-first fold, P6's attachment, and under opt-in policies the patronymic rotations, O6's two family names and `middle_as_family`'s fold): an emitter naming one passes the rest of its sentence as `field_tail` and assemble words the field from the final role (#626, where `Kim Min Do` under `FAMILY_FIRST` was told 'Do' was a middle name with 'Do' in the family). **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a comma that names a family (since #613), in `post_rules` at the end of a name read family-first (#467) and after a comma with nothing before it, where H1 and M4 must read the name first -- so every one of them reports; for two years only the first did. What can still do the shifting is narrow, and #367 is why: a plain title no longer can (`Dr. Van Johnson` reads as `Van Johnson` does and reports from `_assign`), so the `_group` emitter needs a word that is BOTH a title and a particle — measured, `TITLES ∩ particles_ambiguous` is `{freiherr, st}` in the default vocabulary (`do` left TITLES in #296's audit; the particle-line block in decisions/vocabulary-exclusions records the before and after), plus any overlap a caller's config creates — standing ahead of the chained particle as the LEADING NAME word. Titles may precede it, so `Dr. St van Johnson` reaches the emitter and `St van Johnson` does too; a given name may not, so `Jan Freiherr von Richthofen` does not reach it while `Freiherr von Richthofen` and `Dr. Freiherr von Richthofen` do. Two shapes that look like they should reach it and do NOT, both measured by stepping `STAGES` and watching where `ambiguities` grows: `Dr. Do van Johnson` and `Do St Johnson` report from `assign`, not `group`, because `do` is no longer a title and so stays the leading name piece assign reports on — a both-vocabulary word CHAINED (`Jan St Johnson`) reports nothing at all. When checking whether that emitter is dead, a both-vocabulary word in the leading name position is the thing to look for, and the answer is that it is not dead. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful. +- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel (read once in `_group` for a main segment since #614, its picks reported by `_assign` once the roles they took are known, except a pick the particle chain takes into the name, which the chain reports and the read then gives up), the delimiter escape's follow-up in `post_rules` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge_pieces(pieces, ptags, k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all ambiguous particles on "Do Van Jr." (`Dr.` when that was written, before #367 made a plain title transparent and put the shape out of the loop's reach entirely), where the particle stayed a lone leading name piece — the GIVEN name under the default order, the family name under `FAMILY_FIRST` — and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`, or a run of particles group merged into the piece ahead of S2's read, which the chain then claims nothing past, #625) before recording. Structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is not emitted on the `FAMILY_COMMA` path's WHOLLY-FAMILY read -- the comma fixed which piece is the family -- and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". A tail segment is the same case from the other side: assign reads it as suffixes, a maiden marker there included since #601, and since #603 a title word past the second comma as a title (rules/C2, so `Freiherr` below reads title) (`Jane Doe, PhD, Jr nee van Ma` reads suffix `PhD, Jr nee van Ma`, the marker an ordinary word, rules/M2), so group makes no particle chain there at all (#636; its two emitters had been handed no report list there), after either comma — `John Smith, Jr., Freiherr von Richthofen` leaves `von` unchained and reports only `comma-structure` (rules/C2, 2026-09-28). Read that scope narrowly: the comma settles nothing about a particle trailing the given name, so P6's attachment decides that fork on the same path and reports it (#405; in `post_rules` until #613, in `_assign` since, right after the walk that reads the given part), in the kind naming the reading it OVERRODE, which is the reading assign made and not the word's vocabulary: `SUFFIX_OR_NAME` where assign had read the run as a post-nominal (`vd`, `mc`), else `PARTICLE_OR_GIVEN` where the run holds an ambiguous particle (`van`, and `do`, which is in the suffix vocabulary too but in its AMBIGUOUS half, so no credential reading was overridden), else silence. The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct -- all but the FIELD a report names, which a rule in post_rules can still change (H1's move behind a title, P1's family-first fold, P6's attachment, and under opt-in policies the patronymic rotations, O6's two family names and `middle_as_family`'s fold): an emitter naming one passes the rest of its sentence as `field_tail` and assemble words the field from the final role (#626, where `Kim Min Do` under `FAMILY_FIRST` was told 'Do' was a middle name with 'Do' in the family). **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a comma that names a family (since #613), in `post_rules` at the end of a name read family-first (#467) and after a comma with nothing before it, where H1 and M4 must read the name first -- so every one of them reports; for two years only the first did. What can still do the shifting is narrow, and #367 is why: a plain title no longer can (`Dr. Van Johnson` reads as `Van Johnson` does and reports from `_assign`), so the `_group` emitter needs a word that is BOTH a title and a particle — measured, `TITLES ∩ particles_ambiguous` is `{freiherr, st}` in the default vocabulary (`do` left TITLES in #296's audit; the particle-line block in decisions/vocabulary-exclusions records the before and after), plus any overlap a caller's config creates — standing ahead of the chained particle as the LEADING NAME word. Titles may precede it, so `Dr. St van Johnson` reaches the emitter and `St van Johnson` does too; a given name may not, so `Jan Freiherr von Richthofen` does not reach it while `Freiherr von Richthofen` and `Dr. Freiherr von Richthofen` do. Two shapes that look like they should reach it and do NOT, both measured by stepping `STAGES` and watching where `ambiguities` grows: `Dr. Do van Johnson` and `Do St Johnson` report from `assign`, not `group`, because `do` is no longer a title and so stays the leading name piece assign reports on — a both-vocabulary word CHAINED (`Jan St Johnson`) reports nothing at all. When checking whether that emitter is dead, a both-vocabulary word in the leading name position is the thing to look for, and the answer is that it is not dead. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful. - **A kind is worth adding only if a reader would hesitate too**: the test is not "does the code take a branch" but whether a person reading that input would genuinely be unsure. "Smith, John V" reads as a middle initial to anyone -- the comma settles it -- so reporting it would be noise that teaches callers to ignore the field, which costs more than the missing report. Reachability of the second branch is necessary, not sufficient. Prefer leaving a fork silent and documenting the omission over emitting on input nobody finds ambiguous. - **Parser owns config-dependent conveniences**: `Parser.matches`/`Parser.capitalized`/`Parser.revise` exist because the `ParsedName` equivalents fall back to DEFAULT config for str/omitted arguments (documented loudly in both docstrings). `revise` harvests tokens from a full sub-parse of each replacement value (tags kept minus `FOLDED_TAG`, roles forced, the R1 entry pass `suffix_entries` re-run over the forced state so a suffix value's entries follow its own commas, ambiguities discarded); the merge tail is shared with `replace()` via `ParsedName._with_field_tokens`. `Parser.capitalized` delegates through `name.capitalized(self.lexicon)` specifically so `_parser` never imports `_render` — keep it that way. - **Per-word vocabulary fields warn on multi-word entries** (`_normset`/`_normpairs` via `_warn_dead_entry`, UserWarning, never a raise — see the given_name_titles Gotcha for why raising is wrong). `given_name_titles` is the one multi-word-matched field and is exempt; `_edit` passes `warn=False` (add() warns once via the new instance's `__post_init__`; remove() stores nothing). The default vocabulary and every locale pack must stay warning-free (`test_default_lexicon_builds_warning_free`, `test_pack_vocabulary_entries_are_single_words`). diff --git a/docs/customize.rst b/docs/customize.rst index fcdbe7a6..851f522e 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -372,8 +372,13 @@ The full reading of a bare marked acronym, in order: reads as the given name is ``"Smith, Ba"`` (one word, and Title-case has no credential lean) and ``"smith, ba"`` (one word, one case, no lean at all). -- **In brackets**, ``"John Smith (BA)"`` falls through to nickname - parsing. +- **In brackets after the name**, the capitals in a mixed-case name + settle it as the credential too, and so does a period inside the word: + ``"John Smith (BA)"`` and ``"John Smith (B.A)"`` give suffix ``BA`` and + ``B.A``. Any other spelling there, and any spelling inside the name, + falls through to nickname parsing (``"John Smith (Ba)"``, + ``"Jeffrey (JD) Bricken"``), reported as a ``suffix-or-nickname`` + ambiguity. Generation words that are also surnames ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md index 6bb2d4cf..04bb680b 100644 --- a/docs/design/decisions/README.md +++ b/docs/design/decisions/README.md @@ -58,6 +58,7 @@ Entry conventions: - [S10 — the trailing roman numeral](S10-trailing-roman-numeral.md) - [S12 — a credential split across a space](S12-credential-split-across-a-space.md) - [S11 — a suffix word that is also a name](S11-suffix-word-that-is-also-a-name.md) +- [S13 — a bracketed member settled by its writing](S13-bracketed-member-settled-by-its-writing.md) ## Nicknames & quoted names (N) diff --git a/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md b/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md index 0b33a3aa..f2f36fde 100644 --- a/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md +++ b/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md @@ -4,7 +4,7 @@ - The rule says what the escape does: quotes escape as brackets do, and a clause of two or more words is not suffix-shaped unless an ASCII period ends it, so `(PhD MBA)` keeps its nickname reading with nothing reported, as 1.4.0 read it — 2026-10-21 #807. -Elsewhere: an ambiguous member alone in the brackets staying a nickname, reported, is decided under decisions/suffix-acronym-collisions (its Standing line ending 2026-09-25 #540; 2026-10-21) and decisions/ma-do's 2026-07-17 entry. +Elsewhere: an ambiguous member alone in the brackets staying a nickname, reported, is decided under decisions/suffix-acronym-collisions (its Standing line ending 2026-09-25 #540; 2026-10-21) and decisions/ma-do's 2026-07-17 entry; a member after the name whose writing settles it reading as the suffix is decisions/S13 (2026-10-26 #810). ## Log diff --git a/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md b/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md new file mode 100644 index 00000000..70200e31 --- /dev/null +++ b/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md @@ -0,0 +1,24 @@ +# S13 — a bracketed member settled by its writing + +## Standing + +- A member of the ambiguous class alone in brackets or quotes after the name, its writing settling it as the credential, is the suffix, read in one pass, and reports nothing (`John Smith (MA)`, `John Smith (M.A)`, `Smith, John (MA)` → suffix); any other member stays the nickname S1 reads and reports `suffix-or-nickname`, a dotted member inside the name included (`John Smith (Ed)`, `John Smith (Ma)`, `JOHN SMITH (MA)`, `Jean (M.A) Smith`) — 2026-10-26 #810. +- The writing that settles it is read by the rules that read writing: a period inside the word, S3's evidence, whatever case the name is in, and capitals in a name of mixed case, S6's, the case being that of the name's own words and never the clause's, so `john smith (MA)` keeps the nickname where bare `john smith MA` reads suffix. S13's own signal is the position alone — 2026-10-26 #810. +- Only a clause after the name is settled, behind its last name word: one in a name with no word of its own keeps the nickname (`Dr. (MA)`, `(M.A)`, where the review of the one-pass commit found both settled), and one with a name word after it keeps the nickname however it is written (`Jeffrey (JD) Bricken`, `Jean 'JD' Smith`, `Smith (MA), John`), while a suffix, a title or a maiden clause after it does not count, however it is written (`John Smith (MA), PhD`, `John Smith (MA) Ph. D.`, `John Smith (MA) Dr.`), asked of the roles the parse read rather than of the following words' tags — 2026-10-26 #810. +- Where the rest of the name then reads otherwise than beside a bracketed unambiguous credential in the same place, the member's twin, the difference is an accepted cost, recorded in rules/S13's Accepted bullets and counted by tests/v2/test_parser.py's census (`Jack (MA)` → family 'Jack', where `Jack (PhD)` reads given 'Jack') — 2026-10-26 #810. + +## Log + +- **2026-10-26 #810 (Derek, 2026-10-23; the single signal and the one pass Derek, 2026-10-26) — A BRACKETED MEMBER OF THE AMBIGUOUS CLASS AFTER THE NAME, SETTLED BY ITS WRITING, IS THE SUFFIX, READ IN ONE PASS.** Carries out item 16 of decisions/vocabulary-model's 2026-10-23 #752 entry, as phase 2l of #752. SUPERSEDES: in decisions/ma-do, the 2026-07-17 entry's side effect, in part (a parenthesized `(MA)` or `(DO)` no longer escaping to suffix), for a member after the name whose writing settles it. #810 asked whether `John Smith (M.A)` should read as `John Smith M.A` does, and found `(M.A)`, `"M.A"` and `(MA)` all reading as a nickname where 1.4.0 read the bracketed ones as a suffix; 2.0.0 through 2.6.0 read each as a nickname (measured 2026-10-26 on the wheels, `John Smith (MA)`, `John Smith (M.A)`, `Smith, John (MA)`). The escape had refused every member since `ma` joined the class (decisions/ma-do, 2026-07-17), and #796's 33 new members widened its reach. + - **Why the writing, and why after the name.** After the name, brackets hold either a credential or the name the person goes by, and the count of words cannot tell the two apart there, the name being whole without the clause. The writing can: credentials are written in capitals and with their periods, names are not, which is S6's and S3's ground. Inside the name the same member let out of the brackets would read as a middle name, where the brackets more likely hold the name the person goes by: a spike in the design round (2026-10-23, against 638685b4; a dated snapshot, the spike's switch being in no tree) that escaped every member written in capitals or with an interior period moved `JD` from the nickname to the middle name in two committed rows, `Jean 'JD' Smith` and `JEFFREY (JD) BRICKEN`. So a clause with a name word after it is not settled. + - **Why the name's case, not the clause's.** The same spike asked the bracketed word's own case and moved 6 FEC names, every one wrong: the corpus is written wholly in capitals, so capitals there settle nothing, and `(ED)` in such a name is the nickname. Asking the case of the name's own words, the fact S6 records for every other site (`settled_by_case`), moves none of them. The clause's own words are no name of their own, so `john smith (MA)` asks the case of `john smith` and keeps the nickname, where bare `john smith MA` reads suffix; the two differ by construction, the bare word being one of the name's own words. + - **Why the suffix, not the bare reading.** S1's escape lets suffix-shaped content read as if written bare. A settled member cannot, because since #872 a word of two dotted single letters in a script with capitals reads bare as paired initials (decisions/S3), so `John Smith (M.A)` let out of its brackets would read family 'M.A'. The clause is read as the suffix directly, the brackets having already set it apart from the name's words. The decision is made first in post_rules, where every word's role is read, and the `suffix-or-nickname` report moved there with it, from classify, so that it is raised where the fork is decided. Frames counted with `sys.setprofile`, py3.11, 2026-10-26, against master: `Smith, John` 158 and `John Smith` 149 unchanged; `John Smith (MA)` 214 → 210, `Jack (MA)` 174 → 171; a name holding a clause it does not settle pays 2 to 5 (`John (Bob) Smith` 204 → 206, `John Smith (Ed)` 214 → 219). + - **Declined: the twin reading, folding the pipeline again (Derek, 2026-10-26, "the one pass version").** A draft of this PR read a settled member exactly as its twin, a bracketed unambiguous credential in the same place, reads, fields and reports alike: post_rules recorded the settled clauses and the stages folded again from the start, extract escaping each clause and classify giving its word the credential's tag, until no new clause settled (a settled word joining the name's own words can make a one-case name mixed and settle a second clause). Its review rounds found three more places to bring in line with the twin, the part past a second comma among them. It cost a whole parse per settled name (`John Smith (MA)` 214 → 416 frames) and machinery in four stages for a shape that moves 3 real names in the corpora measured here (ctgov-officials'; 35 names hold it, ctgov's 29 and FEC's 6). In one pass the rest of the name is read as beside a nickname, and where that differs from the twin the difference is an accepted cost, recorded in rules/S13's Accepted bullets: a lone name word stays N3's family name (`Jack (MA)`; `Jack (PhD)` reads given 'Jack', reported); a maiden clause after the member stays the maiden name, where the twin's credential run takes it (`John Smith (MA) née Jones`); the case is asked once, so a capitals member settled only once another settled word made the name mixed stays the nickname (`JOHN SMITH (m.a) (MA)`, where the twin `JOHN SMITH (mba) (MA)` reads suffix 'mba MA'); and a part past a second comma is weighed without the member (`Smith, John, X.Y.Z. (MA)` reports nothing where `(MBA)` draws `comma-structure`). tests/v2/test_parser.py's `test_where_a_bracketed_member_reads_otherwise_than_its_twin` records the census of such differences over a grid of comma shapes, heads and the three orders (396 of 1,080 readings on 2026-10-26, by head), so a change that moves the count shows up; the folded draft differed from the twin in none of the settled rows. + - **Why post_rules, found by the code review.** The first draft decided in group and called a word after the clause a name word unless it carried the suffix tag. A suffix the tag does not mark then kept the clause a nickname while its listed twin let it through: `John Smith (MA) VI` and `(MA) Ph. D.` against `(MA) III` and `(MA) PhD`, the same for a member read by its capitals (`(MA) DO`), an unlisted dotted credential (`X.Y.Z.`) and `fils`, and a title after the clause (`John Smith (MA) Dr.`, `Doe, Jane (MA) Mrs`). Asking the roles the parse has read answers for every spelling at once. The same draft scanned the whole name once per clause, quadratic in the number of bracketed members (`"(M.A) " * n` 13.7× for 4× the input, against 3.3× after the fix, py3.11, 2026-10-26); the pass now maps clauses to tokens once, and test_benchmark.py's `bracket_members` row guards it, failing on the first draft twice in two runs. + - **Recorded as a rule of its own, reading one signal (Derek, 2026-10-26).** The position it reads (after the name) is evidence S1's shape signal does not read (docs/design/AGENTS.md, axis 12), so it is not an S1 clause. What settles the member is the writing, and the writing is other rules' evidence, so S13 cites them rather than reading it itself: the period inside the word is S3's, capitals in a mixed-case name S6's, and the escape from the nickname S1's; its Signal line is the position, a hub of S1, S3 and S6. An earlier draft named the period as a second signal of S13's own, a split candidate at birth. S3 reads two dotted letters bare as initials, and S13's Precedence bullet states that the brackets after the name outrank that reading. The dotted arm has no real-name population in any corpus measured here (none of ctgov-officials' 29 bracketed members, nor FEC's 6, is dotted), so it rests on Derek's decision of 2026-10-23 and on 1.4.0's reading of `John Smith (M.A)` as suffix, not on a measurement. A dotted member left the nickname inside the name is reported, as a tagged one is: the report had keyed on the class tag, which a dotted word in a clause does not carry, while S1's bullet, widened to every member, said it was reported (`Jean (M.A) Smith` reported nothing on master and on the first draft). S1's member bullet keeps the nickname reading for the members this rule does not settle, and its Precedence names this rule. + - **Declined: escaping every member.** A spike escaping every bracketed member to its bare reading (2026-10-23, against 638685b4, a dated snapshot like the one above) moved 27 ctgov-officials names and took `Ed` out of the nickname 17 times, `Ed` being the nickname of Edward far more often than a credential written in Title case. + - **Left as found by the second review**: one post rule can still put a name word behind a settled clause, P6's attachment after a comma with nothing before it, which runs after the pass that settles (`, Jones (MA) vd`); the shape has no name before its comma and nothing in the corpora writes it. + - **Kept from the twin draft's reviews.** A clause is the member alone only where its whole text is the member, as extract's escape reads a clause, so a character tokenize drops beside it (an emoji, a zero-width space, a nakaguro) leaves it the nickname, as its twin `(PhD 😀)` is no escape (`John Smith (MA 😀)`); a first draft counted tokens and settled it. A word read as a name after the clause keeps the member a nickname, as it would sit inside the name (`John Smith (MA) Ma`, family 'Ma'), the position being read on the nickname reading. The draft's comma-structure fix is not needed in one pass: the part past a second comma is tested while the clause is still a nickname, so `Smith, John, Jr. (M.A)` reports nothing, as its twin does. + - **Accepted:** the twin differences the declined bullet above lists, in rules/S13's Accepted bullets. + - **Measured** (2026-10-26, against master 12384fae with `--base` on a clean checkout, and again on the one-pass tree with a name word required before the clause, 482691a6, against master 13539131, #897, with the same moves): the committed snapshot moves 3 names, each a member read as the suffix in place of the nickname, its `suffix-or-nickname` report gone (`John Smith (MA)`, `Andrew Perkins (JD)`, `John Smith PhD (MA)` → suffix 'PhD MA'); every CLDR, GND, Wikidata and conventions agreement record matches, the locale views included. Of the local corpora, ctgov-officials moves 3 of the 29 names holding one member alone in brackets or quotes (the other 26: 2 already the suffix by a final period, `Ed.`; 23 in Title case, every one `Ed`, 3 of them inside the name and 1 in a one-case name; and 1 in lower case), each a capitals member in a mixed-case name after a credential behind a comma (shapes `Aa A Aa, AA (AA)`, `Aa Aa, AaAaAa (AA)`, `Aa Aa, Aa.AA(AA)`), read suffix, the credential it is. fec-individuals holds 6 such names, all in one-case names, and none moves; tse-donors and ukec-individuals hold none and match; psc-individuals holds none, and its agreement record matches. + - **Recompute:** `uv run python tools/differential/snapshot.py` and `cldr_agreement.py` from this tree, `snapshot.py --local --base ` for ctgov-officials, fec-individuals, tse-donors and ukec-individuals, and `local_agreement.py psc-individuals --base `; the population is the names in each corpus holding a delimited clause of one word whose period-free fold is in `Lexicon.default().suffix_acronyms_ambiguous`, by delimiter pair, position, case and spelling. diff --git a/docs/design/decisions/ma-do.md b/docs/design/decisions/ma-do.md index 965e9a4e..464f6ec3 100644 --- a/docs/design/decisions/ma-do.md +++ b/docs/design/decisions/ma-do.md @@ -10,11 +10,13 @@ Elsewhere: the rest of the case reading — Title-case `Ma` and `Do` leaning surname, an all-lower member or a one-case name taking the positional count (`ANH DO` → family 'DO', `JOHN SMITH MA` → suffix) — is decisions/S6's 2026-09-14 #289 + #516 entry and decisions/S4's 2026-09-15 (spec review) entry, and two shapes stand outside the case reading altogether: a particle run in front takes `do` whatever its case (`Anh van DO` → family 'van DO', reporting nothing; rules/P2, decisions/S5's 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do` entry), and an unambiguous credential in front of a member speaks for it (`John Smith PhD Ma` → suffix 'PhD Ma'; decisions/S7's 2026-09-27 (Derek), #544 — A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS entry); the count that goes first at the first slot after a comma, a suffix comma included (`Smith, MA` → suffix, `Smith, Ma` → given, `John Smith, Ma` → suffix), is decisions/S6's 2026-09-14 (Derek) and 2026-10-20 (Derek), #761 entries; the pairing that gives `do` its capitals-only credential reading after a family comma, and its accepted cost, is decisions/S5's 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do` entry; the escape that reads a bracketed member as a nickname is decisions/suffix-acronym-collisions' S1 line; the periods gate is rules/S3's since 2026-10-19; whether a word earns the ambiguous marking is decisions/vocabulary-collisions' C-i and decisions/suffix-acronym-collisions', with `do`'s departure from TITLES recorded in the former and further credentials marked ambiguous in the latter (2026-10-21 #796). +Elsewhere: after the name, a member whose writing settles it is the suffix, not the nickname (`John Smith (MA)`, `John Smith (M.A)`; `JOHN SMITH (MA)` and `John Smith (Ma)` keep the nickname) — decisions/S13's 2026-10-26 #810 entry, which supersedes the line above in part. + Open: #792 (S1 and S8: whether a bracketed generation ends the credential run, `W. Russell (Jr.) Withers`), the neighbour of the parenthesized line above, which it does not move: `John Smith (MA) Withers` reads nickname 'MA', family 'Withers'. ## Log -- 2026-07-17 (M12, Derek-approved) (its `Jack Ma` sentence superseded in part by #289 below, for the all-caps spelling) — ma and do joined the ambiguous acronym set because both are common surnames; the two-word "Jack Ma" is kept intact by S2's words-to-spare guard, while the periods gate governs the dotted spellings ("M.A." counts unambiguously). Documented side effect: parenthesized bare +- 2026-07-17 (M12, Derek-approved) (its `Jack Ma` sentence superseded in part by #289 below, for the all-caps spelling; its parenthesized side effect superseded in part by decisions/S13's 2026-10-26 #810 entry, for a member after the name whose writing settles it) — ma and do joined the ambiguous acronym set because both are common surnames; the two-word "Jack Ma" is kept intact by S2's words-to-spare guard, while the periods gate governs the dotted spellings ("M.A." counts unambiguously). Documented side effect: parenthesized bare "(MA)"/"(DO)" no longer escape to suffix as in 1.x. - 2026-09-15 #289 — AMENDS the sentence above for one spelling and leaves it standing for the other. "The two-word 'Jack Ma' is kept intact by S2's words-to-spare guard" is still true of `Jack Ma`, and false of `Jack MA`: an all-caps member of the ambiguous set inside a mixed-case name is now read as a credential with nothing to spare, so `Jack MA` gives given 'Jack', suffix 'MA' — v1's reading was last 'MA', so this is a deliberate parity break and not a restoration. The periods gate is untouched and still governs the dotted spellings. The two ledger comments that quote this sentence carry the same amendment. See decisions/S6. - 2026-09-18 #531 — AND `do` IS NOW SPELLING-DEPENDENT AFTER A FAMILY COMMA, which neither sentence above anticipated: it is the one member of this set that is also particle vocabulary, so `Doe, John DO` reads suffix `DO` (capitals, mixed-case name) while every other spelling attaches to the family under rules/P6. The pairing and its accepted cost are in decisions/S5's 2026-09-18 `do` bullet. diff --git a/docs/design/decisions/suffix-acronym-collisions.md b/docs/design/decisions/suffix-acronym-collisions.md index 4bd32548..0e8ecf21 100644 --- a/docs/design/decisions/suffix-acronym-collisions.md +++ b/docs/design/decisions/suffix-acronym-collisions.md @@ -21,6 +21,8 @@ Closes #342 (a wordlist question) and #454 (a rules.md question) together, becau - `esq` is a suffix word and not an acronym, a contraction rather than an initialism, so no word is both a suffix acronym and a suffix word — 2026-09-08 #316. - The Wikipedia footnote marks `cva[22]`, `gaee[14]` and `np[18]` are out of the suffix acronyms, and a list imported from a wiki is checked for reference marks before it lands — 2026-10-18. +Elsewhere: after the name, a marked member written in capitals in a name of mixed case or with a period inside it is the suffix, not the nickname (`John Smith (MENG)`, `John Smith (M.Eng)`) — decisions/S13's 2026-10-26 #810 entry. + Elsewhere: the case lean that decides the capitals forms above is decisions/S6's 2026-09-14 #289 and 2026-09-14 (Derek) entries; the chunked dotted spelling passing the period gate (`Wang M.Eng.` → suffix) and the credential run that ends in a member keeping its suffix comma (`John Smith, PhD MEng` → suffix 'PhD MEng'), with the run a member leads keeping the cost (`John Smith MEng PhD` → family 'MEng'), are decisions/S7's 2026-09-27 (Derek), #544 entry headed A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS, which this file's 2026-09-27 (#544) entry indexes; the shape-plus-position heuristic this file parked shipped as the two switches of decisions/S9's 2026-09-14 (Derek) entries, the all-caps one reading the comma position by default since its 2026-10-01 (Derek), #564 entry; the roman-numeral accident's narrow retirement is decisions/S3's 2026-09-15 (Derek) entry, which this file's 2026-09-15 twin indexes; and `sa` was a title and suffix dual that position decided by #296's audit (decisions/comma-suffix-arc's audit table, whose `sa` row moving it into the ambiguous set decisions/C8's 2026-08-23 (review of PR #428, four reviewers) entry declined), held out of the marking again by the 2026-10-21 entry, and has been in neither set since decisions/C8's 2026-10-25 #857 entry. Not standing: the #454 bullet's note that the `fix(suffix-routing)` ledger rule stays, the ledgers having been retired for the snapshot (decisions/differential-ledger, #664), and the repair of rules/P6's Accepted clause recorded in the "What #454 got right" bullet, carried out in the same change — 2026-09-07 #342. diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index 0a40b3bc..1bb95b98 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -18,6 +18,7 @@ The vocabulary was a set of base sets plus marker subsets (`particles` and `part - Every numeral rule asks one predicate, `_vocab.reads_as_generation`, the vocabulary's own numerals read inside it; a numeral of two or more letters is the generation wherever the trailing run holds it, while a one-letter one before a credential stays the name word decisions/S10's #721 entry made it — 2026-10-25 #752, phase 2f. - A credential written split across a space is one word, a member of the ambiguous class that the count decides, and the Ph./D. pair starts S8's run from where it stands (decisions/S12) — 2026-10-25 #752, phase 2e. - A listed credential that is also a name, written in capitals in a name of mixed case, is settled as the credential by its writing: it reports nothing and starts the credential's run where no name word stands behind it with no family comma (2026-10-26, Derek), and keeps its other readings, the clash settled at the sites that choose a reading rather than by retagging the word (decisions/S6) — 2026-10-25 #752, phase 2g, #785. +- A member of the ambiguous class alone in brackets or quotes after the name is the suffix where its writing settles it, with a period inside it or in capitals in a name of mixed case, the case being the name's own words' (rules/S13, decisions/S13) — 2026-10-26 #752, phase 2l. - Phase 2's title batches go in order of lone-head breakage in PSC, `buddha`, `siddha` and `gyani` in the default gate and `guru` left to a `ne` pack — 2026-10-24 #752. - `Dean Smith` reads given `Dean`. The contested-head gate covers `dean`, `justice` and `major`, and a title that is also a given name gets a declared given reading one word at a time, in measured batches — 2026-10-15/16; 2026-10-23 #752; 2026-10-25 #752, phase 2i. - Order and shape are separate: a language supplies only a default order, and the shapes (where titles, suffixes, particles and commas sit) belong to each order and live in the core; vocabulary readings are to settle a clash and name readings only to keep the gate on — 2026-10-15/16. @@ -225,3 +226,4 @@ Open: #752 (phase 2, derived ambiguity; its sub-phases 2a to 2m, each its own PR - The set's one reader is rules/H2's shape test, which declines a word in it, so the vocabulary reading outranks the shape-derived one, decision 20 applied in the default. It is not a derived marker: no second vocabulary reading is weighed, only the shape's. `ma`, which already had given, family and credential readings in en.toml, takes its abbreviation reading in fil.toml, and the credential's ambiguous marker is unchanged, its name readings already deriving it. `jo` had no reading before, and its `given` reading derives nothing. - The words, with their counts on each entry: `benj`, `chas`, `edw`, `geo`, `jas`, `jno`, `richd`, `robt`, `saml`, `thos` and `wm` in en.toml (the last keeping its refused title, whose reason now names this reading), `jo` and `ma` in fil.toml. What moved, the vetting and the grid are decisions/H2's entry of this date. Recompute the list: `uv run python -c "from nameparser import Lexicon; print(sorted(Lexicon.default().given_name_abbreviations))"`. Adding the field changes `Lexicon`'s pickle layout, as phases 2h and 2i did. +- **2026-10-26 #752, phase 2l — THE BRACKET ESCAPE READS THE DERIVED CLASH**, carrying out item 16 of the 2026-10-23 #752 entry above (#810). A member of the ambiguous class, the class 2a derives from name readings, alone in brackets or quotes after the name is the suffix where its writing settles it (`John Smith (MA)`, `John Smith (M.A)` → suffix), read in one pass, and stays the nickname otherwise (`John Smith (Ed)`, `JOHN SMITH (MA)`, `Jeffrey (JD) Bricken`). It is recorded as rules/S13, a rule of its own rather than the amendment to S1 item 16 planned, the position it reads being evidence S1's signal does not read; the writing is S3's and S6's evidence, which S13 cites. The record, the measurements and what was declined are decisions/S13's 2026-10-26 #810 entry. diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index c3986881..482fa9f8 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -196,9 +196,9 @@ This catalog converts discovery into a one-time cost — it does not remove disc **How it works.** "Andrew Perkins (MBA)" is not a nickname (rule S1): the parens are masked away and MBA is classified by the normal machinery — reusing the downstream path, so the delimited and bare forms cannot drift. -**Lives in.** nameparser/_pipeline/_extract.py (_suffix_shaped and the inner-span branch). +**Lives in.** nameparser/_pipeline/_extract.py (_suffix_shaped and the inner-span branch); and nameparser/_pipeline/_post_rules.py (`_settle_delimited_members`), the one site where content overrules the delimiter WITHOUT rejoining the bare path: a member of the ambiguous class settled by its writing after the name (rules/S13) is the suffix directly, because a dotted member's bare reading would differ (`M.A` reads as initials bare), so it is decided once the parse's roles are read rather than unmasked here (decisions/S13). -**Reach for it when.** About to add a second code path that duplicates what the bare form already does — #335's fix is this shape (a _maiden_marked sibling predicate). +**Reach for it when.** About to add a second code path that duplicates what the bare form already does — #335's fix is this shape (a _maiden_marked sibling predicate). Not when the bare reading is the wrong answer for the clause: that is S13's shape. ## CURATED-VOCABULARY-ALTERNATION — the config already splits them diff --git a/docs/design/rules/README.md b/docs/design/rules/README.md index 02d9c208..128341fc 100644 --- a/docs/design/rules/README.md +++ b/docs/design/rules/README.md @@ -151,6 +151,7 @@ Terms the rules use without defining them each time, each with the rule that def - [S10 — the trailing roman numeral](S10-trailing-roman-numeral.md) - [S12 — a credential split across a space](S12-credential-split-across-a-space.md) - [S11 — a suffix word that is also a name](S11-suffix-word-that-is-also-a-name.md) +- [S13 — a bracketed member settled by its writing](S13-bracketed-member-settled-by-its-writing.md) ## [Nicknames & quoted names (N)](N-nicknames-and-quoted-names.md) diff --git a/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md b/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md index 33a949c5..3fc054af 100644 --- a/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md +++ b/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md @@ -22,17 +22,19 @@ "John Smith (MBA PhD.)" → suffix="MBA PhD." · boundary ``` -- **A clause that is one member of the ambiguous class (S4) written without periods** is not suffix-shaped, so in a nickname pair it stays a nickname, and the fork is reported, as it is for such a member anywhere in a nickname: the word is also a name, so the brackets may hold the name the person goes by, an accepted cost (decisions/suffix-acronym-collisions). A credential standing in front of the brackets does not speak for it (S7), and a final period makes the clause suffix-shaped, a member's included. +- **A clause that is one member of the ambiguous class (S4)** is not suffix-shaped, so in a nickname pair it stays a nickname, and the fork is reported, as it is for such a member anywhere in a nickname: the word is also a name, so the brackets may hold the name the person goes by, an accepted cost (decisions/suffix-acronym-collisions). A credential standing in front of the brackets does not speak for it (S7), and a final period makes the clause suffix-shaped, a member's included. ``` - "John Smith (MA)" → nickname="MA" - "John Smith (MA)" → ambiguities=("suffix-or-nickname",) - "John Smith PhD (MA)" → nickname="MA" + "John Smith (Ma)" → nickname="Ma" + "John Smith (Ma)" → ambiguities=("suffix-or-nickname",) + "John Smith PhD (Ma)" → nickname="Ma" "John Smith (M.A.)" → suffix="M.A." · boundary "John Smith (MA.)" → suffix="MA." · boundary ``` **Precedence:** +- **A member after the name whose writing settles it** ("John Smith (MA)", "John Smith (M.A)"): S13 reads it as the suffix, and the nickname reading above is left to a member it does not settle. + - **A bracketed clause led by a maiden marker** ("Jane Smith (née Jr.)"): suffix-shaped content is taken here first, and the clause is no maiden name (M3), so M2's delimiter precedence never reaches it. - **A clause M3 reads as a maiden name** ("Jane Smith (née Jones)"): its content is not suffix-shaped, so this rule leaves it to M3; the marker-led clause that is suffix-shaped is the bullet above. diff --git a/docs/design/rules/S13-bracketed-member-settled-by-its-writing.md b/docs/design/rules/S13-bracketed-member-settled-by-its-writing.md new file mode 100644 index 00000000..f4455ddf --- /dev/null +++ b/docs/design/rules/S13-bracketed-member-settled-by-its-writing.md @@ -0,0 +1,82 @@ +# S13 — a bracketed member settled by its writing + +**When:** a clause that is one member of the ambiguous class (S4) stands in brackets or quotes after the name, with no name word after it, though a suffix, a title or a maiden clause may follow however each is written, as in "John Smith (MA)". + +**Then:** where the member's writing settles it as the credential, the way the rules that read the writing say a credential is written, it is no nickname but the suffix, and the clash is not reported. + +- **Written with a period inside it**, whatever case the name is in: the periods are how a credential is written and a name is not (S3). +- **Written in capitals in a name written in more than one case**: the case is asked of the name's own words and never of the clause's, so capitals settle nothing in a name written wholly in one case, and Title case leans to the name (S6). +- **It is the suffix, not the word read as if written bare** (S1): the brackets have already set it apart from the name's words, and bare, two letters each followed by a period but the last read as initials (S3). The rest of the name is read as beside a nickname, which is where it can differ from a bracketed unambiguous credential in the same place, its twin, as the Accepted bullets record. + +**Because:** after the name, brackets hold either a credential or the name the person goes by, and the writing tells the two apart where the count of words cannot: credentials are written in capitals and with their periods, and names are not. Inside the name the same word let out of the brackets would read as a middle name, where the brackets more likely hold the name the person goes by. So only the clause after the name is settled. + +``` +"John Smith (MA)" → suffix="MA" +"John Smith (MA)" → ambiguities=() +"John Smith (M.A)" → suffix="M.A" +"JOHN SMITH (M.A)" → suffix="M.A" +"Smith, John (MA)" → suffix="MA" +"John Smith PhD (MA)" → suffix="PhD MA" +"John Smith (MA), PhD" → suffix="MA, PhD" +"John Smith (MA) Ph. D." → suffix="MA Ph. D." +"John Smith (MA) Dr." → title="Dr." +"John Smith (MA) Dr." → suffix="MA" +"John Smith «MA»" → suffix="MA" +"Jack (M.A)" → suffix="M.A" +"John Smith 「MA」" → suffix="MA" +"John Smith (Ed)" → nickname="Ed" · boundary +``` + +**Except:** + +- **A member in a name written in one case, or not in capitals**, is settled only by a period inside it: its case says nothing (S6), and it stays the nickname S1 reads. + ``` + "JOHN SMITH (MA)" → nickname="MA" + "JOHN SMITH (MA)" → ambiguities=("suffix-or-nickname",) + "john smith (MA)" → nickname="MA" + "John Smith (Ma)" → nickname="Ma" + ``` +- **A member standing inside the name or before it** stays the nickname, however it is written, and so does one in a name with no name word of its own, or one with a word after it the parse reads as a name word, a stray full stop included. + ``` + "Jeffrey (JD) Bricken" → nickname="JD" + "JEFFREY (JD) BRICKEN" → nickname="JD" + "Jean 'JD' Smith" → nickname="JD" + "Jean (M.A) Smith" → nickname="M.A" + "Jean (M.A) Smith" → ambiguities=("suffix-or-nickname",) + "Smith (MA), John" → nickname="MA" + "(MA) John Smith" → nickname="MA" + "Dr. (MA)" → nickname="MA" + "(M.A)" → nickname="M.A" + "John Smith (MA)." → nickname="MA" + "John Smith (MA) Ma" → nickname="MA" + ``` + +**Precedence:** + +- **Two letters each followed by a period but the last** ("John Smith (J.D)"): this rule reads the credential where S3 reads the same word bare as paired initials ("John Smith J.D" gives family 'J.D'), the brackets having already set it apart from the name's words. + +**Accepted:** + +- A lone name word beside a settled member stays the family name it is beside a nickname (N3), where beside its twin, "(PhD)", it is the given name, reported. + ``` + "Jack (MA)" → family="Jack" + "Jack (MA)" → ambiguities=() + ``` +- A maiden clause after a settled member stays the maiden name, where after its twin the credential's trailing run takes it (S8). + ``` + "John Smith (MA) née Jones" → maiden="Jones" + ``` +- The case is asked once, of the name's own words without any clause, so a member in capitals beside a settled one in a name otherwise written in one case stays the nickname, where its twin, joining the name's words, makes the name mixed and settles the other too. + ``` + "JOHN SMITH (m.a) (MA)" → nickname="MA" + "JOHN SMITH (m.a) (MA)" → suffix="m.a" + ``` +- A part past the second comma is weighed without the member, so what stands beside it decides whether the part reports, and the report names that alone, where its twin is weighed as a credential of the part. + ``` + "Smith, John, X.Y.Z. (MA)" → ambiguities=() + "Smith, John, Bob (MA)" → ambiguities=("comma-structure",) + ``` + +--- +**Signal:** position: a member of the ambiguous class alone in brackets or quotes after the name; hub of S1, S3, S6 +**history:** decisions/S13 · **implemented:** nameparser/_pipeline/_post_rules.py diff --git a/docs/design/vocabulary-model-rfc.md b/docs/design/vocabulary-model-rfc.md index d69a7346..f9f03e51 100644 --- a/docs/design/vocabulary-model-rfc.md +++ b/docs/design/vocabulary-model-rfc.md @@ -128,7 +128,9 @@ Phase 2i carried this out as [rules/H9](rules/H9-title-phrase.md) (the title phr ### Brackets -A bracketed or quoted credential is to escape the nickname reading where its writing settles it (#810): with interior periods, or in capitals in a name of mixed case, the case asked of the whole name. Today [rules/S1](rules/S1-bracketed-credential-is-not-a-nickname.md) keeps every member of the ambiguous class written without periods a nickname (`John Smith (MA)` → nickname `MA`), and S1 is amended when the escape lands. `(Ed)` stays a nickname, and so does content standing inside the name rather than after it (`Jean 'JD' Smith`). +A bracketed or quoted credential is to escape the nickname reading where its writing settles it (#810): with interior periods, or in capitals in a name of mixed case, the case asked of the whole name. `(Ed)` stays a nickname, and so does content standing inside the name rather than after it (`Jean 'JD' Smith`). + +Phase 2l carried this out as [rules/S13](rules/S13-bracketed-member-settled-by-its-writing.md) (a bracketed member settled by its writing) rather than as an amendment to [rules/S1](rules/S1-bracketed-credential-is-not-a-nickname.md), since the position it reads is a signal S1 does not: `John Smith (MA)` and `John Smith (M.A)` read suffix, and `JOHN SMITH (MA)` and `Jeffrey (JD) Bricken` keep the nickname. The writing is read as the rules that read it say, S3 for the periods and S6 for the capitals in a name of mixed case, and S13's own signal is the position. A settled member is the suffix, decided in one pass, rather than the word read bare, since bare, `M.A` reads as initials; where the rest of the name then reads otherwise than beside a bracketed unambiguous credential (`Jack (MA)` keeps family 'Jack' where `Jack (PhD)` reads given 'Jack'), S13 records the difference as accepted. ### Readings on the parse diff --git a/docs/release_log.rst b/docs/release_log.rst index 562c8e36..dae86b74 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -17,6 +17,7 @@ Release Log - **Fix the given names Mahdi and Bard reading as titles.** ``Mahdi Karimi`` and ``Bard Larsen`` give first ``Mahdi`` and ``Bard``, where 1.4.0 through 2.6.0 gave title ``Mahdi`` and ``Bard`` with no first name; ``Mr Mahdi Karimi`` and ``Karimi, Mahdi`` read the same. Inside a title phrase either is still a title (``Mahdi of Sudan``), but the epithet with its article is not: ``Muhammad Ahmad, the Mahdi`` gives first ``the Mahdi``, last ``Muhammad Ahmad``, where 1.4.0 through 2.6.0 gave title ``the Mahdi``. :issue:`867`, :issue:`752` - **Fix a lone title qualifier at the head of a name reading as a title.** ``Wing Lau`` and ``Mr Wing Lau`` give first ``Wing``, where 1.4.0 through 2.6.0 gave title ``Wing`` and ``Mr Wing``; a qualifier such as ``Wing``, ``Vice`` or ``Grand`` is a title only with a title after it (``Wing Commander Lau``). After a comma it is still a title (``John Smith, Vice``), except ``Wing``, which is also a given name: ``Lau, Wing Kai`` gives first ``Wing``, middle ``Kai``, where 1.4.0 through 2.6.0 gave title ``Wing``, first ``Kai``. ``John Smith Wing.`` gives last ``Wing.``, as 1.4.0 through 2.2.0 did, where 2.3.0 through 2.6.0 gave title ``Wing.``. Like ``Dean``, it reverses a full name before a comma (``John Smith, Wing`` gives first ``Wing``, last ``John Smith``). :issue:`752` - **Read Chas., Wm. and the other given names written short as given names rather than titles.** ``Chas. Smith`` and ``Ma. Cristina Reyes`` give first ``Chas.`` and ``Ma.``, where 1.4.0 through 2.6.0 gave title ``Chas.`` and ``Ma.``; ``Thos.``, ``Jas.``, ``Geo.``, ``Robt.``, ``Benj.``, ``Jno.``, ``Saml.``, ``Edw.``, ``Richd.``, ``Wm.`` and the Filipino ``Jo.`` read the same way, behind a title too (``Mrs Ma. Liza Cruz`` gives title ``Mrs``, first ``Ma.``). An unlisted abbreviation in front is still a title (``Insp. Jane Morse``), and ``Lexicon.given_name_abbreviations`` lists the words. :issue:`789`, :issue:`752` + - **Read a bracketed credential that is also a name as a suffix where its writing settles it.** ``John Smith (MA)`` and ``John Smith (M.A)`` give suffix ``MA`` and ``M.A``, as 1.4.0 did, where 2.0.0 through 2.6.0 gave nickname ``MA`` and ``M.A``; ``Andrew Perkins (JD)`` gives suffix ``JD``, where 1.4.0 through 2.6.0 gave nickname ``JD``. Only capitals in a mixed-case name or a period inside the word settle it, and only after the name, so ``John Smith (Ed)``, ``JOHN SMITH (MA)`` and ``Jeffrey (JD) Bricken`` keep the nickname and report a ``suffix-or-nickname`` ambiguity. :issue:`810`, :issue:`752` - **A Lexicon or Parser pickled by an earlier release no longer loads.** Loading one raises ``ValueError`` naming the new fields (``suffix_words_ambiguous``, ``agnomens``, ``titles_ambiguous``, ``title_modifiers``, ``title_styles``, ``title_phrases``, ``given_name_abbreviations``); re-pickle after upgrading. ``ParsedName`` and ``HumanName`` pickles still load. :issue:`752` * 2.6.0 - October 25, 2026 diff --git a/docs/usage.rst b/docs/usage.rst index 46f4cf2b..29db8e30 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -843,10 +843,18 @@ parenthesized nicknames that happen to be credentials: The exception to that exception is an *ambiguous* acronym — one that is also a plausible name or set of initials. Standing alone inside -delimiters, it keeps the nickname reading: +delimiters, it keeps the nickname reading unless the way it is written +settles it after the name: in capitals in a name written in mixed case, +or with a period inside it. A name written all in capitals says +nothing about the word, and inside the name the delimiters are read as +a nickname however it is written: .. doctest:: + >>> parse("Andrew Perkins (JD)").suffix + 'JD' + >>> parse("Andrew Perkins (Jd)").nickname + 'Jd' >>> parse("JEFFREY (JD) BRICKEN").nickname 'JD' diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index 750fbd0a..0999b73e 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -7,7 +7,7 @@ second comma asks for it (#289/#516, #613), so this read is the fallback for every other name. Produces: tokens with vocabulary tags added (text/span/role unchanged), -plus ambiguities (SUFFIX_OR_NICKNAME, CONJUNCTION_OR_INITIAL) and +plus ambiguities (CONJUNCTION_OR_INITIAL) and one_case -- whether the name's own words are written in one case, recorded for the later stages that read it (#289/#516) and left alone where an earlier stage already asked. @@ -74,7 +74,7 @@ PendingAmbiguity, WorkToken, copy_with, ) -from nameparser._types import AmbiguityKind, Role +from nameparser._types import AmbiguityKind from nameparser._pipeline._vocab import ( BUILTIN_SUFFIX_DELIMITERS, SPLIT_CHUNK, SPLIT_PAIR_OPENS, _INITIAL, caps_shape_candidate, composed_len, delimiter_cores, glued_stem, is_cased_paired_initials, @@ -257,7 +257,7 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # role is delimited content, decided by extract's escape and never # at the trailing slot -- 'Bridge (A.B)' is the control that # proves the guard load-bearing (without it the nickname reading - # of 'A.B' gains a spurious SUFFIX_OR_NICKNAME report below), and + # of 'A.B' gains a spurious SUFFIX_OR_NICKNAME report in post_rules), and # 'Bridge (1.4)' cannot exercise it, a digit chunk never reaching # the shape verdict at all (period_joined_vocab's alphabetic # gate). @@ -523,21 +523,8 @@ def classify(state: ParseState) -> ParseState: for i, tag in split.items(): written[i] = copy_with(written[i], tags=written[i].tags | {tag}) tokens = tuple(written) - # Delimited content whose vocabulary cannot settle it: extract's - # escape sends an UNambiguous suffix straight through ("(MBA)" -> - # suffix) and keeps everything else as a nickname, so an AMBIGUOUS - # acronym in there was a coin the parser had to call. Reported here - # rather than at the escape itself, which runs before tokenize and - # so has no token index to point at. ambiguities = list(state.ambiguities) for i, token in enumerate(tokens): - if (token.role is Role.NICKNAME - and AMBIGUOUS_ACRONYM_TAG in token.tags): - ambiguities.append(PendingAmbiguity( - AmbiguityKind.SUFFIX_OR_NICKNAME, - f"delimited {token.text!r} is also a post-nominal; read " - f"as a nickname rather than a suffix", - (i,))) # #383/#479: narrows the fork's own "initial" tag with the same # inputs the fork used, rather than re-deciding from scratch. # Only the casedness test is inherited from the tag -- diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 478b4945..c0479202 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -1,8 +1,8 @@ """Stage: group. Consumes: tokens (classified), segments, structure, one_case, extracted -(the role + inner span per delimited region, for the #329 pass below -- -the only stage after tokenize that reads it); and, through +(the role + inner span per delimited region, for the #329 pass below; +post_rules reads it too, for S13); and, through `_comma.decide`, comma_offsets (the own-words span of the name's case contrast) and dropped, which decide extends. Produces: structure -- its head decides a comma form once the words diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 1109c4bd..0f91410d 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -4,8 +4,12 @@ particle fold reads the opening piece of segment 0, or of segment 1 under a family comma (#359). structure was always read here, for the rotation gate. Also comma_offsets and dropped, which R1's entry pass -below reads to find the separators the writer typed (#436/#437). -Produces: tokens with roles adjusted by the post rules, the stable +below reads to find the separators the writer typed (#436/#437), and +extracted and one_case, which S13's pass reads (#810). +Produces: tokens with roles adjusted by the post rules (among them the +SUFFIX role of a delimited member its writing settles after the name, +rules/S13, with the SUFFIX_OR_NICKNAME report on any other member left +a nickname), the stable "joined" tag on a post-nominal continuing the entry before it, and the ambiguity P6's attachment reports at the end of a name read family-first (#467) and after a comma with nothing before it -- after a @@ -15,10 +19,12 @@ Policy.name_order (O6 stands down where the script's order differs from the declared one), Policy.middle_as_family, Policy.extra_suffix_delimiters (R1's entry pass, for the delimiter -cores group drops); Lexicon.given_name_titles. +cores group drops); Lexicon.given_name_titles, and +Lexicon.suffix_acronyms_ambiguous (a dotted member in a clause, which +carries no tag there, S13). -Implements rules H1, M4, P1, P6 (its no-comma site, and after a comma -naming no family), O1, O2, O6, O3 and R1 of docs/design/rules/; +Implements rules S13, H1, M4, P1, P6 (its no-comma site, and after a +comma naming no family), O1, O2, O6, O3 and R1 of docs/design/rules/; each is cited at its code below, and H1/P1/O1/O2's history lives in docs/design/decisions/. `suffix_entries` is the R1 entry pass as a state-in/state-out function, for Parser.revise to run over a @@ -26,17 +32,18 @@ """ from __future__ import annotations +import bisect import re -from nameparser._lexicon import _run_addresses_by_given +from nameparser._lexicon import _normalize, _run_addresses_by_given from nameparser._pipeline._assign import ( _attach_particle_tail, _name_positions, ) from nameparser._pipeline._pieces import ( - is_lone_never_given_particle, particle_tail, + is_lone_never_given_particle, particle_tail, settled_by_case, ) from nameparser._pipeline._state import ( - NAME_ROLES, + AMBIGUOUS_ACRONYM_TAG, NAME_ROLES, ParseState, PendingAmbiguity, Structure, WorkToken, _NEVER_FLIPPED, comma_bucket, copy_with, ) @@ -107,8 +114,9 @@ def _mark_suffix_entries(tokens: list[WorkToken], state: ParseState) -> None: # (mechanisms.md#ONE-PREDICATE-PER-QUESTION). decisions/R1 # (#436/#437). # - # After assign, because it keys on Role.SUFFIX; last in post_rules - # by convention only, since no rule here writes SUFFIX or a + # After assign, because it keys on Role.SUFFIX; last in post_rules, + # after S13's pass, the one rule here that writes SUFFIX (#810), and + # by convention otherwise, since no other writes SUFFIX or a # _RENDERS_ELSEWHERE role. Recorded as a tag because a span-less # token (unpickled, or built by replace()) leaves the render # nothing to scan (mechanisms.md#MARK-DONT-STRIP). Every WorkToken @@ -376,9 +384,69 @@ def _family_settled(state: ParseState, tokens: list[WorkToken], _leading_name_piece(state, tokens), tokens) +# rules/S13: "where the member's writing settles it as the credential, +# the way the rules that read the writing say a credential is written, +# it is no nickname but the suffix" -- Then and its first two bullets, +# with the When's "no name word after it"; +# extract's S1 escape has already sent every suffix-shaped clause +# through (decisions/S13 (#810)) +def _settle_delimited_members(state: ParseState, tokens: list[WorkToken], + ambiguities: list[PendingAmbiguity]) -> None: + """A nickname that is the whole of its clause and a member of the + ambiguous class becomes the suffix where its writing settles it and + it stands after the last name word; any other tagged member stays + the nickname and the fork is reported, a dotted member left a + nickname inside the name included. Asked here, first in this + stage, because only once roles are read is it known whether a name + word follows: a suffix, a title or a maiden clause after the clause + is no name word, whatever spelling put it there. Reported here, + where the escape is decided, rather than in extract, which has no + token index to point at. A dotted member carries no tag in a + clause (classify tags a shape only outside one), so it is found + by its text.""" + last_name = -1 + alone: set[int] = set() + for i, t in enumerate(tokens): + if t.role in NAME_ROLES: + last_name = i + starts = [t.span.start for t in tokens] + for _, clause in state.extracted: + lo = bisect.bisect_left(starts, clause.start) + # the whole clause's text, as extract's escape reads a clause: + # a character tokenize drops beside the member (an emoji, a + # zero-width space) leaves it no member alone + if (bisect.bisect_left(starts, clause.end) == lo + 1 + and state.original[clause.start:clause.end].strip() + == tokens[lo].text): + alone.add(lo) + lex = state.lexicon + for i, token in enumerate(tokens): + if token.role is not Role.NICKNAME: + continue + # a period inside the word ('M.A'); one ending it is extract's + # escape + dotted = ("." in token.text.strip(".") + and _normalize(token.text).replace(".", "") + in lex.suffix_acronyms_ambiguous) + # after the name: behind its last name word, and a name word + # there is, so a clause in a name of no word of its own + # ('Dr. (MA)') is not settled + if 0 <= last_name < i and i in alone and ( + dotted or settled_by_case(token, state.one_case)): + tokens[i] = copy_with(token, role=Role.SUFFIX) + elif dotted or AMBIGUOUS_ACRONYM_TAG in token.tags: + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NICKNAME, + f"delimited {token.text!r} is also a post-nominal; read " + f"as a nickname rather than a suffix", + (i,))) + + def post_rules(state: ParseState) -> ParseState: tokens = list(state.tokens) ambiguities = list(state.ambiguities) + if state.extracted: + _settle_delimited_members(state, tokens, ambiguities) titles = _idx(tokens, Role.TITLE) givens = _idx(tokens, Role.GIVEN) middles = _idx(tokens, Role.MIDDLE) diff --git a/nameparser/_pipeline/_state.py b/nameparser/_pipeline/_state.py index 3e5ccb96..7a3fafc6 100644 --- a/nameparser/_pipeline/_state.py +++ b/nameparser/_pipeline/_state.py @@ -210,8 +210,10 @@ class ParseState: assign -> the remaining token roles AND `order`, the effective order it read them under, AND the folded-middle tag on the particles P6's attachment joins after a family comma (#613); - post_rules -> roles again, and the ambiguity P6's no-comma - attachment reports (#467). + post_rules -> roles again (a delimited member S13 settles becomes + the suffix, #810), and the ambiguity P6's no-comma attachment + reports (#467) and SUFFIX_OR_NICKNAME on a member it leaves a + nickname. Ambiguities are recorded by every stage that DECIDES one -- extract (resolved to a token index by tokenize), segment, script_segment, classify, group, assign, and post_rules -- since a diff --git a/nameparser/_types.py b/nameparser/_types.py index 07cd85e7..2f7a36dc 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -444,7 +444,8 @@ class AmbiguityKind(StrEnum): #: Delimited content is an ambiguous suffix acronym, so it reads #: plausibly as either a post-nominal or a nickname -- "JEFFREY #: (JD) BRICKEN" keeps the nickname reading, where the - #: unambiguous "(MBA)" escapes to suffix on vocabulary alone. + #: unambiguous "(MBA)" escapes to suffix on vocabulary alone, and + #: "Andrew Perkins (JD)" on its capitals, which settle it. SUFFIX_OR_NICKNAME = "suffix-or-nickname" #: A trailing word reads plausibly as either a post-nominal or an #: ordinary name part. Covers an ambiguous acronym written without diff --git a/tests/test_suffixes.py b/tests/test_suffixes.py index ea91869c..044685f6 100644 --- a/tests/test_suffixes.py +++ b/tests/test_suffixes.py @@ -464,26 +464,31 @@ def test_suffix_acronyms_ambiguous_custom_entry_stays_nickname(self) -> None: # A custom suffix_acronyms_ambiguous entry keeps a suffix_acronyms # member classified as a nickname instead of a suffix, confirming # the exception list -- not a hardcoded check -- drives the behavior. + # Title case: capitals in a mixed-case name settle the member as + # the credential (rules/S1, #810). from nameparser.config import Constants C = Constants( suffix_acronyms=['xyz'], suffix_acronyms_ambiguous=['xyz'], ) - hn = HumanName("Andrew Perkins (XYZ)", constants=C) - self.m(hn.nickname, "XYZ", hn) + hn = HumanName("Andrew Perkins (Xyz)", constants=C) + self.m(hn.nickname, "Xyz", hn) self.m(hn.suffix, "", hn) def test_suffix_acronyms_ambiguous_removal_routes_to_suffix(self) -> None: - # Removing 'jd' from a custom suffix_acronyms_ambiguous flips JD + # Removing 'jd' from a custom suffix_acronyms_ambiguous flips Jd # from nickname to suffix. Uses a trailing-position name (unlike the # JEFFREY (JD) BRICKEN regression guard in test_nicknames.py) so # parse_full_name's trailing-run suffix detection actually picks it # up -- see the known mid-name limitation noted on the tests above. + # Title case, which leaves the member a nickname while it is listed + # (rules/S1, #810). from nameparser.config import Constants + assert HumanName("Andrew Perkins (Jd)").nickname == "Jd" C = Constants(suffix_acronyms_ambiguous=[]) - hn = HumanName("Andrew Perkins (JD)", constants=C) + hn = HumanName("Andrew Perkins (Jd)", constants=C) self.m(hn.nickname, "", hn) - self.m(hn.suffix, "JD", hn) + self.m(hn.suffix, "Jd", hn) def test_empty_comma_segment_does_not_drop_following_suffix(self) -> None: # Regression: "Doe, John,, Jr." produced an empty third segment, and diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 5a17157e..6b84a90d 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -8802,6 +8802,182 @@ def _check_cjk_shape_purity(self) -> None: "2026-07-17)"), Case("paren_period_escapes_nickname", "Andrew Perkins (Ret.)", {"given": "Andrew", "family": "Perkins", "suffix": "Ret."}), + # rules/S13 (#810): a member of the ambiguous class alone in the + # brackets after the name, settled by its writing. Each row is + # paired with a control its writing or its position does not settle. + Case("bracket_member_in_capitals_is_the_suffix", "John Smith (MA)", + {"given": "John", "family": "Smith", "suffix": "MA"}, + notes="capitals in a mixed-case name settle it; 2.0.0 through " + "2.6.0 read nickname 'MA', reported"), + Case("bracket_member_in_title_case_stays_the_nickname", + "John Smith (Ed)", + {"given": "John", "family": "Smith", "nickname": "Ed"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="Title case leans to the name (rules/S6); Ed is " + "Edward's nickname far more often than a credential"), + Case("bracket_member_with_a_period_inside_is_the_suffix", + "John Smith (M.A)", + {"given": "John", "family": "Smith", "suffix": "M.A"}, + notes="the suffix, not the bare reading: bare, M.A reads as " + "paired initials (rules/S3)"), + Case("bracket_member_with_a_period_inside_in_one_case", + "JOHN SMITH (M.A)", + {"given": "JOHN", "family": "SMITH", "suffix": "M.A"}, + notes="the period settles it whatever the name's case"), + Case("bracket_member_in_a_one_case_name_stays_the_nickname", + "JOHN SMITH (MA)", + {"given": "JOHN", "family": "SMITH", "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="capitals settle nothing in a name written in one case; " + "1.4.0 read suffix 'MA'"), + Case("bracket_member_in_a_lower_case_name_stays_the_nickname", + "john smith (MA)", + {"given": "john", "family": "smith", "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="the case asked is the name's own words', never the " + "clause's, so this differs from bare 'john smith MA'; " + "1.4.0 read suffix 'MA'"), + Case("bracket_member_after_a_one_case_name_with_a_nickname", + "JOHN SMITH (Bob) (MA)", + {"given": "JOHN", "family": "SMITH", "nickname": "Bob MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="another clause's case is no name's: 'Bob' does not " + "make the name mixed; 1.4.0 read suffix 'MA'"), + Case("bracket_member_jd_is_the_suffix", "Andrew Perkins (JD)", + {"given": "Andrew", "family": "Perkins", "suffix": "JD"}, + classification="fix(#810)", + notes="1.4.0 through 2.6.0 read nickname 'JD'"), + Case("bracket_member_inside_the_name_stays_the_nickname", + "Jeffrey (JD) Bricken", + {"given": "Jeffrey", "family": "Bricken", "nickname": "JD"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="a name word follows: let out, JD would be a middle name"), + Case("bracket_member_with_a_period_inside_the_name", + "Jean (M.A) Smith", + {"given": "Jean", "family": "Smith", "nickname": "M.A"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="inside the name however it is written, and reported as " + "a capitals member is; 1.4.0 read middle 'M.A'"), + Case("bracket_member_before_a_family_comma_stays_the_nickname", + "Smith (MA), John", + {"given": "John", "family": "Smith", "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="the given name after the comma is a name word after " + "it; 1.4.0 read suffix 'MA'"), + Case("bracket_member_after_a_family_comma", "Smith, John (MA)", + {"given": "John", "family": "Smith", "suffix": "MA"}), + Case("bracket_member_before_a_suffix_comma", "John Smith (MA), PhD", + {"given": "John", "family": "Smith", "suffix": "MA, PhD"}, + notes="a suffix after the clause is no name word"), + Case("bracket_member_before_a_maiden_clause", + "John Smith (MA) née Jones", + {"given": "John", "family": "Smith", "suffix": "MA", + "maiden": "Jones"}, + classification="fix(#810)", + notes="a maiden clause after it is no name word, and stays the " + "maiden name, where its twin's credential run takes it " + "(rules/S13's Accepted); 1.4.0 read middle 'Smith MA " + "née', last 'Jones'"), + Case("bracket_member_settled_by_another_in_a_one_case_name", + "JOHN SMITH (m.a) (MA)", + {"given": "JOHN", "family": "SMITH", "suffix": "m.a", + "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="the case is asked once, of the name's own words without " + "either clause, so 'MA' stays the nickname where its twin " + "'JOHN SMITH (mba) (MA)' makes the name mixed " + "(rules/S13's Accepted)"), + Case("bracket_member_settled_by_another_in_a_lower_case_name", + "john smith (M.A) (MA)", + {"given": "john", "family": "smith", "suffix": "M.A", + "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="as the row above; the twin 'john smith (MBA) (MA)' " + "reads suffix 'MBA MA'"), + Case("bracket_member_past_the_second_comma", "Smith, John, Jr. (M.A)", + {"given": "John", "family": "Smith", "suffix": "Jr. M.A"}, + classification="fix(#810)", + notes="no comma-structure report, as its twin '(MBA)' draws " + "none: the part is tested while the clause is still a " + "nickname, and 'Jr.' alone is a suffix"), + Case("bracket_member_past_the_second_comma_after_a_dotted_shape", + "Smith, John, X.Y.Z. (MA)", + {"given": "John", "family": "Smith", "suffix": "X.Y.Z. MA"}, + classification="fix(#810)", + notes="no report, where its twin '(MBA)' draws comma-structure: " + "the part is tested while the clause is a nickname, and " + "'X.Y.Z.' alone is a run of shape-admitted words " + "(rules/S13's Accepted)"), + Case("bracket_member_beside_a_dropped_character", + "John Smith (MA \N{GRINNING FACE})", + {"given": "John", "family": "Smith", "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="not the member alone, as its twin, the same clause " + "with PhD, is no suffix-shaped clause"), + Case("bracket_member_before_a_title_case_member", + "John Smith (MA) Ma", + {"given": "John", "middle": "Smith", "family": "Ma", + "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-name", "suffix-or-nickname"), + notes="read on the nickname reading, 'Ma' is a name word after " + "the clause, where the twin's credential run would take " + "it (rules/S13's Except)"), + Case("bracket_member_with_a_leading_period", "John Smith (.ma)", + {"given": "John", "family": "Smith", "nickname": ".ma"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="a period opening the word is no period inside it"), + Case("bracket_member_in_a_name_of_no_word", "Dr. (MA)", + {"title": "Dr.", "nickname": "MA"}, + classification="fix(#810)", + ambiguities=("suffix-or-nickname",), + notes="no name word, so no clause after the name; the twin " + "'Dr. (MBA)' reads given 'Dr.'"), + Case("bracket_member_in_quotes", 'John Smith "MA"', + {"given": "John", "family": "Smith", "suffix": "MA"}, + notes="every default nickname pair escapes alike (rules/S1)"), + Case("bracket_member_after_a_lone_name_word", "Jack (MA)", + {"family": "Jack", "suffix": "MA"}, + classification="fix(#810)", + notes="the word stays N3's family name, read beside the clause " + "as a nickname, where its twin 'Jack (PhD)' reads given " + "'Jack' (rules/S13's Accepted); 1.4.0 read last 'MA', " + "2.0.0 through 2.6.0 family 'Jack', nickname 'MA'"), + Case("bracket_member_with_a_period_after_a_lone_name_word", + "Jack (M.A)", + {"family": "Jack", "suffix": "M.A"}, + classification="fix(#810)", + notes="as the row above; bare 'Jack M.A' reads family 'M.A'; " + "2.0.0 through 2.6.0 read family 'Jack', nickname 'M.A'"), + Case("bracket_member_before_a_shape_read_suffix", + "John Smith (MA) Ph. D.", + {"given": "John", "family": "Smith", "suffix": "MA Ph. D."}, + classification="fix(#810)", + notes="a suffix after the clause is no name word whatever " + "spelling made it one; the first draft asked the " + "suffix tag, which the split Ph. D. does not carry"), + Case("bracket_member_before_a_numeral", "John Smith (MA) VI", + {"given": "John", "family": "Smith", "suffix": "MA VI"}, + classification="fix(#810)", + ambiguities=("suffix-or-name",), + notes="VI is read by shape, not listed (rules/S10)"), + Case("bracket_member_before_a_trailing_title", + "John Smith (MA) Dr.", + {"title": "Dr.", "given": "John", "family": "Smith", + "suffix": "MA"}, + classification="fix(#810)", + notes="a title after the clause is no name word (rules/H5)"), Case("nickname_quotes", 'John "Jack" Kennedy', {"given": "John", "family": "Kennedy", "nickname": "Jack"}, notes="shape 1's double-quoted Nickname slot, which is the " diff --git a/tests/v2/pipeline/test_classify.py b/tests/v2/pipeline/test_classify.py index f1c519e8..fe9af186 100644 --- a/tests/v2/pipeline/test_classify.py +++ b/tests/v2/pipeline/test_classify.py @@ -576,8 +576,8 @@ def test_delimited_content_never_joins_the_shape_class() -> None: # control that actually load-bears it: without the guard, the # nickname 'A.B' would gain SHAPE_ACRONYM_TAG and (with the switch # on) `vocab:suffix-ambiguous`, and classify's own nickname check - # would then misreport SUFFIX_OR_NICKNAME on a token the escape - # already decided (measured regression, #516 review round). + # would then misreport SUFFIX_OR_NICKNAME (post_rules' report, since + # #810) on a token the escape already decided (measured regression, #516 review round). for text, word in (("Bridge (1.4)", "1.4"), ("Bridge (A.B)", "A.B")): tags = _tags_by_text(text)[word] assert "shape:acronym" not in tags, text diff --git a/tests/v2/pipeline/test_state.py b/tests/v2/pipeline/test_state.py index 51b8b930..d455fe78 100644 --- a/tests/v2/pipeline/test_state.py +++ b/tests/v2/pipeline/test_state.py @@ -281,9 +281,9 @@ def test_stage_field_ownership() -> None: # FAMILY_COMMA opt-out, asked again for itself since #613, and # the hook it consults on a vocabulary decline). "script_segment": {"tokens", "segments", "ambiguities"}, - # classify also emits SUFFIX_OR_NICKNAME: the delimiter escape - # that decides it lives in extract_delimited, which has no token - # index to point at, so the report is raised here instead + # classify emits CONJUNCTION_OR_INITIAL; SUFFIX_OR_NICKNAME + # moved to post_rules with the decision it reports (rules/S13, + # #810) "classify": {"tokens", "ambiguities", "one_case"}, # group also emits PARTICLE_OR_GIVEN: the prefix chain takes # the particle branch of a fork whose given branch _assign @@ -304,7 +304,8 @@ def test_stage_field_ownership() -> None: # post_rules also emits PARTICLE_OR_GIVEN: P6's no-comma # attachment (#467) takes the family branch of a fork whose # other branches assign and group take, so each stage reports - # the side it decides; the comma site is assign's since #613 + # the side it decides; the comma site is assign's since #613; + # and SUFFIX_OR_NICKNAME, with S13's decision (#810) "post_rules": {"tokens", "ambiguities"}, } assert {s.__name__ for s in STAGES} == set(ownership) diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index d8073a3a..6edc76e4 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -53,7 +53,7 @@ "R3": {"words": 483, "then_words": 407, "because_sentences": 2, "except": 1, "precedence": 0, "accepted": 3, "accepted_words": 409}, "R4": {"words": 991, "then_words": 729, "because_sentences": 1, "except": 4, "precedence": 0, "accepted": 4, "accepted_words": 600}, "R5": {"words": 250, "then_words": 154, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, - "S1": {"words": 274, "then_words": 26, "because_sentences": 1, "except": 2, "precedence": 2, "accepted": 0, "accepted_words": 0}, + "S1": {"words": 306, "then_words": 26, "because_sentences": 1, "except": 2, "precedence": 3, "accepted": 0, "accepted_words": 0}, "S2": {"words": 657, "then_words": 119, "because_sentences": 1, "except": 0, "precedence": 7, "accepted": 3, "accepted_words": 214}, "S3": {"words": 960, "then_words": 398, "because_sentences": 3, "except": 4, "precedence": 5, "accepted": 6, "accepted_words": 343}, "S4": {"words": 306, "then_words": 164, "because_sentences": 2, "except": 0, "precedence": 3, "accepted": 0, "accepted_words": 0}, @@ -65,6 +65,7 @@ "S10": {"words": 665, "then_words": 246, "because_sentences": 3, "except": 4, "precedence": 3, "accepted": 3, "accepted_words": 163}, "S11": {"words": 474, "then_words": 266, "because_sentences": 3, "except": 1, "precedence": 1, "accepted": 2, "accepted_words": 107}, "S12": {"words": 674, "then_words": 372, "because_sentences": 2, "except": 3, "precedence": 4, "accepted": 2, "accepted_words": 93}, + "S13": {"words": 431, "then_words": 180, "because_sentences": 3, "except": 2, "precedence": 1, "accepted": 4, "accepted_words": 138}, "T1": {"words": 93, "then_words": 18, "because_sentences": 1, "except": 1, "precedence": 0, "accepted": 3, "accepted_words": 83}, "T2": {"words": 43, "then_words": 14, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "T3": {"words": 56, "then_words": 23, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, diff --git a/tests/v2/test_benchmark.py b/tests/v2/test_benchmark.py index 3ca873e9..515e0a2e 100644 --- a/tests/v2/test_benchmark.py +++ b/tests/v2/test_benchmark.py @@ -294,6 +294,7 @@ def test_a_thousand_names_still_parse_in_reasonable_time( "credential_run": "PhD Ma ", # pieces: S7's anchor pass (#544) "agnomens": "Neto ", # pieces: a gated word scans its front (#890) "carried_agnomens": "MA ma Neto ", # pieces: the scan through carried members + "bracket_members": "(M.A) ", # post_rules: S13's clause pass (#810) } _BASE = 800 diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 0abbeaac..97a77d08 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -271,8 +271,25 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # 1321 + 24 = 1345 on 2026-10-26, #894 (phase 2i) merged over #895 # (phase 2g): 2i's net change from 1568 (1592 - 1568), the two # changes moving disjoint claims. - assert claims == 1345, ( - f"the field sweep checks {claims} claims, recorded as 1345 on " + # 1345 -> 1366 on 2026-10-26, #810 (phase 2l): seven rows whose + # bracketed member stays the nickname, one suffix-or-nickname or + # given-or-family claim each under each order; 1366 -> 1369 the same + # day, its review round: 'Jack (MA)' keeps N3's family and loses its + # given-or-family claims (-3), 'John Smith (MA) VI' (+3) and the + # dotted 'Jean (M.A) Smith', reported now (+3); 1369 -> 1381 the same + # day, the twin reading (Derek): 'Jack (MA)' and 'Jack (M.A)' read + # as 'Jack (PhD)' does, a given-or-family claim each (+6), and + # 'John Smith (MA) née Jones' its two suffix-or-name claims (+6); + # 1381 -> 1390 the same day, its review round's rows: a member + # beside a dropped character (+3) and before 'Ma' (+6); 1390 -> 1384 + # the same day, one pass in place of the twin reading (Derek): + # 'Jack (MA)' and 'Jack (M.A)' lose their given-or-family claims and + # 'John Smith (MA) née Jones' its two suffix-or-name claims (-12), + # and the two clauses a second pass would settle keep the nickname, + # reported (+6); 1384 -> 1390 the same day, its review round's rows + # for a leading period and a name of no word (+6). + assert claims == 1390, ( + f"the field sweep checks {claims} claims, recorded as 1390 on " f"2026-10-26") diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index c17580fb..0a3f4537 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -1,4 +1,5 @@ import dataclasses +import itertools import pickle import re import unicodedata @@ -9,10 +10,11 @@ Lexicon, Locale, Parser, Policy, PolicyPatch, locales, parse, parser_for, ) from nameparser._policy import ( - CapsSuffixes, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST, PatronymicRule, + CapsSuffixes, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST, GIVEN_FIRST, + PatronymicRule, ) from nameparser._types import (UNJOINED_CONJUNCTION_TAG, UNJOINED_TAG, - AmbiguityKind, Role, Segmentation) + AmbiguityKind, ParsedName, Role, Segmentation) def test_parser_defaults_and_properties() -> None: @@ -2184,3 +2186,87 @@ def test_a_declined_cjk_credential_word_does_not_peel_off_a_glued_name( # 長 alone is a borne given name, so 阿部長 can be 阿部 + 長 (#707). assert word not in Parser().lexicon.honorific_tails assert Parser().parse(f"田中{word}").suffix == "" + + +# rules/S13 (#810): where a bracketed member reads differently from its +# twin, a bracketed unambiguous credential in the same place. The twin +# is MBA, a word no head holds, so the twin's reading maps back by +# replacing its text. 1,080 texts, about 2s. +_TWIN_HEADS = ( + "John Smith {c}", "Jack {c}", "{c}", "John {c}", "JOHN SMITH {c}", + "Smith, John {c}", "Smith, {c}", ", {c}", "(Bob), {c}", + "John Smith {c}, PhD", "John Smith, PhD {c}", "John Smith PhD {c}", + "Dr. Jack {c}", "Dr. {c}", "Freiherr von Berg {c}", "Juan de la Vega {c}", + "Jane Doe nee Smith {c}", "John Smith {c} née Jones", + "John Smith {c} Dr.", "John Smith {c} VI", "John Smith {c} Ph. D.", + "Smith, Jane {c} Mrs", "Ortega y Gasset, José {c}", + "Smith, Freiherr von Berg {c}", "Doe, John van {c}", + # a third comma part, and a clause whose settling another makes + # possible, the first settled word making the name mixed case + "Smith, John, Jr. {c}", "JOHN SMITH (m.a) {c}", + "john smith (M.A) {c}", "Smith, John, X.Y.Z. {c}", + "Smith, John, Bob {c}", +) +_TWIN_MEMBERS = ("MA", "M.A", "JD", "DO", "D.O", "MENG") +_TWIN_PAIRS = (("(", ")"), ('"', '"')) +_TWIN_ORDERS = (GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST) + + +def _twin_reading(name: ParsedName, twin: str | None, word: str) -> tuple[ + dict[str, str], list[tuple[str, str]]]: + def back(text: str) -> str: + return text.replace(twin, word) if twin else text + return ({k: back(v) for k, v in name.as_dict().items() if v}, + [(a.kind.value, back(a.detail)) for a in name.ambiguities]) + + +#: The census, by head and by whether the member was settled or left +#: the nickname, recorded 2026-10-26 (#810). rules/S13's Accepted bullets +#: and its Except bullets name each shape: a lone name word stays N3's +#: family name beside a settled member where its twin's is given; the +#: maiden clause stays the maiden name where the twin's credential run +#: takes it; a part past a second comma reports what is left without +#: the member; a name written in one case settles no capitals; a name +#: with no word of its own (a title, a nickname, nothing) settles no +#: clause, where the twin escapes, read as the name ('(MBA)' given +#: 'MBA', 'Dr. (MBA)' given 'Dr.') or as the suffix behind a comma +#: (', (MBA)'); and a clause another +#: settled one would make settle, on a second reading, is left the +#: nickname. +_TWIN_CENSUS = { + ("(Bob), {c}", "nickname"): 36, + (", {c}", "nickname"): 36, + ("Dr. {c}", "nickname"): 36, + ("JOHN SMITH (m.a) {c}", "nickname"): 24, + ("JOHN SMITH {c}", "nickname"): 24, + ("Jack {c}", "settled"): 36, + ("John Smith {c} née Jones", "settled"): 36, + ("John {c}", "settled"): 36, + ("Smith, John, Bob {c}", "settled"): 36, + ("Smith, John, X.Y.Z. {c}", "settled"): 36, + ("john smith (M.A) {c}", "nickname"): 24, + ("{c}", "nickname"): 36, +} + + +def test_where_a_bracketed_member_reads_otherwise_than_its_twin() -> None: + """A census, not a guarantee: one pass decides S13 once the roles are + read, so the rest of the name is read around the clause as a + nickname, and that differs from the twin's reading in the shapes + _TWIN_CENSUS names. A change that moves a count, or adds a head, is + a change to what rules/S13 accepts: re-record it there and here. + The twin reading folded to a fixed point (the PR's fourth draft, + declined by Derek for one pass) differed in none of the settled + rows.""" + census: dict[tuple[str, str], int] = {} + for (open_, close), word, head, order in itertools.product( + _TWIN_PAIRS, _TWIN_MEMBERS, _TWIN_HEADS, _TWIN_ORDERS): + parser = Parser(policy=Policy(name_order=order)) + name = parser.parse(head.format(c=f"{open_}{word}{close}")) + twin = parser.parse(head.format(c=f"{open_}MBA{close}")) + if _twin_reading(name, None, word) != _twin_reading( + twin, "MBA", word): + key = (head, "nickname" if word in name.nickname.split() + else "settled") + census[key] = census.get(key, 0) + 1 + assert census == _TWIN_CENSUS diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index ad760ca8..f4b09108 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -4,6 +4,8 @@ "'Smitty' Jones Jr." "'Smitty' Sir John" "(" +"(M.A)" +"(MA) John Smith" ", Ana García López" ", Dean" ", Dr. fils" @@ -148,6 +150,7 @@ "Dr King Jr" "Dr Smith V" "Dr." +"Dr. (MA)" "Dr. Ana García López" "Dr. Freiherr von Berg, Ed" "Dr. John Smith Prof." @@ -245,8 +248,12 @@ "J.Smith" "JACK MA" "JANE DOE MA R.T." +"JEFFREY (JD) BRICKEN" "JOHN PROF. MA" "JOHN QUINCY SMITH I" +"JOHN SMITH (M.A)" +"JOHN SMITH (MA)" +"JOHN SMITH (m.a) (MA)" "JOHN SMITH JR, AD" "JOHN SMITH MA" "JOHN SMITH MA JONES" @@ -263,6 +270,8 @@ "JOSÉ GARCÍA" "JUAN GARCIA Jr." "JUAN GARCIA Y LOPEZ" +"Jack (M.A)" +"Jack (MA)" "Jack I-V" "Jack I.-V." "Jack MA" @@ -336,11 +345,13 @@ "Jane van der Berg née y Jones" "Jane „JD Smith" "Jean 'JD' Smith" +"Jean (M.A) Smith" "Jean DUPONT" "Jean Marie Fils" "Jean Pierre DUPONT" "Jean Pierre DUPONT-MORETTI" "Jean Pierre MA DUPONT" +"Jeffrey (JD) Bricken" "Jiří Dienstbier ml." "John . Smith" "John Dean Smith" @@ -360,11 +371,20 @@ "John Prof. MA" "John Quincy Smith i" "John Smith" +"John Smith (Ed)" +"John Smith (M.A)" "John Smith (M.A.)" "John Smith (MA PhD)" "John Smith (MA)" +"John Smith (MA) Dr." +"John Smith (MA) Ma" +"John Smith (MA) Ph. D." +"John Smith (MA) née Jones" +"John Smith (MA), PhD" +"John Smith (MA)." "John Smith (MA.)" "John Smith (MBA PhD.)" +"John Smith (Ma)" "John Smith (PhD MBA)" "John Smith 1.4" "John Smith DO Jones" @@ -405,6 +425,7 @@ "John Smith Ph. D. Jones" "John Smith PhD" "John Smith PhD (MA)" +"John Smith PhD (Ma)" "John Smith PhD Ed Ma" "John Smith PhD Jones" "John Smith PhD MEng" @@ -442,6 +463,8 @@ "John Smith XYZ" "John Smith Xyz." "John Smith vi" +"John Smith «MA»" +"John Smith 「MA」" "John Smith 博. 士." "John Smith, A-B" "John Smith, A.B." @@ -720,6 +743,7 @@ "Sita Kumari K.C" "Smith" "Smith (Jones)" +"Smith (MA), John" "Smith (née Jones)" "Smith Jones Jr., J.A." "Smith Jr." @@ -752,6 +776,7 @@ "Smith, Jan van Prof." "Smith, Jane Mrs" "Smith, John" +"Smith, John (MA)" "Smith, John MA Jones" "Smith, John MD/PhD" "Smith, John Ma" @@ -777,12 +802,14 @@ "Smith, John Vi" "Smith, John X" "Smith, John van der Berg Prof." +"Smith, John, Bob (MA)" "Smith, John, MD - née Jones Smith" "Smith, John, PhD - and MD" "Smith, John, PhD - i Soler" "Smith, John, PhD née Jones" "Smith, John, Prof." "Smith, John, Puig - i Soler" +"Smith, John, X.Y.Z. (MA)" "Smith, John, and" "Smith, John; MD" "Smith, Jr." @@ -897,6 +924,7 @@ "john i phd" "john j v" "john j v, PhD" +"john smith (MA)" "john smith EdD" "john smith MEng" "john smith bsc" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 813b262b..ad72fe40 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -7,6 +7,8 @@ {"name": "'Smitty' Sir John", "title": "Sir", "given": "John", "nickname": "Smitty", "initials": "J."} {"name": "(", "ambiguities": ["UNBALANCED_DELIMITER"]} {"name": "()"} +{"name": "(M.A)", "nickname": "M.A", "ambiguities": ["SUFFIX_OR_NICKNAME"]} +{"name": "(MA) John Smith", "given": "John", "family": "Smith", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "John \"Ma\" Smith"} {"name": "(김민준.) John Smith", "given": "김", "middle": "민준. John", "family": "Smith", "initials": "김. 민. J. S."} {"name": ","} {"name": ", Ana García López", "given": "Ana", "middle": "García López", "initials": "A. G. L."} @@ -199,7 +201,7 @@ {"name": "Andrew Perkins \"MBA\"", "given": "Andrew", "family": "Perkins", "suffix": "MBA", "initials": "A. P."} {"name": "Andrew Perkins 'MBA'", "given": "Andrew", "family": "Perkins", "suffix": "MBA", "initials": "A. P."} {"name": "Andrew Perkins (Andy)", "given": "Andrew", "family": "Perkins", "nickname": "Andy", "initials": "A. P."} -{"name": "Andrew Perkins (JD)", "given": "Andrew", "family": "Perkins", "nickname": "JD", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "A. P.", "capitalized_force": "Andrew \"Jd\" Perkins"} +{"name": "Andrew Perkins (JD)", "given": "Andrew", "family": "Perkins", "suffix": "JD", "initials": "A. P."} {"name": "Andrew Perkins (M.D)", "given": "Andrew", "family": "Perkins", "suffix": "M.D", "initials": "A. P."} {"name": "Andrew Perkins (MBA)", "given": "Andrew", "family": "Perkins", "suffix": "MBA", "initials": "A. P."} {"name": "Andrew Perkins (Mgr.)", "title": "Mgr.", "given": "Andrew", "family": "Perkins", "initials": "A. P."} @@ -761,6 +763,7 @@ {"name": "Dr 김민준씨, Jr.", "title": "Dr", "given": "민준", "family": "김", "suffix": "씨, Jr.", "initials": "민. 김."} {"name": "Dr 김민준씨, V.", "title": "Dr", "given": "민준", "family": "김", "suffix": "씨, V.", "initials": "민. 김."} {"name": "Dr.", "title": "Dr."} +{"name": "Dr. (MA)", "title": "Dr.", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "capitalized_force": "Dr. \"Ma\""} {"name": "Dr. Abdul Salam Hassan, MD", "title": "Dr.", "given": "Abdul Salam", "family": "Hassan", "suffix": "MD", "initials": "A. S. H."} {"name": "Dr. Ana García López", "title": "Dr.", "given": "Ana", "middle": "García", "family": "López", "initials": "A. G. L."} {"name": "Dr. Ardelean Dl Daniel Ionescu Dragoș Ioan Jr.", "order": "FAMILY_FIRST", "title": "Dr.", "given": "Dl", "middle": "Daniel Ionescu Dragoș Ioan", "family": "Ardelean", "suffix": "Jr.", "initials": "D. D. I. D. I. A."} @@ -1377,6 +1380,9 @@ {"name": "JOHN PROF. MA", "title": "PROF.", "given": "JOHN", "family": "MA", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. M.", "capitalized": "Prof. John Ma"} {"name": "JOHN QUINCY SMITH I", "given": "JOHN", "middle": "QUINCY", "family": "SMITH", "suffix": "I", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. Q. S.", "capitalized": "John Quincy Smith I"} {"name": "JOHN SMITH", "given": "JOHN", "family": "SMITH", "initials": "J. S.", "capitalized": "John Smith"} +{"name": "JOHN SMITH (M.A)", "given": "JOHN", "family": "SMITH", "suffix": "M.A", "initials": "J. S.", "capitalized": "John Smith M.A"} +{"name": "JOHN SMITH (MA)", "given": "JOHN", "family": "SMITH", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized": "John \"Ma\" Smith"} +{"name": "JOHN SMITH (m.a) (MA)", "given": "JOHN", "family": "SMITH", "suffix": "m.a", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized": "John \"Ma\" Smith M.A"} {"name": "JOHN SMITH JR, AD", "given": "AD", "family": "JOHN SMITH", "suffix": "JR", "initials": "A. J. S.", "capitalized": "Ad John Smith Jr"} {"name": "JOHN SMITH MA", "given": "JOHN", "family": "SMITH", "suffix": "MA", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S.", "capitalized": "John Smith MA"} {"name": "JOHN SMITH MA JONES", "given": "JOHN", "middle": "SMITH MA", "family": "JONES", "initials": "J. S. M. J.", "capitalized": "John Smith Ma Jones"} @@ -1403,6 +1409,8 @@ {"name": "Jacinto López Moreno", "given": "Jacinto", "middle": "López", "family": "Moreno", "initials": "J. L. M."} {"name": "Jack", "given": "Jack", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "J."} {"name": "Jack", "order": "FAMILY_FIRST", "family": "Jack", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "J."} +{"name": "Jack (M.A)", "family": "Jack", "suffix": "M.A", "initials": "J."} +{"name": "Jack (MA)", "family": "Jack", "suffix": "MA", "initials": "J."} {"name": "Jack I-V", "given": "Jack", "family": "I-V", "initials": "J. I."} {"name": "Jack I.-V.", "given": "Jack", "family": "I.-V.", "initials": "J. I."} {"name": "Jack M.A.", "given": "Jack", "suffix": "M.A.", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "J."} @@ -1534,6 +1542,7 @@ {"name": "Jaume Martí i Cabot", "given": "Jaume", "family": "Martí i Cabot", "initials": "J. M. C."} {"name": "Javier López Díaz", "given": "Javier", "middle": "López", "family": "Díaz", "initials": "J. L. D."} {"name": "Jean 'JD' Smith", "given": "Jean", "family": "Smith", "nickname": "JD", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "Jean \"Jd\" Smith"} +{"name": "Jean (M.A) Smith", "given": "Jean", "family": "Smith", "nickname": "M.A", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S."} {"name": "Jean DUPONT", "given": "Jean", "family": "DUPONT", "initials": "J. D.", "capitalized_force": "Jean Dupont"} {"name": "Jean Joseph Antoine Marie de Witte", "given": "Jean", "middle": "Joseph Antoine Marie", "family": "de Witte", "initials": "J. J. A. M. W."} {"name": "Jean Louis Charles d'Orléans-Longueville", "given": "Jean", "middle": "Louis Charles", "family": "d'Orléans-Longueville", "initials": "J. L. C. d.", "capitalized_force": "Jean Louis Charles D'Orléans-Longueville"} @@ -1548,6 +1557,7 @@ {"name": "Jean «Petit» Dupont", "given": "Jean", "family": "Dupont", "nickname": "Petit", "initials": "J. D."} {"name": "Jean-Dominique de Cassini", "given": "Jean-Dominique", "family": "de Cassini", "initials": "J. C."} {"name": "Jean-François Lefebvre de La Barre", "given": "Jean-François", "middle": "Lefebvre", "family": "de La Barre", "initials": "J. L. B.", "capitalized_force": "Jean-François Lefebvre de la Barre"} +{"name": "Jeffrey (JD) Bricken", "given": "Jeffrey", "family": "Bricken", "nickname": "JD", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. B.", "capitalized_force": "Jeffrey \"Jd\" Bricken"} {"name": "Jenny \"JJ\" Baker (Johnson)", "given": "Jenny", "family": "Baker", "nickname": "JJ Johnson", "initials": "J. B.", "capitalized_force": "Jenny \"Jj Johnson\" Baker"} {"name": "Jenny (Johnson) Baker", "given": "Jenny", "family": "Baker", "nickname": "Johnson", "initials": "J. B."} {"name": "Jenny Baker", "given": "Jenny", "family": "Baker", "initials": "J. B."} @@ -1654,11 +1664,20 @@ {"name": "John Q. Smith", "given": "John", "middle": "Q.", "family": "Smith", "initials": "J. Q. S."} {"name": "John Quincy Smith i", "given": "John", "middle": "Quincy", "family": "Smith", "suffix": "i", "initials": "J. Q. S.", "capitalized_force": "John Quincy Smith I"} {"name": "John Smith", "given": "John", "family": "Smith", "initials": "J. S."} +{"name": "John Smith (Ed)", "given": "John", "family": "Smith", "nickname": "Ed", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S."} +{"name": "John Smith (M.A)", "given": "John", "family": "Smith", "suffix": "M.A", "initials": "J. S."} {"name": "John Smith (M.A.)", "given": "John", "family": "Smith", "suffix": "M.A.", "initials": "J. S."} {"name": "John Smith (MA PhD)", "given": "John", "family": "Smith", "nickname": "MA PhD", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "John \"Ma PhD\" Smith"} -{"name": "John Smith (MA)", "given": "John", "family": "Smith", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "John \"Ma\" Smith"} +{"name": "John Smith (MA)", "given": "John", "family": "Smith", "suffix": "MA", "initials": "J. S."} +{"name": "John Smith (MA) Dr.", "title": "Dr.", "given": "John", "family": "Smith", "suffix": "MA", "initials": "J. S."} +{"name": "John Smith (MA) Ma", "given": "John", "middle": "Smith", "family": "Ma", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NAME", "SUFFIX_OR_NICKNAME"], "initials": "J. S. M.", "capitalized_force": "John \"Ma\" Smith Ma"} +{"name": "John Smith (MA) Ph. D.", "given": "John", "family": "Smith", "suffix": "MA Ph. D.", "initials": "J. S."} +{"name": "John Smith (MA) née Jones", "given": "John", "family": "Smith", "suffix": "MA", "maiden": "Jones", "initials": "J. S."} +{"name": "John Smith (MA), PhD", "given": "John", "family": "Smith", "suffix": "MA, PhD", "initials": "J. S."} +{"name": "John Smith (MA).", "given": "John", "middle": "Smith", "family": ".", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S. ..", "capitalized_force": "John \"Ma\" Smith ."} {"name": "John Smith (MA.)", "given": "John", "family": "Smith", "suffix": "MA.", "initials": "J. S."} {"name": "John Smith (MBA PhD.)", "given": "John", "family": "Smith", "suffix": "MBA PhD.", "initials": "J. S."} +{"name": "John Smith (Ma)", "given": "John", "family": "Smith", "nickname": "Ma", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S."} {"name": "John Smith (PhD MBA)", "given": "John", "family": "Smith", "nickname": "PhD MBA", "initials": "J. S.", "capitalized_force": "John \"PhD Mba\" Smith"} {"name": "John Smith 1.4", "given": "John", "middle": "Smith", "family": "1.4", "initials": "J. S. 1."} {"name": "John Smith B.Tech.", "given": "John", "family": "Smith", "suffix": "B.Tech.", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S.", "capitalized_force": "John Smith B.TECH."} @@ -1712,7 +1731,8 @@ {"name": "John Smith Ph. D.", "given": "John", "family": "Smith", "suffix": "Ph. D.", "initials": "J. S."} {"name": "John Smith Ph. D. Jones", "given": "John", "family": "Smith", "suffix": "Ph. D. Jones", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} {"name": "John Smith PhD", "given": "John", "family": "Smith", "suffix": "PhD", "initials": "J. S."} -{"name": "John Smith PhD (MA)", "given": "John", "family": "Smith", "suffix": "PhD", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "John \"Ma\" Smith PhD"} +{"name": "John Smith PhD (MA)", "given": "John", "family": "Smith", "suffix": "PhD MA", "initials": "J. S."} +{"name": "John Smith PhD (Ma)", "given": "John", "family": "Smith", "suffix": "PhD", "nickname": "Ma", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S."} {"name": "John Smith PhD Ed Ma", "given": "John", "family": "Smith", "suffix": "PhD Ed Ma", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S.", "capitalized_force": "John Smith PhD ED MA"} {"name": "John Smith PhD Jones", "given": "John", "family": "Smith", "suffix": "PhD Jones", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} {"name": "John Smith PhD MEng", "given": "John", "family": "Smith", "suffix": "PhD MEng", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S.", "capitalized_force": "John Smith PhD MENG"} @@ -1754,6 +1774,8 @@ {"name": "John Smith Xyz.", "given": "John", "middle": "Smith", "family": "Xyz.", "initials": "J. S. X."} {"name": "John Smith nee Jones R.A.I.", "given": "John", "family": "Smith", "suffix": "R.A.I.", "maiden": "Jones", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} {"name": "John Smith vi", "given": "John", "family": "Smith", "suffix": "vi", "initials": "J. S.", "capitalized_force": "John Smith VI"} +{"name": "John Smith «MA»", "given": "John", "family": "Smith", "suffix": "MA", "initials": "J. S."} +{"name": "John Smith 「MA」", "given": "John", "family": "Smith", "suffix": "MA", "initials": "J. S."} {"name": "John Smith 博. 士.", "given": "John", "middle": "Smith 博.", "family": "士.", "initials": "J. S. 博. 士."} {"name": "John Smith 田.中.", "given": "John", "middle": "Smith", "family": "田.中.", "initials": "J. S. 田."} {"name": "John Smith, 1.4", "given": "1.4", "family": "John Smith", "initials": "1. J. S."} @@ -3112,6 +3134,7 @@ {"name": "Slovák, Ján Matej", "given": "Ján", "middle": "Matej", "family": "Slovák", "initials": "J. M. S."} {"name": "Smith", "given": "Smith", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "S."} {"name": "Smith (Jones)", "family": "Smith", "nickname": "Jones", "initials": "S."} +{"name": "Smith (MA), John", "given": "John", "family": "Smith", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "J. S.", "capitalized_force": "John \"Ma\" Smith"} {"name": "Smith (née Jones)", "family": "Smith", "maiden": "Jones", "initials": "S."} {"name": "Smith , John", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Smith Cecilia Angela", "order": "FAMILY_FIRST", "given": "Cecilia", "middle": "Angela", "family": "Smith", "initials": "C. A. S."} @@ -3165,6 +3188,7 @@ {"name": "Smith, Jan van Prof.", "title": "Prof.", "given": "Jan", "family": "van Smith", "ambiguities": ["PARTICLE_OR_GIVEN"], "initials": "J. S."} {"name": "Smith, Jane Mrs", "title": "Mrs", "given": "Jane", "family": "Smith", "initials": "J. S."} {"name": "Smith, John", "given": "John", "family": "Smith", "initials": "J. S."} +{"name": "Smith, John (MA)", "given": "John", "family": "Smith", "suffix": "MA", "initials": "J. S."} {"name": "Smith, John ,", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Smith, John E, III, Jr", "given": "John", "middle": "E", "family": "Smith", "suffix": "III, Jr", "initials": "J. E. S."} {"name": "Smith, John I", "given": "John", "family": "Smith", "suffix": "I", "initials": "J. S."} @@ -3196,6 +3220,7 @@ {"name": "Smith, John X", "given": "John", "middle": "X", "family": "Smith", "initials": "J. X. S."} {"name": "Smith, John e, III, Jr", "given": "John", "middle": "e", "family": "Smith", "suffix": "III, Jr", "initials": "J. e. S."} {"name": "Smith, John van der Berg Prof.", "title": "Prof.", "given": "John", "middle": "van der Berg", "family": "Smith", "initials": "J. B. S."} +{"name": "Smith, John, Bob (MA)", "given": "John", "family": "Smith", "suffix": "Bob MA", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} {"name": "Smith, John, Extra, Jr.", "given": "John", "family": "Smith", "suffix": "Extra, Jr.", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} {"name": "Smith, John, MD - née Jones Smith", "given": "John", "family": "Smith", "suffix": "MD - née Jones Smith", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S.", "capitalized_force": "John Smith MD - Née Jones Smith"} {"name": "Smith, John, PhD - and MD", "given": "John", "family": "Smith", "suffix": "PhD - and MD", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} @@ -3203,6 +3228,7 @@ {"name": "Smith, John, PhD née Jones", "given": "John", "family": "Smith", "suffix": "PhD née Jones", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S.", "capitalized_force": "John Smith PhD Née Jones"} {"name": "Smith, John, Prof.", "title": "Prof.", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Smith, John, Puig - i Soler", "given": "John", "family": "Smith", "suffix": "Puig - i Soler", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S.", "capitalized_force": "John Smith Puig - I Soler"} +{"name": "Smith, John, X.Y.Z. (MA)", "given": "John", "family": "Smith", "suffix": "X.Y.Z. MA", "initials": "J. S."} {"name": "Smith, John, and", "given": "John", "family": "Smith", "suffix": "and", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} {"name": "Smith, John; MD", "given": "John;", "family": "Smith", "suffix": "MD", "initials": "J. S."} {"name": "Smith, Jr.", "family": "Smith", "suffix": "Jr.", "initials": "S."} @@ -3770,6 +3796,7 @@ {"name": "john j v", "given": "john", "middle": "j", "family": "v", "initials": "j. j. v.", "capitalized": "John J V"} {"name": "john j v, PhD", "given": "john", "family": "j", "suffix": "v, PhD", "initials": "j. j.", "capitalized": "John J V, PhD"} {"name": "john smith", "given": "john", "family": "smith", "initials": "j. s.", "capitalized": "John Smith"} +{"name": "john smith (MA)", "given": "john", "family": "smith", "nickname": "MA", "ambiguities": ["SUFFIX_OR_NICKNAME"], "initials": "j. s.", "capitalized_force": "John \"Ma\" Smith"} {"name": "john smith EdD", "given": "john", "family": "smith", "suffix": "EdD", "initials": "j. s.", "capitalized": "John Smith EdD", "capitalized_force": "John Smith EDD"} {"name": "john smith MA", "given": "john", "family": "smith", "suffix": "MA", "initials": "j. s.", "capitalized": "John Smith MA"} {"name": "john smith MEng", "given": "john", "middle": "smith", "family": "MEng", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s. M.", "capitalized_force": "John Smith Meng"}