Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/design/decisions/H9-title-phrase.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,9 @@
- MEASURED (2026-10-26, against master `12384fae`, py3.11, every local check with `--base`; the shape grid, 26,460 parses, by the design-docs review). No corpus holds a trailing `Wing.` or a full name before `, Wing`. fec-individuals 14 moved, every one a `FAMILY, WING …` donor now read with given 'WING' (`AA, WING` 7, `AA, WING AA…` 6, `AA, WING A. MR.` 1). The committed corpora move one name, this rule's former Accepted example `Lau, Wing` (`snapshot.py`); CLDR and GND agreement unchanged (`cldr_agreement.py`). PSC (1,039,174 scored, the fetch dated 2026-10-22), ctgov-officials and tse-donors do not move: PSC scores its names in spoken order, which decision 10 already reads. Spelling consistency (mechanisms.md#PAIRED-SPELLINGS-PER-CHANGE, per person, against master): jrc-names 0 now alike, 0 apart, of 218,822 persons with two or more spellings; nepal-candidates 0 and 0 of 126,981 pairs. No spelling in either writes `Wing` after a comma.
- THE BOUNDARY LEFT. The modifier test takes any title after the word as its head, a salutation too, so `LAU, WING MR` keeps title 'WING MR' where `LAU, DEAN MR` reads given 'DEAN'. No FEC donor writes it: the one with a salutation writes an initial between (`AA, WING A. MR.`), which breaks the run, and reads right. Pinned as a case row rather than narrowed.
Recompute: `uv run python tools/differential/snapshot.py --local fec-individuals --base <checkout of this commit's parent>` (and `ctgov-officials`, `tse-donors`), `uv run python tools/differential/local_agreement.py psc-individuals --base <the same checkout>`.
- **2026-10-26 #752, phase 2j, `mahdi` and `bard` (#867) — THE SECOND TITLE BATCH: TWO TITLES WRITTEN BEFORE A NAME AS GIVEN NAMES, AND AS EPITHETS WITH THEIR ARTICLE.** The batch decisions/vocabulary-model's 2026-10-24 #752 (Derek) entry put second, `bard` joining `mahdi` from #867. Each gains a dated `given` reading, so each leaves `titles` for `titles_ambiguous` and is a title only inside a title phrase. No listed phrase holds either, so neither reports (the report's reach, above): `Mahdi Karimi`, `Mr Mahdi Karimi`, `Karimi, Mahdi` and `Bard Larsen` read the given name, unreported, where 1.4.0 through 2.6.0 read title 'Mahdi' or 'Bard' (and `Mr Mahdi`). Unlike `wing`, neither is a modifier, so the contested word's partners apply: a modifier or a style before it, or `of` after it (`Grand Mahdi Karimi` and `Mahdi of Sudan` keep the title; `Mahdi Grand Karimi` does not). Like every word H9 reads, each now reads as a name word wherever no phrase holds it: ending the name (`John Karimi Mahdi.` → family 'Mahdi.', as 1.4.0 through 2.2.0 read it, where 2.3.0 through 2.6.0 read a title), alone after a full name and a comma (`John Karimi, Mahdi` → given 'Mahdi', family 'John Karimi', as `John Smith, Dean` reads), and past a second comma (`Karimi, John, Mahdi` → suffix, `comma-structure`, as C2 reads `Smith, John, Dean`).
- WHAT THE TITLE READINGS SERVED, AND THE COST. `mahdi` is a messianic title written as an epithet, never as an honorific before a person's own name; `bard`, from the 2014 title list with no recorded reason (#867), is mostly the Norwegian given name `Bård` with its accent dropped (in jrc-names, 31 of 36 persons with a spelling that starts `Bard` also have one that starts `Bård`). The epithet glued or alone reads the same in 1.4.0, 2.6.0 and this tree (`Muhammad Ahmad al-Mahdi` family 'al-Mahdi', `The Mahdi` given 'The'), but written with its article before a name or after a comma it read as the title in every release and now reads as the given name: `The Mahdi Muhammad Ahmad` → given 'The Mahdi', `Muhammad Ahmad, the Mahdi` → given 'the Mahdi', family 'Muhammad Ahmad', and the same for `The Bard William Shakespeare` and `William Shakespeare, the Bard` (found by the design-docs review). No local corpus writes either shape: 0 spellings holding `the Mahdi` or `the Bard` in jrc-names, fec-individuals, ctgov-officials, psc-individuals or ukec-individuals (2026-10-26), against the 79 PSC records and 26 ctgov and FEC names the given reading fixes. Recorded in rules/H9's Accepted clause. Under both family-first orders the joined epithet before a name is the family name (`The Mahdi Muhammad Ahmad` → family 'The Mahdi'), the comma shapes the given name under all three.
- THE BECAUSE, WIDENED. rules/H9's Because had said a word after a comma is read as written save a modifier also borne as a given name, wrong for `dean` since phase 2i (`John Smith, Dean` → given 'Dean'); it now names every word borne as a given name, and the claim rests on what was measured there: `dean`'s FEC listing forms (phase 2i), `wing`'s 14 and `mahdi`'s and `bard`'s 17 FEC and 6 ctgov names, every one the listing form's given name. `justice`, `major`, `marshal` and `grace` alone after a comma were not measured, and `John Smith, Major`, a rank in a listing, is the shape that would test the claim for a later batch.
- THE COUNTS, recorded on the words (`uv run python tools/name_usage.py Mahdi Bard Bård`, 2026-10-26): `mahdi` given US SSA 2,398 births, Wikidata 141 (a lower bound; Iran 43, Iraq 9), nam_dict band 5 in Arabia/Persia, family Census 1,241, Wikidata 13; `bard` given US SSA 218 (SSA folds `Bård`), Wikidata 10 and 107 more spelled `Bård`, nam_dict band 5 in Norway as `Bård`, family Census 5,953, Wikidata 100. PSC's filers put `Mahdi` in the forename box 84 times as its first word and never in the title box (#853, #867).
- MEASURED (2026-10-26, against master `32049ade`, py3.11, every local check with `--base`). PSC (1,039,174 scored, the fetch dated 2026-10-22): 79 records fixed, none broken, 6 changed, the 6 still disagreeing only because the filer typed two words in the surname box, which the default order reads as middle and family (`Mr MAHDI Aa Aa`, `Aa MAHDI Aa Aa`). ctgov-officials 6 moved, each a `Mahdi Aa, CRED` shape now read with given 'Mahdi'; fec-individuals 20, all fixes: `AA, MAHDI` 10, `AA, BARD` 6, and one each of `AA AA, MAHDI`, `AA, BARD AA`, `AA, BARD A. T` and `BARD AA, AA`, whose family part keeps `BARD` as a surname where H6 had peeled it as a title. The committed corpora, CLDR and GND agreement, and tse-donors do not move: no committed corpus held either word before this change, whose rules examples add two. Spelling consistency (mechanisms.md#PAIRED-SPELLINGS-PER-CHANGE, per person, against master): jrc-names 44 persons now alike and none apart, of 218,822 with two or more spellings, the spellings that read the word as a title and those that did not; nepal-candidates 0 and 0.
Recompute: `uv run python tools/differential/local_agreement.py psc-individuals --base <checkout of this commit's parent>`, and `uv run python tools/differential/snapshot.py --local ctgov-officials --base <the same checkout>` (and `fec-individuals`, `tse-donors`).
1 change: 1 addition & 0 deletions docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ Open: #752 (phase 2, derived ambiguity; its sub-phases 2a to 2m, each its own PR
- `dean`, `justice` and `major` gain an office or military reading beside a dated `given` one; `justice`'s exclusion (#606) is replaced by the reading. The batches of 2j declare the rest.
Adding the four fields changes `Lexicon`'s pickle layout, as phase 2h did.
- **2026-10-26 #752, phase 2i (Derek) — `attaches` MAY NAME A TITLE WORD AS WELL AS A KIND, AND `grace` AND `marshal` JOIN THE CONTESTED TITLES.** Amends the `attaches` bullet of the 2026-10-25 #752, phase 2i entry: a kind reaches every word that later joins it, and once `marshal` became a contested military word the military kind made `Sgt Marshal Smith` a title phrase. A list entry that is no kind must be a word with a title reading (a build check, its mutation in tests/v2/test_vocab.py). `major` attaches to `general` and `sergeant` and `sgt` to `major`. `grace` (style) and `marshal` (military, its #606 exclusion replaced) carry dated `given` readings. The measurement and Derek's answers are decisions/H9's entry of this date.
- **2026-10-26 #752, phase 2j, `mahdi` and `bard` (#867) — THE SECOND TITLE BATCH: both words have a dated `given` reading.** `Mahdi Karimi`, `Mr Mahdi Karimi`, `Karimi, Mahdi` and `Bard Larsen` read the given name, where 1.4.0 through 2.6.0 read a title; no listed phrase holds either word, so nothing is reported. The measurement is decisions/H9's entry of this date.
- **2026-10-26 #752, phase 2j, `wing` — THE FIRST TITLE BATCH: `wing` HAS A DATED `given` READING.** The order is the 2026-10-24 #752 (Derek) entry's. Decision 10 had already read `wing` as a name at the head of the name, so the reading reaches the parts decision 10 left as listed, after a comma and ending the name: `Lau, Wing Kai` reads given 'Wing', and `John Smith Wing.` family 'Wing.'. The rule and the measurement are decisions/H9's entry of this date.
- **2026-10-26 (Derek), #785 — PHASE 2g NARROWED: a capitals member with a name word behind it, and no family comma, does not start the credential's run**, the name word being evidence of initials; `Maria Ana MA Silva`, `Jean Pierre MA DUPONT` and `Mary Jane MA King` keep the readings 1.4.0 through 2.6.0 gave, and `John Smith MA Jones` keeping family 'Jones' is the cost. The record is decisions/S6's entry of this date.
- **2026-10-26 #789, #752, phase 2k — A GIVEN NAME WRITTEN SHORT IS A `given` READING WITH `same_as`, AND IT OUTRANKS rules/H2's SHAPE.** Carries out item 14 of the 2026-10-23 #752 entry above and the second half of #789; the first half, bare `wm` out of the titles, is the 2026-10-22 #789 entry. SUPERSEDES: in the README's property table, `same_as` as a property of every category but `name`. What changes in the vocabulary:
Expand Down
9 changes: 8 additions & 1 deletion docs/design/rules/H9-title-phrase.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
"General Major John Smith" → given="Major"
"Grace Dean Smith" → given="Grace"
"Sgt Marshal Smith" → given="Marshal"
"Mr Mahdi Karimi" → given="Mahdi"
```
- **A modifier in the leading run of titles before the first comma** has its phrase in a head title after it in that run, in `of` after it, or in the object of `of` or `the` there. A modifier outside that run, ending the name or after a comma, is read as any title is, save one that may be a given name, which is read everywhere as it is in that run.
```
Expand All @@ -40,14 +41,15 @@
"Mr Dean Smith" → ambiguities=("title-or-name",)
"Dean Smith, MD" → ambiguities=("title-or-name",)
"Grace Smith" → ambiguities=()
"Bard Larsen" → ambiguities=()
"GRACE SMITH" → ambiguities=()
"SMITH, DEAN MR." → ambiguities=()
", Dean" → ambiguities=()
"Major General John Smith" → ambiguities=()
"John Dean Smith" → ambiguities=()
```

**Because:** a title addresses the name after it, so the front of a name has no count to read, and what tells the person "Dean Smith" from the office in "Dean of Students" is the phrase the word stands in. A word that only qualifies a head title, as wing qualifies commander, says nothing about the person on its own. After a comma the part is read as written there, as decision 10 measured: a lone title word in a part after the name is the person's role, and reading it as a name reversed the name, save a modifier also borne as a given name, which without its head is far more often the listing form's given name, or at the end a surname, than a role.
**Because:** a title addresses the name after it, so the front of a name has no count to read, and what tells the person "Dean Smith" from the office in "Dean of Students" is the phrase the word stands in. A word that only qualifies a head title, as wing qualifies commander, says nothing about the person on its own. After a comma the part is read as written there, as decision 10 measured: a lone title word in a part after the name is the person's role, and reading it as a name reversed the name, save a word also borne as a given name, which the name data measured so far writes there as the listing form's given name, or at the end as a surname, rather than a role.

```
"Dean Smith" → given="Dean"
Expand Down Expand Up @@ -103,6 +105,11 @@
"The Hon. Wing Lau" → title="The Hon. Wing"
"John Smith Wing. Jr." → title="Wing."
```
- An epithet that is also a given name, written with its article before the name or after a comma, reads as a name word, the article joining it; no name data at hand writes either shape.
```
"The Mahdi Muhammad Ahmad" → given="The Mahdi"
"William Shakespeare, the Bard" → given="the Bard"
```
- Once a partner has made the word a title, a coordinating join behind it takes the next word into the title too.
```
"Chief Dean and Mary Smith" → title="Chief Dean and Mary"
Expand Down
1 change: 1 addition & 0 deletions docs/release_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Release Log
- **Read the Brazilian agnomens and French fils as suffixes where the name has a word to spare.** ``Pedro Alves Filho`` and ``Alexandre Dumas fils`` give last ``Alves`` and ``Dumas``, suffix ``Filho`` and ``fils``, where 1.4.0 through 2.6.0 gave middle ``Alves`` and ``Dumas``, last ``Filho`` and ``fils``; ``Neto``, ``Netto``, ``Sobrinho`` and the feminines ``Filha``, ``Neta``, ``Sobrinha`` read the same way. Each is a surname too, so a name of two words keeps it as the last name (``Ana Neto``, ``PEDRO FILHO``) and ``Fils`` in Title case in a mixed-case name stays the last name (``Marie Fils``), the parse reports the choice as a ``suffix-or-name`` ambiguity. ``Lexicon.suffix_words_ambiguous`` lists the words and ``Lexicon.agnomens`` the ones read by the word count alone. :issue:`843`, :issue:`752`
- **Read a credential that is also a name, written in capitals in a mixed-case name, as settled.** ``John Smith MA`` reports no ``suffix-or-name`` ambiguity, where 2.0.0 through 2.6.0 reported one, and ``Smith, John MA Jones`` gives suffix ``MA Jones``, where 2.0.0 through 2.6.0 gave middle ``MA Jones`` and 1.4.0 middle ``Jones``, suffix ``MA``. A name written in one case still reports (``JOHN SMITH MA``), and with no comma a name word behind the capitals keeps them a name word, since they may be initials (``Maria Ana MA Silva`` keeps last ``Silva``). :issue:`785`, :issue:`752`
- **Read a title that is also a given name as a title only inside a title phrase.** ``Chief Justice John Roberts`` gives title ``Chief Justice``, first ``John``, where 1.4.0 through 2.6.0 gave title ``Chief``, first ``Justice``; ``Field Marshal Bernard Montgomery``, ``Dean of Students Jane Smith`` and ``Major General John Smith`` give the phrase as the title too. Outside a phrase the word is still a name, so ``Dean Smith`` and ``Grace Smith`` give first ``Dean`` and ``Grace``; ``Dean Smith`` now reports a ``title-or-name`` ambiguity unless a comma has marked the given name (``Smith, Dean``), while ``Grace``, a title only in a phrase such as ``His Grace``, reports nothing. Listing the word as a title still makes it one everywhere (``Lexicon.default().add(titles={"dean"})``). :issue:`752`, :issue:`853`
- **Fix the given names Mahdi and Bard reading as titles.** ``Mahdi Karimi`` and ``Bard Larsen`` give first ``Mahdi`` and ``Bard``, where 1.4.0 through 2.6.0 gave title ``Mahdi`` and ``Bard`` with no first name; ``Mr Mahdi Karimi`` and ``Karimi, Mahdi`` read the same. Inside a title phrase either is still a title (``Mahdi of Sudan``), but the epithet with its article is not: ``Muhammad Ahmad, the Mahdi`` gives first ``the Mahdi``, last ``Muhammad Ahmad``, where 1.4.0 through 2.6.0 gave title ``the Mahdi``. :issue:`867`, :issue:`752`
- **Fix a lone title qualifier at the head of a name reading as a title.** ``Wing Lau`` and ``Mr Wing Lau`` give first ``Wing``, where 1.4.0 through 2.6.0 gave title ``Wing`` and ``Mr Wing``; a qualifier such as ``Wing``, ``Vice`` or ``Grand`` is a title only with a title after it (``Wing Commander Lau``). After a comma it is still a title (``John Smith, Vice``), except ``Wing``, which is also a given name: ``Lau, Wing Kai`` gives first ``Wing``, middle ``Kai``, where 1.4.0 through 2.6.0 gave title ``Wing``, first ``Kai``. ``John Smith Wing.`` gives last ``Wing.``, as 1.4.0 through 2.2.0 did, where 2.3.0 through 2.6.0 gave title ``Wing.``. Like ``Dean``, it reverses a full name before a comma (``John Smith, Wing`` gives first ``Wing``, last ``John Smith``). :issue:`752`
- **Read Chas., Wm. and the other given names written short as given names rather than titles.** ``Chas. Smith`` and ``Ma. Cristina Reyes`` give first ``Chas.`` and ``Ma.``, where 1.4.0 through 2.6.0 gave title ``Chas.`` and ``Ma.``; ``Thos.``, ``Jas.``, ``Geo.``, ``Robt.``, ``Benj.``, ``Jno.``, ``Saml.``, ``Edw.``, ``Richd.``, ``Wm.`` and the Filipino ``Jo.`` read the same way, behind a title too (``Mrs Ma. Liza Cruz`` gives title ``Mrs``, first ``Ma.``). An unlisted abbreviation in front is still a title (``Insp. Jane Morse``), and ``Lexicon.given_name_abbreviations`` lists the words. :issue:`789`, :issue:`752`
- **A Lexicon or Parser pickled by an earlier release no longer loads.** Loading one raises ``ValueError`` naming the new fields (``suffix_words_ambiguous``, ``agnomens``, ``titles_ambiguous``, ``title_modifiers``, ``title_styles``, ``title_phrases``, ``given_name_abbreviations``); re-pickle after upgrading. ``ParsedName`` and ``HumanName`` pickles still load. :issue:`752`
Expand Down
4 changes: 2 additions & 2 deletions nameparser/config/_compiled.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,6 @@
'banker',
'banner',
'bapak',
'bard',
'baron',
'baroness',
'barrister',
Expand Down Expand Up @@ -469,7 +468,6 @@
'magnate',
'maharajah',
'maharani',
'mahdi',
'maid',
'maj',
'majesty',
Expand Down Expand Up @@ -966,9 +964,11 @@
})

TITLES_AMBIGUOUS: frozenset[str] = frozenset({
'bard',
'dean',
'grace',
'justice',
'mahdi',
'major',
'marshal',
'wing',
Expand Down
4 changes: 3 additions & 1 deletion nameparser/vocab/ar.toml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,9 @@ religious = { note = "addresses by the TITLE, not the given name, and the tradit

[[word]]
text = "mahdi"
religious = {}
religious = { issue = 867, decided = 2026-10-26, note = "a messianic title, written as an epithet with the article or after a name ('the Mahdi', 'al-Mahdi'), not as an honorific before a person's own name; a title only inside a title phrase, so the epithet with its article before a name or after a comma reads as the given name (rules/H9's Accepted clause), its given reading the clash (rules/H9): 'Mahdi Karimi' and 'Mr Mahdi Karimi' read the given name, unreported, since no listed phrase holds it" }
given = { us_ssa = 2398, wikidata = 141, as_of = 2026-10-26, note = "US SSA: male births; Wikidata is a lower bound (Iran 43, Iraq 9); nam_dict band 5 in Arabia/Persia" }
family = { us_census_2010 = 1241, wikidata = 13, as_of = 2026-10-26 }

[[word]]
text = "mufti"
Expand Down
4 changes: 3 additions & 1 deletion nameparser/vocab/en.toml
Original file line number Diff line number Diff line change
Expand Up @@ -1226,7 +1226,9 @@ occupation = {}

[[word]]
text = "bard"
occupation = {}
occupation = { issue = 867, decided = 2026-10-26, note = "a title only inside a title phrase, its given reading the clash (rules/H9): 'Bard Larsen' reads the given name, unreported, since no listed phrase holds it" }
given = { us_ssa = 218, wikidata = 10, as_of = 2026-10-26, note = "mostly the Norwegian Bård with its accent dropped: US SSA folds it to BARD (male births); Wikidata is a lower bound, and holds 107 more spelled 'Bård' (Norway 87); nam_dict band 5 in Norway as 'Bård'" }
family = { us_census_2010 = 5953, wikidata = 100, as_of = 2026-10-26 }

[[word]]
text = "barrister"
Expand Down
Loading
Loading