Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/design/decisions/3-0-reevaluations.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,4 @@ Open: #753 (phase 3, language packs, where the three pack bullets land).
- (A) 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION, cross-referenced rather than parked again. Catalan/Polish `i` joins `conjunctions_ambiguous` beside `e`, so the marked subset now carries two letters and the 2026-08-18 bullet above — which counted six single-letter conjunctions and named the per-word-policy requirement — describes a set of seven, `i` included. Nothing in that bullet's argument moves: the subset IS the per-word policy the bullet asked for, made configurable without packs, and the pack-side half is still what the 2026-09-13 SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY? (A) bullet above asks (this pointer named only its date until 2026-10-22, which two bullets share), since a `ca` or `pl` caller still has to write `Lexicon.default().remove(conjunctions_ambiguous={"i"})` by hand. See `decisions/P3`'s 2026-09-20 bullet.
- (B) 2026-09-23 #459 — THE SHIM RAISES on a v1 config it once accepted: a `capitalization_exceptions` value that does not spell its key's letters and digits raises `ValueError` at the first parse (decisions/R4, 2026-09-23). Cited against the shim's never-raise rule and taken anyway, so it is recorded here: the raise stands on its own — a case mask has no reading for a value that re-spells its word, and the core `Lexicon` raises the same way with no shim involved — and only where it LANDS for a v1 caller, at the first parse rather than at construction, is shim-era. 3.0 has nothing left to decide.
- (B) 2026-10-02 #541 — THE SHIM DROPS, with a warning, every v1 `Constants` entry v1 could never match (decisions/R4, 2026-10-02). Cited from 1.4 parity, and it dies with the shim: the core `Lexicon` normalizes edge whitespace away and has no v1 reading to reproduce, so 3.0 has nothing left to decide.
- (B) 2026-10-26 #752 — THE SHIM KEEPS 2.6.0'S TITLE LIST as v1's default `titles`, frozen in `nameparser/config/_titles_2_6_0.py`, so a title a later release made phrase-only (decisions/H9, 2026-10-26, the v1 title list) is still listed and removable. Cited from v1 parity, and it dies with the shim: the core `Lexicon` keeps the two sets apart and has no v1 list to reproduce, so 3.0 deletes the file with the facade and decides nothing.
6 changes: 4 additions & 2 deletions docs/design/decisions/H9-title-phrase.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- `grace` (a style) and `marshal` (a rank) are titles that may be given names, `sergeant` attaches to the word `major` and `major` to `general` (`attaches` naming a word where a kind would reach too far): `His Grace John Smith`, `Field Marshal Bernard Montgomery` and `Sergeant Major John Smith` read the title; `Mr Justice Smith` keeps given 'Justice', no exception behind a salutation — 2026-10-26 #752, phase 2i (Derek).
- Its partner is a modifier or a style before it, a title before it that attaches to it or a title after it that it attaches to (`attaches`), or `of` after it; a salutation is never one, nor any other title before it (#853's `Mr Dean Smith`), nor a title after it it does not attach to (`SMITH, DEAN MR.`) — 2026-10-25 #752.
- A modifier in the leading run of titles before the first comma is a title only with a head title after it in that run, `of` after it, or in the object of `of` or `the` there; outside that run (ending the name, H5's, or after a comma) it reads as any title does, the given part after a family comma included — 2026-10-25 #752; save a modifier that may be a given name, which the run's test reads in every part (`Lau, Wing Kai` → given 'Wing', `John Smith Wing.` → family 'Wing.') — 2026-10-26 #752, phase 2j, `wing`.
- A title that may be a given name which a caller lists in `titles` is a title wherever it stands, which keeps the 1.4 recipe `c.titles.add("dean")`; a v1 caller who empties `titles` turns the phrase reading off with them, while one who replaces the list keeps it (`c.titles.clear(); c.titles.add('dr')` still reads `Dean of Students` as the title), v1 holding no knob for the set — 2026-10-25 #752.
- A title that may be a given name which a caller lists in `titles` is a title wherever it stands, which keeps the 1.4 recipe `c.titles.add("dean")`; a v1 caller who empties `titles` turns the phrase reading off with them, while one who replaces the list keeps it (`c.titles.clear(); c.titles.add('dr')` still reads `Dean of Students` as the title), v1 holding no knob for the set — 2026-10-25 #752. A title 2.6.0 listed that a later batch made a title only inside a phrase stays in v1's default `titles`, read by the phrase while it is there and as no title once removed, so `c.titles.remove('king')` works as in 2.6.0; listing it in v1 cannot make it a title everywhere, which takes `Lexicon.default().add(titles={'king'})` — 2026-10-26 #752, the v1 title list.

## Log

Expand Down Expand Up @@ -59,7 +59,9 @@
- **2026-10-26 #752, phase 2j, `sultana` and `sultan` (Derek) — THE FIFTH AND SIXTH TITLE BATCHES, COMBINED.** The batches decisions/vocabulary-model's 2026-10-24 #752 (Derek) entry put fifth and sixth; measured apart and together, `sultana` costing nothing and `sultan` only the royal class accepted for `king` (the entry above), Derek combined them on 2026-10-26. Each gains a dated `given` reading (`sultan`'s on its tr.toml entry, the other two files' readings deriving the same marker) and is a title only inside a title phrase; no listed phrase holds either, so nothing is reported. `Sultana Begum`, `Mrs Sultana Begum`, `Sultan Khan`, `Mr Sultan Khan`, `Khan, Sultan` and `Sultan Bin Zayed` read the given name, where 1.4.0 through 2.6.0 read the title (and `Mrs Sultana`, `Mr Sultan`). A style keeps the title (`His Majesty Sultan Hassanal Bolkiah`), as does `of` (`Sultan of Brunei`), and the Ottoman writing after the name was already a name word (`Hurrem Sultan`). The other sites read as for `king`.
- THE COUNTS, recorded on the words (`uv run python tools/name_usage.py Sultana Sultan`, 2026-10-26): `sultana` given US SSA 275 births, Wikidata 19 (Bangladesh 5), nam_dict band 3 in Romania and Arabia/Persia, family Census 3,471, Wikidata 34; `sultan` given US SSA 2,396, Wikidata 151 (Turkey 28, Saudi Arabia 13), nam_dict band 7 in Turkey, family Census 3,363, Wikidata 106 (Ottoman Empire 26, the title written after the name).
- THE COST, `sultan`'s only: a sultan written with the bare title in front reads it as the given name (`Sultan Hassanal Bolkiah` → given 'Sultan', where 1.4.0 through 2.6.0 read title 'Sultan', given 'Hassanal'; so `Sultan Qaboos bin Said`), the class accepted for `king`; in PSC one filer typed `Sultan` in the title box before a name, the one broken record. Sized afterwards, at the review's prompting, in the news text of jrc-names: 135 persons (266 spellings) open with `Sultan` or `Sultana`, by a reading of each about 40 rulers (mostly Ottoman sultans and today's Gulf and Malay rulers), about 80 people whose given name is Sultan, and 13 mosques and other places, a cost about four times `king`'s 10 there.
- FOUND BY THE REVIEW, NOT FIXED HERE: A v1 `remove` OF A WORD 2.6.0 LISTED NOW RAISES. A title that gains a given reading compiles out of `TITLES` (the 2026-10-25 #752, phase 2i entry of decisions/vocabulary-model), so `Constants().titles` no longer holds it and v1's strict `remove` raises `KeyError`: `c.titles.remove('sultan')`, which 2.6.0 accepted (and which then read `Sultan Khan` with first 'Sultan'), raises on this tree, and so does it for every word phase 2j moved out of the titles 2.6.0 listed (`wing`, `mahdi`, `bard`, `buddha`, `siddha`, `gyani`, `king`, `sultana`); phase 2i's own words were never in 2.6.0's titles, so removing them raised before too. The shim is not to raise on a config v1 accepted (AGENTS.md), so this is a defect of the batches, left for its own change before 2.7.0 ships.
- FOUND BY THE REVIEW, NOT FIXED HERE (fixed by the 2026-10-26 #752, the v1 title list entry below): A v1 `remove` OF A WORD 2.6.0 LISTED NOW RAISES. A title that gains a given reading compiles out of `TITLES` (the 2026-10-25 #752, phase 2i entry of decisions/vocabulary-model), so `Constants().titles` no longer holds it and v1's strict `remove` raises `KeyError`: `c.titles.remove('sultan')`, which 2.6.0 accepted (and which then read `Sultan Khan` with first 'Sultan'), raises on this tree, and so does it for every word phase 2j moved out of the titles 2.6.0 listed (`wing`, `mahdi`, `bard`, `buddha`, `siddha`, `gyani`, `king`, `sultana`); phase 2i's own words were never in 2.6.0's titles, so removing them raised before too. The shim is not to raise on a config v1 accepted (AGENTS.md), so this is a defect of the batches, left for its own change before 2.7.0 ships.
- NOT REACHED, for `sheikh`'s batch. `sheikh` is a style, so under rules/H9 it partners a contested word behind it: `Sheikh Sultan Khan` keeps title 'Sheikh Sultan', as master reads it and as `Sheikh Dean Smith` reads, where in Gulf names the word is the person's given name.
- MEASURED (2026-10-26, against master `6ba78d0d`, py3.11, every local check with `--base`). Both words: PSC (1,039,174 scored) 111 records fixed, 1 broken (the title box above), 6 changed, each still wrong for another reason, mostly a two-word surname box; ctgov-officials 41 moved; fec-individuals 21; tse-donors 1; spelling pairs (mechanisms.md#PAIRED-SPELLINGS-PER-CHANGE) jrc-names 9 persons and nepal-candidates 6 pairs now alike, none apart. `sultana` alone: PSC 28 fixed, none broken, 1 changed (a two-word surname box); ctgov 7, every one a fix, 6 of them `Aa Sultana, CRED` names whose credential master read as the given name (`Nasreen Sultana, MBBS` → given 'Nasreen', family 'Sultana', suffix 'MBBS', reported `suffix-or-name`); FEC 5 (`AA, SULTANA`); tse 1; nepal 1. The committed corpora and CLDR and GND agreement do not move.
Recompute: `uv run python tools/differential/local_agreement.py psc-individuals --base <checkout of this commit's parent>`, and `uv run python tools/differential/snapshot.py --local ctgov-officials --base <the same checkout>` (and `fec-individuals`, `tse-donors`).

- **2026-10-26 #752, the v1 title list — A v1 `Constants().titles` HOLDS EVERY TITLE 2.6.0 LISTED THAT IS STILL A TITLE OF EITHER KIND.** The review of phase 2j's fifth and sixth batches found that a word a batch moved from `titles` to `titles_ambiguous` also left v1's default `titles`, which the shim copied from TITLES: `Constants().titles.remove('king')` raised `KeyError` and `'king' in Constants().titles` was False, where 2.6.0 accepted the removal and answered True. Measured against the 2.6.0 wheel (`PYTHONSAFEPATH=1`, `nameparser.__file__` asserted on both sides), word by word over all nine v1 set attributes and the `capitalization_exceptions` keys: `titles` alone had lost words (`bard`, `buddha`, `gyani`, `king`, `mahdi`, `siddha`, `wing`, and `sultan` and `sultana` once the batch above landed), and `suffix_not_acronyms` had gained the seven agnomens and `fils` of phase 2h, a vocabulary addition its release bullet records. `suffix_acronyms_ambiguous` (derived in phase 2a), `given_name_abbreviations` (2k) and the phase 2h sets have no v1 manager that lost a word. The translation: `nameparser/config/_titles_2_6_0.py` freezes 2.6.0's TITLES, generated from the wheel and never edited, and v1's default `titles` is TITLES with every 2.6.0 title now in `titles_ambiguous` added back (`moved`). `Constants._build_snapshot` passes a moved word to `titles_ambiguous` while the caller keeps it, so a default `Constants` still translates to `Lexicon.default()`, and passes it to neither once removed, which turns its modifier, style and phrase readings off with it, as removing any title does: `c.titles.remove('king')` then reads `Hon. King Smith` as title 'Hon.', given 'King', 2.6.0's reading after the same removal. A word 2.6.0 never listed (`dean`, `justice`, `grace`, `major`, `marshal`) is not in `moved`, so `c.titles.add('dean')` still makes it a title everywhere. The gap, accepted because v1's one bit per word cannot carry three readings: adding a moved word in v1 restores the default, the phrase reading, and not 2.6.0's title everywhere, which a caller gets from the 2.0 API (`Lexicon.default().add(titles={'king'})`). Keyed on the compiled sets rather than a hand list, so a later batch that moves another 2.6.0 title (`sheikh`) is covered with no edit, as `sultan` and `sultana` were when the batch above merged in, and `tests/v2/test_config_shim.py` sweeps every moved word by the same derivation: membership, set operators, `remove`, `discard`, re-adding, a replaced list, pickle, deepcopy and copy, 37 of its 39 rows failing on master `6ba78d0d` (the two that pass are the non-empty guard and `dean`'s recipe), with rows added in review for a spelling Lexicon folds onto a moved word (`king。`, `wing。`: the moved test and the modifier, style and phrase intersections compare folded spellings, #582's rule, the second review finding that folding the first alone lost `Wing Commander`) and for a replaced list's unlisted words. The default parse does not move: the snapshot, the CLDR, GND and native agreement records and the facade case rows pass unchanged.
2 changes: 1 addition & 1 deletion docs/release_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Release Log
- **Fix a split Ph. D. in the middle of a name stopping short of the words after it.** ``John Smith Ph. D. Jones`` gives suffix ``Ph. D. Jones``, as ``John Smith PhD Jones`` does, where 1.4.0 through 2.6.0 gave last ``Jones``, suffix ``Ph. D.``; ``John Smith V Ph. D.`` gives last ``V``, as ``John Smith V PhD`` does, where 1.4.0 through 2.6.0 read ``V`` as a suffix. :issue:`827`
- **Read the Brazilian agnomens and French fils as suffixes where the name has a word to spare.** ``Pedro Alves Filho`` and ``Alexandre Dumas fils`` give last ``Alves`` and ``Dumas``, suffix ``Filho`` and ``fils``, where 1.4.0 through 2.6.0 gave middle ``Alves`` and ``Dumas``, last ``Filho`` and ``fils``; ``Neto``, ``Netto``, ``Sobrinho`` and the feminines ``Filha``, ``Neta``, ``Sobrinha`` read the same way. Each is a surname too, so a name of two words keeps it as the last name (``Ana Neto``, ``PEDRO FILHO``) and ``Fils`` in Title case in a mixed-case name stays the last name (``Marie Fils``), the parse reports the choice as a ``suffix-or-name`` ambiguity. ``Lexicon.suffix_words_ambiguous`` lists the words and ``Lexicon.agnomens`` the ones read by the word count alone. :issue:`843`, :issue:`752`
- **Read a credential that is also a name, written in capitals in a mixed-case name, as settled.** ``John Smith MA`` reports no ``suffix-or-name`` ambiguity, where 2.0.0 through 2.6.0 reported one, and ``Smith, John MA Jones`` gives suffix ``MA Jones``, where 2.0.0 through 2.6.0 gave middle ``MA Jones`` and 1.4.0 middle ``Jones``, suffix ``MA``. A name written in one case still reports (``JOHN SMITH MA``), and with no comma a name word behind the capitals keeps them a name word, since they may be initials (``Maria Ana MA Silva`` keeps last ``Silva``). :issue:`785`, :issue:`752`
- **Read a title that is also a given name as a title only inside a title phrase.** ``Chief Justice John Roberts`` gives title ``Chief Justice``, first ``John``, where 1.4.0 through 2.6.0 gave title ``Chief``, first ``Justice``; ``Field Marshal Bernard Montgomery``, ``Dean of Students Jane Smith`` and ``Major General John Smith`` give the phrase as the title too. Outside a phrase the word is still a name, so ``Dean Smith`` and ``Grace Smith`` give first ``Dean`` and ``Grace``; ``Dean Smith`` now reports a ``title-or-name`` ambiguity unless a comma has marked the given name (``Smith, Dean``), while ``Grace``, a title only in a phrase such as ``His Grace``, reports nothing. Listing the word as a title still makes it one everywhere (``Lexicon.default().add(titles={"dean"})``). :issue:`752`, :issue:`853`
- **Read a title that is also a given name as a title only inside a title phrase.** ``Chief Justice John Roberts`` gives title ``Chief Justice``, first ``John``, where 1.4.0 through 2.6.0 gave title ``Chief``, first ``Justice``; ``Field Marshal Bernard Montgomery``, ``Dean of Students Jane Smith`` and ``Major General John Smith`` give the phrase as the title too. Outside a phrase the word is still a name, so ``Dean Smith`` and ``Grace Smith`` give first ``Dean`` and ``Grace``; ``Dean Smith`` now reports a ``title-or-name`` ambiguity unless a comma has marked the given name (``Smith, Dean``), while ``Grace``, a title only in a phrase such as ``His Grace``, reports nothing. Listing the word as a title still makes it one everywhere (``Lexicon.default().add(titles={"dean"})``). A v1 ``Constants().titles`` still lists ``King``, ``Mahdi`` and the other titles 2.6.0 listed, and removing one reads it as a name everywhere, but adding one there no longer makes it a title outside a phrase; that takes the ``Lexicon`` spelling. :issue:`752`, :issue:`853`
- **Fix the given names Sultan and Sultana reading as titles.** ``Sultan Khan`` and ``Sultana Begum`` give first ``Sultan`` and ``Sultana``, where 1.4.0 through 2.6.0 gave title ``Sultan`` and ``Sultana`` with no first name; ``Mr Sultan Khan`` and ``Khan, Sultan`` read the same. A sultan written with the bare title reads it as a first name too (``Sultan Hassanal Bolkiah`` gives first ``Sultan``), where a style keeps it (``His Majesty Sultan Hassanal Bolkiah``). :issue:`752`, :issue:`853`
- **Fix the given name King reading as a title.** ``King Curtis`` and ``Mr King Smith`` give first ``King``, where 1.4.0 through 2.6.0 gave title ``King`` and ``Mr King``; ``SMITH, KING`` reads the same. A monarch written with the bare title reads it as a first name too (``King Henry`` gives first ``King``, last ``Henry``), where a style keeps it (``His Majesty King Henry``). :issue:`752`, :issue:`348`
- **Fix the Nepali given names Buddha, Siddha and Gyani reading as titles.** ``Buddha Bahadur Tamang`` and ``Gyani Maya Gurung`` give first ``Buddha`` and ``Gyani``, middle ``Bahadur`` and ``Maya``, where 1.4.0 through 2.6.0 gave title ``Buddha`` and ``Gyani``, first ``Bahadur`` and ``Maya``; ``Siddha Raj Joshi`` and ``Tamang, Buddha Bahadur`` read the same. ``Guru`` is still a title. The Sikh title written ``Gyani`` reads as a given name too (``Gyani Zail Singh``); its usual spelling ``Giani`` is still a title. :issue:`869`, :issue:`752`
Expand Down
Loading
Loading