diff --git a/docs/design/decisions/H9-title-phrase.md b/docs/design/decisions/H9-title-phrase.md index 705ddb710..064f9bd90 100644 --- a/docs/design/decisions/H9-title-phrase.md +++ b/docs/design/decisions/H9-title-phrase.md @@ -65,3 +65,9 @@ Recompute: `uv run python tools/differential/local_agreement.py psc-individuals --base `, and `uv run python tools/differential/snapshot.py --local ctgov-officials --base ` (and `fec-individuals`, `tse-donors`). - **2026-10-26 #752, the v1 title list — A v1 `Constants().titles` HOLDS EVERY TITLE 2.6.0 LISTED THAT IS STILL A TITLE OF EITHER KIND.** The review of phase 2j's fifth and sixth batches found that a word a batch moved from `titles` to `titles_ambiguous` also left v1's default `titles`, which the shim copied from TITLES: `Constants().titles.remove('king')` raised `KeyError` and `'king' in Constants().titles` was False, where 2.6.0 accepted the removal and answered True. Measured against the 2.6.0 wheel (`PYTHONSAFEPATH=1`, `nameparser.__file__` asserted on both sides), word by word over all nine v1 set attributes and the `capitalization_exceptions` keys: `titles` alone had lost words (`bard`, `buddha`, `gyani`, `king`, `mahdi`, `siddha`, `wing`, and `sultan` and `sultana` once the batch above landed), and `suffix_not_acronyms` had gained the seven agnomens and `fils` of phase 2h, a vocabulary addition its release bullet records. `suffix_acronyms_ambiguous` (derived in phase 2a), `given_name_abbreviations` (2k) and the phase 2h sets have no v1 manager that lost a word. The translation: `nameparser/config/_titles_2_6_0.py` freezes 2.6.0's TITLES, generated from the wheel and never edited, and v1's default `titles` is TITLES with every 2.6.0 title now in `titles_ambiguous` added back (`moved`). `Constants._build_snapshot` passes a moved word to `titles_ambiguous` while the caller keeps it, so a default `Constants` still translates to `Lexicon.default()`, and passes it to neither once removed, which turns its modifier, style and phrase readings off with it, as removing any title does: `c.titles.remove('king')` then reads `Hon. King Smith` as title 'Hon.', given 'King', 2.6.0's reading after the same removal. A word 2.6.0 never listed (`dean`, `justice`, `grace`, `major`, `marshal`) is not in `moved`, so `c.titles.add('dean')` still makes it a title everywhere. The gap, accepted because v1's one bit per word cannot carry three readings: adding a moved word in v1 restores the default, the phrase reading, and not 2.6.0's title everywhere, which a caller gets from the 2.0 API (`Lexicon.default().add(titles={'king'})`). Keyed on the compiled sets rather than a hand list, so a later batch that moves another 2.6.0 title (`sheikh`) is covered with no edit, as `sultan` and `sultana` were when the batch above merged in, and `tests/v2/test_config_shim.py` sweeps every moved word by the same derivation: membership, set operators, `remove`, `discard`, re-adding, a replaced list, pickle, deepcopy and copy, 37 of its 39 rows failing on master `6ba78d0d` (the two that pass are the non-empty guard and `dean`'s recipe), with rows added in review for a spelling Lexicon folds onto a moved word (`king。`, `wing。`: the moved test and the modifier, style and phrase intersections compare folded spellings, #582's rule, the second review finding that folding the first alone lost `Wing Commander`) and for a replaced list's unlisted words. The default parse does not move: the snapshot, the CLDR, GND and native agreement records and the facade case rows pass unchanged. +- **2026-10-26 #752, phase 2j, `shaikh` (Derek) — THE SEVENTH TITLE BATCH: THE FIRST STYLE OF THE BATCHES.** The batch decisions/vocabulary-model's 2026-10-24 #752 (Derek) entry put seventh, `sheikh` left a batch of its own. `shaikh` gains a dated `given` reading and is a title only inside a title phrase; no listed phrase holds it, so nothing is reported. `Shaikh Ahmed Khan`, `Mr Shaikh Ahmed` and `Khan, Shaikh` read given 'Shaikh', where 1.4.0 through 2.6.0 read title 'Shaikh' (and `Mr Shaikh`). A style keeps the title (`His Highness Shaikh Ahmed Khan`, as 1.4.0 read it), as does a modifier (`Grand Shaikh Ahmed`) or `of` (`Shaikh of Bahrain`); the period-marked spelling at the front is a title by H2's shape (`Shaikh. Ahmed`). The other sites read as for `king` (the entry above): before a family comma H6 no longer peels it (`Shaikh Smith, Ahmed` → family 'Shaikh Smith', 1.4.0's reading), ending the name (`Ahmed Khan Shaikh.` → family 'Shaikh.', 1.4.0's), alone after a full name and a comma, past a second comma, and under a family-first order the word before a name is the family. + - A STYLE AS A PARTNER. `shaikh` is a style (the second contested one, after `grace`), so where it is a title it partners a contested word behind it (`His Highness Shaikh Sultan Khan` → title 'His Highness Shaikh Sultan'); where it is read as a name word it partners nothing, being no title there (`Shaikh Sultan Khan` → given 'Shaikh', middle 'Sultan', where master read title 'Shaikh Sultan'). Nor does it partner where only H2's shape makes it a title, the partner test counting a contested word a title only where its own phrase did (`_vocab.title_readings`): `Shaikh. Sultan Khan` → title 'Shaikh.', given 'Sultan', where master and 1.4.0 read title 'Shaikh. Sultan', and `Shaikh. Dean Smith` → given 'Dean', now reported `title-or-name`, as 1.4.0 read it. Master read `Grace. Dean Smith` so already, while a modifier so written partners (`Wing. Dean Smith` → title 'Wing. Dean'); the review of this batch found the shape, which rules/H9's H2 bullet now states. No local corpus writes `Shaikh.` before another word. The 2026-10-26 `sultana` and `sultan` entry's note on `Sheikh Sultan Khan` stands for `sheikh`, still a title. + - THE COUNTS, recorded on the word (`uv run python tools/name_usage.py Shaikh`, 2026-10-26): given US SSA 41 male births, Wikidata none, nam_dict not listed; family Census 6,063 (asian/PI 89.61%), Wikidata 49 (India 7, Pakistan 5). Mostly a South Asian surname, which stands last and was a name word already; first in the name it is a given-name element, as PSC's filers write it (below). + - THE COST, accepted by Derek on 2026-10-27 ("accepted"), shown the share of rulers below beside `king`'s and `sultan`'s and that `His Highness` in front keeps the title: a ruler or a member of a ruling house written with the bare title in front reads it as the given name (`Shaikh Hamad bin Isa Al Khalifa` → given 'Shaikh', middle 'Hamad bin Isa', where 1.4.0 through 2.6.0 read title 'Shaikh', given 'Hamad'), the class accepted for `king` and `sultan`, here larger in news text and smaller in record data. In jrc-names 59 persons have a spelling opening with `Shaikh`: by a reading of each, about 38 are Gulf rulers and members of their houses (`Shaikh` is the Bahraini English press's spelling), about 13 clerics and tribal or political leaders, and about 8 people whose name opens with it. In PSC the forename box holds it first in 38 of the 41 moved records and the title box in 2, beside 3 title-box filings newly set aside (below). + - MEASURED (2026-10-26, against master `fb4cafac`, py3.11, every local check with `--base`). PSC (1,039,171 scored, 3 more title boxes set aside, each `Shaikh` filed as a title) 33 records fixed, 1 broken (`Shaikh` filed in the title box), 7 changed, each still wrong for another reason (among them a two-word surname box and a title-box filing); ctgov-officials 8 moved, 7 fixes (4 `Shaikh A Aa, CRED` read with the initial as the given name, 2 `Aa Shaikh, CRED, CRED` whose credential master read as the given name), 1 still reading its credential as the given name; fec-individuals 2, both fixes after a family comma (`AA, SHAIKH` and `AA, SHAIKH AA`: given 'SHAIKH' where master read the title); tse-donors and nepal-candidates none; spelling pairs (mechanisms.md#PAIRED-SPELLINGS-PER-CHANGE) jrc-names 0 persons now alike and 12 now apart, every one a person spelled both `Shaikh` and `Sheikh`, which `sheikh`'s batch is to rejoin. No committed corpus name held `shaikh` before the batch's three rule examples, so they and CLDR and GND agreement do not move, and could not have. A v1 `Constants().titles` keeps `shaikh` with no edit (the v1 title list entry above), its sweep in `tests/v2/test_config_shim.py` gaining six rows. + Recompute: `uv run python tools/differential/local_agreement.py psc-individuals --base `, and `uv run python tools/differential/snapshot.py --local ctgov-officials --base ` (and `fec-individuals`, `tse-donors`, `nepal-candidates`). diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index 88952799f..bed94e1d7 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -218,6 +218,7 @@ Open: #752 (phase 2, derived ambiguity; its sub-phases 2a to 2m, each its own PR - `dean`, `justice` and `major` gain an office or military reading beside a dated `given` one; `justice`'s exclusion (#606) is replaced by the reading. The batches of 2j declare the rest. Adding the four fields changes `Lexicon`'s pickle layout, as phase 2h did. - **2026-10-26 #752, phase 2i (Derek) — `attaches` MAY NAME A TITLE WORD AS WELL AS A KIND, AND `grace` AND `marshal` JOIN THE CONTESTED TITLES.** Amends the `attaches` bullet of the 2026-10-25 #752, phase 2i entry: a kind reaches every word that later joins it, and once `marshal` became a contested military word the military kind made `Sgt Marshal Smith` a title phrase. A list entry that is no kind must be a word with a title reading (a build check, its mutation in tests/v2/test_vocab.py). `major` attaches to `general` and `sergeant` and `sgt` to `major`. `grace` (style) and `marshal` (military, its #606 exclusion replaced) carry dated `given` readings. The measurement and Derek's answers are decisions/H9's entry of this date. +- **2026-10-26 #752, phase 2j, `shaikh` (Derek) — THE SEVENTH TITLE BATCH: `shaikh` has a dated `given` reading**, so the style is a title only inside a title phrase. `Shaikh Ahmed Khan` and `Mr Shaikh Ahmed` read given 'Shaikh'; a ruler written with the bare title does too (`Shaikh Hamad bin Isa`), the class accepted for `king` and `sultan`, the larger share of rulers Derek accepted on 2026-10-27. `sheikh` keeps its own batch. The measurement is decisions/H9's entry of this date. - **2026-10-26 #752, phase 2j, `sultana` and `sultan` (Derek) — THE FIFTH AND SIXTH TITLE BATCHES, IN ONE PR: both have dated `given` readings**, `sultan`'s on its tr.toml entry, so each is a title only inside a title phrase. `Sultana Begum` and `Sultan Khan` read the given name; a sultan written with the bare title does too (`Sultan Hassanal Bolkiah`), the cost accepted for `king`. Derek combined the two batches on 2026-10-26. The measurement is decisions/H9's entry of this date. - **2026-10-26 #752, phase 2j, `king` (Derek) — THE FOURTH TITLE BATCH: `king` has a dated `given` reading**, US SSA 37,890 births and Census rank 34 as a surname, so it is a title only inside a title phrase. `King Curtis` and `Mr King Smith` read given 'King'; a monarch written with the bare title does too (`King Henry`), the cost Derek accepted on 2026-10-26. `pope` replaces `king` as the worked example of a title that is also a surname wherever the rules and tests used it. The measurement is decisions/H9's entry of this date. - **2026-10-26 #752, phase 2j, `buddha`, `siddha` and `gyani` (#869) — THE THIRD TITLE BATCH: each has a declared `given` reading**, `siddha`'s and `gyani`'s without a count, which their notes explain: the first halves of Nepali compound given names, which no source outside nepal-candidates counts. `Buddha Bahadur Tamang` reads given 'Buddha', middle 'Bahadur'; `guru` stays a title for a `ne` pack (the 2026-10-24 #752 (Derek) entry). The measurement is decisions/H9's entry of this date. diff --git a/docs/design/rules/H9-title-phrase.md b/docs/design/rules/H9-title-phrase.md index ea17f1e9b..03397ea97 100644 --- a/docs/design/rules/H9-title-phrase.md +++ b/docs/design/rules/H9-title-phrase.md @@ -23,6 +23,7 @@ "Buddha Bahadur Tamang" → given="Buddha" "King Curtis" → given="King" "Sultana Begum" → given="Sultana" + "Mr Shaikh Ahmed" → given="Shaikh" ``` - **A modifier in the leading run of titles before the first comma** has its phrase in a head title after it in that run, in `of` after it, or in the object of `of` or `the` there. A modifier outside that run, ending the name or after a comma, is read as any title is, save one that may be a given name, which is read everywhere as it is in that run. ``` @@ -68,11 +69,12 @@ ``` "Hon. Wing Lau" → given="Wing" ``` -- **A period-marked word in the leading run of titles** (H2): the abbreviation shape makes it a title whatever the vocabulary says, save a given name written short or a word a connective joins (below), so the phrase test never reaches it and nothing is reported, at the front of the name or behind another title. +- **A period-marked word in the leading run of titles** (H2): the abbreviation shape makes it a title whatever the vocabulary says, save a given name written short or a word a connective joins (below), so the phrase test never reaches it and nothing is reported, at the front of the name or behind another title; a style that may be a given name, made a title by the shape alone, is no partner of the word behind it, which is then read and reported as at the front, where a modifier is (`Wing. Dean Smith` → title 'Wing. Dean'). ``` "Dean. Smith" → title="Dean." "Wing. Lau" → title="Wing." "Mr Buddha. Tamang" → title="Mr Buddha." + "Shaikh. Sultan Khan" → given="Sultan" ``` - **A title run** (H3, H4): a word this rule reads as a name word is no title word to the run, so the run stops before it and does not count it. ``` @@ -120,10 +122,11 @@ "Gyani Zail Singh" → given="Gyani" "Lord Buddha Tamang" → given="Buddha" ``` -- A monarch written with the bare title in front reads it as the given name, a style before it or `of` after it keeping the title; the record data at hand writes the word there almost only as a given name, and monarchs that way mostly in news text. +- A ruler written with the bare title in front reads it as the given name, a style before it or `of` after it keeping the title; the record data at hand writes the word there almost only as a given name, and rulers that way mostly in news text. ``` "King Henry" → given="King" "Sultan Hassanal Bolkiah" → given="Sultan" + "Shaikh Hamad bin Isa" → given="Shaikh" "His Majesty King Charles" → title="His Majesty King" ``` - Once a partner has made the word a title, a coordinating join behind it takes the next word into the title too. diff --git a/docs/release_log.rst b/docs/release_log.rst index 1d9673793..235e6dcbf 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -18,6 +18,7 @@ Release Log - **Read the Brazilian agnomens and French fils as suffixes where the name has a word to spare.** ``Pedro Alves Filho`` and ``Alexandre Dumas fils`` give last ``Alves`` and ``Dumas``, suffix ``Filho`` and ``fils``, where 1.4.0 through 2.6.0 gave middle ``Alves`` and ``Dumas``, last ``Filho`` and ``fils``; ``Neto``, ``Netto``, ``Sobrinho`` and the feminines ``Filha``, ``Neta``, ``Sobrinha`` read the same way. Each is a surname too, so a name of two words keeps it as the last name (``Ana Neto``, ``PEDRO FILHO``) and ``Fils`` in Title case in a mixed-case name stays the last name (``Marie Fils``), the parse reports the choice as a ``suffix-or-name`` ambiguity. ``Lexicon.suffix_words_ambiguous`` lists the words and ``Lexicon.agnomens`` the ones read by the word count alone. :issue:`843`, :issue:`752` - **Read a credential that is also a name, written in capitals in a mixed-case name, as settled.** ``John Smith MA`` reports no ``suffix-or-name`` ambiguity, where 2.0.0 through 2.6.0 reported one, and ``Smith, John MA Jones`` gives suffix ``MA Jones``, where 2.0.0 through 2.6.0 gave middle ``MA Jones`` and 1.4.0 middle ``Jones``, suffix ``MA``. A name written in one case still reports (``JOHN SMITH MA``), and with no comma a name word behind the capitals keeps them a name word, since they may be initials (``Maria Ana MA Silva`` keeps last ``Silva``). :issue:`785`, :issue:`752` - **Read a title that is also a given name as a title only inside a title phrase.** ``Chief Justice John Roberts`` gives title ``Chief Justice``, first ``John``, where 1.4.0 through 2.6.0 gave title ``Chief``, first ``Justice``; ``Field Marshal Bernard Montgomery``, ``Dean of Students Jane Smith`` and ``Major General John Smith`` give the phrase as the title too. Outside a phrase the word is still a name, so ``Dean Smith`` and ``Grace Smith`` give first ``Dean`` and ``Grace``; ``Dean Smith`` now reports a ``title-or-name`` ambiguity unless a comma has marked the given name (``Smith, Dean``), while ``Grace``, a title only in a phrase such as ``His Grace``, reports nothing. Listing the word as a title still makes it one everywhere (``Lexicon.default().add(titles={"dean"})``). A v1 ``Constants().titles`` still lists ``King``, ``Mahdi`` and the other titles 2.6.0 listed, and removing one reads it as a name everywhere, but adding one there no longer makes it a title outside a phrase; that takes the ``Lexicon`` spelling. :issue:`752`, :issue:`853` + - **Fix the name Shaikh reading as a title.** ``Shaikh Ahmed Khan`` and ``Mr Shaikh Ahmed`` give first ``Shaikh``, where 1.4.0 through 2.6.0 gave title ``Shaikh`` and ``Mr Shaikh``; ``Khan, Shaikh`` reads the same. A ruler written with the bare title reads it as a first name too (``Shaikh Hamad bin Isa`` gives first ``Shaikh``), where a style keeps it (``His Highness Shaikh Hamad``); the spellings ``Sheikh`` and ``Shekh`` are still titles. :issue:`752`, :issue:`853` - **Fix the given names Sultan and Sultana reading as titles.** ``Sultan Khan`` and ``Sultana Begum`` give first ``Sultan`` and ``Sultana``, where 1.4.0 through 2.6.0 gave title ``Sultan`` and ``Sultana`` with no first name; ``Mr Sultan Khan`` and ``Khan, Sultan`` read the same. A sultan written with the bare title reads it as a first name too (``Sultan Hassanal Bolkiah`` gives first ``Sultan``), where a style keeps it (``His Majesty Sultan Hassanal Bolkiah``). :issue:`752`, :issue:`853` - **Fix the given name King reading as a title.** ``King Curtis`` and ``Mr King Smith`` give first ``King``, where 1.4.0 through 2.6.0 gave title ``King`` and ``Mr King``; ``SMITH, KING`` reads the same. A monarch written with the bare title reads it as a first name too (``King Henry`` gives first ``King``, last ``Henry``), where a style keeps it (``His Majesty King Henry``). :issue:`752`, :issue:`348` - **Fix the Nepali given names Buddha, Siddha and Gyani reading as titles.** ``Buddha Bahadur Tamang`` and ``Gyani Maya Gurung`` give first ``Buddha`` and ``Gyani``, middle ``Bahadur`` and ``Maya``, where 1.4.0 through 2.6.0 gave title ``Buddha`` and ``Gyani``, first ``Bahadur`` and ``Maya``; ``Siddha Raj Joshi`` and ``Tamang, Buddha Bahadur`` read the same. ``Guru`` is still a title. The Sikh title written ``Gyani`` reads as a given name too (``Gyani Zail Singh``); its usual spelling ``Giani`` is still a title. :issue:`869`, :issue:`752` diff --git a/nameparser/config/_compiled.py b/nameparser/config/_compiled.py index 393d32898..7c5a7a6e7 100644 --- a/nameparser/config/_compiled.py +++ b/nameparser/config/_compiled.py @@ -673,7 +673,6 @@ 'sgtmaj', 'sgtmajmc', 'shaik', - 'shaikh', 'shayk', 'shaykh', 'sheik', @@ -968,6 +967,7 @@ 'mahdi', 'major', 'marshal', + 'shaikh', 'siddha', 'sultan', 'sultana', diff --git a/nameparser/vocab/ar.toml b/nameparser/vocab/ar.toml index 0b979b7ae..c6efc149d 100644 --- a/nameparser/vocab/ar.toml +++ b/nameparser/vocab/ar.toml @@ -18,7 +18,9 @@ style = { addresses = "given", gender = "m", same_as = "sheikh" } [[word]] text = "shaikh" -style = { addresses = "given", gender = "m", same_as = "sheikh" } +style = { addresses = "given", gender = "m", same_as = "sheikh", issue = 752, decided = 2026-10-26, note = "a title only inside a title phrase, its given reading the clash (rules/H9): 'Shaikh Ahmed' and 'Mr Shaikh Ahmed' read the given name, unreported, since no listed phrase holds it" } +given = { us_ssa = 41, as_of = 2026-10-26, note = "US SSA: male births; Wikidata holds no given-name statement for it, a lower bound" } +family = { us_census_2010 = 6063, wikidata = 49, as_of = 2026-10-26, note = "Census: asian/PI 89.61%; Wikidata: India 7, Pakistan 5" } [[word]] text = "shayk" diff --git a/tests/v2/cases.py b/tests/v2/cases.py index a0b136636..7a0fc7a71 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -12629,6 +12629,40 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Sheikh Sultan", "family": "Khan"}, notes="'sheikh' is a style, so it partners the word: master's " "reading, left for sheikh's own batch"), + Case("given_name_title_shaikh_at_the_head", "Shaikh Ahmed Khan", + {"given": "Shaikh", "middle": "Ahmed", "family": "Khan"}, + classification="fix(#752)", + notes="H9; read title 'Shaikh', given 'Ahmed' 1.4.0 through " + "2.6.0 (#752 phase 2j)"), + Case("given_name_title_shaikh_after_a_salutation", "Mr Shaikh Ahmed", + {"title": "Mr", "given": "Shaikh", "family": "Ahmed"}, + classification="fix(#853)", + notes="H9: a salutation is no partner; read title 'Mr Shaikh' " + "before"), + Case("given_name_title_shaikh_after_a_family_comma", "Khan, Shaikh", + {"given": "Shaikh", "family": "Khan"}, classification="fix(#752)", + notes="H9; read title 'Shaikh' before"), + Case("given_name_title_shaikh_before_a_family_comma", + "Shaikh Smith, Ahmed", + {"given": "Ahmed", "family": "Shaikh Smith"}, + notes="H6 peels only a title, so the surname keeps the word, as " + "1.4.0 read it; master read title 'Shaikh'"), + Case("shaikh_behind_a_style_keeps_the_title", + "His Highness Shaikh Ahmed Khan", + {"title": "His Highness Shaikh", "given": "Ahmed", "family": "Khan"}, + notes="H9: a style is a partner, as master and 1.4.0 read it"), + Case("ruler_with_a_bare_leading_shaikh", + "Shaikh Hamad bin Isa Al Khalifa", + {"given": "Shaikh", "middle": "Hamad bin Isa", + "family": "Al Khalifa"}, + classification="fix(#752)", + notes="H9's Accepted clause, the class accepted for 'king' and " + "'sultan'; read title 'Shaikh' before"), + Case("sultan_behind_a_bare_leading_shaikh", "Shaikh Sultan Khan", + {"given": "Shaikh", "middle": "Sultan", "family": "Khan"}, + classification="fix(#752)", + notes="H9: a style read as a name word is no title there, so it " + "partners nothing; read title 'Shaikh Sultan' before"), Case("given_name_modifier_after_a_style", "Hon. Wing Lau", {"title": "Hon.", "given": "Wing", "family": "Lau"}, notes="H9: the modifier's test, not a contested word's style " diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 9db2cd10c..16d910ff6 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -25,7 +25,7 @@ "H6": {"words": 316, "then_words": 79, "because_sentences": 3, "except": 4, "precedence": 0, "accepted": 2, "accepted_words": 100}, "H7": {"words": 343, "then_words": 274, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 54}, "H8": {"words": 431, "then_words": 68, "because_sentences": 3, "except": 3, "precedence": 3, "accepted": 3, "accepted_words": 128}, - "H9": {"words": 660, "then_words": 263, "because_sentences": 3, "except": 0, "precedence": 5, "accepted": 9, "accepted_words": 342}, + "H9": {"words": 704, "then_words": 263, "because_sentences": 3, "except": 0, "precedence": 5, "accepted": 9, "accepted_words": 342}, "M1": {"words": 231, "then_words": 120, "because_sentences": 2, "except": 1, "precedence": 2, "accepted": 0, "accepted_words": 0}, "M2": {"words": 714, "then_words": 435, "because_sentences": 1, "except": 4, "precedence": 1, "accepted": 6, "accepted_words": 209}, "M3": {"words": 234, "then_words": 84, "because_sentences": 2, "except": 2, "precedence": 2, "accepted": 1, "accepted_words": 80}, diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index ce71c6b6d..9e10baf06 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -645,6 +645,7 @@ "Mr Dean Smith" "Mr Justice Smith" "Mr Mahdi Karimi" +"Mr Shaikh Ahmed" "Mr Smith" "Mr Wing Lau" "Mr. Jack Jill" @@ -732,6 +733,8 @@ "Sean O'Connor" "Sergeant Major John Smith" "Sgt Marshal Smith" +"Shaikh Hamad bin Isa" +"Shaikh. Sultan Khan" "Sheik Abu Bakar" "Sheik abdul Jr Smith" "Sheik abdul salam" diff --git a/tools/differential/local_corpora.py b/tools/differential/local_corpora.py index 8bcc64ed9..2c904a525 100644 --- a/tools/differential/local_corpora.py +++ b/tools/differential/local_corpora.py @@ -677,7 +677,7 @@ class GroupCounts: #: The words of the titles, and of the suffixes, of a person some of #: whose spellings read one and others none, folded by word_key, #: counted once a person: two spellings reading two titles (Sheikh, - #: Shaikh) with no untitled spelling beside them add nothing. + #: Shekh) with no untitled spelling beside them add nothing. titles: Counter[str] suffixes: Counter[str] diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 55cf57372..ded919eaa 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -2582,6 +2582,7 @@ {"name": "Mr MA", "title": "Mr", "family": "MA", "initials": "M.", "capitalized_force": "Mr Ma"} {"name": "Mr Mahdi Karimi", "title": "Mr", "given": "Mahdi", "family": "Karimi", "initials": "M. K."} {"name": "Mr Md Karim Uddin", "title": "Mr", "given": "Md", "middle": "Karim", "family": "Uddin", "initials": "M. K. U."} +{"name": "Mr Shaikh Ahmed", "title": "Mr", "given": "Shaikh", "family": "Ahmed", "initials": "S. A."} {"name": "Mr Smith", "title": "Mr", "family": "Smith", "initials": "S."} {"name": "Mr Wing Lau", "title": "Mr", "given": "Wing", "family": "Lau", "initials": "W. L."} {"name": "Mr XXX", "title": "Mr", "family": "XXX", "initials": "X.", "capitalized_force": "Mr Xxx"} @@ -3075,6 +3076,8 @@ {"name": "Señora María García", "title": "Señora", "given": "María", "family": "García", "initials": "M. G."} {"name": "Sgt Marshal Smith", "title": "Sgt", "given": "Marshal", "family": "Smith", "ambiguities": ["TITLE_OR_NAME"], "alternatives": ["TITLE_OR_NAME: title as title (vocabulary)"], "initials": "M. S."} {"name": "Shahnon Ahmad", "given": "Shahnon", "family": "Ahmad", "initials": "S. A."} +{"name": "Shaikh Hamad bin Isa", "given": "Shaikh", "middle": "Hamad", "family": "bin Isa", "initials": "S. H. I."} +{"name": "Shaikh. Sultan Khan", "title": "Shaikh.", "given": "Sultan", "family": "Khan", "initials": "S. K."} {"name": "Shanti Gaonkar", "given": "Shanti", "family": "Gaonkar", "initials": "S. G."} {"name": "Shehu Muhammed", "given": "Shehu", "family": "Muhammed", "initials": "S. M."} {"name": "Sheik Abu Bakar", "title": "Sheik", "given": "Abu Bakar", "initials": "A. B."}