[derek73]

Phase 2m of #752: every ambiguity report of a fork now carries the reading the parse did not take, as Ambiguity.alternative. Targets 2.7. Token.reading, the reading taken, stays for 3.0 as Derek decided on 2026-10-26.

>>> parse("JOHN SMITH MA").ambiguities[0].alternative
Reading('name' as 'family', 'declared')
>>> parse("John Smith PhD Jones").ambiguities[0].alternative
Reading('name' as 'family', 'position')
>>> parse("Van Buren").ambiguities[0].alternative
Reading('particle' as 'family', 'vocabulary')

What ships

• Reading, a new public type (frozen, slotted, picklable, exported from nameparser with ReadingKind and ReadingSource). It has three fields:
  • kind: credential, generation, agnomen, post-nominal, title, particle, conjunction, initial or name.
  • role: the field the word would land in, with every other word of the name read as the parse read it.
  • source: vocabulary, declared (the vocabulary also lists the word as a name), shape (a roman numeral, dotted letters, an initial) or position (an ordinary word nothing lists).
• Ambiguity.alternative: Reading | None = None, declared compare=False and left out of comparison_key, so ambiguities and names compare as before. The three shape kinds (comma-structure, unbalanced-delimiter, segmentation) carry None.
• Every emitter fills it in at its decision site, about 40 of them across 10 pipeline modules. How each kind of reading gets its field:
  • A non-name reading names its field by what it is: credential, generation and agnomen → suffix, title → title, particle → family.
  • A name reading takes the field the positional read gives one more name word in that place. If the name piece in front would take the word in (a particle chain, a lone particle that doesn't open the part, or a bound given name whose join a family comma or a given-name title licenses), it takes that piece's final field instead. A never-given particle opening the part folds the word into the family (P1).
  • O5's lone word gets the other primary field, named from its final role.
  • Classify's connective and group's particle chain run before the name has positions, so they defer the field (_state.Deferred) and assign names it. A connective whose join would merge two pieces takes the first's place (JOSE E MARIA SANTOS → given); one inside its own piece (MARIA DA SILVA E SOUZA, now a case row), or in a three-word name where P3 (connectives) joins nothing (JOHN E SMITH → middle), keeps its own field. A report assign can't place gets no alternative rather than a guess. That happens in two shapes: a connective letter that opens its part (E SMITH JONES, now a case row), which has no word before it to join, so no field could hold the reading; and the chain in Freiherr von Berg Dr. and Ed. Prof.. rules/A1 and the docs state the first.
• Pickles. _guarded_setstate now accepts a missing field only if the field carries metadata={ADDED_IN: "2.7"} and has a default. Everything the narrow guard refused is still refused: a missing field without the marker (with or without a default, e.g. Token.tags), an unknown field, a renamed field. Each of those is a row in _REFUSED_SKEWS. I ran all seven rows against the pre-2.7 guard and every one raises there too. The admitted row and a real 2.6.0 pickle (tests/v2/data/parsedname_v26.pickle, made with the released wheel using the PYTHONSAFEPATH=1 recipe) fail under the old guard and load under the new one. I also checked by hand that pickles from the 2.0.0 and 2.3.0 wheels load.

Decisions for review

1. A fourth source, position. The RFC's "Readings on the parse" had three sources (vocabulary, name, shape). A word the credential run takes by position alone (Jones) has no declared name and no shape. Its name reading is just the reading any word has, so it needs a source of its own. That is what lets JOHN SMITH MA (declared) be told apart from John Smith PhD Jones (position) without a new ambiguity kind.
2. No languages field. It would need a per-word reading index compiled into _compiled.py. Measured with bytecode cached: about +0.35 ms on an 18.2 ms import, about 2% (1,790 words, 1,975 readings by word and kind), plus a _normalize call per report. The bigger problem is that the index describes the default TOML, not the Lexicon the parse used, so a pack's or a caller's words would report wrong languages. It waits for 3.0 alongside Token.reading, which needs the same index.
3. Enum values don't collide. ReadingKind uses post-nominal (not suffix) and ReadingSource uses declared (not name), so the repo's pinned rule that StrEnum values are disjoint still holds. The one exception is deliberate: ReadingKind.TITLE == Role.TITLE, and the disjointness test now covers all five enums with that overlap allowed.
4. The snapshot records alternatives, one string per report that carries one, kept in report order (GIVEN_OR_FAMILY: name as family (position)). Without that, a later change that moves an alternative's field would be invisible, and losing one of two identical alternatives still moves the line (DOE, JOHN MA JD holds two).
   • A header says what was recorded. The snapshot's first line, {"recorded": [...]}, names the keys its recording tree could record. A comparison leaves out a key either side did not record, judged by that header rather than by whether any row holds one. A snapshot without the header (a tag's, or a local corpus's from before this change) records no alternatives.
   • compare.py --baseline follows the same rule. Its worker says whether the wheel has Ambiguity.alternative. Against a release that doesn't, the key is left out of both sides, and the report says so.
5. Accepted limits. A listed suffix word other than a generation or agnomen reads as post-nominal, so jr and esq look alike; only H4's input of nothing but post-nominals reaches this. Also, because the field holds every other word fixed, it differs from a re-parse with a plain stand-in word in 14 of the sweep's (row, order) pairs. In 11, the stand-in changes a neighbour's reading (John Smith VI Jones). In 3, it is Andrew Perkins (X.Y.Z.): rules/S1 reads that dotted content as if written bare, so the name reading is the bare word's field (family), while a plain word in brackets stays a nickname.

Measurements (2026-10-26, against master 957509d; re-taken against 6ba78d0 after the review fixes)

• No reading moves.

  • Committed snapshot: matched before re-recording; the re-record adds the alternatives key to 969 of 6,476 rows and changes nothing else.
  • Every agreement record (CLDR, GND, Wikidata, conventions, native, both locale views) matches.
  • Local corpora with --base master: all seven snapshots (ctgov-officials, fec-individuals, jrc-names, nepal-candidates, tse-donors, tw-donors, ukec-individuals) and the PSC agreement match the base.
  • The one new snapshot name is the case row E SMITH JONES; it reads as 1.4.0 does.

• Frames. The reference row (_CALL_BASELINE, tools/perf/call_count.py) keeps 305 and Smith, John keeps 158. Over the snapshot's names, each of the 789 names with one report pays 2 to 13 frames more (JOHN SMITH MA 215 → 223). 7 of the 4,840 names with no report pay 2 to 7, for a report that is made and then withdrawn. The serial benchmark band passes.

• Field test. test_an_alternative_names_the_field_the_other_reading_gives runs every case row under all three orders. A name reading is checked by re-parsing with a plain stand-in word in its place: 436 reports checked, 299 skipped because the stand-in still lands in the taken field (a decision made by position). Negative controls: 63 failures with the trailing name reading pinned to the family, 16 without the front-piece rule. The reports the oracle can't reach (a credential run's, whose stand-in the same run reads; a one-case connective's, whose stand-in is an initial too) are pinned with their kind, role and source in _EXPECTED_ALTERNATIVES, one row per branch that chooses a kind or source, plus the middle_as_family fold.

• Review shapes. Two code reviews found wrong fields on shapes no case row held. The first found four: PhD VI, Smith, John van VI, abdul VI and Juan de la Vega van. The second, run on the first round's fixes, found three more: Smith, Abu Mary van (a bound join mistaken for a particle chain), Sir abdul VI (a given-name title also licenses P5's join), and Dr. mc VI (P1's fold). All are rows of _REVIEW_SHAPES, which fails 34 of 48 parses on the first draft and 14 on the first fix. A fix in the second round briefly made three scaling rows quadratic; the serial benchmark caught it and it is fixed.

• PR review (f1f391d). A four-agent review of this PR found:

  • capitalized() dropped every alternative. It is fixed, and a test now compares the alternatives themselves, since equality can't see them.
  • A connective inside a joined piece got no alternative: 1,651 of the 4,967 connective reports in the tse-donors local corpus, and 1 after the fix (the opening letter's).
  • The given-name title licence was a hand copy. It now asks group's _run_addresses_by_given over the leading run (abdul VI Sir., Sir Dr. abdul VI); _REVIEW_SHAPES fails 3 of its 57 parses without that fix.

  A review of those fixes found the connective's field ignoring P3's three-word carve-out (JOHN E SMITH gave given where its twin reads middle). That moved four snapshot alternatives given → middle, all intended.

• Known gaps, recorded in decisions/A1. Under two_family_names, the field of a chain in front includes O6's move (John Freiherr V Freiherr under FAMILY_FIRST). A bound given name behind a title-particle joins at group, which the model doesn't see (Freiherr Abu VI under FAMILY_FIRST). A one-case connective between two particles is taken by the chain in front rather than joined (JOHN VAN E VAN), so its family alternative is a reading the parse would not give either.

• Release figures (snapshot-tool fixes, e65feeb9 and its amend). compare.py --baseline 2.6.0 reported 1,088 moved entries with alternatives compared; 867 of those were only the key appearing. With the fix it reports 221, the role and ambiguity moves alone, and says the alternatives were not compared. snapshot.py --since v2.6.0 reports 357 both before and after. The new tests (duplicate alternatives, a header with no alternatives in its rows, a headerless snapshot, a baseline without the field) each fail on f1f391d. The review of the fix commit found no defects; I changed one hint so it names only the case views.

• Merged master with Read King as a given name outside a title phrase (#752 phase 2j, #348) #901 (king, 2j batch 4). Every new title-or-name report carries a title alternative. The local corpora match the new master (6ba78d0) as well.

• Checks. uv run pytest passes (31,326). The serial suite passes. Python 3.15, mypy, ruff, the sphinx doctest and HTML builds, and the README doctest are all clean.

Docs

usage.rst gains "The reading it did not take", with a recipe doctest. concepts.rst links to it. modules.rst adds autodoc for the three new types. The release log has an Additions bullet with the pickle note. decisions/A1 has the full entry, and decisions/vocabulary-model has a pointer. rules/A1 gains a Then bullet (rules_budget.json re-recorded; the clause reads no new signal). mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE, the RFC's "Readings on the parse" and AGENTS.md's ambiguity and pickling bullets are updated.

Refs #752.

🤖 Generated with Claude Code