Skip to content

Read credentials joined by a hyphen or a slash (#752, phases 2c and 2d) - #887

Merged
derek73 merged 9 commits into
masterfrom
claude/phase2cd-joined-credentials-a4d3c9
Oct 25, 2026
Merged

derek73 merged 9 commits into
masterfrom
claude/phase2cd-joined-credentials-a4d3c9

Conversation

@derek73

@derek73 derek73 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Phase 2c and 2d of #752, one commit each: the first two Phase 2 changes that move readings. Both carry out item 5 of decisions/vocabulary-model's 2026-10-23 #752 entry, which Derek agreed on 2026-10-23.

2c: a credential joined by a hyphen or a slash, every chunk listed

S3 (credentials written in joined chunks; the rule was "credentials written with interior periods" and is renamed for its wider When) already reads a word with interior periods by its chunks. It now reads chunks joined by - or / too, with a stricter test: the periods need any chunk to be listed, but a joined word needs every chunk to count as a suffix as written, which a bare member of the ambiguous class (MA, Ed) does not. The reason is that a hyphen also joins names (Smith-Jones).

The test lives in suffix_as_written. That way classify's tag, the comma's decision and every predicate that tests a word's text all read a joined word the same way.

Input 1.4.0 through 2.6.0 Now
John Smith, MD/PhD given MD/PhD, family John Smith given John, family Smith, suffix MD/PhD
John Smith MD-PhD middle Smith, family MD-PhD family Smith, suffix MD-PhD
John Smith, M.D./Ph.D. reversed suffix M.D./Ph.D.
John Smith-Jones · John Smith MD-Jones · John Smith, MA/PhD · Jack I-V · Kim, Ae-Ra unchanged unchanged

S3's single-character exception covers a joined word too, so a word made only of single-character chunks (I-V) is not a credential.

A hyphenated word written as a name stays a name word (Derek's call). A hyphen-joined word stays a name word when its first chunk is in Title case and every other chunk is in Title case or lower case. That holds even when every chunk is a listed credential. The case that needed it: ae and ra are listed as unambiguous credentials, so without the cue Kim, Ae-Ra lost its given name.

  • Kim, Ae-Ra and Shin Ae-ra keep their 1.4.0 readings. Revised Romanization writes the second syllable in lower case, which is why the cue also accepts lower-case later chunks.
  • PhD-Md, MD-PhD, md-phd and slash joins keep the joined reading. A slash joins no names.
  • Accepted costs: the capitals spelling KIM, AE-RA still reads suffix AE-RA. A credential typed in Title case throughout (John Smith, Phd-Md) now reads as a name word, because the writing is the cue; none of the 34 distinct hyphen-joined credentials in the local corpora is written that way.
  • A word holding a slash anywhere takes no cue, and a word with an interior period is read by its periods first.
  • The spaced Kim, Ae Ra already lost its given name on master; that is the ae/ra vocabulary question and is left as it was.
  • No local corpus holds a joined credential written in Title case, so the cue gives back none of 2c's ctgov moves.

2d: an unlisted word of joined capitals after a comma (#797's half A)

An unlisted word of capitals whose letter chunks are joined by - or /, with one chunk of two or more letters, is now S9's caps shape. It joins the ambiguous class exactly where and as the unjoined word does: after a comma, gated by C4's name-word count and C6's case contrast.

Input 1.4.0 through 2.6.0 Now
John Smith, PU-PH given PU-PH, family John Smith family Smith, suffix PU-PH, reported
JOHN SMITH, PU-PH · Smith, PU-PH · John Smith, A-B · John Smith, Pu-Ph unchanged unchanged

Reachability. All three call sites of caps_shape_candidate (the comma's candidate list, C12's case-repair marks, and classify under EVERYWHERE) ask isalpha() in C before calling it. The joined spelling fails that test, so each pre-filter widens with the predicate. This trap is now a field note in mechanisms.md.

Under CapsSuffixes.EVERYWHERE the joined word reads as the unjoined one does in every slot (Jean Pierre DUPONT-MORETTI as Jean Pierre DUPONT), which is that setting's documented cost. Limiting it to the comma would have made one shape into two by position, so I didn't.

The oddity the spike found. In Jean Claude MD DUPONT, PU-PH, the capitals word between MD and the comma goes into the suffix. That is not 2d's doing. Jean Claude MD DUPONT with no comma already reads suffix MD DUPONT (S8's run, reported). 2d makes the comma a suffix comma, and the part before it then reads as the comma-less name does (C1). The XYZ and PhD twins read the same way. One ctgov entry takes this shape. I recorded it in decisions/S9 as S8's question and changed nothing.

Measured

Each commit was measured against its parent, with the local snapshots recorded from the parent using --tree.

2c vs 2af8da8 2d vs 2c
committed corpora (snapshot.py) rule examples only rule examples only
CLDR / GND / Wikidata agreement matches matches
psc-individuals (local_agreement.py) 0 fixed, 0 broken, 0 changed 0 / 0 / 0
ctgov-officials 1,333 of 352,104 529
fec-individuals 1 of 1,893,964 0 (one case: C6 never admits)
tse-donors 0 of 848,351 0

2c in ctgov:

  • 1,319 moves are a joined credential after a comma that had been read as the given name, with the name reversed around it.
  • 2 are a joined credential ending a name.
  • 12 move only a report.
  • There are 82 distinct joined words, all degree combinations: MD-PhD 494, MD/PhD 473, PhD/MD 65, MS-PT 53.
  • 68 entries hold a chunk written with its periods (M.D./Ph.D.). The spike's letters-only test did not read those, which is why its count was 1,254.

2d in ctgov:

  • 519 moves are a joined capitals word after a comma, now the suffix with a report.
  • 10 move only in capitalized(force=True).
  • There are 104 distinct words, all credentials or job titles: PU-PH 258, MS-OMPT 32, MCU-PH 30. None is a name.
  • The spike's 759 included about 230 that 2c now reads first.

Re-measured after merging #886 (phase 2f) against master 1f0b07c, with each baseline recorded right before its check. ctgov still moves 1,862 entries, every one holding a joiner; fec moves 1; tse and psc-individuals move none. Names with both a numeral and a joined credential (John Smith VI MD/PhD, John Smith V, MD-PhD, Smith, John VI, PU-PH, Smith, John V MD/PhD) read as their spaced or XYZ twins.

Sweeps: a grid of four comma shapes (including an empty head and a nickname-only head) × three orders × joiner spellings (glued, spaced, glued to one side) × hyphenated surnames before and after the comma, compared base against tree. Every 2d move in that grid reads the same as its unjoined XYZ twin under all three orders.

Frame cost: a word with no joiner pays nothing (John Smith 150, Smith, John 158, unchanged). A hyphenated word pays 3 frames at each site that asks for its suffix reading (Mary Smith-Jones 156 → 159). A joined capitals word pays what an unjoined capitals word already pays.

Docs in this PR

  • rules/S3 gets a joined bullet, an Accepted cost (John Smith, MA/PhD) and a Precedence bullet (in capitals, S9 reads the word). rules/S9 gets the joined spelling, and rules/C6 gets examples.
  • Dated entries in decisions/S3 and decisions/S9, with pointers in decisions/vocabulary-model. The RFC's "today" sentence is updated.
  • One mechanisms.md field note (the pre-filter trap) and the AGENTS.md spelling sweep.
  • Case rows, the release-log bullet, and a doctest each in customize.rst's delimiter and all-caps sections.
  • Not done here: migrate.rst's "Changed in" list. Its heading names the release, so it belongs to the release checklist's step 0.

Leads, not done: 63 ctgov entries join two listed credentials with &. MD/ PhD, with the joiner glued to one side, keeps MD/ as a name word (10 entries).

Refs #752. Refs #797, but does not close it: this answers its half A only.

🤖 Generated with Claude Code

derek73 and others added 5 commits October 25, 2026 03:08
…unk is listed (#752)

Phase 2c. rules/S3 (credentials written in joined chunks, renamed
from "credentials written with interior periods") extends its chunk
reading to '-' and '/', with a stricter test than the periods' any:
every chunk counts as a suffix as written, so a bare member of the
ambiguous class ('MA/PhD') and a hyphenated name ('Smith-Jones') are
left as they were. The test lives in suffix_as_written, so classify's
tag, the comma's decision and every predicate that tests a word's
text read a joined word alike.

'John Smith, MD/PhD' reads suffix 'MD/PhD' where 1.4.0 through 2.6.0
read given 'MD/PhD', family 'John Smith'. Measured against 2af8da8:
ctgov-officials 1,333 entries, fec-individuals 1, tse-donors,
psc-individuals, the committed corpora and the agreement records
none (decisions/S3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#752)

Phase 2d, #797's half A. An unlisted word of capitals whose letter
chunks are joined by '-' or '/', one of two or more letters, is the
caps shape (rules/S9): 'John Smith, PU-PH' reads suffix 'PU-PH',
reported, where 1.4.0 through 2.6.0 read given 'PU-PH', family 'John
Smith'. It joins the class where and as the unjoined word does, so
C4's count and C6's case contrast gate it after a comma by default
('JOHN SMITH, PU-PH' and 'Smith, PU-PH' keep the given name), and
EVERYWHERE takes it at every suffix position.

The three call sites' C-level pre-filters (the comma's candidates,
C12's case-repair marks, classify under EVERYWHERE) asked isalpha()
ahead of the predicate and widen with it; without that nothing moves
(mechanisms.md's field notes).

Measured against the 2c commit: ctgov-officials 529 entries; fec,
tse, psc-individuals, the committed corpora and the agreement
records none (decisions/S9). A capitals word between MD and the
comma is taken into the suffix as the comma-less name already reads
it (rules/S8's run), recorded there rather than changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e Korean cost (#752)

Review of phases 2c and 2d. A joined word whose every chunk is a
single ASCII character ('Jack I-V') read as a suffix with no family;
rules/S3's exception binds it now, as it binds 'Jack X.Y.I.'. S3's
Then states the joined reading, and its bullet names what joins
nothing (other dashes, 'MD/Ph. D.'). Accepted: a Korean given name
hyphenated between two listed credentials ('Kim, Ae-Ra') reads as
the credential, as the spaced spelling already did on the base
(decisions/S3). The release bullet names C4's two name words.

ctgov-officials still moves 1,862 entries over both phases, fec 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second review of the fix: 'Jack I.-V.' still read suffix with no
family, the length test counting the chunk's period where a word
split at its periods never does. AGENTS.md's spelling sweep records
the miss.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added enhancement design vocabulary Per-word vocabulary model and language packs labels Oct 25, 2026
@derek73 derek73 self-assigned this Oct 25, 2026
derek73 and others added 4 commits October 25, 2026 10:57
Codecov's patch check found it uncovered: suffix_as_written calls
_joined_credential only for a word holding a joiner, so the split
always gives two chunks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Derek's decision on the Korean given names the code review found. A
word joined by a hyphen whose first chunk is in Title case and whose
other chunks are in Title case or lower case is a name word, whatever
its chunks are listed as: 'Kim, Ae-Ra' and 'Shin Ae-ra' keep their
1.4.0 readings. A chunk with an inner capital ('PhD-Md'), capitals
('MD-PhD'), a lower-case opening ('md-phd') and a slash keep the
joined reading. Accepted in its place: 'KIM, AE-RA' reads suffix.

No local corpus holds a Title-case joined credential, so phase 2c's
1,333 ctgov moves all stand (decisions/S3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…752)

Review of the cue: a word holding a slash took it ('Kim, Ae-Ra/Md'),
which the rule says a slash does not; a one-letter lower-case tail
('Md-v') did not, which the rule says it does. S3 now says a word
with an interior period is read by its periods first, and records a
credential typed in Title case throughout ('Phd-Md') as the cue's
cost; no local corpus writes one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ned-credentials-a4d3c9

# Conflicts:
#	docs/design/decisions/vocabulary-model.md
#	docs/release_log.rst
#	tests/v2/rules_budget.json
#	tests/v2/test_cases.py
#	tools/differential/corpus_rules.jsonl
@derek73 derek73 added this to the 2.7 milestone Oct 25, 2026
@derek73
derek73 merged commit f2e31d4 into master Oct 25, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design enhancement vocabulary Per-word vocabulary model and language packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant