Skip to content

Release canonical name normalization 0.4.0#19

Draft
sergeyf wants to merge 2 commits into
mainfrom
codex/canonical-all-people
Draft

Release canonical name normalization 0.4.0#19
sergeyf wants to merge 2 commits into
mainfrom
codex/canonical-all-people

Conversation

@sergeyf

@sergeyf sergeyf commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • adds canonical person-name results while preserving structured first/middle/last/suffix roles
  • adds general non-Chinese normalization and East Asian name-order routing
  • adds curated Japanese/Korean/Vietnamese lexicon assets and reproducible builders
  • tightens Chinese-vs-non-Chinese evidence, including guarded Korean vetoes and contextual Taiwanese romanization
  • adds the Timo v2 interface while preserving the existing entry point
  • bumps the package version to 0.4.0

Why

Sinonym previously normalized Chinese names well but did not provide a reliable, structured contract for the much larger non-Chinese population. Source roles, punctuation, casing, East Asian order, and mixed-script evidence could be lost or misclassified.

Impact

Callers can consume structured canonical-name metadata for both Chinese and non-Chinese people. Conservative source-role preservation remains the default, and culturally inferred routing is gated by high-confidence evidence.

Validation

  • uv run ruff check .
  • uv build
  • repository status checker: expected 15 known failures, no new failures
  • performance check: 3,295 names/second
  • focused classification and normalization suites passed
  • three disjoint random 1K evaluations plus targeted Korean/Taiwanese negative coverage

@atalyaalon atalyaalon force-pushed the codex/canonical-all-people branch from 9f80f85 to a035ea2 Compare July 14, 2026 19:13
@atalyaalon

atalyaalon commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

@sergeyf Reviewed the non-Chinese / Latin-script path with an LLM judge (+ Tier-A priors; production DB not treated as ground truth).

  • ~94% of non-Chinese occ is reproduced identically to production — a reproduction match, not a correctness measure on its own.
  • Correctness: stratified, occ-reweighted judge puts the error in the low single digits (~3%, rough estimate — wide CI, LLM-judged), down from ~5% before these fixes.

Shipped (stacked PR #21): dangling-and, particles (ten/ur/ud), HTML entities, credential-collision givens, Senior/Junior surnames, org + non-English-org rejection, initial-dup, spaced-kanji reorder, EA-swap diacritic gate, Center surname gate (Company kept hard-org). Each net-positive by a wide margin, with small documented casualties (standalone … Company surnames; some Portuguese … Junior).

Open / proposed (need a call):

  • Compound-surname grouping (M. Calvo GomezCalvo Gomez) — largest open class (≳0.9% of non-Ch occ, floor — detector only catches compounds production stored whole in last); ambiguous (judge 39/48/14); proposed INE∩corpus lexicon.
  • EA-swap residual — ASCII Western (Kim Dam-Johansen) + already-given-first JP.
  • ASCII Vietnamese family-first (Nguyen Thi …) — only the diacritic form routes.
  • Portuguese … Junior (suffix) sometimes kept as surname.
  • Mononym / lone-initial contract — non-Chinese path emits surname-empty persons; Chinese path doesn't.
  • CJK-script foreign names rejected — scope/contract call.

Details + numbers in #21.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants