Implement iterative precision-based leveraging with tiebreaking

[wadimw]

Leveraging

Improved the leveraging logic to be more intuitive and reliable. Unit matching now attempts matching in descending precision order (MD5 > name+content > name-only/content-only). If multiple matches are found at the same precision level, tiebreaking is applied on a per-locale basis select the best available translation.

Match level ordering per mode:

• MD5: MD5 only
• NAME: MD5 > name+content > name-only
• EXACT: MD5 > name+content > content-only

Once any matches are found at a given precision level, iteration ends.

Within each level, tiebreaking picks:

• used over unused
• higher translation status
• most recently translated

Added "effectively unambiguous" check to UNIQUE status preservation mode: if leveraging finds matches across multiple TUs, but all matches agree on source content (across matches for all locales) and translations (per-locale), then they are effectively the same and it's safe to preserve the status of leveraged translation.

Additionally, added missing decision details into the logged leveraging comment which are displayed in UI (to help figure out why something didn't get applied correctly). Examples:

Scenario	Comment
Unique MD5 match, PRECISION mode	Leverage by MD5 - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: unique, status: preserved (high precision, unique)
Unique name-only match, PRECISION mode	Leverage by NAME_ONLY - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: unique, status: downgraded (low precision)
Non-unique name+content match, PRECISION mode, used won	Leverage by NAME_AND_CONTENT - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: used over unused, status: downgraded (high precision, ambiguous)
Non-unique name+content match, UNIQUE mode, effectively unambiguous (same source + same translations)	Leverage by NAME_AND_CONTENT - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: most recently translated, status: preserved (effectively unambiguous)
Non-unique name+content match, UNIQUE mode, translations disagree	Leverage by NAME_AND_CONTENT - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: higher status, status: downgraded (ambiguous)
Non-unique name-only match, UNIQUE mode, sources disagree (name-only)	Leverage by NAME_ONLY - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: used over unused, status: downgraded (ambiguous)
Single candidate, complementary locale (gap-fill from unused TU)	Leverage by NAME_AND_CONTENT - from tmTextUnitId: 789, tmTextUnitVariantId: 101, tiebreaker: unique, status: preserved (unique)
ALL mode (forced status preservation)	Leverage by NAME_ONLY - from tmTextUnitId: 123, tmTextUnitVariantId: 456, tiebreaker: most recently translated, status: preserved (forced)

Also added some debug logs for the leveraging logic.

Tests

• copyByNamePrefersHigherPrecisionEvenWhenAllUnused represents the situation that bit us (leveraging across assets within the same repository causing name+content matching to be skipped due to USED-only filtering)
• bunch of new tests that prove logic outlined above (precision levels, tiebreaking on used and recency)
• refactored parts of the leveraging suite a bit by extracting some shared helpers corresponding to typical "actions" like push asset, translate etc.; improves readability and introduces consistency for how different tests access the underlying data model

Followup

CopyTmLeverager diverged from the AbstractLeverager model, so a followup refactor PR #1064 gets rid of it completely in favour of separate, self-contained leveragers.

[socket-security]

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

[wadimw]

Showerthought: need tiebreaking by status before recency

[wadimw]

To discuss: Do we want to leveraging on a per-locale basis, or stick to whole-unit (all locales together) matches?

Consequences of per-locale matching for each match level:

• MD5 and NAME_AND_CONTENT — source text is identical across candidates. Translations should be equivalent regardless of which TU they come from. Per-locale mixing is safe.
• CONTENT_ONLY — source text is identical (just different key names). Same reasoning — per-locale mixing is safe.
• NAME_ONLY — this is the problematic one. Two TUs named "greeting" with content "Hello" vs "Hello world" are semantically different. Picking fr-FR from TU A ("Hello" -> "Bonjour") and de-DE from TU B > ("Hello world" -> "Hallo Welt") creates a target TU with translations derived from different source strings per locale. That's inconsistently wrong rather than consistently wrong.

That said, NAME_ONLY matches are already flagged TRANSLATION_NEEDED (unless forcibly preserved) — so they'll all be reviewed regardless. And the single-TU approach also picks arbitrarily, just consistently arbitrarily.

Two options:

1. Keep per-locale for all levels — accept the inconsistency since NAME_ONLY is already flagged for review, and the gap-filling benefit outweighs it
2. Per-locale only for high-confidence levels (MD5, NAME_AND_CONTENT, CONTENT_ONLY), fall back to single-TU for NAME_ONLY — prevents inconsistent fuzzy matches

EDIT: We've agreed on per-locale for all levels

[ehoogerbeets]

1. Let's say there is a unit that has matches in 2 different assets by name + content. Now asset 1 has an accepted translation in language 1, and asset 2 has an accepted translation in language 2. Now we get the situation where the matches are considered not unique, so the status will be downgraded to "needs translation", even though there is only 1 variant available for each language that is in the accepted state. Maybe we should be counting uniqueness by locale instead of by text unit? The extreme case is when one of the assets has a match but no translations in it, causing a non-unique match with another asset that does have translations.

2. Instead of downgrading to "needs translation", why don't we downgrade to "needs review"?

A "needs translation" state means we have no translation at all, or there is an old translation that we are really not sure about, or there is an old translation and the source has changed so the translation may need to be updated. In all these scenarios, the translator has to do some work. A "needs review" state means that we have a valid translation, but it may not be correct in this context and therefore it needs to be checked again.

I think it would be reasonable for unique MD5 matches to get downgraded from "accepted" to "needs review" because of the new context, but to go all the way to "needs translation" seems excessive.

3. What happens to locales that inherit from a parent? Do they have their own variants? Do they get leveraged?

[wadimw]

Question 1: Per-TU vs per-translation uniqueness check

Let's say there is a unit that has matches in 2 different assets by name + content. Now asset 1 has an accepted translation in language 1, and asset 2 has an accepted translation in language 2. Now we get the situation where the matches are considered not unique, so the status will be downgraded to "needs translation", even though there is only 1 variant available for each language that is in the accepted state. Maybe we should be counting uniqueness by locale instead of by text unit? The extreme case is when one of the assets has a match but no translations in it, causing a non-unique match with another asset that does have translations.

Maybe?

---

Case 1: high-precision name+content match with full translations

• TU A:
  • repo: Source
  • asset: one.properties
  • id: application.greeting
  • content: "Hello"
  • translations:
    • DE "Hallo" APPROVED
    • FR "Bonjour" APPROVED
• TU B:
  • repo: Source
  • asset: other.properties
  • id: application.greeting
  • content: "Hello"
  • translations:
    • DE "Hallo" NEEDS-REVIEW
    • FR "Bonjour" NEEDS-REVIEW
• Target TU:
  • repo: Target
  • asset: target.xliff
  • id: application.greeting
  • content: "Hello"

Leveraging with source Source, target Target, mode NAME, preserve-status UNIQUE

Current outcome: both DE and FR leveraged, but downgraded to NEEDS-TRANSLATION due to per-TM non-uniqueness.
Expected outcome: both leveraged and APPROVED; both TUs have equal content so the match is "effectively unique", same for the translations.

Case 2: high-precision name+content match with complementary translations

• TU A:
  • repo: Source
  • asset: one.properties
  • id: application.greeting
  • content: "Hello"
  • translations:
    • DE "Hallo" APPROVED
    • FR NULL NEW
• TU B:
  • repo: Source
  • asset: other.properties
  • id: application.greeting
  • content: "Hello"
  • translations:
    • DE NULL NEW
    • FR "Bonjour" APPROVED
• Target TU:
  • repo: Target
  • asset: target.xliff
  • id: application.greeting
  • content: "Hello"

Leveraging with source Source, target Target, mode NAME, preserve-status UNIQUE

Current outcome: both DE and FR leveraged, but downgraded to NEEDS-TRANSLATION due to per-TM non-uniqueness.
Expected outcome: both leveraged and APPROVED; both TUs have equal content so the match is again "effectively unique", translations are actually unique; we're fine with mixing translations from across different assets, since this is a high-precision match so there's little risk of contextual mismatch.

Case 3: low precision name-only match with complementary translations

(this is our primary usecase)

• TU A:
  • repo: Source
  • asset: one.properties
  • id: application.greeting
  • content: "Hello {name}"
  • translations:
    • DE "Hallo {name}" APPROVED
    • FR NULL NEW
• TU B:
  • repo: Source
  • asset: other.properties
  • id: application.greeting
  • content: "Hello {name}"
  • translations:
    • DE NULL NEW
    • FR "Bonjour {name}" APPROVED
• Target TU:
  • repo: Target
  • asset: target.xliff
  • id: application.greeting
  • content: "Hello { name }" (different spacing)

Leveraging with source Source, target Target, mode NAME, preserve-status UNIQUE

Current outcome: both DE and FR leveraged, but downgraded to NEEDS-TRANSLATION due to per-TM non-uniqueness.
Expected outcome: debatable (both leveraged and APPROVED?)

Here we have a low-precision match with complementary translations, but the target TU now has a different spacing than the source TUs. This is a case where we know we want to leverage the translations and preserve their status (hence we use --preserve-status=UNIQUE).

Case 4 (counter-example): low precision name-only match across factually different content

• TU A:
  • asset: one.properties
  • id: application.greeting
  • content: "Hello"
  • translations:
    • DE "Hallo" APPROVED
    • FR NULL NEW
• TU B:
  • asset: other.properties
  • id: application.greeting
  • content: "Hello {name}"
  • translations:
    • DE NULL NEW
    • FR "Bonjour {name}" APPROVED
• Target TU:
  • repo: Target
  • asset: target.xliff
  • id: application.greeting
  • content: "Hello { name }" (different spacing)

Leveraging with source Source, target Target, mode NAME, preserve-status UNIQUE

Current outcome: DE "Hallo" and FR "Bonjour {name}" leveraged, both downgraded to NEEDS-TRANSLATION due to per-TM non-uniqueness.
Expected outcome: same as current; these should NOT get upgraded to APPROVED since they are factually different.

So in this case, if we were to move to a simple per-locale uniqueness check, we'd accidentally upgrade status for mismatched translations. Maybe we could layer this somehow, like:

• for md5 matches, just simply NEVER downgrade
• for name+content, maybe also never downgrade? depends on if we consider them good enough
• for low precision name/content only
  • check for effectively unique TUs i.e. Case 3 - content A equals content B
  • then check for effectively unique translations - though if we already have effectively unique TUs, maybe downgrading isn't even needed anymore

I was actually considering including "effectively unique" checks during implementation, but eventually decided to defer it to a separate PR due to complexity. For now, checking per-TU is the safer approach (more false downgrades, but less risk of false upgrades).

Question 2: Downgrading to "needs review"

Instead of downgrading to "needs translation", why don't we downgrade to "needs review"? A "needs translation" state means we have no translation at all, or there is an old translation that we are really not sure about, or there is an old translation and the source has changed so the translation may need to be updated. In all these scenarios, the translator has to do some work. A "needs review" state means that we have a valid translation, but it may not be correct in this context and therefore it needs to be checked again. I think it would be reasonable for unique MD5 matches to get downgraded from "accepted" to "needs review" because of the new context, but to go all the way to "needs translation" seems excessive.

TBD

Question 3: Leveraging inherited locales

What happens to locales that inherit from a parent? Do they have their own variants? Do they get leveraged?

TBD

[wadimw]

We do want to implement effectively unique checks according to the following:

TU A id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo"
TU B id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo"

TUs effectively unique, translations effectively unique => not downgrade

TU A id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo"
TU B id: app.greeting content: "Hello" comment: "A greeting" FR: "Bonjour"

TUs effectively unique, translations actually unique => not downgrade

TU A id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo!"
TU B id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo."

TUs effectively unique, translations differ => DO downgrade

TU A id: app.greeting content: "Hello" comment: "A greeting" DE: "Hallo"
TU B id: app.greeting content: "Hello {name}" comment: "A greeting" FR: "Bonjour {name}"

TUs NOT unique, translations actually unique (case 4) => DO downgrade

[wadimw]

For Q2:
Case id application.status content Free translation in A means Free (not occupied) APPROVED translation in B means Free (no cost) APPROVED => target TU id equal content equal (name+content match) but translations differ => we want to get NEEDS_REVIEW (cause it's a valid translation, but we're not sure if it's valid for the context)

Case:

TU A id: app.greeting content: "Hello" comment: "A greeting"
TU B id: app.greeting content: "Hello {name}" comment: "A greeting with name"
Target TU: app.greeting content: "Hello { name }" comment: "A greeting with name"
Expected: prefer B due to matching comment
counter:
one.properties modal.title content "Popup" comment "title of a modal" FR: "French popup" APPROVED
other.properties modal.title content "Please log in" comment "title of a modal" DE: "Bitte einloggen" APPROVED
if we downgrade to NEEDS_REVIEW due to comment equality rather than NEEDS_TRANSLATION, we get FR "French popup" DE "Bitte einloggen"

TU A id: app.greeting content: "Hello {name}" comment: "A greeting" DE: "Hallo"
TU B id: app.greeting content: "Hello {name}" comment: "A greeting with name" DE: "Hallo."
Target TU: app.greeting content: "Hello {name}" comment: "A greeting with name"
Outcome: matching does not consider TU A because TU B is an MD5 match (TU A would be name+content i.e. lower tier)

EDIT: After discussion, we decided that for now we're NOT implementing "partial downgrade" to NEEDS_REVIEW

[wadimw]

Q3: Fallback to parent is applied during mojito pull - in leveraging it's simply: if a translation for frCA exists (as in a distinct frCA one), then copy it over

[wadimw]

Effectively unique implemented in daa07c5 - ready for rereview @ehoogerbeets