Localization: wire AI plural translation into the GlotPress reverse fold

.buildkite/commands/test-localization-tooling.sh

@@ -0,0 +1,22 @@
+#!/bin/bash -eu
+
+# Runs the localization tooling's pure-Ruby unit suites (stdlib minitest — no Xcode, no app build, no bundle).
+# Intentionally always runs (no should-skip-job guard): these guard the fastlane localization helpers, and the
+# `validation` skip rule skips on tooling-only changes — exactly when these tests matter most.
+
+echo "--- :test_tube: Localization tooling unit tests"
+
+shopt -s nullglob
+tests=(fastlane/lanes/*_test.rb)
+if [[ ${#tests[@]} -eq 0 ]]; then
+  echo "No *_test.rb files found under fastlane/lanes/."
+  exit 0
+fi
+
+status=0
+for test in "${tests[@]}"; do
+  echo "+++ :ruby: ${test}"
+  ruby "${test}" || status=1
+done
+
+exit "${status}"

.buildkite/pipeline.yml

@@ -163,6 +163,13 @@ steps:
           - github_commit_status:
               context: "Verify String Catalog Coverage"
 
+      - label: ":test_tube: Localization Tooling Unit Tests"
+        command: .buildkite/commands/test-localization-tooling.sh
+        plugins: [$CI_TOOLKIT_PLUGIN]
+        notify:
+          - github_commit_status:
+              context: "Localization Tooling Unit Tests"
+
   #################
   # Claude Build Analysis - dynamically uploaded so Build result conditions evaluate at runtime after the wait
   #################

AGENTS.md

@@ -46,7 +46,7 @@ WordPress-iOS uses a modular architecture with the main app and separate Swift p
 ### Important Considerations
 - **Multi-site Support**: Code must handle both WordPress.com and self-hosted sites
 - **Accessibility**: Use proper accessibility labels and traits
-- **Localization**: follow best practices from @docs/localization.md
+- **Localization**: follow best practices from @docs/localization.md. For how strings flow through GlotPress and the AI translation tier (the `human ?? AI ?? English` floor), see @docs/localization-pipeline.md.
 
 ## Xcode Schemes
 - `WordPress` builds the WordPress iOS app and runs `WordPressUnitTests.xctestplan` — default for builds and the full unit test suite. Use this scheme to run unit tests.

Gemfile

@@ -2,6 +2,8 @@
 
 source 'https://rubygems.org'
 
+# Official Anthropic SDK — backs the AI translation tier of the localization pipeline (fastlane/lanes/ai_translator.rb).
+gem 'anthropic', '~> 1.50'
 gem 'danger-dangermattic', '~> 1.3'
 gem 'dotenv'
 # 2.223.1 includes a fix for an ASC-interfacing issue

Gemfile.lock

@@ -5,6 +5,10 @@ GEM
     abbrev (0.1.2)
     addressable (2.9.0)
       public_suffix (>= 2.0.2, < 8.0)
+    anthropic (1.50.0)
+      cgi
+      connection_pool
+      standardwebhooks
     artifactory (3.0.17)
     ast (2.4.3)
     atomos (0.1.3)
@@ -33,6 +37,7 @@ GEM
     bigdecimal (4.1.2)
     buildkit (1.6.1)
       sawyer (>= 0.6)
+    cgi (0.5.2)
     chroma (0.2.0)
     claide (1.1.0)
     claide-plugins (0.9.2)
@@ -43,6 +48,7 @@ GEM
     colored2 (3.1.2)
     commander (4.6.0)
       highline (~> 2.0.0)
+    connection_pool (3.0.2)
     cork (0.3.0)
       colored2 (~> 3.1)
     csv (3.3.5)
@@ -348,6 +354,7 @@ GEM
       CFPropertyList
       naturally
     singleton (0.3.0)
+    standardwebhooks (1.0.1)
     terminal-notifier (2.0.0)
     terminal-table (3.0.2)
       unicode-display_width (>= 1.1.1, < 3)
@@ -376,6 +383,7 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
+  anthropic (~> 1.50)
   danger-dangermattic (~> 1.3)
   dotenv
   fastlane (~> 2.236)

docs/localization-pipeline.md

@@ -0,0 +1,81 @@
+# Localization translation pipeline
+
+How user-facing strings get from English source into every shipped locale. This is the **release/tooling** view (the fastlane lanes under `fastlane/lanes/`); for how to *write* localizable strings in app code, see [localization.md](./localization.md).
+
+> The contract for every shipped string is **`human ?? AI ?? English`**: a human (GlotPress) translation if one exists, otherwise a machine translation, otherwise the English source. Nothing ships a broken placeholder — machine output that fails the format-specifier gate falls back to English.
+
+## The round trip
+
+Strings make two trips, both driven from fastlane.
+
+### Forward (code freeze) — English → GlotPress
+
+Run as part of `complete_code_freeze` (`generate_strings_file_for_glotpress`):
+
+- **Regular strings** are extracted from source (`ios_generate_strings_file_from_code`, i.e. `genstrings` over `NSLocalizedString` / `AppLocalizedString`) into `WordPress/Resources/en.lproj/Localizable.strings`, then the manually-maintained `.strings` files are merged in. These English originals are uploaded to the [apps/ios GlotPress project](https://translate.wordpress.org/projects/apps/ios/dev/).
+- **Plurals** are authored in `WordPress/Classes/Plurals.xcstrings` (English `one`/`other`). The forward lane (`generate_plural_strings_for_glotpress`) flattens each plural form into an independent string keyed `<key>|==|plural.<cldr-category>` and merges those originals into the same `Localizable.strings`, so they ride the same GlotPress project as everything else.
+
+Translators then do their work in GlotPress.
+
+### Reverse (release prep) — GlotPress → app
+
+`download_localized_strings` (called by `complete_code_freeze` / `finalize_release`) runs, in order:
+
+1. **Download** each locale's `Localizable.strings` from GlotPress (`ios_download_strings_files_from_glotpress`) into `WordPress/Resources/<locale>.lproj/`, and commit. The export filter is `status: current`, so **only translated strings come back** — untranslated ones are *omitted entirely* (not emitted as empty values; the action even errors if it finds an empty value). This is why `pl` ships ~1,650 of ~4,280 keys while `fr` ships ~all of them.
+2. **Re-dispatch** the relevant subset back to the manually-maintained `.strings` files (`ios_extract_keys_from_strings_files`), and commit.
+3. **Plural fold** (`download_localized_plurals`): pull the flat plural translations back out of the downloaded `Localizable.strings`, fold them into `Plurals.xcstrings`, and fill the gaps with the AI tier (below).
+
+Step 3 runs via `run_plural_step`, which logs and continues on failure — the AI tier can never break a release.
+
+## The AI tier
+
+The machine-translation rung of the floor. It is **injected and gated**, never mandatory:
+
+- **Gate**: `ANTHROPIC_API_KEY`. Absent ⇒ the AI tier is skipped entirely and untranslated cells keep their English fallback — i.e. exactly the pre-AI behavior. Providing the key (e.g. in the release environment) is what turns it on.
+- **Placeholder gate**: every machine cell must preserve the source's `printf`/`NSString` format specifiers exactly (count + type; positional `%1$@` may reorder). A mismatch is rejected and the cell falls back to English. So the AI tier can only ever produce a *safe* translation or nothing.
+- **Model**: `claude-opus-4-8` by default (see `AITranslator::DEFAULT_MODEL`).
+
+The reusable primitives live in `fastlane/lanes/`: `AITranslator` (prompt building + validation; `translate` / `translate_plural` / `translate_all` / the async Message-Batches path), `TranslationValidator` (the placeholder gate), `Glossary` (brand do-not-translate list + per-locale terms), and `AnthropicBatch` (SDK glue). All the logic is pure and unit-tested with a canned-reply lambda; only `AITranslator.with_anthropic` touches the network.
+
+## What's wired today: plurals
+
+The plural reverse-fold (`PluralStrings.fold_translations!`) fills each `(key, locale)` cell of `Plurals.xcstrings` as `human ?? AI ?? English` — human ⇒ `translated`; AI / English ⇒ `needs_review`. The AI tier is called **once per `(key, locale)` form-set** (`AITranslator#translate_plural`), not per cell, with the already-human-translated forms passed as **anchors**. Translating the whole set in one request keeps a single consistent stem across the forms — a per-category call lets the model drift between synonyms (Polish `słowo` → `wyrazy` → `słów`), which it structurally can't prevent.
+
+**`Plurals.xcstrings` is a String Catalog, which is why this works**: the catalog carries a real `needs_review` state, so a machine cell is recorded as machine output and a human translation supersedes it on the next download.
+
+> **This does not ship machine translations yet.** `Plurals.xcstrings` is built into the app but **not consumed at runtime** — no code reads from it; the app still renders plurals the legacy way. The fold *pre-populates* the catalog so it's ready when plurals cut over to it. Until that cutover, the AI plural translations sit in the catalog unused.
+
+## What's deferred: regular strings
+
+Regular (non-plural) strings are **not** machine-translated, by design. The app still ships the legacy `WordPress/Resources/<locale>.lproj/Localizable.strings` for them — `Localizable.xcstrings` (`generate_strings_catalog`) is generated as the future backing store but isn't the runtime store yet. A machine translation written into the legacy `.strings` would be **live immediately**, and we don't want machine-translated regular strings shipping before the catalog cutover.
+
+So regular-string MT waits for the same shape as plurals: once `Localizable.xcstrings` becomes the runtime store, a regular-string **catalog reverse-fold** folds the human translations in and AI-fills the `needs_review` gaps, staged in the catalog (not shipped) until cutover — exactly as the plural fold does today.
+
+When that's built, two facts established here will carry over:
+
+- **"Undefined by GlotPress" = absent**, not empty. The export omits untranslated strings (`status: current`; verified no empty-valued entries), so absence is the untranslated signal.
+- **Humans always supersede MT**, and machine output never returns to GlotPress — so there's no translation-memory pollution and no manual reconciliation, as long as MT lives in a state-bearing store (the catalog's `needs_review`).
+
+## Why these choices
+
+- **Why translate whole plural form-sets at once?** Per-category calls let the model pick different synonyms for different forms of the same word. One request for the whole set, with human forms as anchors, keeps one stem.
+- **Why is the AI tier gated and non-fatal?** Cost and safety: it runs only where a key is configured, and a failure logs and continues rather than breaking a release.
+- **Why does regular-string MT need the catalog, not legacy `.strings`?** The catalog's `needs_review` state lets a machine translation be *staged* (built but not shipped until cutover) and lets humans supersede it automatically. Legacy `.strings` has no state and is live, so anything written there ships immediately — which is exactly what we don't want before cutover.
+
+## Operational notes
+
+- **Eyeball one string against the live model** (needs `ANTHROPIC_API_KEY` + `bundle install`):
+  `ruby fastlane/lanes/ai_translator.rb fr "You have %1$d new posts" "Notification text. %1$d is the count."`
+- **Tests** are pure stdlib minitest and run in CI (`.buildkite/commands/test-localization-tooling.sh`): `ruby fastlane/lanes/*_test.rb`.
+
+## Code map
+
+| Concern | File |
+| --- | --- |
+| Translation tier (prompts, validation, `translate*`) | `fastlane/lanes/ai_translator.rb` |
+| Placeholder safety gate | `fastlane/lanes/translation_validator.rb` |
+| Brand do-not-translate + per-locale terms | `fastlane/lanes/translation_glossary.rb` |
+| Anthropic SDK glue + Message Batches | `fastlane/lanes/anthropic_batch.rb` |
+| Plural fold (`Localizable.strings` ⇄ `Plurals.xcstrings`) + AI wiring | `fastlane/lanes/plural_strings_helper.rb`, `fastlane/lanes/localization_plurals.rb` |
+| Catalog generation (future regular-string backing store) | `fastlane/lanes/localization_catalog.rb` |
+| Download/upload orchestration | `fastlane/lanes/localization.rb` |