Skip to content

Fix: match only gemini-1.<digit> in isGemini1Model (adk-python parity) - #617

Open
AmaadMartin wants to merge 1 commit into
mainfrom
fix/gemini-1-model-version-regex
Open

Fix: match only gemini-1.<digit> in isGemini1Model (adk-python parity)#617
AmaadMartin wants to merge 1 commit into
mainfrom
fix/gemini-1-model-version-regex

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    Closes: #issue_number
    Related: #issue_number
  2. Or, if no issue exists, describe the change:
    Problem: isGemini1Model — the switch that routes a request down ADK's legacy Gemini 1.x code paths — used a bare prefix test with no digit boundary:
return modelName.startsWith('gemini-1');

That accepts any model id whose version field merely begins with the character 1, and accepts a bare major version with no minor at all. adk-python's is_gemini_1_model (src/google/adk/utils/model_name_utils.py) requires a literal . followed by at least one digit: re.match(r'^gemini-1\.\d+', model_name). So the two SDKs disagree on what "Gemini 1.x" means.

Five input classes are misclassified today:

input before after (and adk-python)
gemini-10.0-pro true false
gemini-1 true false
gemini-1-pro true false
gemini-1. true false
gemini-1x-foo true false

gemini-10.0-pro is the exact shape of a real future release id. The moment such a model ships, adk-js silently routes it down the Gemini 1.x paths with no code change and no warning: GoogleSearchTool would append the legacy googleSearchRetrieval: {} and throw Google search tool can not be used with other tools in Gemini 1.x., and applyGoogleMapsGrounding, VertexAiSearchTool.processLlmRequest and applyEnterpriseWebSearch would each throw their Gemini 1.x-only errors for a Gemini 10 model.

It also breaks a structural invariant adk-python pins in TestModelNameUtilsIntegration::test_model_classification_consistency: a model must never be classified as both Gemini 1.x and Gemini 2.0+. Today gemini-10.0-pro satisfies both, because parseVersion('10') yields major 10 ≥ 2.

The existing doc comment already claimed the behaviour the code did not implement — "Check if the model is a Gemini 1.x model using regex patterns".

Solution: Replace the prefix test with the same regex adk-python uses, hoisted to a module-level constant next to the existing MODEL_NAME_PATTERN:

const GEMINI_1_MODEL_PATTERN = /^gemini-1\.\d+/;

export function isGemini1Model(modelString: string): boolean {
  const modelName = extractModelName(modelString);

  return GEMINI_1_MODEL_PATTERN.test(modelName);
}

Two files change, +76/−1. Notes on the choices:

  • No /g flag. A global-flagged regex carries mutable lastIndex state across .test() calls, which would make this pure predicate return alternating results for the same input. A non-global literal is stateless and safe to hoist to module scope.
  • MODEL_NAME_PATTERN is deliberately left as a string. It is consumed by String.prototype.match and belongs to extractModelName's hunk; harmonising the two would be an out-of-scope edit.
  • Signature stays modelString: string. adk-python accepts Optional[str] and short-circuits on falsy input, but all four call sites already guard with if (!llmRequest.model) return;, and '' returns false under the new regex without an explicit guard. An undefined branch would be dead code.
  • Cross-language parity, and where it yields. Parity wins here because this is an observable classification that both SDKs must agree on. One divergence is accepted and deliberate: Python's \d matches Unicode decimal digits, JavaScript's is ASCII-only. ASCII-only is the desired behaviour for model identifiers, so this is not chased with the u flag or \p{Nd}.
  • Behaviour is unchanged for every real Gemini 1.x id. All existing tests that exercise a gemini-1.* model use gemini-1.5-* or gemini-1.0-pro and are unaffected.

Scope. Only isGemini1Model changes. extractModelName, isGeminiModel, isGemini2OrAbove, isGemini3xFlashLive, parseVersion and isGeminiModelIdCheckDisabled are untouched, the four call sites are untouched, and isGemini1Model stays unexported from core/src/common.ts and core/src/index.ts. Because extractModelName does not yet strip provider prefixes, adk-python's provider-prefixed cases (gemini/gemini-1.5-flash, vertex_ai/gemini-1.5-flash, openrouter/google/gemini-1.5-pro:online) are intentionally excluded from the test plan — porting them here would force an out-of-scope edit.

Prior-art / collision check. Before writing anything I scanned all 516 open PRs on the fork (gh pr list --limit 1000) and inspected the three that touch core/src/utils/model_name.ts. PR #615 (fix/model-name-eap-and-path-parity) bundles an equivalent Gemini-1 boundary fix, but it is stacked three deep on separate tasks (#615#471 extractModelName widening → #372 EAP gate → main) and carries those unrelated changes with it. This PR is the standalone, main-based version of just the boundary fix, reviewable and revertable on its own. If #615 lands first this becomes a no-op and can be closed; if this lands first, #615's model_name.ts hunk becomes redundant.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

A new describe('isGemini1Model', ...) block was added to core/test/utils/model_name_test.ts. The two pre-existing describe blocks (isGemini2OrAbove, isGemini3xFlashLive) are unchanged — no existing test was edited, weakened, skipped or deleted. The block follows the file's existing table-driven style and covers 16 inputs plus 2 invariants:

  • truegemini-1.5-flash, gemini-1.0-pro, gemini-1.5-pro-preview, gemini-1.9-experimental, and two Vertex path forms (exercising composition with the unmodified extractModelName).
  • false — the five regressions above, plus gemini-2.5-flash, claude-3-sonnet, my-gemini-1.5-model (present but not anchored), '', and a path-based gemini-2.5-flash.
  • invariants — no model is classified as both Gemini 1.x and Gemini 2.0+ (the adk-python test_model_classification_consistency invariant), and every Gemini 1.x model is also a Gemini model.

Targeted runs (no full-repo suite):

npx vitest run --project unit:core \
  core/test/utils/model_name_test.ts \
  core/test/tools/google_search_tool_test.ts \
  core/test/tools/google_maps_grounding_tool_test.ts \
  core/test/tools/vertex_ai_search_tool_test.ts \
  core/test/tools/enterprise_web_search_tool_test.ts
→ Test Files 5 passed (5) | Tests 79 passed (79)

npx vitest run --project unit:core
→ Test Files 168 passed (168) | Tests 2369 passed (2369)

The four tool suites are the integration-level regression signal for the changed predicate; all pass unchanged. New-line and new-branch coverage of the changed statement is 100% — the test block executes it on both outcomes.

Falsifiability proof (test run against the unfixed code). The new block was written before the fix and run against the original startsWith('gemini-1') implementation. It failed with exactly the predicted signature — the five boundary cases, plus the mutual-exclusivity invariant:

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 6 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > invalid models > should return false for model: gemini-10.0-pro
 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > invalid models > should return false for model: gemini-1
 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > invalid models > should return false for model: gemini-1-pro
 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > invalid models > should return false for model: gemini-1.
 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > invalid models > should return false for model: gemini-1x-foo
AssertionError: expected true to be false // Object.is equality
- Expected
+ Received
- false
+ true

 FAIL  core/test/utils/model_name_test.ts > isGemini1Model > classification invariants > should never classify a model as both Gemini 1.x and Gemini 2.0+
AssertionError: expected [ 'gemini-10.0-pro' ] to deeply equal []
- Expected
+ Received
- []
+ [
+   "gemini-10.0-pro",
+ ]

 Test Files  1 failed (1)
      Tests  6 failed | 38 passed (44)

The other 10 cases pass both before and after, which is expected — they pin the behaviour the change must not alter.

Manual End-to-End (E2E) Tests:
No E2E test is applicable: isGemini1Model is a pure, module-internal, string-in/boolean-out predicate with no I/O, network or DI surface, and exercising the headline case end-to-end would require a published gemini-10.x model, which does not exist. The behaviour table above, reproduced as executable assertions, is the complete verification.

To reproduce the fix locally:

npm ci && npm run build
npx vitest run --project unit:core core/test/utils/model_name_test.ts

Standalone sanity check of the regex itself:

node -e "const r=/^gemini-1\.\d+/; for (const m of ['gemini-1.5-flash','gemini-10.0-pro','gemini-1','gemini-1-pro','gemini-1.','gemini-1x-foo']) console.log(m, r.test(m));"
gemini-1.5-flash true
gemini-10.0-pro false
gemini-1 false
gemini-1-pro false
gemini-1. false
gemini-1x-foo false

Lint and format gates on the changed files pass:

npx eslint core/src/utils/model_name.ts core/test/utils/model_name_test.ts   → exit 0
npx prettier --check core/src/utils/model_name.ts core/test/utils/model_name_test.ts
  → All matched files use Prettier code style!

Disclosure on npm run ts:check: it is red on this branch, but it is equally red on the base commit — npx tsc --noEmit --pretty false reports 281 errors both with and without this change, none of them in either file touched here (all are pre-existing BASE_AGENT_SIGNATURE_SYMBOL assignability errors in core/test/a2a/* and similar). This PR neither introduces nor fixes any of them; fixing them is out of scope.

No suppressions of any kind were added: git diff <base> -U0 | grep -E '@ts-expect-error|@ts-ignore|eslint-disable|as any|: any|v8 ignore' returns nothing.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

CI status

All checks green on the pushed commit: run-tests passes on ubuntu-latest, windows-latest and macos-latest, plus the standalone run-tests, check-license and auto-assign jobs.

macos-latest failed on the first two attempts, both times on a pre-existing flake unrelated to this change, and passed on re-run:

FAIL  integration  tests/integration/app_loader/app_loader_test.ts
  > AgentLoader discovery and loading integration
  > should discover apps vs agents across directories and standalone files
Error: Test timed out in 40000ms.

Test Files  1 failed | 223 passed | 20 skipped (244)

Recording it here rather than quietly re-running, since it is worth knowing the job is unstable:

No change was made to accommodate it; fixing it here would mean editing an unrelated integration test in this diff.

isGemini1Model used a bare `startsWith('gemini-1')` prefix test with no
digit boundary, so it accepted any id whose version field merely begins
with the character `1`. A future `gemini-10.0-pro` would be routed down
the legacy Gemini 1.x paths in GoogleSearchTool, VertexAiSearchTool,
applyGoogleMapsGrounding and applyEnterpriseWebSearch, and would also be
classified as both Gemini 1.x and Gemini 2.0+ at the same time.

Replace the prefix test with the `^gemini-1\.\d+` regex adk-python's
is_gemini_1_model already uses, hoisted to a module-level non-global
constant so `.test()` stays stateless. This makes the existing doc
comment ("using regex patterns") honest and restores mutual exclusivity
with isGemini2OrAbove.

Behaviour is unchanged for every real Gemini 1.x id; only gemini-10.0-pro,
gemini-1, gemini-1-pro, gemini-1. and gemini-1x-foo flip to false.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant