Test: pin the EAP pattern boundaries and version-path parity negatives (stacked on #372) - #618
Open
AmaadMartin wants to merge 4 commits into
Open
Conversation
added 4 commits
August 3, 2026 22:30
…tives Extends the isGemini2OrAbove suite with the edge cases the EAP change leaves unpinned, so a later loosening of the pattern cannot pass silently: - regex shape: 'gemini-flash-early-experiment' and 'gemini-flash-early-expX' fail if the pattern loses its trailing anchor; 'gemini-early-exp3' fails if the <variant> segment becomes optional. - version path (adk-python parity negatives): 'gemini-one', 'gemini-0.9-test', 'gemini-2.' and 'gemini-' fail if an unparseable version is admitted or the major >= 2 bound is dropped. - the falsy guard, which no test reached before. - 'gemini-2.5-flash-early-exp' -> true, pinning that the EAP branch only short-circuits on a match rather than returning its own verdict, which would demote versioned names carrying the suffix. Every case was verified to fail under a targeted mutation of the line it pins.
…L-context suite
The EAP case added a 15-line makeToolContext() factory with a single caller,
pulling five more imports into the file, to build an InvocationContext that
UrlContextTool.processLlmRequest never reads - it destructures only
{llmRequest}. The assertion it enabled was already covered elsewhere, so the
scaffolding was cost without signal.
The alternative of passing `toolContext: {} as never`, as the file's six other
call sites do, was rejected: it trades a real object for an unchecked cast, and
the cast is only avoidable here by not needing the argument at all.
EAP admission at a consumer gate stays pinned by
built_in_code_executor_test.ts, which needs no context and no cast, and the
predicate itself is exhaustively covered in utils/model_name_test.ts.
The convention and its two examples were spelled out both on EAP_MODEL_NAME_PATTERN and again in the isGemini2OrAbove doc. Keep the normative description next to the pattern and have the function reference it, mirroring how adk-python cross-references its helper. Also drops the note that the literal carries no `g` flag: it documented the absence of a bug that a reader of a non-`g` literal would not suspect.
Removing the inert makeToolContext() scaffolding in the previous commit took the test case with it, which went too far. UrlContextTool is the call site that *throws* on a false negative -- "URL context tool requires Gemini 2 or above, but got gemini-flash-early-exp" is the user-visible symptom this change exists to remove -- and no test guarded that wiring once the case was gone. The unit tests on isGemini2OrAbove cover the predicate, not the tool's use of it. The case is back in the one-line form the file's six other call sites already use, so it costs 11 lines with no helper and no new imports, rather than the 37 the factory version cost. Verified to fail without the fix: deleting the EAP short-circuit from isGemini2OrAbove makes this test fail with the exact production error above.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
Collision check (run before any code was written).
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000surfaced two live PRs on thisground:
fix/gemini-eap-model-name-parity(OPEN, unmerged, basemain). It already lands theentire fix I was assigned: the module-level
EAP_MODEL_NAME_PATTERN(
/^gemini-[a-z0-9_]+(?:-[a-z0-9_]+)*-early-exp\d*/— anchored, nogflag) incore/src/utils/model_name.ts, short-circuited insideisGemini2OrAboveahead of thegemini-prefix check and the version parse, plus EAP cases in the model-name,
BuiltInCodeExecutorandUrlContextToolsuites. Building a second implementation of it would be pure duplicate work, sothis PR does not touch
core/src/at all — it stacks on Fix: recognise Gemini EAP model names in isGemini2OrAbove (adk-python parity) #372 and contributes only the testcases Fix: recognise Gemini EAP model names in isGemini2OrAbove (adk-python parity) #372 leaves unpinned.
fix/model-name-eap-and-path-parity(OPEN, stacked onfix/extract-model-name-parity).Overlaps more broadly: it exports a new
isGeminiEapOr2OrAboveand widens the accepted pathforms. It also overlaps the separately-queued
extractModelNamewidening. Whoever lands Fix: recognise Gemini EAP model names in isGemini2OrAbove (adk-python parity) #372should reconcile Fix: tighten the Gemini-1 boundary and split the EAP gate out of isGemini2OrAbove (adk-python parity) #615 against it; this PR deliberately stays out of that.
Problem: #372 fixes the defect, but its suite pins the happy shape of the EAP pattern more
than its boundaries. Several ways of loosening the regex, and several ways of breaking the
version-parsing path that the EAP branch now sits in front of, still pass on #372's head. Concretely,
none of these were covered: a suffix that merely starts with
early-exp(
gemini-flash-early-experiment), a non-digit character afterearly-exp(gemini-flash-early-expX),a missing
<variant>segment combined with a numeric suffix (gemini-early-exp3), the adk-pythonparity negatives for the version path (
gemini-one,gemini-0.9-test,gemini-2.,gemini-), andthe
if (!modelString)guard, which no test reached.Solution: 17 added lines in
core/test/utils/model_name_test.ts, appended only — no existingentry, ordering or assertion was edited or removed. Two follow-up commits then address a complexity
review of the whole stack (see Complexity review below); together they take the stack from
4 files / +124-3 to 4 files / +99-2.
gemini-flash-early-experiment,gemini-flash-early-expXnonEapModelsearly-expis exact, not a prefixgemini-early-exp3nonEapModels<variant>segment is mandatory even with a numeric suffixgemini-one,gemini-0.9-test,gemini-2.,gemini-invalidModelsmajor < 2)isGemini2OrAbove('')itininvalid modelsgemini-2.5-flash-early-exp→trueitinEAP modelsThat last one is the load-bearing one and is worth spelling out. The natural way to write this
feature wrong is
if (name.includes('early-exp')) return EAP_PATTERN.test(name);— which reads fine,passes every other test in the file, and silently demotes a versioned name carrying the suffix
from
truetofalse. It is the only test that catches that, and it is what enforces the change'smonotonicity property (nothing that returned
truebefore may returnfalsenow).I did not reshape #372's implementation. The elaborated design I was given sketches a private
isGeminiEapModel(modelString)wrapper that callsextractModelNameitself; #372 instead tests thepattern against the already-extracted name inside
isGemini2OrAbove. Same regex, same orderingrelative to the prefix check and version parse, identical truth table — rewriting a sibling PR's
just-reviewed hunk into an equivalent shape is churn, not an improvement.
One deliberate, pre-existing divergence from adk-python, inherited from #372 and left alone: Python's
$also matches immediately before a trailing newline, sore.matchaccepts'gemini-flash-early-exp\n'; the JS$without themflag does not. JS is stricter and morecorrect here, so it is not emulated and no test asserts the Python quirk.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
Proof each new test can fail. Every case was run against a targeted mutation of the exact line it
pins; each failed, and the source was restored and re-verified green (50/50) after each.
core/src/utils/model_name.ts$from the patterngemini-flash-early-experiment,gemini-flash-early-expX(+ #372's…-exp-001)[a-z0-9_]+(?:-[a-z0-9_]+)*-early-exp→(?:[a-z0-9_]+-)*early-exp(variant optional)gemini-early-exp3(+ #372'sgemini-early-exp)return parsedVersion.valid && parsedVersion.major >= 2→return parsedVersion.validgemini-0.9-test(+gemini-1.5-pro,gemini-1.0-pro)return !parsedVersion.valid || parsedVersion.major >= 2gemini-one,gemini-2.,gemini-(+ 9 others)if (!modelString) return false→return trueshould return false for an empty model string(only that one)if (name.includes('early-exp')) return EAP_PATTERN.test(name)should admit a versioned name carrying the EAP suffix(only that one)Example failure output (mutation A):
Two honest caveats rather than overclaiming:
it, but deleting the guard outright leaves all 50 green —
''falls through to''.startsWith('gemini-')and returnsfalseanyway. So it buys branch coverage of that guard andlocks its behaviour; it is not a regression signal for a reachable defect. Stated plainly so nobody
counts it as more than it is.
gemini-2.5-flash-early-expis a genuine signal, but only against mutation G. Widening thecharacter class to include
.(so a versioned name does match the EAP pattern) leaves it passing,because the answer is
truedown either branch; that variant is caught by Fix: recognise Gemini EAP model names in isGemini2OrAbove (adk-python parity) #372's existinggemini-1.5-flash-early-expnegative instead. The two tests are complements.Repo-wide
npm run ts:checkis red on this branch — and equally red on the base, unrelated to thischange.
npx tsc --noEmit --pretty falsereports 280 errors with this diff and the same 280 on#372's pristine head, 0 of them in any file this PR touches. They are all the
core/dist/types/... is not assignable to core/src/...dual-identity artifact that appears once abuilt
core/distexists in the working tree (removingcore/distmakes it worse, 741, since@google/adkthen resolves to nothing).npx tsc --noEmit -p core/tsconfig.json— the project thatactually covers
core/src— exits 0. Nothing here is mine to fix and I have not touched it.Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
Run against the real compiled build (
npm run build, then import fromcore/dist/esm/index.js— nomocks, no stubs), exercising the predicate and both downstream gates that the EAP fix unlocks:
Observed, in full:
Per the design note, this probe is not committed as a test file: the two consumers are gated by the
same predicate and already have their own suites (#372 extends both), and a queued follow-up will be
editing exactly those two files.
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.
Complexity review — findings addressed (two rounds)
A complexity audit of the full stack (
fcfd0436..HEAD) raised two items. Both sit in #372's commitsrather than in my test additions, but since they are in this branch's delta I fixed them here.
1.
makeToolContext()inurl_context_tool_test.ts— factory removed, test case kept(net +11 lines instead of the original +37). The finding was that a 15-line factory with one
caller built an
InvocationContextthe code under test never reads (processLlmRequestdestructures only
{llmRequest}), expanding a one-line import into a ten-line block.My first attempt at this deleted the factory and the test case, on the reasoning that an
assertion needing 15 lines of inert scaffolding was not worth its cost. That was wrong and the
follow-up review was right to block it.
url_context_tool.ts:43is the one call site ofisGemini2OrAbovethat throws on a false negative, andURL context tool requires Gemini 2 or above, but got gemini-flash-early-expis precisely theuser-visible symptom this change exists to remove. Deleting the case left that wiring with no
regression guard — the
model_name_test.tscases cover the predicate, not the tool's use of it.Coverage at a sibling consumer is not a substitute for the one that throws.
The case is restored in the one-line form the file's six other call sites already use: no helper,
no new imports, +11 lines rather than +37.
Proof the restored case is load-bearing. Deleting the EAP short-circuit from
isGemini2OrAbovemakes it fail with the production error verbatim:
2. Duplicated EAP prose in
model_name.ts— collapsed (-4 lines). The convention and bothexamples were spelled out on
EAP_MODEL_NAME_PATTERNand again in theisGemini2OrAbovedoc. Thenormative description now lives once, next to the pattern; the function references it in plain prose
(not
{@link}— TypeDoc runs with--treatWarningsAsErrorsand a link from an exported symbol to anon-exported one warns). Also dropped the "without the
gflag so.test()stays stateless" note,which documented the absence of a bug. No logic changed — the source diff is comments only plus
#372's original three-line short-circuit.
Everything the reviewer explicitly cleared was left alone: the regex is still verbatim the upstream
pattern, the check still precedes the
startsWith('gemini-')guard, and the inlined form is keptrather than ported into a wrapper.
CI status: absent (not green) — validated locally instead
No test job ran on this PR, and none will. Every test-bearing workflow in
.github/workflowsisgated on
pull_request: branches: [main](cross-language-integration.yml,license-check.yml);this PR's base is the stacked branch
fix/gemini-eap-model-name-parity, so they never trigger. Theonly check that reported is
auto-assign, which comes fromauto-assignment.yml(
pull_request: types: [opened], no branch filter) and is not validation of anything.Rather than call that green, here is the local run on the exact pushed commit
11e47a5, workingtree clean:
Full CI will run against these tests once #372 merges to
mainand this branch is retargeted.