Test: pin the CFC model gate semantics in the runner (adk-python parity) - #716
Open
AmaadMartin wants to merge 2 commits into
Open
Test: pin the CFC model gate semantics in the runner (adk-python parity)#716AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 6, 2026 04:13
The runner gates supportCfc on isGemini2OrAbove, so it accepts Gemini major
2 and above and unwraps a Vertex publisher path. adk-python uses a literal
startsWith('gemini-2') on the raw id, so it rejects gemini-3.0-pro and the
path form. Neither runtime has a test on this gate.
Add core/test/runner/runner_cfc_test.ts to pin the adk-js semantics, plus a
comment above the gate explaining why it uses the shared predicate. No
production behaviour changes.
The removed lines restated isGemini2OrAbove's own contract, claimed a
provider-wrapper strip that extractModelName does not do, and described a
startsWith('gemini-2') implementation this file never had.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Link to an existing issue (if applicable):
N/A
Or, if no issue exists, describe the change:
Problem: The
supportCfcmodel gate disagrees across the two SDKs, and neither SDK tests it. adk-jsrunner.tscallsisGemini2OrAbove, so it acceptsgemini-3.0-proand a Vertex publisher path. adk-pythonrunners.pycallsstartswith('gemini-2')on the raw id, so it rejects both. An untested gate can be "fixed" on the wrong side.Solution: Pin the adk-js semantics with tests. CFC is supported if, and only if, the canonical model id names a Gemini model of major version 2 or above, after the Vertex publisher path is stripped. adk-python is the side that needs correcting: its gate must admit what its own
BuiltInCodeExecutoradmits, and that executor usesis_gemini_eap_or_2_or_above. adk-js behaviour is unchanged; this is one new test file plus a one-line comment.Scope notes:
extractModelName(open PR Fix: widen extractModelName to the adk-python model-id grammar (models/, apigee/, provider-prefixed) #471 widens it).core/src/runner/runner.ts. Open PR Fix: tighten the Gemini-1 boundary and split the EAP gate out of isGemini2OrAbove (adk-python parity) #615 (fix/model-name-eap-and-path-parity) adds aRunner CFC model gateblock tocore/test/runner/runner_test.tscovering two reject ids. This PR uses a separate file,runner_cfc_test.ts, so the two branches do not conflict and the stub classes do not collide. Merging the two homes is a follow-up once Fix: tighten the Gemini-1 boundary and split the EAP gate out of isGemini2OrAbove (adk-python parity) #615 lands.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
New file
core/test/runner/runner_cfc_test.ts, 4 cases. Each drives a realRunnerover anInMemorySessionServicewithrunConfig: {supportCfc: true}and asserts on the gate's own observable,agent.codeExecutor.gemini-2.5-flashgemini-3.0-proprojects/.../publishers/google/models/gemini-2.5-flashclaude-3-5-sonnetThe accept cases stop downstream at
CFC is not yet supported in callLlmAsync, because the CFC call path is not implemented (llm_agent.ts). The tests assert that stop; they do not assert a successful run.Proof the tests can fail. The only source change is a comment, so mutation testing is the only evidence these tests carry signal. I ran both mutations against the new file.
Mutation 1, the adk-python literal. Replace
!isGemini2OrAbove(modelName)with!modelName.startsWith('gemini-2'). 2 of 4 fail:Mutation 2, remove the gate. Replace the condition with
false. 1 of 4 fails:Mutation 2 also shows why the gate should track the executor: with the gate gone,
BuiltInCodeExecutorrejects the same model one layer later.Both mutations were reverted. Coverage: the source change is a comment, so it adds no executable line. The tests cover both branches of the gate.
Manual End-to-End (E2E) Tests:
None. adk-js does not implement the CFC call path, so there is no end-to-end CFC run to exercise. The mutation runs above are the substitute.
Local validation on the pushed commit:
npm run ts:checkreports pre-existing errors in the core test tree (open PR #248 fixes them). It reports none for the two files in this PR.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.