Fix: validate --log_level against a closed vocabulary and stop discarding LogLevel.DEBUG - #592
Open
AmaadMartin wants to merge 3 commits into
Open
Conversation
added 3 commits
August 3, 2026 13:56
`--log_level` had no validator, so a typo such as `debbug` silently resolved to INFO on every command that carries the option, including the deploy commands, which bake the raw string into the generated Dockerfile entrypoint and the gcloud `--verbosity` argv. The bad value then degrades logging for the lifetime of the deployment with no signal to the user. `--log_level debug` was also broken independently: `LogLevel.DEBUG` is 0, so the `||` fallback in `getLogLevelFromOptions` discarded it and the CLI ran at INFO. Only `-v/--verbose` reached DEBUG. Attach the level vocabulary to the shared `LOG_LEVEL_OPTION` instance so all seven commands validate at parse time, and switch the fallback to `??`. The choices are derived from `LOG_LEVEL_MAP`, which stays the single source of truth. A custom `argParser` keeps commander's help and error text while accepting any letter case and normalizing to the canonical lower-case name, matching the Python SDK's `click.Choice(..., case_sensitive=False)` and keeping today's `--log_level DEBUG` invocations working.
…evels `parseLogLevel` tested membership with `name in LOG_LEVEL_MAP`. `in` walks the prototype chain, so `constructor` and `__proto__` — the two `Object.prototype` keys that survive lower-casing — passed the validator. `adk web --log_level constructor` exited 0 and started the server with the `Object` constructor stored as the log level, which makes every subsequent `logLevel > level` guard a NaN comparison; and `deploy cloud_run --log_level __proto__` reached `deployToCloudRun`, baking `--log_level=__proto__` into the generated container entrypoint. Route both the validator and `getLogLevelFromOptions` through one prototype-safe `resolveLogLevel` helper built on `Object.hasOwn`. Sharing the lookup also closes the same hole in the defensive fallback, so that fallback is finally total for programmatic callers as intended, and adds no separately-tested code path. The rejection suite now covers `constructor`, `__proto__` and their upper-case forms, and asserts an inherited key cannot reach the Cloud Run deploy path. `expectNoActionRan` also asserts `runIntegrationTests` was not called, so the `integration conformance` row of the rejection table is no longer vacuous.
A Map has no prototype chain to walk, so `constructor` and `__proto__` are ordinary misses. That removes the need for the hand-rolled `Object.hasOwn`-based lookup and the comment explaining why the prototype chain was a hazard: `has` and `get` are the guard. `Map` preserves insertion order, so `LOG_LEVEL_CHOICES` and the help text are unchanged. Also drop the `.toLowerCase()` in `getLogLevelFromOptions`. It is dead now that `LOG_LEVEL_OPTION` is the only writer of `options.log_level` and `parseLogLevel` returns the canonical lower-case name (commander's `'info'` default bypasses `parseArg` already lower-cased), and keeping it implied unvalidated input could reach that line.
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A — no public issue is tracking this.
Problem: Two distinct defects in the same four lines of
dev/src/cli/cli.ts.Defect 1 —
--log_levelhas no validator.LOG_LEVEL_OPTIONwas declared with nochoices()and noargParser(), andgetLogLevelFromOptionsresolved the value withLOG_LEVEL_MAP[value] || LogLevel.INFO. Any string outsidedebug|info|warn|error(
debbug,DEBUG2,trace,"") silently fell through to INFO with no warning andexit code 0. Every command carrying the shared option was affected:
web,api_server,run,deploy cloud_run,deploy agent_engine,deploy reasoning_engine, andintegration conformance.The deploy paths were the worst case, because the bad value is persisted:
deploy_utils.tsappends--log_level=${options.logLevel}verbatim into the entrypointbaked into the generated Dockerfile, and
cli_deploy_cloud_run.tslowercases the samevalue into the
gcloud run deploy --verbosityargv. Verified against the pre-fix build —adk deploy cloud_run --log_level DEBUG2produced:so a typo degrades logging for the lifetime of the deployment.
Defect 2 —
--log_level debugnever enabled debug logging.LogLevelis a numericenum with
DEBUG = 0, which is falsy, soLOG_LEVEL_MAP['debug'] || LogLevel.INFOevaluated to
0 || 1 === 1.adk web --log_level debugran at INFO. Only-v/--verbosereached DEBUG, because it takes an early return and never touches the map. Confirmed
end-to-end against the pre-fix build (full table below). No existing test covered this —
the only
LogLevel.DEBUGassertion incli_test.tsgoes through--verbose.Solution: Attach the vocabulary to the single shared
LOG_LEVEL_OPTIONinstance soall seven commands validate at parse time, and switch the fallback from
||to??.LOG_LEVEL_CHOICES = [...LOG_LEVEL_MAP.keys()]— the level table stays the single sourceof truth, so the advertised choices cannot drift from the levels we can resolve. No second
literal
['debug','info','warn','error']is written anywhere..choices(LOG_LEVEL_CHOICES).argParser(parseLogLevel)—.choices()populatesargChoices, which is what--helpand commander'sAllowed choices are ...messagerender;
.argParser()then replaces commander's case-sensitive validator with acase-insensitive one that normalizes to the canonical lower-case name. Order matters and
is called out in a comment. Errors go through commander's own channel
(
InvalidArgumentError→Command.error()); no bespoke error type, no hand-rolledconsole.error+process.exit.??instead of||ingetLogLevelFromOptions, soLogLevel.DEBUG === 0survives.LOG_LEVEL_MAPis aMap<string, LogLevel>rather than an object literal. AMaphas noprototype chain to walk, so
constructorand__proto__are ordinary misses forhas/get— no guard and no explanation needed. See the review-round notes below.Deliberate decision:
--log_level DEBUGkeeps working. A bare.choices()iscase-sensitive and would reject upper-case values that work today — a second, unrelated
breaking change smuggled into a bug fix. The Python SDK validates the same flag with
click.Choice(..., case_sensitive=False), and the CLI surface is observable across thelanguage boundary, so parity wins here over a 3-line implementation. Normalizing in the
parser also means the value baked into a generated Dockerfile is always canonical, so the
containerised
adk api_server— which after this change validates its own--log_level—can never reject an image the CLI just produced. Alternative considered and rejected:
plain
.choices(LOG_LEVEL_CHOICES)with no custom parser.Deliberately out of scope (independent of this fix, queued separately):
--log_level warnproduces the invalid gcloud verbositywarn(gcloud wantswarning);--verboseissilently ignored on the deploy commands and beats an explicit
--log_levelelsewhere,contrary to adk-python; and the JS/Python vocabularies differ (
warnvsWARNING/CRITICAL). Nothing from the approved spec was dropped.This is a breaking CLI change. Behaviour changes worth calling out to users:
--log_level <anything outside debug|info|warn|error>now exits non-zero. Previously--log_level trace(or any typo) fell back to INFO silently. Failing loudly is the pointof this PR, but a script that passed a bad level and relied on it being ignored will now
break — with a message naming the fix.
--log_level debugnow actually produces DEBUG. That is the advertised behaviour finallytaking effect, but it is a real change in log volume for anyone already passing it.
deployToCloudRun/deployToAgentEnginestill take anunvalidated
logLevel: string(deliberately — the spec keeps the programmatic API open).If a programmatic caller passes something like
'warning',deploy_utils.tsbakes--log_level=warninginto the entrypoint and the containerisedadk api_serverwill nowreject it at boot rather than degrading to INFO. Garbage-in, and arguably the better
failure mode, but it is a behaviour change outside the CLI and worth stating.
Review round 1 — prototype-chain bypass. The first revision validated with
if (!(normalized in LOG_LEVEL_MAP)).inwalks the prototype chain, soadk web --log_level constructorexited 0, started the server, and stored theObjectconstructor as the log level (making every later
this.logLevel > levelguard aNaNcomparison), while
deploy cloud_run --log_level __proto__reacheddeployToCloudRunandbaked
--log_level=__proto__into the generated entrypoint — reproducing both of thedefects this PR exists to eliminate.
constructorand__proto__are the onlyObject.prototypekeys that survive.toLowerCase(), so they were the whole exposure.Review round 2 — the level table is now a
Map. Round 1 fixed the bypass with anObject.hasOwn-basedresolveLogLevelhelper; that was a hand-rolled own-key lookup plus acomment explaining the hazard. A
Maphas no prototype chain at all, sohas/getare theguard and both the helper and its rationale comment are gone — the bug class is removed
structurally rather than guarded against (net −15 lines in
cli.ts). This also makes specinvariant 2 true for the first time:
LOG_LEVEL_MAP['constructor'] ?? LogLevel.INFOreturned
Objectonmain, so "returnsLogLevel.INFOfor an unrecognised string" wasnever actually the case.
Mappreserves insertion order, soLOG_LEVEL_CHOICESand thehelp text are byte-identical and no test changed. The
??stays — it is the Defect 2 fixand is still required by
get'sLogLevel | undefinedreturn.While there, the
.toLowerCase()ingetLogLevelFromOptionsis dropped as dead:LOG_LEVEL_OPTIONis the only writer ofoptions.log_levelat everyaddOptionsite,parseLogLevelalready returns the canonical lower-case name, and commander's'info'default bypasses
parseArgalready lower-cased. Keeping it implied unvalidated input couldreach that line. Removing it also strengthened a mutation:
parseLogLevelreturningvalueinstead of
normalizednow fails 3 tests rather than 1.Collision check (required before implementation).
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000returned 490 open PRs; none implements--log_levelvalidation. The nearest neighbours were reviewed and are unrelated: #431(
getLogLevel()accessor on the core logger), #557 / #349 (pinning the log level insidevitest workers), #432 (winston logger dedup) — all in
core/, none touchingdev/src/cli/cli.ts.This PR is stacked on #451 (
fix/cli-file-type-choices-validation, itself stacked on#358), which touches the same two files for the sibling
--file_typeflag. It is not acompeting implementation — different option, different code region — but it lands the
applyExitOverride/argvFor/findCommand/expectNoActionRantest helpers thischange needs, so stacking reuses them instead of adding a second copy that would conflict
on merge. Base is
fix/cli-file-type-choices-validation; please merge #358 and #451 first.Files changed:
dev/src/cli/cli.tsonly, on the production side.deploy_utils.ts,cli_deploy_cloud_run.ts,cli_deploy_agent_engine.ts,core/src/utils/logger.tsandAGENT_FILE_MODULE_TYPEare untouched. In particular the now-redundant.toLowerCase()incli_deploy_cloud_run.tsstays:deployToCloudRunis also callable programmatically,bypassing commander.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
29 new cases in
dev/test/cli/cli_test.ts, in a newdescribe('option: --log_level validation')block. No existing test was modified, weakened, skipped, or deleted; theshared
beforeEachis untouched. The one edit to shared test scaffolding is additive:expectNoActionRan()now also assertsrunIntegrationTestswas not called, so theintegration conformancerow of the rejection table is not vacuous.Coverage:
dev/src/cli/cli.tsis covered, except one:the
LogLevel.INFOright-hand arm of?? LogLevel.INFO(line 70, col 46-65) reports 0hits. That is unreachable by design: commander now guarantees the CLI only ever hands
getLogLevelFromOptionsa validated, normalized level. The fallback is deliberately keptso the function stays total for programmatic callers (
deployToCloudRun/deployToAgentEnginestill accept an unvalidatedlogLevel: stringand bypasscommander). Per the repo guideline, the safety net is kept and the measured shortfall
reported rather than deleting a guard to move a number. Measured with
--coverage.include='dev/src/cli/cli.ts'and read out ofcoverage-final.json;parseLogLevelis hit 22 times with both arms exercised.typeof options.log_level === 'string') is pre-existing code, unreachable becauseLOG_LEVEL_OPTIONhas.default('info'). Not introduced or changed here.Proof the tests can fail. Each mutation below was applied to
dev/src/cli/cli.tsandthe targeted suite re-run:
??reverted to||applies --log_level debugandaccepts --log_level DEBUG regardless of letter case:AssertionError: expected "spy" to be called with arguments: [ +0 ](i.e.LogLevel.DEBUG; INFO was passed).argParser(parseLogLevel)removedaccepts --log_level DEBUG…,accepts --log_level Warn…,normalizes the value handed to the Cloud Run deploy path.choices(LOG_LEVEL_CHOICES)removedlists the accepted levels in \` help` case.choices()and.argParser()removed (pre-fix wiring)parseLogLevelreturnsvalueinstead ofnormalizednormalizes the value handed to the Cloud Run deploy path(expected { …(15) } to match object { logLevel: 'debug' }, actual"logLevel": "DEBUG") plus both case-insensitivity casesLOG_LEVEL_MAPreverted to a plain object literal consulted withinrejects the inherited key --log_level constructor / __proto__ / CONSTRUCTOR / __PROTO__andkeeps an inherited key out of the Cloud Run deploy path:AssertionError: promise resolved "undefined" instead of rejectingThe inherited-key list is exactly the discriminating set:
constructorand__proto__arethe only
Object.prototypekeys that survive.toLowerCase(), so cases liketoStringwere dropped from the table rather than shipped as tests that pass either way.
Manual End-to-End (E2E) Tests:
Please describe the tests that you ran to verify your changes.
All of the following were run against the real built CLI (
npm run build, thennode dev/dist/esm/cli_entrypoint.js …) with nothing mocked.Rejection is total, on every option holder. Each exits
1, prints commander'sstandard message, and runs no action:
Same for
run agent.ts --log_level trace,api_server --log_level verbose,integration conformance --log_level '', anddeploy cloud_run --log_level DEBUG2(which sets
allowUnknownOption()— confirmed it does not bypass the validator).Also re-verified for the prototype-chain keys after the review fix —
web --log_level constructor,__proto__,CONSTRUCTOR,__PROTO__anddeploy cloud_run --log_level __proto__all exit1with the standard message.Nothing is written on the deploy path when the level is invalid.
deploy cloud_run --log_level DEBUG2 --temp_folder <dir>exits 1 and<dir>is nevercreated. Same for
--log_level __proto__.Level resolution, real CLI, before vs after. Driven through
createProgram()fromdev/dist/with a recording logger installed via the publicsetLogger()extensionpoint:
adk webadk web --log_level debugadk web --log_level DEBUGadk web --log_level Warnadk web --log_level erroradk web --log_level debbugadk web --log_level constructorObject, server startedadk web --log_level __proto__{}, server startedadk web --verboseGenerated Dockerfile is normalized.
deploy cloud_run --log_level DEBUG …, with thetemp folder captured before cleanup:
(pre-fix, the same run with
DEBUG2baked in--log_level=DEBUG2).Help advertises the choices.
node dev/dist/esm/cli_entrypoint.js web --help:CI note — no test job will run on this PR.
.github/workflows/validation.yamlisgated on
pull_request: branches: [main], and this PR targetsfix/cli-file-type-choices-validation(see the stacking note above), so the workflow nevertriggers. Validated locally instead, on the exact pushed commit:
The one failure across the whole
dev/test/clidirectory iscli_create_test.ts > should handle Vertex AI selection with gcloud defaults, which ispre-existing and environment-dependent — it reads the developer's ambient gcloud project
instead of the mocked one. Confirmed it fails identically with this branch's changes
stashed, and it is unrelated to
--log_level(see the open hermeticity fixes #569 / #576 /#589).
(
npm run ts:checkis not a CI gate and reports ~308 pre-existing errors incore/test/**andtests/integration/**on this base and ~331 onmain; none are in thefiles touched here, and the count is unchanged by this PR.)
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.