Ci: report high and critical production dependency advisories in validation - #799
Open
AmaadMartin wants to merge 2 commits into
Open
Ci: report high and critical production dependency advisories in validation#799AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 8, 2026 03:22
The validation workflow never looked at the dependency tree, so GHSA-xcpc-8h2w-3j85 shipped in three releases of @google/adk before a downstream user reported it (google#601). Add an audit-dependencies job that runs npm audit --omit=dev --audit-level=high and writes the report to the run summary. The job reports and does not gate: npm audit cannot acknowledge a finding, so a blocking gate would fail every open pull request the moment a new advisory lands against a dependency we already ship.
Drop the setup-node step: it supplies no node-version, so it pins nothing over the npm the runner already provides, and license-check.yml is checkout-plus-run for the same reason. Drop `if: always()` and the missing-log fallback. `continue-on-error` already lets the publish step run, and `tee` creates the log before the audit can fail, so neither guard has a reachable case. Move the --omit=dev and advisory-policy prose out of the workflow comment; CONTRIBUTING.md is the one copy. Tighten that section too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Related: adm-zip range ^0.5.17 pins consumers to a version with an unpatched DoS (GHSA-xcpc-8h2w-3j85) google/adk-js#601
Problem: The
validationworkflow checks secrets, build, tests, coverage, lint, format and docs, but it never looks at the dependency tree. A high-severity advisory (GHSA-xcpc-8h2w-3j85,adm-zip < 0.6.0) therefore shipped in three releases of@google/adkbefore a downstream user reported it. CI gave no signal at any point.Solution: Add an
audit-dependenciesjob that runsnpm audit --omit=dev --audit-level=highand writes the report to the run summary. The job reports and does not gate.npm audithas no way to acknowledge a finding, so a blocking gate would fail every open pull request the moment a new advisory lands against a dependency we already ship. The escalation path is one line: once the production tree is clean athigh, dropcontinue-on-error.Two decisions worth stating:
run-tests.run-testsis a 3-OS matrix, so a step would audit three times for an OS-independent check. Its steps are also sequential, so an audit step would not run when the build or the tests fail. The job needs nonpm installand nosetup-node, becausenpm auditresolves the tree frompackage-lock.jsonand the runner already provides npm.@mikro-orm/sqlite@7.1.11bump, which a CI-configuration change cannot take. An allowlist was rejected:npm audithas no ignore flag, so it would mean a custom script plus a JSON file, and it would still red every open pull request on the next new advisory.Scope is CI configuration and
CONTRIBUTING.mdonly. Theadm-ziprange bump is tracked separately and is not in this change.One thing I kept. A simplicity review asked me to delete the
Publish audit reportstep, because the job log already prints the same bytes. I kept it: writing the report to the run summary is the point of the job, and it is what makes the finding visible without expanding a job log. I took the rest of that review — thesetup-nodestep, theif: always()guard, the missing-log fallback, and the duplicated policy prose in the workflow comment are all gone.Collision check. I listed the 300 open pull requests on the fork and diffed every plausibly adjacent one. No open or closed pull request adds a dependency audit. Six open pull requests touch
.github/workflows/validation.yaml(#507, #649, #651, #669, #713, #756) and none of them audits dependencies; #651 is the closest, and it asserts that declared production dependencies install, which is a different check. I branched frommainand did not stack.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[ ] I have added or updated unit tests for my change.
[ ] All unit tests pass locally.
No unit tests. The change adds no executable source: two jobs' worth of workflow configuration and a Markdown section. The vitest coverage
includeglobs reach onlycore/src,dev/srcandintegrations/src, so no coverage number moves. I verified the shell instead, as below.Manual End-to-End (E2E) Tests:
I executed the two step bodies exactly as committed, read out of the YAML file, under
bash -ewithRUNNER_TEMPandGITHUB_STEP_SUMMARYset to temp files. My local npm mirror has no audit endpoint, so the local runs pass--registry=https://registry.npmjs.org/; the workflow does not.found 0 vulnerabilities(no audit output captured), fence closedThe audit reports
36 vulnerabilities (3 low, 22 moderate, 10 high, 1 critical)over 552 production packages, includingadm-zipwithGHSA-xcpc-8h2w-3j85.Both guards were mutation-tested, and both mutations produce a wrong result:
set -o pipefail: the audit step exits0over 11 findings, so the summary printsNo high or critical advisories in the production dependency tree.directly above the report listing them.--audit-level=high: in a scratch package whose only advisory is moderate,npm audit --omit=devexits1where--audit-level=highexits0. The flag is what makes the verdict line mean "high or critical" rather than "any advisory".steps.audit.outcomeis used rather thanconclusionbecause GitHub documentsconclusionas the value aftercontinue-on-erroris applied, which is alwayssuccesshere.On this pull request the job ran green in 12 seconds with no install step, and its log reports the same 36 vulnerabilities including
adm-zip/GHSA-xcpc-8h2w-3j85. The threerun-testslegs are unchanged and pass.Reproduce locally:
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] New and existing unit tests pass locally with my changes.