Skip to content

Ci: pin Node 24, install with npm ci, and fail validation when package-lock.json is out of date - #467

Open
AmaadMartin wants to merge 2 commits into
mainfrom
feat/ci-lockfile-drift-check
Open

Ci: pin Node 24, install with npm ci, and fail validation when package-lock.json is out of date#467
AmaadMartin wants to merge 2 commits into
mainfrom
feat/ci-lockfile-drift-check

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    N/A — no existing issue.
  2. Or, if no issue exists, describe the change:

Problem: Nothing enforces that the committed package-lock.json is what npm
actually resolves from the committed package.json. The validation workflow
installs with npm install, which silently repairs a stale lockfile in the
runner's throwaway workspace and then discards the repair. Lockfile rot
therefore never surfaces in the PR that caused it; it accumulates and lands as
collateral churn in somebody else's PR. Concrete precedent: the
package-lock.json entry for adm-zip@0.5.17 carried a stale "dev": true
flag long after adm-zip had become a production dependency. It was finally
removed as collateral in 00f37755, "fix(deps): hoist @google/genai 2.x to the
workspace root" (#564) — 142 lines of lockfile churn in a PR about something
else, which a contributor then had to justify by hand.

The install was also not reproducible: actions/setup-node@v6 ran with no
inputs at all, so the job installed nothing and used whatever Node — and
therefore whatever bundled npm — the runner image happened to ship.

Solution: three edits to .github/workflows/validation.yaml, and nothing
else. The PR changes exactly one file.

  1. Pin the toolchain: node-version: '24'. Node 24 (Krypton) has been
    Active LTS since 2025-10-28 and is supported until 2028-04-30; Node 20
    (Iron) reached end-of-life on 2026-04-30 (verified against nodejs/Release
    schedule.json). Quoted deliberately — YAML parses an unquoted two-segment
    version such as 20.10 as the float 20.1.
  2. Install with npm ci instead of npm install, so the committed lockfile
    is authoritative rather than something the runner quietly repairs.
  3. Add a Check package-lock.json is up to date step immediately after the
    install: regenerate the lockfile and fail with an actionable annotation if
    the result differs from what is committed.

The pin and the gate are a package deal and ship together on purpose: without
the pin the gate would be non-deterministic across runner-image updates and
would eventually red-line every PR for no reason.

Three design points a reviewer should not have to re-derive:

  • The gate regenerates before diffing, and that is load-bearing. npm ci
    never writes a package-lock (npm docs: "It will never write to package.json
    or any of the package-locks: installs are essentially frozen"
    ), so npm ci
    followed by a bare git diff --exit-code is a check that can never fail —
    worse than adding nothing, because it looks like protection. The step runs
    npm install --package-lock-only first. The mutation test below proves this
    empirically rather than by argument.
  • The gate is not redundant with npm ci. npm ci only errors when the
    lock cannot satisfy the manifests' dependency specs. It exits 0 on both
    drift classes tested below — including the motivating one, a stale
    "dev": true, which is resolution metadata that only regeneration
    recomputes.
  • It runs once, on ubuntu-latest. Lockfile content does not vary by
    platform under lockfileVersion: 3, and the repo has no .gitattributes, so
    running the diff on windows-latest would risk whole-file noise from CRLF
    conversion.

The if ! git diff …; then … fi wrapper is required rather than a bare
git diff --exit-code: multi-line run: blocks execute under bash -e, which
would abort the step before the echo could emit the annotation, losing the
remediation message. Both outputs are kept — the diff says what drifted, the
annotation says what to do. The diff is scoped to package-lock.json so an
unrelated file touched by an earlier step cannot trip it.

Collision check. Per the pre-work check
(gh pr list --repo AmaadMartin/adk-js --state open --limit 100, then
gh pr diff <n> --name-only on every plausibly adjacent PR), six open PRs touch
CI install/toolchain configuration:

PR Overlap
#428, #416, #406 Also pin Node in validation.yaml ('24' / .nvmrc 22 / '22'), and all three additionally add an npm cache. They compete with each other as much as with this PR.
#415 Also changes npm installnpm ci in validation.yaml.
#427, #393 cross-language-integration.yml only — deliberately untouched here.
#450 Adds a separate deps:check step at the end of the same job; no conflict.

No open PR adds a lockfile drift gate, which is the substance of this
change, so this is not a duplicate of any of them. The pin and the npm ci
line do overlap #428/#416/#406 and #415 respectively. This PR was initially
stacked on #428's branch to avoid restating the pin, but that base also carries
edits to .github/workflows/cross-language-integration.yml and
CONTRIBUTING.md, and the cross-language-integration.yml change would collide
line-for-line with the separately queued task for that file. Keeping this PR to
a single file was worth more than avoiding the three-line overlap, so it is
based on main and carries the pin itself. If any of those PRs lands first the
overlapping hunks are identical or near-identical one-liners and rebase
trivially; only the pin's spelling could change, and the gate is unaffected.

Not in scope: .github/workflows/cross-language-integration.yml also runs a
bare npm install under an unpinned setup-node. It is queued separately and
is deliberately not edited here so the two changes cannot conflict. No npm cache
is added either — that is unrequested here and is what #428/#416/#406 are for.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:
[ ] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

Unit tests are not applicable: this change ships no executable code — it is
a GitHub Actions workflow document. There is nothing to import and assert on, so
no test file was added and vitest.config.ts is untouched. Coverage is
discharged instead by exercising both branches of the new gate locally, plus
a mutation test. Every run below executes the shipped script, extracted
straight out of the YAML and run under bash -e exactly as GitHub runs a
multi-line run: block:

python3 -c "import yaml; d=yaml.safe_load(open('.github/workflows/validation.yaml')); \
  open('/tmp/gate.sh','w').write(next(s for s in d['jobs']['run-tests']['steps'] \
  if s['name']=='Check package-lock.json is up to date')['run'])"

All runs are on Node 24.18.1 / npm 11.16.0 — the exact pair the pin resolves
to (nodejs.org/dist/index.json).

1. Positive control — the gate passes on a clean tree.

$ rm -rf node_modules && npm ci
npm ci exit=0
$ bash -e /tmp/gate.sh
POSITIVE CONTROL exit=0

The committed 15,680-line lockfile round-trips byte-identically under the pinned
toolchain, so the gate is green on main today. (This was the precondition to
check before writing any YAML: had it reported drift, either a regenerated
lockfile or a '22' pin would have had to ship with this PR. Neither is needed.)

2. Negative control — the gate fails on real drift. Drift introduced the way
a contributor introduces it: edit a manifest, do not regenerate the lock.

$ npm pkg set 'devDependencies.@types/js-yaml=^4.0.8'   # was ^4.0.9
$ npm ci --dry-run
exit=0                              # npm ci does NOT catch this
$ bash -e /tmp/gate.sh
diff --git a/package-lock.json b/package-lock.json
-        "@types/js-yaml": "^4.0.9",
+        "@types/js-yaml": "^4.0.8",
::error file=package-lock.json::package-lock.json is out of date. Run 'npm install' locally and commit the regenerated package-lock.json.
NEGATIVE CONTROL exit=1

Worth noting: npm ci exits 0 here, because the locked 4.0.9 still satisfies
^4.0.8. The install step alone would not have caught this. Both the diff and
the annotation are present, which is the check that the if ! … wrapper works —
a bare git diff --exit-code under bash -e would have exited before the
annotation was emitted. Manifest and lockfile were restored afterwards and
git status --porcelain verified empty.

3. Mutation test — proof the gate is not the permanently-green no-op. The
mutation is the historical bug itself: the stale "dev": true on
adm-zip@0.5.17 was re-injected into package-lock.json and committed, so a
pristine checkout sees the rot exactly as CI would, with package.json
untouched. Three commands were then run against that identical tree:

$ npm ci --dry-run
exit=0        # passes: the lock still satisfies every manifest spec

$ bash -e -c 'if ! git diff --exit-code package-lock.json; then echo "::error"; exit 1; fi'
exit=0        # the gate with the regeneration line DELETED: cannot fail

$ bash -e /tmp/gate.sh          # the shipped gate
     "node_modules/adm-zip": {
       "version": "0.5.17",
-      "dev": true,
::error file=package-lock.json::package-lock.json is out of date. Run 'npm install' locally and commit the regenerated package-lock.json.
exit=1

The mutant scores exit=0 on rot the shipped gate catches, so the
npm install --package-lock-only line is load-bearing and this test would fail
if it were removed. npm ci also scores exit=0, so the gate is not redundant
with the install step. The recovered hunk is the same - "dev": true, line
that had to be cleaned up by hand in #564. The throwaway commit was dropped
(git reset --hard HEAD~1) and the tree verified pristine.

4. Workflow lints, parses, and preserves the rest of the file.

$ actionlint .github/workflows/validation.yaml
exit=0

Asserted programmatically: node-version is the YAML string '24' (not a
float) and is the only input to setup-node; the gate sits immediately after
Install dependencies and before Check for secrets leaks, guarded by
matrix.os == 'ubuntu-latest'; and NODE_OPTIONS, the three-OS matrix and the
setup-python 3.11 pin are unchanged.

['Checkout code', 'Use Node.js', 'Setup Python', 'Install dependencies',
 'Check package-lock.json is up to date', 'Check for secrets leaks',
 'Build packages', 'Run tests and check code coverage', 'Run lint check',
 'Run format check', 'Run documentation build check']
STRUCTURE OK: node-version is the string 24, sole setup-node input; invariants 2 and 5 hold

5. npm ci does not break the workspace layout. npm ci deletes
node_modules and rebuilds from the lock, so the three link: true workspace
symlinks were re-checked and the pipeline re-run end to end on a clean tree:

$ rm -rf node_modules && npm ci                    exit=0
node_modules/@google/adk              -> ../../core
node_modules/@google/adk-devtools     -> ../../dev
node_modules/@google/adk-integrations -> ../../integrations
$ npm run build                                    exit=0
$ npm run lint                                     exit=0
$ npm run format:check                             exit=0
$ bash -e /tmp/gate.sh                             exit=0
$ git status --porcelain                           (empty)

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

To reproduce the gate firing, from a clean checkout on Node 24:

npm ci
npm pkg set 'devDependencies.@types/js-yaml=^4.0.8'   # any manifest edit will do
npm install --package-lock-only
git diff --exit-code package-lock.json ; echo "exit=$?"   # prints exit=1
git checkout package.json package-lock.json               # restore

This PR is its own end-to-end test, and it has now run. The workflow runs on
pull_request against main and GitHub evaluates the workflow file from the
PR's merge ref, so the change validates itself. Observed on this PR — all three
legs green:

leg Install dependencies Check package-lock.json is up to date tests
ubuntu-latest success (npm ci) success success
windows-latest success (npm ci) skipped success
macos-latest success (npm ci) skipped success

From the ubuntu-latest log, confirming the pin resolved and the gate ran as
designed:

  node-version: 24
Found in cache @ /opt/hostedtoolcache/node/24.18.0/x64
node: v24.18.0
##[group]Run npm ci
##[group]Run npm install --package-lock-only
  if ! git diff --exit-code package-lock.json; then
  ...
shell: /usr/bin/bash -e {0}

That last line is the bash -e this step's if ! … wrapper exists for, visible
in the real runner rather than inferred. (The runner resolved 24.18.0; local
verification above used 24.18.1 — same pin, one patch apart.)

One flake, ruled out as unrelated. The first attempt failed on
macos-latest and was fail-fast-cancelled on windows-latest, both in
Run tests and check code coverage — never in a step this PR adds. The cause
was tests/integration/app_loader/app_loader_test.ts hitting the 40s vitest
timeout (2677/2678 passed). This is pre-existing and independent of this
change: the identical app_loader_test.ts timeout failed run 30707362154 on
the unrelated branch feat/apihub-toolset-part1, which touches no CI config and
still ran npm install under an unpinned Node. A plain re-run of the failed
jobs went green on all legs with no code change. It has been filed as separate
follow-up work rather than papered over here.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

Amaad Martin added 2 commits August 1, 2026 10:26
`actions/setup-node@v6` was invoked with no inputs, so the job installed
nothing and ran on whatever Node -- and therefore whatever bundled npm -- the
runner image happened to ship. Pin `node-version: '24'`: Node 24 (Krypton) has
been Active LTS since 2025-10-28 and is supported until 2028-04-30, while
Node 20 (Iron) reached end-of-life on 2026-04-30. The value is quoted because
YAML parses an unquoted two-segment version such as 20.10 as a float.

Install with `npm ci` rather than `npm install`. `npm ci` installs strictly
from the lockfile and hard-errors when the manifests and the lock disagree,
instead of silently repairing the lockfile in the runner's throwaway workspace
and discarding the repair.
Nothing enforced that the committed package-lock.json is what npm resolves from
the committed package.json, so lockfile rot never surfaced in the pull request
that caused it. It accumulated and landed as collateral churn elsewhere: the
stale `"dev": true` flag on adm-zip@0.5.17 was finally removed inside
google#564, a change about hoisting @google/genai, as 142 lines of
lockfile diff a contributor then had to justify by hand.

Regenerate the lockfile after the install and fail with an actionable
annotation when the result differs from what is committed.

The regeneration is load-bearing: npm ci never writes a package-lock, so
`npm ci` followed by a bare `git diff --exit-code` is a check that can never
fail -- worse than adding nothing, because it looks like protection. The gate
is also not redundant with npm ci, which only verifies that the manifests'
dependency specs are satisfied by the lock; resolution metadata like the
adm-zip flag is recomputed only by regeneration.

The `if ! git diff ...; then ... fi` wrapper is required because a multi-line
run block executes under `bash -e`, which would abort the step before the echo
could emit the annotation. Both outputs are kept: the diff says what drifted,
the annotation says what to do about it.

Lockfile content does not vary by platform under lockfileVersion 3, so the gate
runs once on ubuntu-latest; the repo has no .gitattributes, so running the diff
on windows-latest would risk whole-file noise from CRLF conversion.
@AmaadMartin AmaadMartin changed the title Ci: fail validation when package-lock.json is out of date (stacked on #428) Ci: pin Node 24, install with npm ci, and fail validation when package-lock.json is out of date Aug 1, 2026
@AmaadMartin
AmaadMartin changed the base branch from fix/ci-pin-node-version-npm-cache to main August 1, 2026 17:30
@AmaadMartin
AmaadMartin force-pushed the feat/ci-lockfile-drift-check branch from 71c4dcd to ebcd44b Compare August 1, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant