Skip to content

Fix: install CI dependencies with npm ci so the committed lockfile is authoritative - #338

Open
AmaadMartin wants to merge 1 commit into
mainfrom
feat/ci-npm-ci-deterministic-install
Open

Fix: install CI dependencies with npm ci so the committed lockfile is authoritative#338
AmaadMartin wants to merge 1 commit into
mainfrom
feat/ci-npm-ci-deterministic-install

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    No tracking issue exists for this change.

  2. Or, if no issue exists, describe the change:

Problem: Both CI workflows install dependencies with npm install, which treats the committed package-lock.json as a suggestion rather than a specification. npm install may re-resolve semver ranges and it rewrites the lockfile in place — silently, since CI never reports the edit and throws the runner workspace away.

Two consequences, both of which make the CI signal weaker than it looks:

  • A PR that edits a package.json without committing the regenerated lockfile goes green. main then carries a lockfile that does not describe the tree, and the next contributor's clean install gets something different.
  • A transitive dependency that publishes inside an existing ^ range can be picked up by CI while the lockfile still pins the old version, so a breakage is attributed to the PR under test rather than to the upgrade.

This is not hypothetical for this repo. On a clean checkout of the base commit (1210acc7) with no node_modules, a plain npm install modifies the committed lockfile:

--- a/package-lock.json
+++ b/package-lock.json
@@ -5207,7 +5207,6 @@
       "version": "0.5.17",
       "resolved": "https://registry.npmjs.org/adm-zip/-/adm-zip-0.5.17.tgz",
       "integrity": "sha512-+Ut8d9LLqwEvHHJl1+PIHqoyDxFgVN847JTVM3Izi3xHDWPE4UtzzXysMZQs64DMcrJfBeS/uoEP4AD3HQHnQQ==",
-      "dev": true,
       "license": "MIT",

No version changed in that particular run — the point is the write. Under npm install the committed lockfile is not authoritative in CI.

Solution: Change the Install dependencies step in both workflows from npm install to npm ci. That is the entire diff — two files, two lines:

- name: Install dependencies
  run: npm ci

npm ci installs strictly from the lockfile, refuses to mutate it, and aborts with a named diagnostic if package-lock.json and any of the four workspace manifests (package.json, core/, dev/, integrations/) disagree.

The precondition holds: exactly one lockfile is tracked (git ls-files '*package-lock.json'package-lock.json), it is lockfileVersion: 3 with 1167 packages entries, and it is already in sync — npm ci exits 0 on a clean checkout (see Testing Plan).

Deliberately not included, to keep the diff to the one decision under review:

  • No --no-audit / --no-fund / --prefer-offline flags on the CI step. Each is a separate opinion and would muddy the before/after install-duration comparison below.
  • No continue-on-error and no || npm install fallback. The hard failure is the feature.
  • No Node version pin. Both workflows use actions/setup-node@v6 with no with: block, so they get the runner default, which is well past the npm 7 that lockfileVersion: 3 needs. Pinning Node is a separate change with its own tradeoffs.
  • No lockfile regeneration. package-lock.json is byte-identical to main in this PR (git diff main --stat shows only the two workflow files). The stale "dev": true on node_modules/adm-zip visible in the diff above is a real pre-existing latent bug — adm-zip is a production dependency of core (core/src/skills/loader.ts), so npm ci --omit=dev would drop it — but it does not affect this change (CI never passes --omit=dev, and the dev flag is metadata that is not part of npm's sync check), and fixing it here would put a 562 KB lockfile diff in front of a two-line review. It is tracked separately.
  • CONTRIBUTING.md is untouched. It tells local contributors to run npm install, which remains correct: developers need to be able to add a dependency and have the lockfile update. This change is CI-only.
  • The tests/integration/**fixtures are untouched. Several suites run their ownnpm install from inside test code against fixture projects that have no committed lockfiles (tests/\*\*/package-lock.jsonis gitignored), sonpm ci does not apply there.

Behaviour change contributors should know about. A PR that edits any package.json without committing the regenerated package-lock.json will now fail at the install step instead of going green. The fix is npm install && git add package-lock.json. The failure is self-describing:

npm ERR! code EUSAGE
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! Missing: <pkg>@<version> from lock file

Collision check (run before writing anything). gh pr list --state open --limit 100 on the fork, then gh pr diff --name-only on every plausibly adjacent PR. Six open PRs touch a workflow file — #296 and #306 (cache: npm on the setup-node step), #250 (deps:check step), #237 (a new integration:slow job), #235 (fail-fast), and #245 (root scripts only) — and none of them changes npm install to npm ci; grepping each diff for npm ci / npm install / Install dependencies returns nothing but #237's new job, which adds another npm install rather than replacing one. So this change is uncontested and is branched from main rather than stacked. The two cache PRs edit the Use Node.js step, a different step from the one edited here, and compose cleanly either way: npm ci reads the same npm HTTP cache npm install does, so whichever lands first, the other still applies unchanged. #237's new job would need the same npm installnpm ci treatment if both land; that is a one-line follow-up on whichever merges second, not a conflict.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:

[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

New-code coverage: N/A — CI configuration change, no product code added. No test file and no Vitest project was added to manufacture a coverage number for a two-line YAML edit. The existing suite still gates the change: npm run test:coverage runs unchanged in validation.yaml immediately after the install step, so any regression in the installed tree surfaces as an ordinary test failure.

What replaces a coverage figure is a negative test and its counterfactual — the equivalent of proving a test can fail. Both were run locally on Linux, Node v22.22.2 / npm 9.2.0, from a clean checkout of this branch.

1. The sync check passes (the precondition).

$ rm -rf node_modules && npm ci --no-audit --no-fund
added 1089 packages in 15s     # exit 0
$ git status --porcelain package-lock.json
                               # empty — npm ci did not touch the lockfile

npm's own summary count is not stable across runs (a second identical npm ci reported added 1081 packages for a byte-identical tree), so the exit code is the assertion, not the count.

2. The mutation — the guard actually fires. Adding a bogus "left-pad": "^1.3.0" to core/package.json's dependencies and re-running the same command:

npm ERR! code EUSAGE
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! Missing: left-pad@1.3.0 from lock file

exit 1. So the pass in (1) is a real pass across all four manifests, not a check that never runs.

3. The counterfactual — the old command passes the same mutation. With core/package.json still drifted, the command this PR replaces:

$ npm install --no-audit --no-fund
added 1 package, and changed 1 package in 4s     # exit 0
$ git status --porcelain package-lock.json
 M package-lock.json                             # left-pad silently written into the lock

This is the whole argument in three lines: identical drifted input, npm install goes green and rewrites the lockfile, npm ci fails loudly. The scratch edit to core/package.json and the resulting lockfile write were both reverted (git checkout core/package.json package-lock.json); neither is in this PR.

4. The installed tree does not change. Comparing the trees produced by npm ci and by npm install from an empty node_modulesname@version for every package at every nesting depth, 1089 entries each — diff reports no differences, and a second npm ci reproduces the first byte for byte. So this PR does not alter what CI tests today; it removes the mechanism by which that could silently drift.

5. The tree works, and the CI gates pass on it. All run against the exact commit pushed, after a npm ci install:

command result
npm run build exit 0
npx vitest run --project unit:core core/test/skills/loader_test.ts 30 passed (exercises adm-zip, a lockfile-resolved production dep)
npm run lint exit 0
npm run format:check All matched files use Prettier code style!
npm run docs:check exit 0
npx secretlint <the two workflow files> exit 0

Per the repo's testing guidance the full suite was not run locally; CI runs it. (npm run ts:check fails on this branch, but it fails identically on main — it is not a step in either workflow, and this diff contains no TypeScript.)

6. The prepare script is unaffected. The root prepare script (husky) runs under npm ci exactly as under npm install — visible in the install output above — so git-hook setup behaviour is unchanged.

Manual End-to-End (E2E) Tests:

Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

The PR's own CI run is the e2e test. To verify:

  1. Confirm validation is green on ubuntu-latest, windows-latest and macos-latest, and Cross-Language Tests is green on macos-latest.

  2. Expand the Install dependencies step on each leg and confirm it runs npm ci and logs no lockfile-rewrite warning.

  3. Compare install-step durations against the last main run, which still uses npm install, via the Actions API rather than the UI:

    gh run list --workflow=validation.yaml --branch <branch> --limit 1
    gh api repos/<owner>/adk-js/actions/runs/<run-id>/jobs \
      --jq '.jobs[] | {os: .name, step: (.steps[] | select(.name=="Install dependencies")
            | {started: .started_at, completed: .completed_at})}'

To reproduce steps 1–3 of the Testing Plan locally, run the commands as given from a clean checkout; step 2 requires temporarily adding a package to a workspace manifest that is not in the lockfile, and step 3 requires reverting package-lock.json afterwards.

Results of this PR's own CI run (validation run 30582056937, Cross-Language run 30582056925):

  • validation passes on all three matrix legsubuntu-latest, windows-latest, macos-latest. Cross-Language Tests passes on macos-latest.

  • The Install dependencies step succeeded on all four legs. The ubuntu log shows exactly what it should:

    ##[group]Run npm ci
    ...
    > adk@1.4.0 prepare
    > husky
    added 1089 packages, and audited 1093 packages in 19s
    

    No lockfile-rewrite warning, no EUSAGE, and the prepare/husky hook installs normally. Runner toolchain: node v22.23.1, npm 10.9.8.

  • One Windows flake, unrelated to this change, and proven so. The first Windows attempt failed in tests/integration/tools/run_skill_script_tool_test.tssuccessfully executes a real PowerShell skill script and captures stderr from a failing PowerShell skill script, both Test timed out in 5000ms. Re-running that job on the identical commit, with no changes, passed. The margin is the cause: on another current PR's Windows leg (which still uses npm install) the same PowerShell test passes in 2719ms against the same 5000 ms budget. This is a latency-sensitive test on a slow shell, not an install-method effect — the installed tree is identical either way (§4) — and it is already being addressed by separate open PRs about PowerShell spawn cost and Windows shell test timeouts.

Install-step duration, before vs after, per OS. Pulled from the Actions API (/actions/runs/<id>/jobs, Install dependencies started_at/completed_at), not the UI.

workflow OS before (npm install) after (npm ci)
validation ubuntu-latest 19 s 19 s
validation macos-latest 26 s 25 s
validation windows-latest 61 s ⁽¹⁾ 84 s / 36 s ⁽²⁾
Cross-Language Tests macos-latest 23 s 20 s

⁽¹⁾ From run 29888027719 (main @ 85d0321), because the Windows leg of the main run at this PR's base commit (30405795959, 1210acc7) was cancelled by matrix fail-fast and recorded no step timings. The ubuntu/macos "before" figures are from that base-commit run.
⁽²⁾ Two attempts of the same commit: 84 s on attempt 1, 36 s on attempt 2.

Read this as an observation, not a benchmark. One sample per cell, and the Windows cell alone varies by a factor of 2.3 between two runs of an identical commit — a range that brackets the baseline, so no conclusion about Windows is available from this data. Ubuntu and macOS are flat within noise. Locally the two verbs were also indistinguishable (npm ci 15.5 s vs npm install 16.0 s from an empty node_modules with a warm HTTP cache). This change is not a speed optimization and none is claimed — the justification is determinism.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

On the fourth box: this change adds no product code to unit-test, so what is offered instead is the negative test in §2 plus its counterfactual in §3 — the drifted-manifest input that npm ci rejects and npm install accepts. That pair is what proves the change is effective.

npm install treats the committed package-lock.json as a suggestion: it may
re-resolve semver ranges and rewrite the lockfile in place, so CI can pass
against a tree that no contributor's clean install reproduces.

npm ci installs strictly from the lockfile, refuses to mutate it, and aborts
with a named diagnostic when package-lock.json and any workspace manifest
disagree. A package.json edit landed without its regenerated lockfile now
fails at the install step instead of going green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant