Test: add a repo-level release version consistency test - #297
Open
AmaadMartin wants to merge 1 commit into
Open
Conversation
release-please keeps core, dev, integrations and the root package in one linked-versions group, but nothing enforced that invariant: a partially applied release could leave the workspaces and .release-please-manifest.json on different versions with fully green CI. This test reads the four package.json files, the manifest and release-please-config.json at runtime and asserts they all declare one version, and that the set of packages it checks is exactly the linked-versions group, so adding a workspace to the release group without extending this test fails loudly.
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
Problem:
adk-jspublishes every package as one linked group:release-please-config.jsonconfigures thenode-workspaceandlinked-versionsplugins so that the root package,core,devandintegrationsalways move to the same version. Nothing in the test suite enforces that invariant. A partially applied release — an interruptedrelease-pleaserun, a hand-edited.release-please-manifest.json, a manual bump of one workspace — can leavecoreat one version anddevat another and still produce a fully green CI run. The failure only surfaces at publish time, as a broken npm release where@google/adk-devtools@Xdepends on@google/adk@Y, or on the next release, which then computes the wrong next version from a stale manifest.Solution: Add one repo-level integration test,
tests/integration/release/version_consistency_test.ts, that reads the release inputs from disk at test time and asserts:package.jsonversionfields and the four.release-please-manifest.jsonentries are one and the same string (all eight values compared in a singletoEqualover a record keyed by source label, so a failure diff names every file that drifted rather than only the first)./^\d+\.\d+\.\d+(?:-[\w.]+)?$/, so the equality above can never pass vacuously on an empty or malformed value.release-please-config.json'spackages, the keys of the manifest and the keys of the test's ownLINKED_PACKAGESmap are the same set; each package's configuredcomponentmatches; and thelinked-versionsplugin'scomponentslist is the same set of component names. Adding a fifth workspace to the release group without extending this test is therefore itself a test failure, rather than a silently unchecked package.Design notes:
integrationvitest project'sincludeglob (tests/integration/**/*_test.ts), sovitest.config.tsis untouched. It imports nothing fromcore/src,dev/srcorintegrations/src, so it needs no build step and cannot move the coverage thresholds (theirincludeis*/src/**only).resolveJsonModuleis not enabled in the tsconfig chain, so the JSON is read withnode:fs+JSON.parserather than imported.JSON.parsereturnsany, so every parse result is assigned to an explicitlyunknown-typed local and narrowed through theisJsonObjecttype guard and the smallreadStringField/readObjectField/readArrayFieldaccessors. There is noany,as any,as unknown as,@ts-expect-errororeslint-disableanywhere in the file..github/workflows/validation.yamlruns the suite on[ubuntu-latest, windows-latest, macos-latest], so the repo root is resolved withfileURLToPath(import.meta.url)+path.resolve(the existing idiom intests/integration/adk_web/webui_test.ts), neverprocess.cwd(), and package paths go throughpath.join.version, a non-object JSON document, a missingpackages/pluginskey or an absentlinked-versionsplugin fails with a message naming the file and field (e.g.Expected "version" to be a string in core/package.json) instead of lettingundefinedpropagate into an assertion that would then pass vacuously. These are raised withexpect.fail(...)rather thanthrow new Error(...)so the runner reports them as assertion failures (and so TypeScript narrows the value afterwards).Out of scope on purpose: the per-package
src/version.tsliterals andintegrations/test/version_test.tsare not touched here.Collision check (required before implementation):
gh pr list --state open --limit 100was inspected, plus the file lists of the two plausibly adjacent PRs. #258 (Test: add self-maintaining version consistency tests for core and dev) addscore/test/version_test.tsanddev/test/version_test.ts, which pin each package's exportedversionagainst its ownpackage.json; #245 touchesintegrations/test/version_test.tsand the rootpackage.jsonscripts. Both are single-package by construction and cannot observe cross-package drift or manifest drift. No file overlap with this change (which adds exactly one new file), so this branches frommainrather than stacking.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
The change is the test. Targeted run:
Proof the tests can fail. A consistency test that has never been observed failing is indistinguishable from one that reads the wrong paths. Every assertion and every guard branch in the new file was mutated and observed failing, then reverted (
git statusclean afterwards):dev/package.jsonversion→9.9.9AssertionError: expected {…} to deeply equal {…}, diff line- "dev/package.json": "1.4.0"/+ "dev/package.json": "9.9.9".release-please-manifest.json"core"→9.9.9.release-please-manifest.json#core"integrations"from the manifestExpected "integrations" to be a string in .release-please-manifest.json; test 2:expected [ '.', 'core', 'dev' ] to deeply equal [ '.', 'core', 'dev', 'integrations' ]component: "devtools"→"devtool"expected {…} to deeply equal {…}, diff- "dev": "devtools"/+ "dev": "devtool""integrations"from thelinked-versionscomponentsexpected [ 'adk', 'devtools', 'main' ] to deeply equal [ 'adk', 'devtools', 'integrations', 'main' ]1.4(equality would still hold)expected '1.4' to match /^\d+\.\d+\.\d+(?:-[\w.]+)?$/— the anti-vacuity guardcore/package.jsonversion→ the number140Expected "version" to be a string in core/package.json[]Expected .release-please-manifest.json to hold a JSON objectpackages→[]Expected "packages" to be an object in release-please-config.jsonplugins→{}Expected "plugins" to be an array in release-please-config.jsonlinked-versionsplugin entryExpected a "linked-versions" plugin in release-please-config.jsonlinked-versionscomponents→ a stringExpected "components" to be an array in release-please-config.json#linked-versionsOn coverage: the new file is test code and is not instrumented —
vitest.config.ts's coverageincludeiscore/src/**,dev/src/**,integrations/src/**, so this change cannot move the thresholds. Bothitblocks execute every non-guard line on a healthy repo; the guard branches are unreachable by design when the repo is consistent, and mutations 3 and 6–12 above are the direct evidence that each of them is live and produces a named message.Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
From the repo root:
To watch it catch a real drift, edit any one of
package.json,core/package.json,dev/package.json,integrations/package.jsonor.release-please-manifest.jsonto a different version and re-run — the failure diff names the file you changed. Revert afterwards.Full local validation run on the pushed commit:
The three failing suites (
build_setup,agent_loader/agent_dirname_test.ts,skills/script_js) are the fixture-installing suites; they fail in this sandbox withnpm ERR! code E401 Incorrect or missing passwordbecause the environment has no reachable npm registry. They fail identically without this change, and the new test spawns no process and touches no network.CI on this PR — read this before re-running it.
run-testsis green onubuntu-latestandmacos-latest. Thewindows-latestleg is red, and it is red on a pre-existing failure that this change does not touch:The new test itself passed on all three legs, Windows included:
That PowerShell case is
it.skipIf(!IS_WINDOWS)and spawns a realpowershell -File <tmp>/script.ps1child process against vitest's default 5000 mstestTimeout, which the repo never overrides. It is marginal, not broken: on a recent passing Windows run of a sibling PR the same file took 5105 ms for its 12 cases, i.e. the spawn lands within a few hundred milliseconds of the budget. Because vitest packs files across a fixed set of forked workers, adding any file to theintegrationproject reshuffles that packing, which is enough to tip a test sitting that close to its limit — which also explains why re-running the job reproduces the same result rather than flaking green: the same file set schedules the same way.This PR deliberately does not "fix" that by widening the timeout here. Doing so would (a) bundle an unrelated change into a PR whose entire diff is one new file, and (b) duplicate #210, which is open against exactly this file and raises those four Windows-only cases to an explicit 60 s. This change should land on top of, or after, that one.
Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.