ci: run cross-language integration tests on ubuntu-latest - #648
Open
AmaadMartin wants to merge 1 commit into
Open
ci: run cross-language integration tests on ubuntu-latest#648AmaadMartin wants to merge 1 commit into
AmaadMartin wants to merge 1 commit into
Conversation
The Cross-Language Tests workflow was the only workflow in the repository whose sole runner was macos-latest. The suite it runs is loopback HTTP between a Node process and a Go process and has no macOS-specific surface: the only platform branches in core/src, dev/src, integrations/src and tests are Windows branches, and the repository's own test code treats Linux and macOS as a single behavioural class (IS_UNIX in tests/integration/tools/run_skill_script_tool_test.ts:22). macOS is the tightest GitHub-hosted runner pool an account has, so the job was queueing against a scarce shared resource for signal ubuntu-latest already produces. validation.yaml keeps the repository's macOS coverage via its ubuntu/windows/macos matrix. The workflow name (Cross-Language Tests), the job id (run-tests), the triggers and all 7 steps are unchanged, so the check-run name that branch-protection rules match on is preserved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Closes: #issue_number
Related: #issue_number
No existing issue; description follows.
Problem:
.github/workflows/cross-language-integration.ymlwas the only workflow in the repository whose sole runner ismacos-latest. macOS is the tightest GitHub-hosted runner pool an account has — the Actions limits cap concurrent macOS jobs at 5 (Free/Pro/Team) or 50 (Enterprise) account-wide, against a total concurrent-job budget of 20–500 — so every macOS job queues behind, and delays, every other macOS job in the account. It is also the most expensive runner class (billing docs: macOS$0.062/min vs Linux$0.006/min, ~10.3x), which matters if this repository is ever made private or moved to larger runners. The suite that job runs has no macOS-specific surface, so it was paying that premium for signalubuntu-latestalready produces.Solution: Flip
runs-ontoubuntu-latestand record why the runner is Linux in a three-line comment, so a future contributor does not "restore" macOS coverage by reflex. This is the whole change — 4 insertions, 1 deletion, in one file:Why this is safe — the suite has no macOS-specific surface. An exhaustive search of
core/src,dev/src,integrations/srcandtestsforprocess.platform,os.platform()and'darwin'returns four hits, and every one is a Windows branch:core/src/code_executors/unsafe_local_code_executor.ts:21—const IS_WINDOWS = os.platform() === 'win32';dev/src/utils/agent_loader.ts:634—process.platform === 'win32' ? 'junction' : 'dir'tests/integration/tools/run_skill_script_tool_test.ts:21-22—IS_WINDOWS, plusconst IS_UNIX = os.platform() === 'linux' || os.platform() === 'darwin';That last line is the decisive one: the repository's own test code already treats Linux and macOS as a single behavioural class. There is no code path anywhere that macOS reaches and Linux does not. The tests this workflow runs are loopback HTTP between a Node process and a Go process (
go_server.tsbinds127.0.0.1,test_api_server.tsuseslocalhost), the two Go modules have pure-Go dependency sets with no cgo and no darwin-only packages, andtests/integration/test_case_utils.ts'sBaseTestServercontains no platform branching at all.For the record, the native-addon fixtures are not real native addons and are not run by this workflow (they belong to the
integrationvitest project, driven byvalidation.yaml):tests/integration/build_setup/ts_esm_native_addon/.../binding.nodeis a plain ASCII text file containingplaceholder native addon binary, and nothing is everdlopened on any OS. They already run onvalidation.yaml'subuntu-latestleg today.macOS coverage for the repository is retained.
.github/workflows/validation.yaml:17still runs its full matrix —os: [ubuntu-latest, windows-latest, macos-latest]— and is left byte-for-byte alone by this PR.Invariants preserved. The workflow name (
Cross-Language Tests), the job id (run-tests), the trigger block, and all 7 steps are unchanged, so the GitHub check-run name that branch-protection rules and rulesets match on is preserved and no admin coordination is required. Verified structurally, not by eye — see check 1 below. No source file, test file,package.json,vitest.config.tsorgo.modis touched.Deliberately out of scope (each queued as its own task): gating
validation.yaml'smacos-latestleg on the event (that one does remove a check name from PR runs and so needs branch-protection coordination), and adding npm caching / switchingnpm installtonpm ci.Collision check. Ran
gh pr list --repo AmaadMartin/adk-js --state open --limit 1000(547 open PRs) and diffed every plausibly adjacent one. 16 open PRs touchcross-language-integration.yml(#505, #427, #428, #416, #406, #403, #393, #306, #415, #338, #574, #509, #593, #571, #572, #504) — they variously pin Node, add npm/Go caching, switch tonpm ci, pin actions to commit SHAs, addtimeout-minutesand add concurrency groups. Grepping each of those diffs forruns-on|macos|ubuntuinside thecross-language-integration.ymlhunk returned zero hits: none of them changes the runner, so this change is orthogonal to all of them and does not stack on any one of them.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Note on unit tests / coverage: the deliverable is a GitHub Actions workflow definition, so there is no new line of executable product code to cover.
vitest.config.ts'scoverage.includeglobs arecore/src/**,dev/src/**,integrations/src/**—.github/**is not and cannot be instrumented. No coverage threshold was touched and no unit test was invented for a YAML file. The verification burden is met by the three checks below instead, all run locally on Linux x86_64 (Node v22.22.2, Go 1.26.5).Check 1 — the workflow still parses and still has the intended shape, and this check is discriminating (it reports the old value against the unchanged file, i.e. it fails before the fix). Run against the pre-change file (
git show fork/main:...) and the post-change file:The step count does not move, and the name/job-id/trigger invariants hold. A stronger structural assertion was also run — load both YAML documents, set the before document's
runs-ontoubuntu-latest, and deep-compare:so the diff provably changes nothing else semantically.
Check 2 — the diff is exactly what it claims to be. (The PR base is the fork's
main, i.e.fork/main; the localmainref in this clone is stale.)One file, and the only non-comment change is the
runs-onvalue. Thego.sumfiles thatgo mod tidygenerates undertests/cross_language/a2a/*/go*/are gitignored (.gitignore:11) and are not committed;git statusis clean after a full install/build/test cycle.Check 3 — the real suite passes on Linux, run with exactly the command sequence the workflow uses:
The set of tests executed is identical before and after (the
cross-languagevitest project,tests/cross_language/**/*_test.ts). Both loopback directions were exercised — the Go server boundhttp://127.0.0.1:40973and the TS ADK API server came up onlocalhost— so the one plausible Linux/macOS difference (whetherlocalhostresolves to::1first) is empirically a non-issue here; noECONNREFUSED ::1occurred in either direction.Unit Tests:
[x] I have added or updated unit tests for my change. — N/A and deliberately so: zero new lines of executable code, and
.github/**is not instrumentable. Substituted by the mandatory structural checks 1 and 2 above.[x] All unit tests pass locally.
Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.
This pull request is its own end-to-end test.
pull_requestworkflows execute the workflow file from the PR's merge commit, so the PR verifies the change against real GitHub infrastructure. This has now been observed, not merely predicted — the five acceptance criteria and their measured results:The
Cross-Language Tests / run-testscheck ran on this PR and passed.The runner image changed from macOS to Ubuntu. Read from the
Set up jobstep of each run:Operating SystemImagemacos-26-arm64ubuntu-24.04That is the primary acceptance evidence. One
macos-latestjob is now removed from everypush-to-mainrun and everypull_requestrun, freeing one slot out of the account-wide macOS concurrency cap of 5 (or 50) per run — and the job is not slower for it (1m31s vs 1m48s; sampling three prior macOS runs gives 1m48s / 1m54s / 1m48s).The suite is green on the Ubuntu runner, with the same test count as before:
The check-run name is unchanged —
run-testsunderCross-Language Tests. No previously-reporting check disappeared from this PR's check list.No
macos-latestjob remains in this workflow's run, andvalidation.yamlis unperturbed — this PR's check list still shows all three of its legs:run-tests (ubuntu-latest),run-tests (windows-latest),run-tests (macos-latest).To reproduce locally instead, run the Check 3 command sequence above on a Linux host with Go >= 1.25, Node, and network access to the npm registry and the Go module proxy.
If this suite ever fails on
ubuntu-latestfor a genuine platform reason, the right response is a Linux+macOS matrix and a bug report — not an OS-conditional skip, and not reverting to macOS.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.