Fix: sweep the script_js integration fixture by name shape - #766
Open
AmaadMartin wants to merge 2 commits into
Open
Fix: sweep the script_js integration fixture by name shape#766AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 7, 2026 02:34
The skill write path de-duplicates against existing files: it appends `_2`, `_3`, ... before the extension when the target name is taken. A teardown keyed on the three exact names therefore leaves every variant behind, and those variants are not gitignored. Sweep the fixture directory by name shape instead, in `beforeAll` and in `afterAll`. Sweeping first also stops the content assertions comparing a previous run's output, which is what kept the leak invisible.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A
Problem: The
script_jsintegration teardown removes three exact filenames, but the skill write path de-duplicates:materializeFilesappends_2,_3, ... before the extension when the target name is taken (core/src/utils/file_utils.ts). A run that starts from a dirty fixture directory therefore writesindex_2.htmland friends, which the teardown never names and.gitignorenever covers, so they pile up as untracked files. The same dirt hides a second defect: the content assertions read the three exact names, which on a dirty run are the previous run's files, so the suite stays green while it compares stale output.Solution: The test now sweeps the fixture directory by name shape.
isGeneratedOutputstrips a trailing_<n>from the basename and compares the result against the three literal names, andremoveGeneratedOutputsdeletes every match.beforeAllsweeps beforenpm installand lets a failure reject, so the run always starts clean;afterAllreplaces the three exact removals with one sweep. Sweeping first also fixes the stale comparison, because de-duplication can only trigger on a dirty directory.Safety of the predicate: it strips a suffix rather than prefix-matching, so
indexed.htmlis not a match. The tracked fixture entriesagent.ts,agent_test.ts,model_responses.json,package.jsonandexpected/are all rejected —path.extname('expected')is'', so the comparison is'expected'against the list and misses.fs.rmis called withoutrecursive: trueon purpose, so a future mis-match can never take a directory with it.No existing assertion was modified, reworded or deleted. The diff is one file under
tests/, which the Vitest coverageincludelist (core/src/**,dev/src/**,integrations/src/**) excludes, so the coverage thresholds cannot move.Collision check:
gh pr list --repo AmaadMartin/adk-js --state openplus a scan of every branch touching this file. #662 rewrites the sameafterAllto report each removal failure, and #730, #409, #523 changematerializeFiles; none of them sweeps by name shape. This PR branches frommainand keeps the.catch(() => {})teardown style byte-identical so it stays separable from #662.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
The change adds one assertion to the existing integration test, which runs the two new helpers on every execution. Command:
npx vitest run --project integration tests/integration/skills/script_js/agent_test.ts— 1 passed.Manual End-to-End (E2E) Tests:
A. Clean run — passes,
git status --porcelain tests/integration/skills/script_jsprints nothing.B. Dirty run (the reported bug). Seed a leftover and an orphan variant, then run:
Before (
git status --porcelain tests/integration/skills/script_js):After: 1 passed, and
git status --porcelain tests/integration/skills/script_jsprints nothing.C. Back-to-back runs — both pass, tree clean after each.
D. Proof that the change can fail.
Against the unfixed code.
git stash, then scenario B. The suite still passes, because it compares the staleindex.html, and leaks:Against the new assertion. With the fix applied, delete
await removeGeneratedOutputs();frombeforeAlland run scenario B. The new assertion fails:E. Static gates.
npx eslint tests/integration/skills/script_js/agent_test.tsandnpx prettier --check ...both pass.npm run ts:checkoutput is byte-identical with and without this change, so it adds no type error.npm run buildsucceeds.Note on the local hook timeout. On a cold npm cache the fixture
npm installtakes about 68 seconds on my machine, over the file's own 60 sbeforeAllbudget, so the hook times out before the test body runs. This is pre-existing and unrelated: it reproduces on unmodifiedmain, and #662 already proposes letting the hook inherit the project's 120 shookTimeout. I pre-installed the fixture dependencies before each run above so the hook fits its budget.Note on CI.
run-testsis green on ubuntu-latest, windows-latest and macos-latest, andtests/integration/skills/script_js/agent_test.tspassed on all three platforms in every run. The Windows leg is flaky for reasons unrelated to this change: across four runs it failed three times, each in a different test this diff does not touch —a2a/stream/stream_test.ts("CLI exited prematurely"),app_loader/app_loader_test.ts, andcore/test/code_executors/unsafe_local_code_executor_test.ts(a 5 s unit-test timeout). Each cleared on a re-run. The same workflow fails on unmodifiedmain(9360bf2) on macOS inapp_loader_test.ts, and sibling PR #765 fails on Windows in the same test, so the flake is repo-wide.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.