Skip to content

test(selftest-cli): give the group child small groups and the suite's own time - #435

Merged
StepenkoAnatoli merged 1 commit into
mainfrom
fix/selftest-cli-timeout
Oct 21, 2026
Merged

StepenkoAnatoli merged 1 commit into
mainfrom
fix/selftest-cli-timeout

Conversation

@StepenkoAnatoli

Copy link
Copy Markdown
Owner

Fixes selftest-cli > multiple valid groups still run every matching test file, which failed the full suite under the commit gate's load: its child ran past a fixed 60 s timeout. The fix is to the test only.

What changed

  • fetchproof/test/selftest-cli.test.mjs:
    • The two checks that run more than one group now run ordering witness instead of warc similarity. Both read only the kit, and the child takes a few seconds.
    • The child's spawnSync timeout is the harness's TEST_TIMEOUT (FETCHPROOF_TEST_TIMEOUT, default 60 s) instead of a fixed 60000 ms.
    • A failed check names the child's error (for example ETIMEDOUT) before its output, instead of a bare null !== 0.

Why

  • The child for warc similarity takes 31-34 s on an idle PC. Under the gate's load on 2026-10-21 (the hand-run gate for the browser /stalled fix), it ran past 60 s, was killed with status null, and failed the suite.
  • The fixed 60 s also ignored FETCHPROOF_TEST_TIMEOUT, which the gate's operator raises on a slow machine.
  • Set aside:
    • Only raising the fixed timeout: that keeps a 30 s child in every run and just moves the line.
    • action-pins as the second group (1.8 s with ordering): it reads ../.github/scripts, outside the kit.
  • Test mechanics, so no ADR (AGENTS.md rule 3). No README count change (3 tests before and after) and no CHANGELOG entry.

What you verified

  • Red first, on the unchanged test. A preload held the two-group child for 35 s without using CPU. Result: 1 passed, 2 failed (null !== 0) in 124.8 s.
  • Green on the fix, with the same 35 s hold: 3 passed, 0 failed in 89.7 s.
  • Alone: 3 passed, 0 failed in 18.2 s while another suite ran. It took 64.5 s before the fix.
  • Forced timeout (FETCHPROOF_TEST_TIMEOUT=5000, child held 8 s): both checks fail and say ETIMEDOUT.
  • git diff --check is clean.
  • Full suite through the renamed kit's gate, run by hand on 763ea36 (node fetchproof/bin/gate.mjs --gate commit): allowed in 2647 s. The installed hook is still the pre-rename kit's, which skips the suite on fetchproof/ paths. So the gate was run by hand first, and the commit then went through that hook.
  • Rebased onto main after handoff: the OK line says how old the snapshot is; a stale page end to end (discussion #413, ADR-0187) #434 merged (30b77c9). handoff: the OK line says how old the snapshot is; a stale page end to end (discussion #413, ADR-0187) #434 changes none of the files this test runs, and selftest.mjs selftest-cli on the rebased head: 3 passed, 0 failed in 6.3 s. CI's suite is the full run on this head.

What I got wrong and fixed

  • I first picked action-pins for its speed before checking what it reads. It was replaced by witness before the commit.

🤖 Generated with Claude Code

… own time

what changed: The two multi-group checks in selftest-cli.test.mjs run `ordering witness` instead of `warc similarity`, the child's spawnSync timeout is the harness's TEST_TIMEOUT instead of a fixed 60000 ms, and a failed check names the child's error (ETIMEDOUT) before its output.

why: The child for `warc similarity` takes 31-34 s on an idle PC, so under the commit gate's load it ran past the fixed 60 s, was killed with status null and failed the full suite (2026-10-21, the hand-run gate for the browser /stalled fix); the fixed value also ignored FETCHPROOF_TEST_TIMEOUT. Set aside: raising the fixed timeout alone, which keeps a 30 s child inside every run and only moves the line. Both new groups read only the kit; action-pins was tried and set aside because it reads ../.github.

what you verified: Red first on the unchanged test with a preload that holds the two-group child 35 s without CPU: 1 passed, 2 failed (`null !== 0`) in 124.8 s. Green with the same hold: 3 passed, 0 failed in 89.7 s. Alone: 3 passed, 0 failed in 18.2 s while another suite ran (64.5 s before). Forced timeout (FETCHPROOF_TEST_TIMEOUT=5000, child held 8 s): both checks fail and say ETIMEDOUT. git diff --check clean. Full suite: the renamed kit's gate run by hand on 763ea36 (`node fetchproof/bin/gate.mjs --gate commit`) allowed the commit in 2647 s; the installed hook is the pre-rename kit's and skips the suite on fetchproof/ paths.

what you got wrong: I first picked action-pins as the second group for its speed (1.8 s with ordering) before checking what it reads; it imports ../.github/scripts, outside the kit, so it was replaced by witness before any commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StepenkoAnatoli
StepenkoAnatoli merged commit 44026e1 into main Oct 21, 2026
7 checks passed
@StepenkoAnatoli
StepenkoAnatoli deleted the fix/selftest-cli-timeout branch October 24, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant