Stream live test-progress from forked test JVMs to the mvnd client (mvnd-1.x) - #1670
Stream live test-progress from forked test JVMs to the mvnd client (mvnd-1.x)#1670ammachado wants to merge 15 commits into
Conversation
c3ab044 to
0819f84
Compare
- Extract buildRequestEnvironment() in DefaultClient for subclass override - Strip MAVEN_ARGS from JvmTestClient and NativeTestClient to block per-test flag overrides - Write an isolated settings.xml in MvndTestExtension so tests never fall back to the runner's user settings (fixes InteractiveTest when actions/setup-java sets interactiveMode) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fix on mvnd-1.x Reshape the live terminal display: a drawn progress bar with percent on the status line, concise arrow-style worker lines (> :module goal (execution)), and dimmed > IDLE slots for free threads. At addProjectLine, append a styled live test-progress suffix (tests/failures/errors/skipped counts plus the current test class#method) driven by the new PROJECT_TEST_PROGRESS message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
0819f84 to
5aeb95c
Compare
gnodet
left a comment
There was a problem hiding this comment.
Review: Test Progress Reporting Feature
Well-designed feature with sound architecture. The maker/checker review process confirmed several items worth the author's attention.
Medium Severity
1. supportsForkNode version check logic gap
The version check in supportsForkNode applies the M5 milestone threshold to ALL 3.x versions, not just 3.0.0. For example, "3.1.0-M2" produces group(4)="2" and returns false, even though the milestone guard should only apply to the 3.0.0-Mx series. Practical impact is negligible since Surefire never published milestone versions beyond 3.0.0-Mx (3.1.0+, 3.2.x, 3.5.x all shipped as GA), but the fix (apply milestone check only when minor==0 && patch==0) is trivial and improves defensive correctness.
2. clearDisplay() no longer called on build failure
The base branch BUILD_FINISHED handler calls clearDisplay() unconditionally. This PR wraps it in if (failures.isEmpty()), so on failure the JLine display's last render (project status lines) remains painted in the terminal. The shouldShowProjectDetails() method and its comment suggest this is intentional (keep failure info visible), but this is a behavior change from prior code — please confirm this is the desired UX.
Low Severity
3. Mutable ArrayList references in getter methods
ProjectTestProgressEvent.getFlakyTests(), getFailedTests(), and getErroredTests() return the mutable internal lists directly. The constructor makes defensive copies, but the getters expose the internal state. Currently no callers mutate the returned lists, but wrapping in Collections.unmodifiableList() would be a safe defensive measure.
4. BANNED_MARKER fragile string matching
The string "This project has been banned from the build due to previous failures." is compared via equals() after stripping ANSI/level prefixes. If Maven changes this message text, the filter silently breaks. This is inherent to text-based output matching (no programmatic API alternative), but worth noting as a maintenance risk.
5. Pattern.compile on each invocation of supportsForkNode
The regex is compiled inline each call. Called once per surefire/failsafe plugin per project, so allocation count is small, but the pattern could be a static final field at no cost.
6. Duplicated isEnabled logic
Both Server.handle() and MvndTestProgressLifecycleParticipant.isEnabled() contain identical logic: Environment.MVND_TEST_PROGRESS.asOptional().map(Boolean::parseBoolean).orElse(Boolean.TRUE). Could be extracted to a shared static method.
7. Magic indices in int[] snapshot array
The TestBuildSummary snapshot array is packed as {completed, failures, errors, skipped, retrying, flaky} and unpacked with raw indices 0–5 (notably skipping index 4/retrying intentionally). A record or named-constant approach would make the skip of "retrying" explicit and prevent index-drift bugs.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
Fix the supportsForkNode milestone check so it only guards the 3.0.0-Mx series instead of every 3.x version, always clear the JLine display at build end (including failures), make ProjectTestProgressEvent's list getters unmodifiable, dedupe the test-progress enabled check between Server and MvndTestProgressLifecycleParticipant, hoist the version regex to a static field, replace magic indices in TestBuildSummary's snapshot array with named constants, and document the BANNED_MARKER string-match as a maintenance risk. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Thanks for the thorough review! Addressed all 7 points in acbb147: Medium
Low |
gnodet
left a comment
There was a problem hiding this comment.
Re-review: All Previous Findings Resolved
The fix commit (acbb147) correctly addresses all 7 findings from the previous review:
- supportsForkNode version check — milestone guard now only applies to the
3.0.0-Mxseries (checks minor/patch before applying). New testmilestoneGuardOnlyAppliesToThe300MilestoneSeriescovers"3.1.0-M2"and"3.2.5-M1". - clearDisplay() — again called unconditionally in the
BUILD_FINISHEDhandler. - Mutable list getters —
getFlakyTests/getFailedTests/getErroredTestsnow returnCollections.unmodifiableListwrappers. - BANNED_MARKER — field now has a maintenance-risk comment documenting the fragility.
- Pattern.compile —
VERSION_PATTERNis now astatic finalfield. - Duplicated isEnabled —
Server.handle()delegates toMvndTestProgressLifecycleParticipant.isTestProgressEnabled()instead of duplicating the logic. - Magic array indices —
IDX_COMPLETED/IDX_FAILURES/IDX_ERRORS/IDX_SKIPPED/IDX_FLAKYconstants replace magic indices, withIDX_RETRYING=4explicitly documented as not folded.
The fix commit is well-scoped — touches only the files relevant to the review feedback with no unrelated changes. Nice work!
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
gnodet
left a comment
There was a problem hiding this comment.
Re-review: New Commit (3eedb48)
The new commit adds a defensive clamp to renderBar(int percent) — Math.max(0, Math.min(100, percent)) — ensuring out-of-range inputs can't under/overfill the progress bar. A corresponding test (renderBarClampsOutOfRangeInput) covers both negative and >100 values.
Trivially correct, well-scoped change. LGTM.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
gnodet
left a comment
There was a problem hiding this comment.
Well-engineered feature adding live test-progress streaming from Surefire/Failsafe forks to the mvnd client. The classloading bridge design, thread safety (AtomicReference for cross-thread listener bridge, per-fork-channel accumulators), and Surefire version guarding are all sound. Test coverage is comprehensive.
Two items worth addressing before merge:
1. BannedSkipFilter not flushed on CANCEL_BUILD / BUILD_EXCEPTION (medium)
The BUILD_FINISHED handler (line 319) correctly calls bannedSkipFilter.flush(log::accept) before closing the log. However, the CANCEL_BUILD handler (line 258) and BUILD_EXCEPTION handler (line 272) skip this flush. Any reactor-level lines buffered in the filter (blank lines, separators, "Skipping X" preambles) will be silently dropped during cancelled or failed builds.
Suggested fix — add the flush before projects.values().stream()... in both handlers:
if (hideBannedProjectSkips) {
bannedSkipFilter.flush(log::accept);
}2. Stale TODO comment on PROJECT_TEST_PROGRESS (medium)
The comment at Message.java line 69-73 says "the daemon-side feed that emits this message is not implemented on mvnd-1.x yet", but this PR fully implements it across 12 commits (dispatch, bridge listener, forkNode injection, etc.). The TODO should be replaced with documentation describing the implemented feature.
Minor note
- TestState.displayName() null safety (low): Returns
nullwhen bothtestClassandtestMethodare null, which would causeNullPointerExceptioninflakyDetail()vianew StringBuilder(displayName()). Practically unreachable since Surefire always provides source names, but a defensive"(unknown)"fallback would be more robust.
Note: This PR is stacked on PR #1666 — some findings (like BannedSkipFilter) may have been introduced in the base PR.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
Applies the review-driven fixes from the sibling 2.x PR (apache#1667): - PROJECT_TEST_PROGRESS now sorts after PROJECT_STARTED/MOJO_STARTED in the sendQueue, avoiding a race where the progress message could be dequeued before its project's PROJECT_STARTED event. - ProjectTestProgressEvent reuses the existing readUTF/writeUTF null-sentinel convention instead of a redundant boolean-prefixed nullable encoding. - Restores the plugin groupId/version in the worker line display, which was dropped by the visual redesign. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds the mvnd.testProgress opt-out flag (default on) and the cross-realm MvndTestProgress bridge interface + static listener registry that connects the Surefire plugin realm to the daemon realm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
New module contributing the Surefire plugin-realm pieces of the live test-progress feed: - TestProgressAccumulator: pure per-fork counter (unit-tested) - MvndForkNodeFactory: decorates Surefire's EventHandler<Event>, accumulates counts, pushes through the MvndTestProgress bridge, and always delegates to the real handler (never breaks the test run) - MvndSurefireProgressLocator: zero-dependency marker the daemon uses to locate this jar for plugin-realm injection Verified compiling and testing against the real Surefire 3.5.x extensions SPI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Adds ClientDispatcher.testProgress to enqueue PROJECT_TEST_PROGRESS messages, and wires Server.handle to register an MvndTestProgress listener (per build, when mvnd.testProgress is enabled) that forwards fork events to the dispatcher; the listener is cleared in finally. Also adds the daemon dependency on mvnd-surefire-progress. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Wires the Maven 3.9 test-progress feed: - DaemonMavenCli: export org.mvndaemon.mvnd.testprogress to the core realm so the bridge Class is shared with the surefire plugin realm - MvndTestProgressLifecycleParticipant: afterProjectsRead injects <forkNode implementation=MvndForkNodeFactory><projectId>..> into surefire/failsafe config (the Maven 3.9-native mechanism) - InvalidatingPluginRealmCache: addURL the mvnd-surefire-progress jar onto the surefire/failsafe plugin realm - dist: bundle mvnd-surefire-progress; daemon: plexus-xml provided (Xpp3Dom) - integration-tests: TestProgressTest + test-progress JUnit 5 project Runtime validation of the mandatory 1.x spike (PROJECT_TEST_PROGRESS emitted end-to-end) follows in a later commit. Running the integration tests requires a full dist build first, since the harness spawns a real daemon against dist/target/maven-mvnd-*. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
The 1.x spike (TestProgressTest) exposed a classloader bug that broke every project running Surefire (e.g. UpgradesInBomTest, ExecOutputTest): Cannot load implementation hint MvndForkNodeFactory: NoClassDefFoundError: SurefireForkNodeFactory Root cause: the factory lived under org.mvndaemon.mvnd.testprogress, the package exported to the core realm for the shared MvndTestProgress bridge. ClassWorlds package imports are hierarchical (prefix-based), so the export also captured the factory's (sub)package; the surefire plugin realm imported it from the core realm, which has no Surefire, so linking the factory's SurefireForkNodeFactory superclass failed. Fix: - Move MvndForkNodeFactory / TestProgressAccumulator / locator to a sibling package org.mvndaemon.mvnd.forknode that is NOT under the exported prefix, so the plugin realm loads the factory from its own (addURL'd) jar and links against the plugin realm's Surefire. Bridge stays in the exported org.mvndaemon.mvnd.testprogress. - Guard injection on Surefire >= 3.0.0-M5 (forkNode SPI floor) so older Surefire/Failsafe are never touched. Add supportsForkNode unit test. Validated end-to-end on mvnd-1.x: TestProgressTest emits increasing PROJECT_TEST_PROGRESS with correct class/counts, the opt-out emits none, and ExecOutputTest / UpgradesInBomTest (Surefire 3.0.0-M8) pass unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Regenerated completion list to include the new mvnd.testProgress property. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Extend the live test-progress feed beyond basic counts so a failing build
tells you which tests broke and why, directly in mvnd's output.
- Accumulate retrying/flaky state and capture FAILED/ERRORED test identities
plus a compact failure message (Surefire smartTrimmedStackTrace, sanitized),
streamed alongside flaky names in ProjectTestProgressEvent.
- Render a live "Flaky tests:" summary at build end, and inject a
"Failed tests:" / "Errored tests:" block ("<projectId> Class#method: message")
directly above Maven's BUILD FAILURE banner, with a build-finished fallback.
- Add mvnd.hideBannedProjectSkips (default true) to drop the per-project
"Skipping X / banned from the build" reactor blocks while preserving the
final reactor "... SKIPPED" rows; disable with =false.
- Cover with accumulator, message round-trip, and TerminalOutput unit tests
(ordering + banned-block filter), plus a new TestProgressFailureTest IT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ging Move end-of-build test summary rendering from the client (string injection keyed on a "BUILD FAILURE" text match) to the daemon, where LoggingExecutionListener logs it through its own SLF4J logger just before the Reactor Summary. This routes the summary through the same MvndSimpleLogger pipeline as every other Maven console line, so ANSI coloring, [LEVEL] prefixes, and -q gating all come for free instead of being reimplemented client-side. ClientDispatcher now collects failed/errored/flaky test identities and per-fork numeric totals into a new TestBuildSummary as testProgress() events arrive; BuildEventListener exposes foldTestProgress()/ getTestSummary() so the Maven-realm listener can fold and render them. TerminalOutput drops the now-daemon-owned summary machinery, keeping only the live per-project progress line and the banned-skip filter. The -q client-side plumbing (MAVEN_QUIET, TerminalOutput's quiet field) is removed since the daemon's own logger level already handles it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Fix the supportsForkNode milestone check so it only guards the 3.0.0-Mx series instead of every 3.x version, always clear the JLine display at build end (including failures), make ProjectTestProgressEvent's list getters unmodifiable, dedupe the test-progress enabled check between Server and MvndTestProgressLifecycleParticipant, hoist the version regex to a static field, replace magic indices in TestBuildSummary's snapshot array with named constants, and document the BANNED_MARKER string-match as a maintenance risk. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dback renderBar() already produced a sane bar for out-of-range input, but the invariant wasn't self-documenting; clamp explicitly and add a regression test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
3eedb48 to
629790e
Compare
gnodet
left a comment
There was a problem hiding this comment.
Re-review after commit b574e61 — the ported fixes (message ordering, serialization cleanup, plugin version restore) are correct. However, 3 findings from the previous review remain open:
1. BannedSkipFilter not flushed on CANCEL_BUILD / BUILD_EXCEPTION (medium)
TerminalOutput.java — BUILD_FINISHED (line 318) correctly calls bannedSkipFilter.flush(log::accept), but CANCEL_BUILD (line 256) and BUILD_EXCEPTION (line 269) close the log without flushing the filter first. Buffered cosmetic lines (blank lines, separators, "Skipping X" preambles) would be silently dropped during cancelled or failed builds.
2. Stale TODO on PROJECT_TEST_PROGRESS (medium)
Message.java line 71 says "TODO: the daemon-side feed … is not implemented on mvnd-1.x yet" but this PR fully implements it. Should be replaced with documentation describing the feature.
3. TestState.displayName() null safety (low)
TestProgressAccumulator.java line 289 — when both testClass and testMethod are null, displayName() returns null → flakyDetail() calls new StringBuilder(null) → NPE. Practically unreachable but a defensive "(unknown)" fallback would be more robust.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of @gnodet
Flush BannedSkipFilter on CANCEL_BUILD/BUILD_EXCEPTION (previously only BUILD_FINISHED did), replace the stale PROJECT_TEST_PROGRESS TODO now that the daemon-side feed is implemented, and give TestState.displayName() a defensive fallback so it can't return null into flakyDetail(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Thanks for the re-review! Addressed all 3 remaining findings in 9822b4c:
|
gnodet
left a comment
There was a problem hiding this comment.
Re-review after commit 9822b4c: all 3 remaining findings are now addressed. ✅
-
BannedSkipFilter flush —
CANCEL_BUILDandBUILD_EXCEPTIONhandlers now callbannedSkipFilter.flush(log::accept)guarded byhideBannedProjectSkips, matching the existingBUILD_FINISHEDpattern. No more silently dropped cosmetic lines on cancelled/failed builds. -
Stale TODO replaced — The
Message.javaTODO now accurately describes the implemented data flow (forked JVM → daemon → client display). -
displayName() null safety — Returns
"(unknown)"when bothtestClassandtestMethodare null, preventing NPE inflakyDetail().
All changes are minimal and consistent with existing patterns. No new issues introduced.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of @gnodet
Fix the supportsForkNode milestone check so it only guards the 3.0.0-Mx series instead of every 3.x version, always clear the JLine display at build end (including failures), make ProjectTestProgressEvent's list getters unmodifiable, dedupe the test-progress enabled check between Server and MvndTestProgressLifecycleParticipant, hoist the version regex to a static field, replace magic indices in TestBuildSummary's snapshot array with named constants, and document the BANNED_MARKER string-match as a maintenance risk. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Flush BannedSkipFilter on CANCEL_BUILD/BUILD_EXCEPTION (previously only BUILD_FINISHED did), replace the stale PROJECT_TEST_PROGRESS TODO now that the daemon-side feed is implemented, and give TestState.displayName() a defensive fallback so it can't return null into flakyDetail(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Important
This PR is stacked on top of #1666 (
feature/worker-line-display-1x). It should be reviewed and merged after #1666. Until #1666 merges, the diff here includes its commits as ancestors; the base will effectively rebase ontomvnd-1.xonce #1666 lands.What this adds
The test-progress feed for mvnd 1.x: a bridge that streams live JUnit/Surefire test progress from forked test JVMs back to the daemon and out to the client. This is the data source that PR #1666's worker-line suffix renders.
Highlights
-Dmvnd.testProgressflag (Environment) to opt into the feature, wired into bash completion.mvnd-surefire-progressmodule providing a custom SurefireForkNodeFactory(MvndForkNodeFactory) plus aTestProgressAccumulatorthat tallies per-fork test events.MvndTestProgressLifecycleParticipantinjects the fork-node factory into the Surefire mojo configuration, and the daemon dispatches accumulated progress to the client (ClientDispatcher,Server).InvalidatingPluginRealmCacheloadsMvndForkNodeFactoryin the Surefire plugin realm so the class resolves at fork time.TestProgressTest) driving a sample project end to end.Files
21 files changed, ~1000 insertions across the common module, daemon, surefire-progress module, distribution, and integration tests.
🤖 Generated with Claude Code