Skip to content

Stream live test-progress from forked test JVMs to the mvnd client (mvnd-1.x) - #1670

Open
ammachado wants to merge 15 commits into
apache:mvnd-1.xfrom
ammachado:feature/test-progress-feed-1x
Open

Stream live test-progress from forked test JVMs to the mvnd client (mvnd-1.x)#1670
ammachado wants to merge 15 commits into
apache:mvnd-1.xfrom
ammachado:feature/test-progress-feed-1x

Conversation

@ammachado

Copy link
Copy Markdown

Important

This PR is stacked on top of #1666 (feature/worker-line-display-1x). It should be reviewed and merged after #1666. Until #1666 merges, the diff here includes its commits as ancestors; the base will effectively rebase onto mvnd-1.x once #1666 lands.

What this adds

The test-progress feed for mvnd 1.x: a bridge that streams live JUnit/Surefire test progress from forked test JVMs back to the daemon and out to the client. This is the data source that PR #1666's worker-line suffix renders.

Highlights

  • -Dmvnd.testProgress flag (Environment) to opt into the feature, wired into bash completion.
  • mvnd-surefire-progress module providing a custom Surefire ForkNodeFactory (MvndForkNodeFactory) plus a TestProgressAccumulator that tallies per-fork test events.
  • Daemon-side wiring: MvndTestProgressLifecycleParticipant injects the fork-node factory into the Surefire mojo configuration, and the daemon dispatches accumulated progress to the client (ClientDispatcher, Server).
  • Plugin-realm fix: InvalidatingPluginRealmCache loads MvndForkNodeFactory in the Surefire plugin realm so the class resolves at fork time.
  • Integration test (TestProgressTest) driving a sample project end to end.

Files

21 files changed, ~1000 insertions across the common module, daemon, surefire-progress module, distribution, and integration tests.

🤖 Generated with Claude Code

@ammachado
ammachado force-pushed the feature/test-progress-feed-1x branch 2 times, most recently from c3ab044 to 0819f84 Compare July 21, 2026 13:03
ammachado and others added 2 commits July 21, 2026 09:08
- Extract buildRequestEnvironment() in DefaultClient for subclass override
- Strip MAVEN_ARGS from JvmTestClient and NativeTestClient to block per-test flag overrides
- Write an isolated settings.xml in MvndTestExtension so tests never fall back to
  the runner's user settings (fixes InteractiveTest when actions/setup-java sets interactiveMode)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fix on mvnd-1.x

Reshape the live terminal display: a drawn progress bar with percent on the
status line, concise arrow-style worker lines (> :module goal (execution)),
and dimmed > IDLE slots for free threads. At addProjectLine, append a styled
live test-progress suffix (tests/failures/errors/skipped counts plus the
current test class#method) driven by the new PROJECT_TEST_PROGRESS message.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ammachado
ammachado force-pushed the feature/test-progress-feed-1x branch from 0819f84 to 5aeb95c Compare July 21, 2026 13:10

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Test Progress Reporting Feature

Well-designed feature with sound architecture. The maker/checker review process confirmed several items worth the author's attention.

Medium Severity

1. supportsForkNode version check logic gap
The version check in supportsForkNode applies the M5 milestone threshold to ALL 3.x versions, not just 3.0.0. For example, "3.1.0-M2" produces group(4)="2" and returns false, even though the milestone guard should only apply to the 3.0.0-Mx series. Practical impact is negligible since Surefire never published milestone versions beyond 3.0.0-Mx (3.1.0+, 3.2.x, 3.5.x all shipped as GA), but the fix (apply milestone check only when minor==0 && patch==0) is trivial and improves defensive correctness.

2. clearDisplay() no longer called on build failure
The base branch BUILD_FINISHED handler calls clearDisplay() unconditionally. This PR wraps it in if (failures.isEmpty()), so on failure the JLine display's last render (project status lines) remains painted in the terminal. The shouldShowProjectDetails() method and its comment suggest this is intentional (keep failure info visible), but this is a behavior change from prior code — please confirm this is the desired UX.

Low Severity

3. Mutable ArrayList references in getter methods
ProjectTestProgressEvent.getFlakyTests(), getFailedTests(), and getErroredTests() return the mutable internal lists directly. The constructor makes defensive copies, but the getters expose the internal state. Currently no callers mutate the returned lists, but wrapping in Collections.unmodifiableList() would be a safe defensive measure.

4. BANNED_MARKER fragile string matching
The string "This project has been banned from the build due to previous failures." is compared via equals() after stripping ANSI/level prefixes. If Maven changes this message text, the filter silently breaks. This is inherent to text-based output matching (no programmatic API alternative), but worth noting as a maintenance risk.

5. Pattern.compile on each invocation of supportsForkNode
The regex is compiled inline each call. Called once per surefire/failsafe plugin per project, so allocation count is small, but the pattern could be a static final field at no cost.

6. Duplicated isEnabled logic
Both Server.handle() and MvndTestProgressLifecycleParticipant.isEnabled() contain identical logic: Environment.MVND_TEST_PROGRESS.asOptional().map(Boolean::parseBoolean).orElse(Boolean.TRUE). Could be extracted to a shared static method.

7. Magic indices in int[] snapshot array
The TestBuildSummary snapshot array is packed as {completed, failures, errors, skipped, retrying, flaky} and unpacked with raw indices 0–5 (notably skipping index 4/retrying intentionally). A record or named-constant approach would make the skip of "retrying" explicit and prevent index-drift bugs.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of gnodet

ammachado added a commit to ammachado/maven-mvnd that referenced this pull request Jul 28, 2026
Fix the supportsForkNode milestone check so it only guards the
3.0.0-Mx series instead of every 3.x version, always clear the JLine
display at build end (including failures), make ProjectTestProgressEvent's
list getters unmodifiable, dedupe the test-progress enabled check between
Server and MvndTestProgressLifecycleParticipant, hoist the version regex
to a static field, replace magic indices in TestBuildSummary's snapshot
array with named constants, and document the BANNED_MARKER string-match
as a maintenance risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ammachado

Copy link
Copy Markdown
Author

Thanks for the thorough review! Addressed all 7 points in acbb147:

Medium

  1. supportsForkNode version gap — now only applies the milestone guard when minor == 0 && patch == 0, so 3.1.0-M2/3.2.5-M1 etc. are correctly treated as GA. Added a regression test for this.
  2. clearDisplay() on failure — reverted to unconditional clearDisplay() at BUILD_FINISHED; the guarded version wasn't intentional.

Low
3. Mutable list getters — getFlakyTests()/getFailedTests()/getErroredTests() now return Collections.unmodifiableList(...).
4. BANNED_MARKER fragile match — agreed there's no programmatic alternative; added a comment documenting it as a maintenance risk so it's not silently missed if Maven changes the message.
5. Pattern.compile per call — hoisted to a static final Pattern VERSION_PATTERN.
6. Duplicated isEnabled logic — consolidated into MvndTestProgressLifecycleParticipant.isTestProgressEnabled(), called from both Server.handle() and the lifecycle participant.
7. Magic indices in the int[] snapshot — replaced with named IDX_* constants (records aren't available at this module's Java 8 target).

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: All Previous Findings Resolved

The fix commit (acbb147) correctly addresses all 7 findings from the previous review:

  1. supportsForkNode version check — milestone guard now only applies to the 3.0.0-Mx series (checks minor/patch before applying). New test milestoneGuardOnlyAppliesToThe300MilestoneSeries covers "3.1.0-M2" and "3.2.5-M1".
  2. clearDisplay() — again called unconditionally in the BUILD_FINISHED handler.
  3. Mutable list gettersgetFlakyTests/getFailedTests/getErroredTests now return Collections.unmodifiableList wrappers.
  4. BANNED_MARKER — field now has a maintenance-risk comment documenting the fragility.
  5. Pattern.compileVERSION_PATTERN is now a static final field.
  6. Duplicated isEnabledServer.handle() delegates to MvndTestProgressLifecycleParticipant.isTestProgressEnabled() instead of duplicating the logic.
  7. Magic array indicesIDX_COMPLETED/IDX_FAILURES/IDX_ERRORS/IDX_SKIPPED/IDX_FLAKY constants replace magic indices, with IDX_RETRYING=4 explicitly documented as not folded.

The fix commit is well-scoped — touches only the files relevant to the review feedback with no unrelated changes. Nice work!

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of gnodet

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: New Commit (3eedb48)

The new commit adds a defensive clamp to renderBar(int percent)Math.max(0, Math.min(100, percent)) — ensuring out-of-range inputs can't under/overfill the progress bar. A corresponding test (renderBarClampsOutOfRangeInput) covers both negative and >100 values.

Trivially correct, well-scoped change. LGTM.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of gnodet

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-engineered feature adding live test-progress streaming from Surefire/Failsafe forks to the mvnd client. The classloading bridge design, thread safety (AtomicReference for cross-thread listener bridge, per-fork-channel accumulators), and Surefire version guarding are all sound. Test coverage is comprehensive.

Two items worth addressing before merge:

1. BannedSkipFilter not flushed on CANCEL_BUILD / BUILD_EXCEPTION (medium)

The BUILD_FINISHED handler (line 319) correctly calls bannedSkipFilter.flush(log::accept) before closing the log. However, the CANCEL_BUILD handler (line 258) and BUILD_EXCEPTION handler (line 272) skip this flush. Any reactor-level lines buffered in the filter (blank lines, separators, "Skipping X" preambles) will be silently dropped during cancelled or failed builds.

Suggested fix — add the flush before projects.values().stream()... in both handlers:

if (hideBannedProjectSkips) {
    bannedSkipFilter.flush(log::accept);
}

2. Stale TODO comment on PROJECT_TEST_PROGRESS (medium)

The comment at Message.java line 69-73 says "the daemon-side feed that emits this message is not implemented on mvnd-1.x yet", but this PR fully implements it across 12 commits (dispatch, bridge listener, forkNode injection, etc.). The TODO should be replaced with documentation describing the implemented feature.

Minor note

  • TestState.displayName() null safety (low): Returns null when both testClass and testMethod are null, which would cause NullPointerException in flakyDetail() via new StringBuilder(displayName()). Practically unreachable since Surefire always provides source names, but a defensive "(unknown)" fallback would be more robust.

Note: This PR is stacked on PR #1666 — some findings (like BannedSkipFilter) may have been introduced in the base PR.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of gnodet

ammachado and others added 12 commits July 31, 2026 18:18
Applies the review-driven fixes from the sibling 2.x PR (apache#1667):
- PROJECT_TEST_PROGRESS now sorts after PROJECT_STARTED/MOJO_STARTED in the
  sendQueue, avoiding a race where the progress message could be dequeued
  before its project's PROJECT_STARTED event.
- ProjectTestProgressEvent reuses the existing readUTF/writeUTF null-sentinel
  convention instead of a redundant boolean-prefixed nullable encoding.
- Restores the plugin groupId/version in the worker line display, which was
  dropped by the visual redesign.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds the mvnd.testProgress opt-out flag (default on) and the
cross-realm MvndTestProgress bridge interface + static listener
registry that connects the Surefire plugin realm to the daemon realm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
New module contributing the Surefire plugin-realm pieces of the live
test-progress feed:
- TestProgressAccumulator: pure per-fork counter (unit-tested)
- MvndForkNodeFactory: decorates Surefire's EventHandler<Event>,
  accumulates counts, pushes through the MvndTestProgress bridge, and
  always delegates to the real handler (never breaks the test run)
- MvndSurefireProgressLocator: zero-dependency marker the daemon uses to
  locate this jar for plugin-realm injection

Verified compiling and testing against the real Surefire 3.5.x extensions SPI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Adds ClientDispatcher.testProgress to enqueue PROJECT_TEST_PROGRESS
messages, and wires Server.handle to register an MvndTestProgress
listener (per build, when mvnd.testProgress is enabled) that forwards
fork events to the dispatcher; the listener is cleared in finally.
Also adds the daemon dependency on mvnd-surefire-progress.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Wires the Maven 3.9 test-progress feed:
- DaemonMavenCli: export org.mvndaemon.mvnd.testprogress to the core realm
  so the bridge Class is shared with the surefire plugin realm
- MvndTestProgressLifecycleParticipant: afterProjectsRead injects
  <forkNode implementation=MvndForkNodeFactory><projectId>..> into
  surefire/failsafe config (the Maven 3.9-native mechanism)
- InvalidatingPluginRealmCache: addURL the mvnd-surefire-progress jar onto
  the surefire/failsafe plugin realm
- dist: bundle mvnd-surefire-progress; daemon: plexus-xml provided (Xpp3Dom)
- integration-tests: TestProgressTest + test-progress JUnit 5 project

Runtime validation of the mandatory 1.x spike (PROJECT_TEST_PROGRESS emitted
end-to-end) follows in a later commit. Running the integration tests requires
a full dist build first, since the harness spawns a real daemon against
dist/target/maven-mvnd-*.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
The 1.x spike (TestProgressTest) exposed a classloader bug that broke
every project running Surefire (e.g. UpgradesInBomTest, ExecOutputTest):

  Cannot load implementation hint MvndForkNodeFactory:
  NoClassDefFoundError: SurefireForkNodeFactory

Root cause: the factory lived under org.mvndaemon.mvnd.testprogress, the
package exported to the core realm for the shared MvndTestProgress bridge.
ClassWorlds package imports are hierarchical (prefix-based), so the export
also captured the factory's (sub)package; the surefire plugin realm
imported it from the core realm, which has no Surefire, so linking the
factory's SurefireForkNodeFactory superclass failed.

Fix:
- Move MvndForkNodeFactory / TestProgressAccumulator / locator to a sibling
  package org.mvndaemon.mvnd.forknode that is NOT under the exported prefix,
  so the plugin realm loads the factory from its own (addURL'd) jar and
  links against the plugin realm's Surefire. Bridge stays in the exported
  org.mvndaemon.mvnd.testprogress.
- Guard injection on Surefire >= 3.0.0-M5 (forkNode SPI floor) so older
  Surefire/Failsafe are never touched. Add supportsForkNode unit test.

Validated end-to-end on mvnd-1.x: TestProgressTest emits increasing
PROJECT_TEST_PROGRESS with correct class/counts, the opt-out emits none,
and ExecOutputTest / UpgradesInBomTest (Surefire 3.0.0-M8) pass unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Regenerated completion list to include the new mvnd.testProgress property.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZU9mLBgwG78z2WeAi4z47
Extend the live test-progress feed beyond basic counts so a failing build
tells you which tests broke and why, directly in mvnd's output.

- Accumulate retrying/flaky state and capture FAILED/ERRORED test identities
  plus a compact failure message (Surefire smartTrimmedStackTrace, sanitized),
  streamed alongside flaky names in ProjectTestProgressEvent.
- Render a live "Flaky tests:" summary at build end, and inject a
  "Failed tests:" / "Errored tests:" block ("<projectId> Class#method: message")
  directly above Maven's BUILD FAILURE banner, with a build-finished fallback.
- Add mvnd.hideBannedProjectSkips (default true) to drop the per-project
  "Skipping X / banned from the build" reactor blocks while preserving the
  final reactor "... SKIPPED" rows; disable with =false.
- Cover with accumulator, message round-trip, and TerminalOutput unit tests
  (ordering + banned-block filter), plus a new TestProgressFailureTest IT.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ging

Move end-of-build test summary rendering from the client (string
injection keyed on a "BUILD FAILURE" text match) to the daemon, where
LoggingExecutionListener logs it through its own SLF4J logger just
before the Reactor Summary. This routes the summary through the same
MvndSimpleLogger pipeline as every other Maven console line, so ANSI
coloring, [LEVEL] prefixes, and -q gating all come for free instead of
being reimplemented client-side.

ClientDispatcher now collects failed/errored/flaky test identities and
per-fork numeric totals into a new TestBuildSummary as testProgress()
events arrive; BuildEventListener exposes foldTestProgress()/
getTestSummary() so the Maven-realm listener can fold and render them.
TerminalOutput drops the now-daemon-owned summary machinery, keeping
only the live per-project progress line and the banned-skip filter.
The -q client-side plumbing (MAVEN_QUIET, TerminalOutput's quiet field)
is removed since the daemon's own logger level already handles it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Fix the supportsForkNode milestone check so it only guards the
3.0.0-Mx series instead of every 3.x version, always clear the JLine
display at build end (including failures), make ProjectTestProgressEvent's
list getters unmodifiable, dedupe the test-progress enabled check between
Server and MvndTestProgressLifecycleParticipant, hoist the version regex
to a static field, replace magic indices in TestBuildSummary's snapshot
array with named constants, and document the BANNED_MARKER string-match
as a maintenance risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dback

renderBar() already produced a sane bar for out-of-range input, but the
invariant wasn't self-documenting; clamp explicitly and add a regression
test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ammachado
ammachado force-pushed the feature/test-progress-feed-1x branch from 3eedb48 to 629790e Compare July 31, 2026 22:43

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after commit b574e61 — the ported fixes (message ordering, serialization cleanup, plugin version restore) are correct. However, 3 findings from the previous review remain open:

1. BannedSkipFilter not flushed on CANCEL_BUILD / BUILD_EXCEPTION (medium)
TerminalOutput.javaBUILD_FINISHED (line 318) correctly calls bannedSkipFilter.flush(log::accept), but CANCEL_BUILD (line 256) and BUILD_EXCEPTION (line 269) close the log without flushing the filter first. Buffered cosmetic lines (blank lines, separators, "Skipping X" preambles) would be silently dropped during cancelled or failed builds.

2. Stale TODO on PROJECT_TEST_PROGRESS (medium)
Message.java line 71 says "TODO: the daemon-side feed … is not implemented on mvnd-1.x yet" but this PR fully implements it. Should be replaced with documentation describing the feature.

3. TestState.displayName() null safety (low)
TestProgressAccumulator.java line 289 — when both testClass and testMethod are null, displayName() returns null → flakyDetail() calls new StringBuilder(null) → NPE. Practically unreachable but a defensive "(unknown)" fallback would be more robust.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of @gnodet

Flush BannedSkipFilter on CANCEL_BUILD/BUILD_EXCEPTION (previously only
BUILD_FINISHED did), replace the stale PROJECT_TEST_PROGRESS TODO now that
the daemon-side feed is implemented, and give TestState.displayName() a
defensive fallback so it can't return null into flakyDetail().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ammachado

Copy link
Copy Markdown
Author

Thanks for the re-review! Addressed all 3 remaining findings in 9822b4c:

  1. BannedSkipFilter now flushes on CANCEL_BUILD and BUILD_EXCEPTION, matching the existing BUILD_FINISHED behavior — no more silently dropped cosmetic reactor lines on cancelled/failed builds.
  2. Replaced the stale PROJECT_TEST_PROGRESS TODO in Message.java with a description of the implemented feature.
  3. TestState.displayName() now falls back to "(unknown)" instead of returning null when both testClass and testMethod are absent, so flakyDetail() can't NPE.

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after commit 9822b4c: all 3 remaining findings are now addressed. ✅

  1. BannedSkipFilter flushCANCEL_BUILD and BUILD_EXCEPTION handlers now call bannedSkipFilter.flush(log::accept) guarded by hideBannedProjectSkips, matching the existing BUILD_FINISHED pattern. No more silently dropped cosmetic lines on cancelled/failed builds.

  2. Stale TODO replaced — The Message.java TODO now accurately describes the implemented data flow (forked JVM → daemon → client display).

  3. displayName() null safety — Returns "(unknown)" when both testClass and testMethod are null, preventing NPE in flakyDetail().

All changes are minimal and consistent with existing patterns. No new issues introduced.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of @gnodet

ammachado added a commit to ammachado/maven-mvnd that referenced this pull request Aug 1, 2026
Fix the supportsForkNode milestone check so it only guards the
3.0.0-Mx series instead of every 3.x version, always clear the JLine
display at build end (including failures), make ProjectTestProgressEvent's
list getters unmodifiable, dedupe the test-progress enabled check between
Server and MvndTestProgressLifecycleParticipant, hoist the version regex
to a static field, replace magic indices in TestBuildSummary's snapshot
array with named constants, and document the BANNED_MARKER string-match
as a maintenance risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ammachado added a commit to ammachado/maven-mvnd that referenced this pull request Aug 1, 2026
Flush BannedSkipFilter on CANCEL_BUILD/BUILD_EXCEPTION (previously only
BUILD_FINISHED did), replace the stale PROJECT_TEST_PROGRESS TODO now that
the daemon-side feed is implemented, and give TestState.displayName() a
defensive fallback so it can't return null into flakyDetail().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants