Add headless game-test/unit-test harness with script-side AssertJ assertions and coverage by hspragg-godaddy · Pull Request #1153 · kube-mods/kubejs

hspragg-godaddy · 2026-06-16T21:08:50Z

Description

Grows the initial "one BlockEvents.broken test" into a full headless test harness for KubeJS — game tests, script-side AssertJ assertions, a bootstrapped JUnit layer, and merged coverage reporting — plus a small 26.1 port bug fix the harness surfaced.

Game-test harness

KubeJSGameTests (the testmod entry point) stands up NeoForge's TestFramework; AlmostGradle's testMod/gameTests/testFramework/jUnit sets are enabled. runGametest boots a headless dedicated server that loads the testmod mod plus the KubeJS scripts under src/test/kubejs/.
TestRuntime (bound into scripts by TestRuntimePlugin) bridges a script and the game test driving it: pass/passed/count/clear/passStartup markers, plus check/verify and assertThat/assertDefined (below).
Event coverage across groups — each test drives a world action with a mock server player and asserts the KubeJS event fired (or that its event object resolves): BlockEventTests, EntityEventTests, LevelEventTests, ServerEventTests, ItemEventTests, PlayerEventTests. Events that can't fire on a headless dedicated server (client/GUI/networking, stage mutation) are deliberately omitted.

Script-side AssertJ assertions

TestRuntime.assertThat(...) exposes typed AssertJ entry points to scripts (String/number/boolean/Object/Iterable), plus custom KubeEntityEventAssert (hasEntityType/hasPlayer/hasNoPlayer) and LevelBlockAssert (hasId/hasProperty) for event objects.
Since the event dispatcher swallows exceptions thrown in handlers, assertions run inside TestRuntime.check(id, () => …), which captures the failure; the paired game test calls TestRuntime.verify(id) to re-throw it on the game-test thread with AssertJ's message — so a wrong value fails the test instead of silently timing out.
kjs_reads.js + KjsReadTests exercise the script-exposed *KJS convenience getters (EntityKJS/LivingEntityKJS/ServerPlayerKJS/LevelBlock/ItemStackKJS) and assert each resolves (assertDefined).
builder_fixtures.js registers a spread of blocks/items/fluids to cover the builder classes during startup-registry load.

Unit tests

BootstrapExtension runs Bootstrap.bootStrap() once so JUnit tests can hit registry-backed script API (wrappers, block lookups) without standing up a server.
Over two dozen unit-test classes over pure/registry-backed helpers: util (ID, Tristate, TimeJS, TickDuration, IntBounds, CountingMap, Object2LongEntry, ErrorStack, TinyMap, JsonUtils, UtilsJS, NBTUtils, ListJS, RegExpKJS), wrappers (Item/Block/Color/NBT/Misc/Text), recipe components (Number/Boolean/String/Character), and MapColorHelper.

Coverage

jacoco (0.8.15 — the first version supporting Java 25 class files) attached to both the JUnit test JVM and the runGametest server, scoped to dev.latvian.mods.kubejs.*. coverageReport merges both into one HTML+CSV report; coverageSummaryJson derives coverage-summary.json (instruction/branch/line %). A Coverage workflow runs both suites on PRs and posts the summary. Whole-mod instruction coverage is ~26% and rises with each added test.

Bug fix

DetectorBlock failed to register on 26.1 (its block id was never stamped on Properties), which crashed world load; fixed, which also unblocks the detector game tests.

Example Script

Server scripts can now assert on event objects, with failures surfaced to the driving game test:

BlockEvents.broken(event => {
	TestRuntime.check('block.broken.assert', () => {
		TestRuntime.assertThat(event.block).hasId('minecraft:dirt');
		TestRuntime.assertThat(event).hasPlayer();
	});
});

The paired @GameTest breaks a dirt block, waits for the marker, then calls TestRuntime.verify("block.broken.assert") to re-throw any captured AssertJ failure on the test thread.

Example Command

./gradlew runGametest          # headless game-test server
./gradlew test                 # JUnit unit tests
./gradlew coverageSummaryJson  # both suites + merged coverage report/JSON

Output lands in build/reports/coverage/ (HTML, CSV, coverage-summary.json); game-test logs under build/tmp/gametestRuns/gametest/logs/; JUnit results under build/reports/tests/test/.

Other details

Adds a test-only org.assertj:assertj-core dependency (testImplementation).
The harness is server-side only; client/GUI/rendering/recipe-viewer bindings aren't exercised.
Most coverage comes from the mod booting during the game test, so the number grows as more targeted tests are added.
The PR-comment coverage step only runs for branches in the main repository, since fork PRs receive a read-only token.

Enable AlmostGradle's game tests and run them headlessly via NeoForge's gameTestServer (./gradlew runGametest), scoped to the testmod namespace. Adds a testmod game test where a fake player breaks dirt with an empty hand; a KubeJS server script reacting to BlockEvents.broken reports the result through a TestRuntime script binding, which the test asserts.

Enable AlmostGradle's JUnit and testframework test sets, attach JaCoCo to both the `test` (JUnit) and `runGametest` (game-test server) JVMs, and merge their execution data into one report (HTML + CSV + a JSON summary). Coverage tasks force a fresh run of both test JVMs so the report always reflects a real execution. Adds a coverage CI workflow that runs both suites, uploads the HTML report, and posts the coverage summary as a PR comment.

@testholder

Mirror how NeoForge structures its own tests: stand up a TestFramework in the testmod bootstrap and write the block-break test as an annotated @GameTest/@EmptyTemplate/@testholder method in a @ForEachTest holder, replacing the hand-rolled GameTestInstance + manual RegisterGameTestsEvent. The test now simulates a real player join via makeTickingMockServerPlayerInCorner and breaks the block through the player game-mode path, following NeoForge's decoratedPotBreaking sequence idiom.

rlnt · 2026-06-17T06:31:08Z

 	}
+
+	runs {
+		// AlmostGradle's gametest run launches as a plain server here; use Neo's headless type and run only our tests.


Is this required because you want to run the unit tests when the game tests are running? I could add a flag to AG that enables this automatically if required.

Without this section it starts up a regular minecraft server rather than a gametest server. Even kicks off the old server GUI.

I have now fixed this in Almost Gradle 2.2.0. When you update, this should no longer be necessary. The game test property name was updated, which I must have missed. Please report it the next time you discover something like this. 😄

I also added a new option to run the unit tests in the game test run config, which you configured here as well.
https://github.com/AlmostReliable/almostgradle#unit-tests-in-game-tests

However, as written in the README, I would not encourage enabling this behavior. Unit tests are invoked in the test Gradle task, which is part of the check Gradle task. And check is invoked when build is run. So a successful build already guarantees that all unit tests passed.

I just pushed changes for the latest version and removed the section like you had suggested.
I also isolated the game tests to not run on build in favor of just the tests running on build.
That way builds don't trigger game tests.

The `<T>` in the injected interface name leaks into the raw class name that NeoForge's TransformerClassWriter feeds to Class.forName during runtime class-hierarchy computation. As the loader does not strip generics, this throws ClassNotFoundException for dev.latvian.mods.kubejs.core.TagLoaderKJS<T> while loading the vanilla datapack, crashing the game test server. The generic remains in the Java source/signature; only the injection name needs the raw type.

AlmostGradle sets it implicitly when the tests {} block is used, per upstream review feedback.

AlmostGradle 2.2.0 launches the gametest run as a headless gameTestServer, so the custom runs{} override is no longer needed.

Adds NeoForge TestFramework game tests exercising every BlockEvents and EntityEvents handler, driven by paired server/startup scripts through a hardened TestRuntime bridge (concurrent, per-marker clears, count and startup-scoped markers). DetectorBlock could not be constructed on 26.1: it built its Properties from bedrock without setting the block id that BlockBehaviour now requires, crashing registration and world load. It now stamps the id the same way BlockBuilder does, which also unblocks the detector event tests.

Add typed TestRuntime.assertThat entry points backed by AssertJ so game-test scripts can assert on event object contents: custom KubeEntityEventAssert and LevelBlockAssert plus String/boolean/double/Object entry points. check() captures failures (the event dispatcher swallows handler exceptions) and verify() re-throws them on the game-test thread. Adds block_broken_asserts, block_placed_asserts and entity_death_asserts covering the new API.

Add game tests for level, server, item, and player events (tick, explosion, command, item drop/right-click/entity-interact/pickup) reusing TestRuntime + assertThat, plus an assertThat(Iterable) overload. Add pure-logic unit tests for util classes (ID, Tristate, TimeJS, TickDuration, IntBounds, CountingMap, Object2LongEntry, ErrorStack, TinyMap). Bump the changelog PR numbers to kube-mods#1153.

Add a coverageGate Gradle task that measures coverage over the script-exposed API surface (partitioned into reachable / client-only / excluded-internal), prints the top uncovered reachable classes, and fails below a configurable threshold (-PscriptCoverageGate). Add a BootstrapExtension so unit tests can exercise registry-backed script API without a game server, and unit tests for wrappers (Item/Color/NBT/Misc/Text), recipe components, and util (JsonUtils/UtilsJS/NBTUtils). Raises reachable script-API coverage from 21.8% to 24.8%.

Register a spread of blocks (basic + all custom types: slab/stairs/fence/wall/ fence_gate/pressure_plate/button/falling/carpet/door/trapdoor/cardinal/pillar/crop), items, and fluids via startup-registry scripts, exercising BlockBuilder, block.custom.*, ItemBuilder, and FluidBuilder during headless startup. Raises reachable script-API coverage from 24.8% to 28.8%.

Server scripts that read the vanilla-injected *KJS convenience getters (EntityKJS, LivingEntityKJS, ServerPlayerKJS, LevelBlock, ItemStackKJS) off the event objects the existing game tests already drive. Raises reachable script-API coverage from 28.8% to 29.8%.

… gate Make the *KJS getter fixtures real tests: TestRuntime.assertDefined verifies each exposed property resolves (not null / not JS undefined), driven+verified by new KjsReadTests, rather than merely calling getters. Add bootstrapped BlockWrapper and pure MapColorHelper unit tests. Reachable script-API coverage now 30.1%. Remove the temporary coverageGate task from the build script now that it has served its purpose.

Address review feedback by re-categorising the test suite so behaviour is exercised through JS scripts in real game-test scenarios rather than trivial Java unit tests: - Wrapper coverage now drives raw JS values across typed boundaries (TestRuntime.as*), exercising the registered type wrappers end to end. - Replace the kjs_reads getter fixtures with an entity-behaviour test: a script gives a spawned zombie a held item and moves a cow, verified Java-side. - Add JS binding tests (incl. JsonIO/NBTIO file creation) and syntax tests. - Assert exact event fire counts, and route Java assertions through AssertJ (GameAsserts) so failures carry actual-vs-expected messages. - Trim builder fixtures to a few actually integrated into game tests. - Drop the JUnit unittest package in favour of the JS-driven tests.

…mples

MaxNeedsSnacks · 2026-07-10T11:04:27Z

+ServerEvents.tick(event => {
+	if (wrapperChecksRun) {
+		return;
+	}


why use a single-run ServerEvents.tick instead of firing a custom event when the game test world is ready?

Exceptions were being thrown on first tick, this was a mitigation to that.

hspragg-godaddy added 3 commits June 16, 2026 13:35

rlnt reviewed Jun 17, 2026

View reviewed changes

hspragg-godaddy and others added 13 commits June 17, 2026 17:06

Remove redundant tests.enabled flag

84ee77a

AlmostGradle sets it implicitly when the tests {} block is used, per upstream review feedback.

Updated build script such that unit tests run before game test

eb09321

Bump AlmostGradle to 2.2.0 and drop redundant gametest run config

9ddaf10

AlmostGradle 2.2.0 launches the gametest run as a headless gameTestServer, so the custom runs{} override is no longer needed.

Merge branch '2601' into feat/headless-game-tests

1fa06a8

Add AssertJ test dependency for game-test assertions

00c1f2d

hspragg-godaddy changed the title ~~Add headless game test harness with a BlockEvents.broken test~~ Add headless game-test/unit-test harness with script-side AssertJ assertions and coverage Jul 7, 2026

MaxNeedsSnacks requested changes Jul 9, 2026

View reviewed changes

hspragg-godaddy added 3 commits July 9, 2026 17:00

Add game tests running wiki recipe, tag, modification and builder exa…

cbcd0d8

…mples

Add game tests for Item, Ingredient, Text, NBT and JsonUtils bindings

b94742c

MaxNeedsSnacks reviewed Jul 10, 2026

View reviewed changes

Comment thread src/test/kubejs/server_scripts/syntax_checks.js

MaxNeedsSnacks reviewed Jul 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add headless game-test/unit-test harness with script-side AssertJ assertions and coverage#1153

Add headless game-test/unit-test harness with script-side AssertJ assertions and coverage#1153
hspragg-godaddy wants to merge 19 commits into
kube-mods:2601from
hspragg-godaddy:feat/headless-game-tests

hspragg-godaddy commented Jun 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

rlnt Jun 17, 2026

Uh oh!

hspragg-godaddy Jun 17, 2026 •

edited

Loading

Uh oh!

rlnt Jun 18, 2026

Uh oh!

Hunter19823 Jun 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MaxNeedsSnacks Jul 10, 2026

Uh oh!

hspragg-godaddy Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

hspragg-godaddy commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Example Script

Example Command

Other details

Uh oh!

Uh oh!

rlnt Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

hspragg-godaddy Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rlnt Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

Hunter19823 Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MaxNeedsSnacks Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

hspragg-godaddy Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hspragg-godaddy commented Jun 16, 2026 •

edited

Loading

hspragg-godaddy Jun 17, 2026 •

edited

Loading