Skip to content

perf(kotlin): resolve operand paths lazily, and fix numeric comparison - #56

Merged
rapatao merged 4 commits into
mainfrom
perf/lazy-path-resolution
Aug 31, 2026
Merged

perf(kotlin): resolve operand paths lazily, and fix numeric comparison#56
rapatao merged 4 commits into
mainfrom
perf/lazy-path-resolution

Conversation

@rapatao

@rapatao rapatao commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

Replaces the Kotlin engine's per-evaluation input flattening with lazy path resolution, then fixes two numeric
correctness bugs the new test coverage exposed. Adds allocation and GC tracking to the benchmark harness, plus a wide
input variant that separates a cost scaling with the input from one scaling with the rule.

Lazy path resolution

KotlinEvaluator.call walked the whole input graph on every evaluate and materialised a flat map of every path, so
a rule reading one field of a wide object paid for every other field and every element of every list.

InputPath.resolve(root, path) now walks the input on demand, visiting only the nodes a path names. The path is
scanned in place, so a.b[0].c allocates one String per named segment and nothing else. memberProperties is
reflected once per class into a cache that also serves the name lookup.

The absence rules were read off the old parseKeys and reproduced exactly, because they decide what throws and
therefore what onFailure swallows:

path shape exists when
"" the root is a value, a Collection or an Array
name under a Map containsKey(name), or a key whose toString() matches
name under an object javaClass.kotlin.memberProperties reports it
name under a value, a null, a Collection or an Array never
[i] the node is a Collection or Array and i is in range, by iteration order

KotlinPathResolutionTest covers the table.

Three differences from the flattened implementation, none covered by an existing test:

  • KotlinContext's constructor takes Any instead of Map<String, Any?>. Source-compatible for callers passing a
    map, binary-incompatible.
  • A Map key containing . or [ was a literal key in the flat map and is read as a path now.
  • A property whose getter throws under reflection failed every evaluate of every rule, because flattening touched
    every property. It fails only when a rule names it now.

Numeric fixes

BigDecimal.valueOf(this.toLong()) truncated every BigDecimal operand to its integer part, on both sides of a
comparison, and overflowed above Long.MAX_VALUE:

price = BigDecimal("1.5")
"price" equalsTo "1.9"      -> true    (should be false)
"price" greaterThan "1.4"   -> false   (should be true)

A BigDecimal is now kept as it is and a BigInteger is converted directly. Equality compares numbers by value
rather than by representation, so 10 and 10.00 are the same number, which comparisons already did through
compareTo. Normalization also reaches into collections, guarded so a list without numbers is not rebuilt, which
makes listOf(1, 2) expContains 1 match.

The Rhino engine diverged on the same case: Contains delegated to indexOf, which on a java.util.List is Java
equality, so a JS number never matched a boxed Integer. It scans with == now, and throws for a non-container so
onFailure behaves as it does on the Kotlin engine.

NumberCases covers fractions, scale and numeric lists, and runs against all three engines through
BaseEvaluatorTest.

Benchmark harness

  • alloc/op, total allocation, GC collections and time, and the standard deviation of an iteration are reported per
    run. Allocation comes from the JVM per-thread counter and reports n/a where it is unavailable.
  • -PbenchWide=N runs the same rules against the same fields under a root carrying N extra scalar fields and an N
    element list. Wired into all three bench tasks, which now take the same arguments in the same order.
  • BaseEvaluatorTest.assertWideInputMatchesDefaultInput asserts every case gives the same result against both inputs
    on every engine, so the variant measures the same work.

Measured

2000 iterations of the 173 expression suite, Apple M3 Pro with Amazon Corretto 21.0.11, three runs per configuration
in one session, medians:

engine ops/s avg per iteration p50 p99 relative cost alloc/eval
Kotlin 1,506,886 115us 83us 425us 1x 893 B
Rhino 288,132 600us 490us 1.41ms ~5.2x 11,741 B
GraalJS (reused ctx) 240,662 719us 580us 2.58ms ~6.3x 5,850 B
GraalJS 8,959 19.31ms 19.09ms 21.08ms ~168x 128,763 B

Against the flattening implementation on its own suite, lazy resolution measured 2.1x on the benchmark input and
23.8x on a wide one, where widening the input cost the flattened version 11.7x with the rules unchanged.

Input width, same session:

engine ops/s default ops/s wide(200) wide is alloc default alloc wide(200) wide allocates
Kotlin 1,506,886 1,768,634 unchanged 893 B 872 B unchanged
Rhino 288,132 138,658 ~2.1x slower 11,741 B 23,788 B ~2.0x more
GraalJS (reused ctx) 240,662 16,313 ~14.8x slower 5,850 B 85,596 B ~14.6x more
GraalJS 8,959 5,875 ~1.5x slower 128,763 B 208,161 B ~1.6x more

Both JS engines scale with input width: they inject every top-level entry of the input into the scope on every
evaluate, and the allocation factor tracks the throughput factor on both. Neither scales with depth. Width is the
largest per-evaluation cost left on reused-context GraalJS.

Test plan

  • ./gradlew test green on all three engines, with the existing cases unmodified
  • ./gradlew detekt koverVerify green
  • ./gradlew :kotlin-evaluator:bench -PbenchIterations=2000 and the same with -PbenchWide=200

Operand paths are now resolved on demand against the input, visiting only
the nodes a path names, instead of eagerly flattening the entire input into
a map keyed by dotted paths. Cost scales with the rule, not the input width.

InputPath handles the resolution: maps by key (or `toString()` for non-string
keys), collections and arrays by index, and plain objects by cached Kotlin
reflection. A path that does not exist (missing key, out-of-range index, or
traversal past a value) throws, which `onFailure` turns into a rule result,
preserving the original semantics.

Benchmarks show ~2.7x throughput gain on the default input (2.1M → 0.78M
ops/s before this, now 2.1M again), no cost for wide inputs (previously
degraded by 11.7x), and elimination of the classloader pinning from the
unbounded property cache (now scoped to the engine instance).
A list written in an expression holds operands, but only String operands
were resolved, so a quoted element kept its quotes and never matched the
resolved right operand.

Elements keep their own type: the Number to BigDecimal normalization stays
on the operand as a whole, matching lists read from the input data.
Measures bytes allocated per evaluation and garbage collection counts
during benchmark runs, using HotSpot ThreadMXBean for thread-local
allocation and ManagementFactory for JVM-wide GC stats. Also adds
standard deviation to latency distribution reporting.

Allocation metrics reveal whether engine overhead is bound by memory
allocation or execution cost. Current results show allocation ranging
from 866 B per evaluation on Kotlin to 128 KB on default GraalJS.
Numbers now compare by value using BigDecimal.compareTo instead of
equals, so 10 and 10.00 are the same number and fractions are never
truncated. All numbers normalize to BigDecimal, including elements in
collections, so listOf(1, 2) expContains 1 matches. Updated contains
operator to compare elements by value. Added NumberCases test suite
and updated benchmark results to reflect new test count (173
expressions, up from 150).
@rapatao
rapatao merged commit 65da6bb into main Aug 31, 2026
4 checks passed
@rapatao
rapatao deleted the perf/lazy-path-resolution branch August 31, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant