Reduce cached codec metaclass registration overhead

[jamesfredley]

The Problem

Fixes #15374.

Grails codec registration dynamically adds encodeAs* and decode* methods onto globally hot metaclasses - String, GStringImpl, StringBuffer, StringBuilder, and Object. On Groovy Indy, repeatedly mutating those globally-used ExpandoMetaClass types invalidates call sites and creates avoidable startup/runtime overhead. The clearest, fully compatible win is to stop re-adding the exact same cached codec method when the same CodecFactory is already registered for the same target metaclass.

The Fix

Skip redundant cached metaclass writes while preserving every existing behavior.

• CodecMetaClassSupport now tracks cached registrations by target class + codec method name + CodecFactory identity.
• Factory identity is a weak identity key (not the codec name, not raw identityHashCode alone), so distinct factories that expose the same codec name are not collapsed.
• Repeated cached registration for the same factory/method skips the second and later ExpandoMetaClass write; the first still installs the normal closures.
• If a target metaclass is removed and recreated, the method is re-added even when the same factory identity was seen before.
• Duplicate checks/writes are synchronized per target ExpandoMetaClass so parallel registration cannot race into duplicate writes.

What is deliberately preserved

• Non-cached (development/reload) registration is unchanged - codecs still re-resolve and reattach at invocation time.
• Distinct factory instances stay distinct even with the same codec name.
• Aliases still register through the same path.
• No public API, config, dependency, or migration change - apps keep calling value.encodeAsHTML(), value.encodeAsURL(), value.decodeSomeCodec() exactly as before.

Measured impact

Micro-benchmark (:grails-encoder:codecMetaClassBenchmark, 10,000 same-factory registrations):

Scenario	Baseline 8.0.x	This branch	Result
Same factory, 10k registrations	~7.99s	~1.23s	~6.5x fewer registration ns; only 2 metaclass writes
New factory per registration (control)	~9.96s	~14.4s	20,000 writes - distinct factories intentionally not collapsed

Real Grails app (grails-test-examples-app1, opt-in realAppCodecBenchmark, 5 full app starts):

Metric	Baseline 8.0.x	This branch	Improvement
Startup mean (5 starts)	16.36s	11.26s	31.2% lower / 1.45x
Startup median (5 starts)	10.03s	5.86s	41.6% lower / 1.71x
Direct dynamic codec median	8.11s	6.48s	20.1% lower / 1.25x
HTTP codec endpoint median	2.10s	1.53s	26.8% lower / 1.37x

Branch counter shows exactly 110 codec metaclass writes per full app1 startup.

Scope

This is the codec-registration slice of the broader #15374 investigation. GORM dynamic methods, taglib dispatch, static-compilation opportunities, artefact indexing, and metaclass freeze/warning mechanisms were examined but are not changed here - they carry a larger compatibility/review surface. No user-facing docs change is required (no new setting, workflow, API, or syntax).

Testing

• :grails-encoder:test (adds CodecMetaClassSupportSpec: same-factory idempotence, re-registration after metaclass replacement, distinct-factory isolation, non-cached re-registration).
• Functional: :grails-test-examples-gsp-layout:integrationTest GSP encodeAsHTML prevents XSS / encodeAsURL encodes URL parameters pass through a real browser-driven app surface.

[Copilot]

Pull request overview

This pull request reduces startup/runtime overhead from repeated cached codec registration by making cached encodeAs* / decode* ExpandoMetaClass writes idempotent per (target class, codec method name, codec factory identity), while preserving non-cached (development/reload) behavior and distinct-factory semantics.

Changes:

• Add a weak-identity registration key and per-ExpandoMetaClass synchronization to skip duplicate cached meta-method writes.
• Add focused Spock coverage for cached idempotence, metaclass replacement, distinct factories, and non-cached re-registration.
• Add a benchmark harness plus a :grails-encoder:codecMetaClassBenchmark Gradle task to measure registration cost and write counts.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
grails-encoder/src/main/groovy/org/grails/encoder/CodecMetaClassSupport.groovy	Adds cached-registration dedupe bookkeeping and synchronized duplicate checks to avoid repeated ExpandoMetaClass mutation.
grails-encoder/src/test/groovy/org/grails/encoder/CodecMetaClassSupportSpec.groovy	Adds targeted tests covering idempotence, concurrency, metaclass replacement, stale-key pruning, and non-cached behavior.
grails-encoder/src/test/groovy/org/grails/encoder/CodecMetaClassBenchmark.groovy	Adds a runnable benchmark main for codec registration and encode-path timing/counters.
grails-encoder/build.gradle	Registers a JavaExec benchmark task and forwards grails.codec.benchmark.* system properties.
grails-test-suite-uber/src/test/groovy/org/grails/commons/DefaultGrailsCodecClassTests.groovy	Adds regression coverage calling configureCodecMethods() twice and validates public dynamic methods still work; improves metaclass cleanup.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 73.52941% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.4050%. Comparing base (7863ae3) to head (ac31469).
⚠️ Report is 148 commits behind head on 8.0.x.

Files with missing lines	Patch %	Lines
...vy/org/grails/encoder/CodecMetaClassSupport.groovy	73.5294%	5 Missing and 4 partials ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##                8.0.x     #15800        +/-   ##
==================================================
+ Coverage     49.3866%   49.4050%   +0.0184%     
- Complexity      16810      16828        +18     
==================================================
  Files            1993       1993                
  Lines           93495      93531        +36     
  Branches        16360      16369         +9     
==================================================
+ Hits            46174      46209        +35     
+ Misses          40189      40186         -3     
- Partials         7132       7136         +4     

Files with missing lines	Coverage Δ
...vy/org/grails/encoder/CodecMetaClassSupport.groovy	71.4286% <73.5294%> (+15.6146%)	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jdaugherty]

I'm worried about the memory impacts this has on large applications. It's rolling it's own caching mechanism when elsewhere in the code base we've used other solutions like caffeine, etc.

[jdaugherty]

From using fabled to review this:

Review: PR #15800 — Reduce cached codec metaclass registration overhead

First, confirming one premise: repeated emc."${name}" << closure does silently replace
(Groovy's registerIfClosure only throws for native Java methods), so the "redundant
writes invalidate call sites" problem is real. The fix's goal is sound; the
implementation has several problems.

Correctness / performance problems

1. The distinct-factory path is now quadratic — and the PR's own control benchmark
proves it regressed. removeStaleMetaMethodRegistrationKeys()
(CodecMetaClassSupport.groovy:224-231) runs a full scan of every registered key under
the global lock on every single registration:

private static void removeStaleMetaMethodRegistrationKeys() {
    CodecFactoryKey codecFactoryKey = (CodecFactoryKey) STALE_CODEC_FACTORIES.poll()
    while (codecFactoryKey != null) {
        REGISTERED_META_METHODS.remove(codecFactoryKey)
        codecFactoryKey = (CodecFactoryKey) STALE_CODEC_FACTORIES.poll()
    }
    REGISTERED_META_METHODS.keySet().removeIf { CodecFactoryKey key -> key.stale }
}

With N distinct factories that's O(N²), which is exactly why the PR's control case went
from ~9.96s to ~14.4s (+45%). The removeIf sweep also makes the ReferenceQueue
completely redundant — two stale-cleanup mechanisms where the queue alone (O(1)
amortized) would do.

2. synchronized (emc) locks globally shared metaclasses
(CodecMetaClassSupport.groovy:178). In production these are the metaclasses of
String, Object, StringBuilder, etc. Groovy runtime internals and arbitrary user
code can synchronize on those same monitors; taking them and then nesting the global
REGISTERED_META_METHODS lock inside creates a contention and lock-ordering hazard on
some of the hottest objects in the JVM. All codec registration also serializes through
the one static map lock.

3. The registration key uses the class name, not the class (registrationKey,
line 235):

private static MetaMethodRegistrationKey registrationKey(ExpandoMetaClass emc, String methodName) {
    new MetaMethodRegistrationKey(emc.getTheClass().getName(), methodName)
}

Under dev reload / plugin classloaders, a reloaded class with the same name collides
with the old entry, so its registration is skipped and only the getMetaMethod
fallback saves it — and that fallback is unreliable (next point).

4. The fallback check is name-only and cross-factory (line 219):

registeredMetaMethodKeys(codecFactory).add(key) || emc.getMetaMethod(methodName, EMPTY_ARGS) == null

After a metaclass is replaced, factory A's re-registration is skipped if any factory
has already attached a same-named method to the new EMC — so "distinct factories with
the same codec name stay distinct" (the PR's own claim) breaks across metaclass
replacement, silently changing which encoder wins. Worse, with
ExpandoMetaClass.enableGlobally() a freshly created String EMC can inherit the
expando method from Object's EMC, making getMetaMethod non-null and suppressing the
re-attach entirely.

5. This is ~150 lines of hand-rolled weak-identity caching — custom WeakReference
key with identity equals/hashCode, a ReferenceQueue, manual stale sweeps, hand-written
equals/hashCode on two key classes, double-checked locking.
Caffeine.newBuilder().weakKeys() gives exactly weak identity keys, thread-safe, with
none of this — and Caffeine is already used in grails-web-url-mappings,
grails-datastore-core, grails-rest-transforms, etc. All of that machinery exists to
dedupe what the PR itself says is ~110 writes per startup.

6. Is this even the right layer? DefaultCodecLookup.reInitialize() →
GrailsCodecClass.configureCodecMethods() is where the duplicates originate.
DefaultGrailsCodecClass already tracks an initialized flag for exactly this "called
more than once" situation (DefaultGrailsCodecClass.java:324-330). A per-codec-class
"already registered cached methods" guard there would be a few lines with no global
static registry at all. The PR never identifies where the same-factory duplicate calls
come from in a real app, which is the question that determines the simplest fix.

Evidence problems

7. The headline numbers aren't reproducible from the branch. The
realAppCodecBenchmark (5 app starts, 31–41% startup improvement) is not in this diff,
and no baseline write count is given — "110 writes on the branch" is meaningless without
knowing the baseline count (if baseline was ~130, the dedup saved ~20 writes and can't
explain a 5-second startup change). A mean of 16.36s vs median of 10.03s across 5
samples means one or two massive outliers dominated the mean; that's noise, not a
measurement. The PR description also still references a
:grails-encoder:codecMetaClassBenchmark JavaExec task that was removed in the third
commit.

Test / process problems

8. Tests violate the repo's own rules (CLAUDE.md rule 9 — test via public APIs) and
the production class is polluted with test instrumentation.
META_METHOD_REGISTRATION_COUNT is incremented on every registration forever in
production solely so tests can read it; clearMetaMethodRegistrationState() and
getMetaMethodRegistrationKeyCount() exist only for tests. The spec then goes further
and reflects into the private field (CodecMetaClassSupportSpec.groovy:211-225):

private static Set registeredMetaMethodKeys() {
    def field = CodecMetaClassSupport.getDeclaredField('REGISTERED_META_METHODS')
    field.accessible = true
    ((Map) field.get(null)).keySet()
}

Tests that can only assert via private internals are a signal the design isn't
observable through its public behavior.

9. Global-state test hygiene. The spec clears process-wide static registration state
in setup/cleanup, and DefaultGrailsCodecClassTests now removes the metaclasses of
String, GStringImpl, StringBuffer, StringBuilder, and Object in tearDown —
global mutations in a suite the project explicitly warns runs in parallel. TestLens
already flagged a flaky test on this PR's CI run.

10. Minor. grails-encoder/build.gradle reads System.properties at configuration
time for every Test task (configuration-cache unfriendly); the benchmark spec carries
a main() method and println reporting inside a Spock spec; hand-rolled
equals/hashCode instead of @EqualsAndHashCode.

Bottom line

The problem is real, but I'd push back on the shape of the fix:

• (a) Identify where the duplicate same-factory configureCodecMethods calls
  actually come from and dedupe at the caller
  (DefaultGrailsCodecClass/DefaultCodecLookup) if possible.
• (b) If a registry is genuinely needed, use Caffeine weakKeys() keyed on the
  factory with the target Class (not its name) in the value key, and drop the
  synchronized(emc) / global-lock / manual sweep machinery.
• (c) Remove the production-side counters and make tests observe public behavior.
• (d) Either include the real-app benchmark harness in the PR or restate the impact
  with the baseline write count and a defensible measurement.

[testlens-app]

✅ All tests passed ✅

🏷️ Commit: ac31469
▶️ Tests: 10798 executed
⚪️ Checks: 61/61 completed

---

Learn more about TestLens at testlens.app.

[jamesfredley]

The problem this targets is real - repeated emc."${name}" << closure does silently replace and invalidate call sites, and reducing those redundant startup writes is worth doing (it's the codec slice of the Indy/metaclass performance topic, Codebase 2.3 / Google Doc 1.1). But the review is right that the current implementation needs reshaping, and I'd rather fix the shape than defend it:

• The stale-sweep is O(N²). removeStaleMetaMethodRegistrationKeys() does a full removeIf scan of every registered key under the global lock on every registration, which is why the control benchmark regressed ~+45%. The ReferenceQueue alone is O(1) amortized and makes the sweep redundant.
• synchronized (emc) locks globally-shared metaclasses (String, Object, StringBuilder...), which Groovy internals and user code can also synchronize on - a real contention / lock-ordering hazard, made worse by nesting the global map lock inside it.
• Name-based registration key collides under dev-reload / plugin classloaders (a reloaded same-named class overwrites the old entry), and the name-only cross-factory fallback can silently change which encoder wins.
• ~150 lines of hand-rolled weak-identity caching duplicates what Caffeine.newBuilder().weakKeys() gives for free - and Caffeine is already used in grails-web-url-mappings / grails-datastore-core.

So I agree with the direction you outlined:

1. Find where the duplicate same-factory configureCodecMethods calls actually originate (DefaultCodecLookup.reInitialize() -> GrailsCodecClass.configureCodecMethods()) and dedupe at the caller - DefaultGrailsCodecClass already tracks an initialized flag for exactly this, so a per-codec-class guard there may remove the need for a global registry entirely.
2. If a registry is still needed, use Caffeine weakKeys() keyed on the factory + target Class (not its name), and drop the synchronized(emc) / global-lock / manual-sweep machinery.
3. Move the test instrumentation (META_METHOD_REGISTRATION_COUNT, the reflect-into-private-field spec) out of the production class and assert via public behavior (repo test-via-public-API rule).
4. Either include the real-app benchmark harness in the PR or restate the impact with the baseline write count and a defensible measurement (a mean of 16.36s vs a 10.03s median is outlier-dominated).

I'll rework it this way. Thanks for the thorough review - this one's a genuinely better design after the feedback.

[jdaugherty]

@jamesfredley let me know once all of these changes are pushed and I'll review again.