Repository navigation
feat(messagepack): MessagePack value serializer package - #405
Open
cosmin-staicu wants to merge 1 commit into
Open
cosmin-staicu wants to merge 1 commit into
cosmin-staicu wants to merge 1 commit into
Conversation
cosmin-staicu
requested review from
alinahornet,
florin-munteanu-uipath,
litheon,
lucianaparaschivei and
razvalex
as code owners
November 12, 2026 01:20
|
🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off. Strong signals
This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.
|
cosmin-staicu
force-pushed
the
perf/value-serializer-wiring
branch
from
November 13, 2026 13:39
9c17a18 to
e368986
Compare
cosmin-staicu
force-pushed
the
perf/messagepack-serializer
branch
from
November 13, 2026 13:39
3abbcdb to
25ebdbf
Compare
cosmin-staicu
force-pushed
the
perf/value-serializer-wiring
branch
4 times, most recently
from
November 14, 2026 03:46
9378c89 to
f4a9d49
Compare
cosmin-staicu
force-pushed
the
perf/messagepack-serializer
branch
from
November 14, 2026 03:53
25ebdbf to
1e5409d
Compare
cosmin-staicu
force-pushed
the
perf/value-serializer-wiring
branch
from
November 14, 2026 04:17
f4a9d49 to
a32ea16
Compare
Adds the UiPath.Caching.MessagePack package, a value serializer that a policy picks with Serializer.Write = "messagepack". It is registered with AddMessagePackSerializer, which takes an optional name and a configure callback for MessagePackValueSerializerOptions. Values below CompressionThresholdBytes (default 1024) are written plain with format id 0x02. At or above it the value is tried with LZ4, stored as id 0x03 when the result is smaller and as plain 0x02 otherwise. A threshold of 0 tries every value. Reads accept both ids and reject a compressed value that claims more than 255 times its stored size or more than MaxDecompressedBytes (default 512 MiB), which is also applied as MessagePack's own decompression limit. A value larger than that cap is written plain so it stays readable, and an OutOfMemoryException wrapped by MessagePack is rethrown unwrapped. The default serializer options use a contractless resolver with MessagePackSecurity.UntrustedData, and supplied options that carry TrustedData are made untrusted. The serializer is internal and owns only ids 0x02 and 0x03, registered through an internal method that UiPath.Caching exposes to the package with InternalsVisibleTo. The public AddValueSerializer still accepts consumer ids only, so the package depends on the exact same UiPath.Caching version. Nothing changes until an app registers the package and a policy names it. Once values are written as MessagePack, every reader of those keys needs the package registered, readers first in a rolling deploy. Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
cosmin-staicu
force-pushed
the
perf/messagepack-serializer
branch
from
November 14, 2026 04:35
1e5409d to
e77fc7f
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fourth in the fast value serializer stack, on top of #404. Adds the
UiPath.Caching.MessagePackpackage: a value serializer that a policy picks withSerializer.Write = "messagepack".The serializer is internal and owns only ids 0x02 and 0x03. Library ids are registered through an internal method, so
UiPath.Cachinggives the packageInternalsVisibleTo. The publicAddValueSerializerkeeps accepting consumer ids only. Because of that, the package depends on the exact sameUiPath.Cachingversion ([3.2.0]in the nuspec).Evidence
Through
RedisCachewith a fake connector, MessagePack against JSON (BenchmarkDotNet short run; times are noisy, allocations are exact):The fake connector now keeps its own copy of each value, so Set allocations include one stored-size copy for both formats. An earlier Large MessagePack read measured a miss on a reused pooled buffer; that number is gone.
Compression: MessagePack's built-in
Lz4BlockArraywas within noise of a plain-then-K4os wrapper and allocates far less on write, so no new dependency. Compressible values shrink about 2-3x, writes cost about 2-3x, reads about the same.A value is serialized once. Below the threshold it is copied; at or above it, the plain bytes go through MessagePack's LZ4 block array and the smaller form is kept. Below the threshold this is 20% faster. Above it, short runs showed 0 to 12% slower (16 KB worst), inside those runs' noise.
Mutations caught: compressed value reported as 0x02, the "smaller" check dropped, labelling by length only, defaults without
UntrustedData, suppliedTrustedDatakept, null written as zero bytes, a threshold of 0 passed through, the 255x ratio moved either way, no absolute cap, pooled arrays returned uncleared, a non-binary first block accepted, MessagePack's 64 MiB limit left in place, out-of-memory left wrapped.End to end against Redis: a key written JSON, then MessagePack, then JSON reads correctly each time; hash cache, batch, multilayer invalidation and peer field updates with real MessagePack ids.
Full suite with Redis: 4948, 0 failed, 1 skipped.
-c Release -warnaserrorbuilds clean.Merge Danger
Door: two-way
A new package. Nothing changes until an app registers it and a policy names it. Once values are written as MessagePack, every reader of those keys needs this package registered (readers first in a rolling deploy, as
docs/how-to/extending.mdsays).Blast Radius: storage
Only keys whose policy names the MessagePack serializer.
PackageValidationBaselineVersionis cleared in the new csproj because no baseline package exists yet. Set it after the first release.