Repository navigation
feat(negotiation): switch the value format once every node can read it - #413
Open
cosmin-staicu wants to merge 1 commit into
Open
cosmin-staicu wants to merge 1 commit into
cosmin-staicu wants to merge 1 commit into
Conversation
cosmin-staicu
requested review from
alinahornet,
florin-munteanu-uipath,
litheon,
lucianaparaschivei and
razvalex
as code owners
November 12, 2026 08:10
|
🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off. Strong signals
Other signals
This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.
|
cosmin-staicu
force-pushed
the
perf/binary-event-formatter
branch
from
November 13, 2026 13:39
edd0081 to
7aef2f6
Compare
cosmin-staicu
force-pushed
the
perf/format-negotiation
branch
from
November 13, 2026 13:39
252edb9 to
d5d8a19
Compare
cosmin-staicu
force-pushed
the
perf/binary-event-formatter
branch
2 times, most recently
from
November 14, 2026 07:03
dda7521 to
55b9067
Compare
cosmin-staicu
force-pushed
the
perf/format-negotiation
branch
from
November 14, 2026 07:13
d5d8a19 to
3c4c09c
Compare
A policy's non-JSON value format is now written only once every live node is known to read it. The switch happens in one deploy with no operator step. Every node with a non-JSON serializer registered advertises, every 15 seconds, the formats its reads decode, stamped with Redis TIME in a hash. A policy whose reads do not look for a header advertises json only, which blocks the switch. Reader-only and Force nodes advertise too, and a stopping node marks its advertisement as left. One decision is kept per registered serializer, refreshed every 15 seconds from the advertisements and the Streams consumer groups. The format is written when every live node advertises it and no node joined within UpgradeHold. A node that leaves or crashes keeps the format. A node that cannot read it, or a live group with no advertisement, sends writes back to JSON at once. An unreadable node list keeps the decision for MaxViewAge and then falls back to JSON. A node that cannot see its peers, as with broadcast off or Pub/Sub only, writes JSON unless AssumeAllNodesCanRead is set. Mode = Force writes the policy's format as before, and raw or custom proxy policies always write their own format. UpgradeHold defaults to 1 hour and 0 is allowed. A rollback within the hold never wrote the new format, and after the switch it makes old nodes miss and rewrite those keys as JSON, never read wrong values. FormatFor reads a cached decision with no Redis call. An app with no serializer registered writes no key, starts no timer and allocates nothing extra. New options live under RedisCacheOptions.FormatNegotiation: Mode, AdvertiseInterval, ViewInterval, LivenessWindow, UpgradeHold, MaxViewAge and AssumeAllNodesCanRead. New metrics are cache.format.effective, cache.format.fallback, cache.format.nodes and cache.format.view.age. Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
cosmin-staicu
force-pushed
the
perf/format-negotiation
branch
from
November 14, 2026 07:47
3c4c09c to
19fd942
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sixth in the fast value serializer stack, on top of #412. A policy's non-JSON value format is written only once every live node is known to read it. The switch happens in one deploy, with no operator step.
UpgradeHolddefaults to 1 hour, and 0 is allowed. A rollback within the hold never wrote the new format, so old nodes see no misses. After the switch, a rollback makes old nodes miss and rewrite those keys as JSON. They never read wrong values.An app with no serializer registered does nothing new. It writes no key, starts no timer and allocates nothing extra on the JSON path. The Pub/Sub node list and binary events follow in the next PR.
New options are under
RedisCacheOptions.FormatNegotiation:Mode,AdvertiseInterval,ViewInterval,LivenessWindow,UpgradeHold,MaxViewAgeandAssumeAllNodesCanRead. New metrics arecache.format.effective,cache.format.fallback,cache.format.nodesandcache.format.view.age.Evidence
The JSON default path allocates what the base (
edd0081) allocates.SerializerBenchmarkwith no registry, B:Two nodes against Redis:
Mutations caught:
FormatFor;Forceignored;Integration tests that expect a headered write through the container set
Mode = Force, sinceAutowrites JSON until the hold passes.Full suite with Redis: 5168, 0 failed, 1 skipped (a pre-Redis-7 case on Redis 7.4).
-c Release -warnaserrorbuilds clean.Merge Danger
Door: two-way
Apps with no serializer registered are unchanged. Apps that already write MessagePack now write JSON until every node advertises and the hold passes, or until they set
Mode = Force. All 3.2 nodes read both formats, so either way is safe.Blast Radius: storage
The value format of keys whose policy writes a non-JSON serializer, plus one advertisement hash per keyspace.