Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

### Added

- **A seam for value serializers.** `ICacheValueSerializer` and `CacheValueFormats` define a buffer-based serializer whose output starts with a one-byte format id, `AddValueSerializer` registers one by name, and `CachePolicy.Serializer` (`Write`, `Read`, `MigrateOnRead`) names it per policy. `AddValueSerializer` throws at the call for a name used twice on one builder or for `json`. A policy that names an unregistered or blank serializer, has an empty `Read`, or writes a format its `Read` leaves out fails with an `OptionsValidationException`, as does a serializer whose ids are outside the consumer ranges; these are checked when the policy factory is first resolved, on first cache use rather than at host start. A custom `ICachePolicyFactory` whose policy names an unregistered serializer fails the same way when the Redis provider builds its cache. Serializers the container created are disposed with it, through `DisposeAsync` when they only implement `IAsyncDisposable`, so a factory must return a new instance rather than one resolved from DI. The default stays JSON with no header, and `AddDistributedCache` ignores `Serializer`: its own policy always writes and reads JSON with no migration, so the caller's bytes never get a format header.
- **A seam for value serializers.** `ICacheValueSerializer` and `CacheValueFormats` define a buffer-based serializer whose output starts with a one-byte format id, `AddValueSerializer` registers one by name, and `CachePolicy.Serializer` (`Write`, `Read`, `MigrateOnRead`) names it per policy. `AddValueSerializer` throws at the call for a name used twice on one builder or for `json`. A policy that names an unregistered or blank serializer, has an empty `Read`, or writes a format its `Read` leaves out fails with an `OptionsValidationException`, as does a serializer whose ids are outside the consumer ranges; these are checked when the policy factory is first resolved, on first cache use rather than at host start. A custom `ICachePolicyFactory` whose policy names an unregistered serializer fails the same way when the Redis provider builds its cache. Serializers the container created are disposed with it, through `DisposeAsync` when they only implement `IAsyncDisposable`, so a factory must return a new instance rather than one resolved from DI. A policy that sets `Write` makes `RedisCache` and `RedisHashCache` write their values, hash fields and hash metadata with that serializer, and every read picks the serializer by the format id, whatever the policy writes. A value in a format no registered serializer owns is a miss, counted in `cache.serializer.unreadable` and removed only if the key still holds the bytes that were read, so a newer value written in between survives; a hash with such a field misses whole and is left in place. A value whose registered serializer throws is a logged miss and the key stays, like one that is not headered and fails to parse as JSON, because the bytes may be good for another version of the type during a rolling deploy; this departs from the design spec, which removed it, and an `OutOfMemoryException` from its payload is never swallowed, while one from a value with no header (JSON) is a logged miss as it is with no serializer registered. `RedisHashCache.TryDecodeFields` now treats any exception except `OutOfMemoryException` as a refresh, since a registered serializer can throw any type, where it caught four. A node with no serializer registered logs the `JsonException` and keeps the key. A policy passed to a single call that names an unregistered serializer throws `OptionsValidationException` before anything is written. The reader only looks for a header with the default JSON serializer or under a policy whose effective `Write` is a serializer other than `json` (`Read` does not count), so `byte[]` values stored through `RawByteSerializerProxy` or `AddDistributedCache` are never reinterpreted. A policy that sets no serializer, or only `Read`, takes the default policy's `Write`. The default stays JSON with no header, and `AddDistributedCache` ignores `Serializer`: its own policy always writes and reads JSON with no migration, so the caller's bytes never get a format header. A hash write that names a serializer leaves `_metadata_` out when the metadata is empty and nulls are not cached, and a `HashReplace` one deletes the field an earlier write left. Hash system field names are compared as bytes instead of strings. See [value serializers](docs/how-to/extending.md#value-serializers) and [rolling a serializer out](docs/concepts.md#rolling-a-serializer-out).

### Changed

Expand Down
60 changes: 60 additions & 0 deletions benchmarks/UiPath.Caching.Benchmarks/SerializerBenchmark.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using Microsoft.Extensions.Logging.Abstractions;
using StackExchange.Redis;
using UiPath.Caching.Config;
using UiPath.Caching.Policies;
using UiPath.Caching.Redis;
using UiPath.Caching.Serialization;
using UiPath.Caching.Telemetry;

namespace UiPath.Caching.Benchmarks;

Expand All @@ -11,12 +17,23 @@ namespace UiPath.Caching.Benchmarks;
[CategoriesColumn]
public class SerializerBenchmark
{
private static readonly TimeSpan Expiration = TimeSpan.FromMinutes(30);

private readonly Func<CancellationToken, Task<CustomObject?>> _neverRuns = _ => throw new InvalidOperationException("A hit ran its generator");

private CustomObject _obj = default!;
private RedisValue _payload;
private FakeRedisConnector _redis = default!;
private RedisCache _cache = default!;
private CachePolicy? _policy;

[Params("Small", "Medium", "Large")]
public string Size { get; set; } = "Medium";

/// <summary>Whether a consumer serializer is registered, with a policy that names <c>json</c> so that every read looks for a format header; off is the app that never registered one.</summary>
[Params(false, true)]
public bool Registered { get; set; }

[GlobalSetup]
public void Setup()
{
Expand All @@ -27,8 +44,33 @@ public void Setup()
_ => CustomObject.RandomMedium(),
};
_payload = JsonSerializer.SerializeToUtf8Bytes(_obj);

var registry = Registered ? new ValueSerializerRegistry([("bench", new UnusedSerializer(), false)]) : ValueSerializerRegistry.Empty;
_policy = Registered ? new CachePolicy { Serializer = new SerializerPolicy { Write = "json" } } : null;
_redis = new FakeRedisConnector();
_cache = new RedisCache(
_redis,
new SystemJsonByteSerializerProxy(),
EmptyResiliencePipelineProvider.Instance,
NullTelemetryProvider.Instance,
new RedisCacheOptions(),
new CacheOptions { AppShortName = "bench" },
new DefaultCachePolicyFactory(null, null, valueSerializers: registry),
TimeProvider.System,
NullLogger<RedisCache>.Instance)
{
ValueSerializers = registry,
};
_cache.SetAsync("bench:serializer", _obj, Expiration, _policy).AsTask().GetAwaiter().GetResult();
if (_cache.GetAsync<CustomObject>("bench:serializer", _policy).AsTask().GetAwaiter().GetResult() is null)
{
throw new InvalidOperationException("The fake connector did not serve the read");
}
}

[GlobalCleanup]
public void Cleanup() => _redis.Dispose();

[BenchmarkCategory("Serialize"), Benchmark(Baseline = true)]
public RedisValue Serialize_String() => JsonSerializer.Serialize(_obj);

Expand All @@ -48,4 +90,22 @@ public void Setup()
var reader = new Utf8JsonReader(payload);
return JsonSerializer.Deserialize<CustomObject>(ref reader);
}

[BenchmarkCategory("RedisCacheGet"), Benchmark]
public ValueTask<CustomObject?> RedisCache_Get() => _cache.GetAsync<CustomObject>("bench:serializer", _policy);

[BenchmarkCategory("RedisCacheGetOrAdd"), Benchmark]
public ValueTask<CustomObject?> RedisCache_GetOrAdd_Hit() => _cache.GetOrAddAsync<CustomObject>("bench:serializer", _neverRuns, Expiration, _policy);

[BenchmarkCategory("RedisCacheSet"), Benchmark]
public ValueTask<bool> RedisCache_Set() => _cache.SetAsync("bench:serializer", _obj, Expiration, _policy);

private sealed class UnusedSerializer : ICacheValueSerializer
{
public IReadOnlyList<int> FormatIds { get; } = [0x10];

public int Serialize<T>(T value, IBufferWriter<byte> target) => throw new NotSupportedException();

public T? Deserialize<T>(int formatId, ReadOnlySequence<byte> source) => throw new NotSupportedException();
}
}
87 changes: 87 additions & 0 deletions benchmarks/UiPath.Caching.Benchmarks/ValueReadBenchmark.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
using System.Buffers;
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using StackExchange.Redis;
using UiPath.Caching.Serialization;

namespace UiPath.Caching.Benchmarks;

/// <summary>A single-key read of a value that carries a format header: the bytes StackExchange.Redis allocates, or a leased buffer it hands back.</summary>
[MemoryDiagnoser]
[MarkdownExporterAttribute.GitHub]
public class ValueReadBenchmark
{
private const int FormatId = 0x10;

private readonly string _run = Guid.NewGuid().ToString("N");
private readonly ValueSerializerRegistry _registry = new([("bench", new JsonBody(), false)]);

private ConnectionMultiplexer _redis = default!;
private IDatabase _database = default!;
private ValueReader _reader = default!;
private RedisKey _key;

[Params("Small", "Medium", "Large")]
public string Size { get; set; } = "Medium";

[GlobalSetup]
public async Task Setup()
{
_redis = await ConnectionMultiplexer.ConnectAsync("127.0.0.1:6379,abortConnect=false");
_database = _redis.GetDatabase();
_reader = new ValueReader(new SystemJsonByteSerializerProxy(), _registry);
_key = $"bench:valueread:{_run}:{Size}";
var value = Size switch
{
"Small" => CustomObject.RandomSmall(),
"Large" => CustomObject.RandomLarge(),
_ => CustomObject.RandomMedium(),
};
var body = JsonSerializer.SerializeToUtf8Bytes(value);
var stored = new byte[body.Length + 1];
stored[0] = FormatId;
body.CopyTo(stored, 1);
await _database.StringSetAsync(_key, stored, TimeSpan.FromMinutes(30));
}

[GlobalCleanup]
public async Task Cleanup()
{
await _database.KeyDeleteAsync(_key);
await _redis.CloseAsync();
_redis.Dispose();
}

[Benchmark(Baseline = true)]
public async Task<CustomObject?> Bytes()
{
var value = await _database.StringGetAsync(_key);
return _reader.Read<CustomObject>(((byte[]?)value)!).Value;
}

[Benchmark]
public async Task<CustomObject?> Lease()
{
using var lease = await _database.StringGetLeaseAsync(_key);
var memory = lease!.Memory;
if (!ValueHeader.TryRead(memory.Span, out var formatId, out var headerLength) || !_registry.TryGetByFormat(formatId, out var serializer))
{
throw new InvalidOperationException("unexpected value");
}

return serializer.Deserialize<CustomObject>(formatId, new ReadOnlySequence<byte>(memory[headerLength..]));
}

private sealed class JsonBody : ICacheValueSerializer
{
public IReadOnlyList<int> FormatIds { get; } = [FormatId];

public int Serialize<T>(T value, IBufferWriter<byte> target) => throw new NotSupportedException();

public T? Deserialize<T>(int formatId, ReadOnlySequence<byte> source)
{
var reader = new Utf8JsonReader(source);
return JsonSerializer.Deserialize<T>(ref reader);
}
}
}
52 changes: 52 additions & 0 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,58 @@ See also: [how-to/resilience.md#named-policies](how-to/resilience.md#named-polic

---

## Value serializers and the format header

Values are JSON unless a policy says otherwise; a call with no policy takes the default policy's choice. A
policy that sets `Serializer.Write` to a registered serializer writes its values in that serializer's format, behind a one-byte *format id*; JSON has no header,
and no JSON text starts with a byte a header can be. The reader looks at the first byte: a format id picks
the serializer registered for it, anything else is JSON. It does this whatever the reading policy writes, so
a key written as JSON, then MessagePack, then JSON again reads correctly each time. Hash fields and a hash's
`_metadata_` field follow the same rule; `_version_` stays a plain number and is never inspected.

The reader looks only where it can tell a header from caller bytes, and only when a serializer is registered.
It does so when the cache's serializer is the default `SystemJsonByteSerializerProxy` itself, or when the
call's effective policy writes a serializer other than JSON: its own `Serializer.Write`, else the default
policy's, compared without regard to case. `Write = "json"` does not open it, and neither does what
`Serializer.Read` lists. A subclass, a custom `ISerializerProxy<byte[]>` and `RawByteSerializerProxy` (which
`AddDistributedCache` uses) keep their bytes as they are otherwise, so a stored `byte[]` that starts with
`0x02` reads back unchanged, also under a policy that writes JSON or lists other formats to read.

A hash write that names a serializer leaves out `_metadata_` when the metadata is empty and nulls are not
cached; the entry then reads with no metadata rather than an empty dictionary, and a `HashReplace` write also
deletes the field an earlier write left, in the same transaction as the fields (a replacement too large for one command is
written in chunks and deletes it right after). The JSON default keeps writing it. Events stay JSON for every policy, and a peer that applies another node's field update decodes the field
bytes by their header like any other read, with the gate its own policy for that key sets. Where serializers are registered and
the key's policy is not known to the peer, it does not apply the update and reads the hash instead.

### Rolling a serializer out

Reading comes first, writing second. Every node from 3.2.0 reads every registered format; nothing writes a
new one until a policy sets `Serializer`, so set it only after every node runs 3.2.0 or later with the
serializer registered.

A node that cannot read a format treats the value as a miss, and what happens to the key depends on why:

- A node before 3.2.0, or a 3.2.0 node with no serializer registered, hits a `JsonException` (logged at
Warning) and keeps the key; the generator rewrites it as JSON. In a batch read one `JsonException` misses
every key of the batch, and a hash misses whole, its `_metadata_` included.
- A 3.2.0 node with a serializer registered, reading an id none of its serializers owns, removes the key and
counts it in `cache.serializer.unreadable`, but only if the key still holds the bytes it read. A newer value
another node wrote in between, or a lagging replica's stale read, survives the removal. A hash with such a
field misses whole and is left in place, still counted: removing it could drop what another node just
wrote, and removing the one field would leave the rest looking like a complete hash.
- A registered serializer that throws on a value it owns is a logged miss, and the key stays, as for JSON that
does not parse. The bytes may be good for another version of the type that other nodes still run during a
rolling deploy, and a serializer disposed at shutdown says nothing about the data. This departs from the
design spec, which removed such a value. An `OutOfMemoryException` from a registered serializer's payload is never treated as a failed read: it propagates from `Get`, `GetOrAdd` and the hash reads instead of becoming a miss that runs the generator. A value with no header (JSON) behaves as it does with no serializer registered, so an `OutOfMemoryException` there is a logged miss.

All of it shows up as factory load and churn during the deploy, never as a wrong value. Switching a policy back
to JSON is safe at any time: new writes are JSON and the older values still read.

See also: [how-to/extending.md#value-serializers](how-to/extending.md#value-serializers).

---

## Telemetry

`ICachingTelemetryProvider` is the instrumentation seam. The library never takes
Expand Down
Loading
Loading