Skip to content

Read either cache record format in the redis backend - #631

Merged
folbricht merged 1 commit into
masterfrom
cache-format-unification
Sep 5, 2026
Merged

Read either cache record format in the redis backend#631
folbricht merged 1 commit into
masterfrom
cache-format-unification

Conversation

@folbricht

@folbricht folbricht commented Sep 5, 2026

Copy link
Copy Markdown
Owner

The memory backend stores a cache entry as a packed blob. The Redis backend has its own binary record for the same thing. Unifying them is a two step change and this is the first: the reader. The writer follows in #632, which is stacked on this branch.

Two encodings of one concept had drifted apart:

blob Redis record
version byte 0 1
timestamp unix nanoseconds unix seconds
expiry stored not stored, Redis TTL only
prefetch flag blobMetaPrefetchEligible flagPrefetchBit, same bit

Why the reader ships alone

A Redis cache is shared between instances, which is the reason the backend exists, so a record in a format an instance does not recognise is a cache miss and an error log line on every lookup that finds it, once per hit for as long as the mixed-version window lasts.

Shipping the reader a release ahead of the writer closes that window. By the time anything emits the shared layout, the instances sharing a database can already read it, and a rollback lands on a build that can read it too. The cost is one extra release. The alternative, flipping both at once, is correct but noisy in exactly the deployment this backend exists to serve.

Nothing writes version 2 here. decodeRecord dispatches on the version byte and version 2 is currently reached only from tests, deliberately.

Version constants

The reader has to recognise exactly what the writer will later produce, so the constants are fixed here rather than in the second PR.

blobVersion goes to 2, past the 1 the Redis record already uses, and is now written into the blob rather than left to the zero value a fresh allocation happens to carry. Both 0 and 1 are taken, and a record that either backend may read cannot be ambiguous about which layout it is in.

Raw cache file

Writing blobVersion makes the raw file's records version 2, so rawCacheVersion goes to 2 with them. A version 1 file holds the same layout under blobVersion 0, and is now refused by the header check rather than losing every record one at a time to the per-record check, so it reports a cold start instead of silently coming up empty.

Those two constants have to move together and nothing in the types can say so, so a test pins both values. It is the only thing that catches them drifting apart: a round trip writes and reads with the same constant, so it passes either way, and the damage only shows on a file written earlier. Confirmed by bumping blobVersion alone and checking that this is the sole test that fails.

That format is unreleased, no tag contains #614, so this costs nothing outside a dev build.

Tests

A version 2 record read back through the dispatch, over both prefetch values, which is the path the writer will exercise once it lands; a version 1 record still readable, which is what this backend writes today; the dispatch refusing an unknown version and an empty record; a version 1 raw file refused at the header, which had no coverage; and the constant pinning described above.

The memory backend stores an entry as a packed blob, the redis backend
has its own binary record for the same thing, and the two have drifted:
the version byte 0 against 1, a timestamp in nanoseconds against seconds,
an expiry stored in one and not the other, and the prefetch flag defined
twice on the same bit. Unifying them is a two step change, and this is
the first step: the reader.

A redis cache is shared between instances, which is the reason the
backend exists, so a record in a format an instance does not recognise is
a miss and an error log line on every lookup that finds it. Shipping the
reader a release before the writer means that by the time anything emits
the shared layout, the instances sharing a database can already read it,
and a rollback lands on a build that can too. Nothing writes version 2
yet; the dispatch is here so that a later release can.

That fixes the version constants now rather than later, since the reader
has to recognise exactly what the writer will produce. blobVersion goes
to 2, past the 1 the redis record already uses, and is written into the
blob rather than left to the zero value an allocation happens to carry.
0 and 1 are both taken, and a record that either backend may read cannot
be ambiguous about which layout it is in.

Writing blobVersion makes the raw cache file's records version 2, so the
file version goes to 2 with them. A version 1 file holds the same layout
under blobVersion 0 and is now refused by the header check rather than
losing every record one at a time to the per-record check, which reports
a cold start instead of silently coming up empty. Those two constants
have to move together and nothing in the types can say so, so a test pins
both. It is the only thing that catches them drifting: a round trip
writes and reads with the same constant and passes either way.

That format is unreleased, no tag contains #614, so this costs nothing
outside a dev build.
@folbricht
folbricht force-pushed the cache-format-unification branch from 8dba57f to 57faad2 Compare September 5, 2026 09:09
@folbricht folbricht changed the title Share one record layout between the cache backends Read either cache record format in the redis backend Sep 5, 2026
@folbricht
folbricht merged commit 011ef6b into master Sep 5, 2026
16 checks passed
@folbricht
folbricht deleted the cache-format-unification branch September 5, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant