Read either cache record format in the redis backend - #631
Merged
Conversation
The memory backend stores an entry as a packed blob, the redis backend has its own binary record for the same thing, and the two have drifted: the version byte 0 against 1, a timestamp in nanoseconds against seconds, an expiry stored in one and not the other, and the prefetch flag defined twice on the same bit. Unifying them is a two step change, and this is the first step: the reader. A redis cache is shared between instances, which is the reason the backend exists, so a record in a format an instance does not recognise is a miss and an error log line on every lookup that finds it. Shipping the reader a release before the writer means that by the time anything emits the shared layout, the instances sharing a database can already read it, and a rollback lands on a build that can too. Nothing writes version 2 yet; the dispatch is here so that a later release can. That fixes the version constants now rather than later, since the reader has to recognise exactly what the writer will produce. blobVersion goes to 2, past the 1 the redis record already uses, and is written into the blob rather than left to the zero value an allocation happens to carry. 0 and 1 are both taken, and a record that either backend may read cannot be ambiguous about which layout it is in. Writing blobVersion makes the raw cache file's records version 2, so the file version goes to 2 with them. A version 1 file holds the same layout under blobVersion 0 and is now refused by the header check rather than losing every record one at a time to the per-record check, which reports a cold start instead of silently coming up empty. Those two constants have to move together and nothing in the types can say so, so a test pins both. It is the only thing that catches them drifting: a round trip writes and reads with the same constant and passes either way. That format is unreleased, no tag contains #614, so this costs nothing outside a dev build.
folbricht
force-pushed
the
cache-format-unification
branch
from
September 5, 2026 09:09
8dba57f to
57faad2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The memory backend stores a cache entry as a packed blob. The Redis backend has its own binary record for the same thing. Unifying them is a two step change and this is the first: the reader. The writer follows in #632, which is stacked on this branch.
Two encodings of one concept had drifted apart:
blobMetaPrefetchEligibleflagPrefetchBit, same bitWhy the reader ships alone
A Redis cache is shared between instances, which is the reason the backend exists, so a record in a format an instance does not recognise is a cache miss and an error log line on every lookup that finds it, once per hit for as long as the mixed-version window lasts.
Shipping the reader a release ahead of the writer closes that window. By the time anything emits the shared layout, the instances sharing a database can already read it, and a rollback lands on a build that can read it too. The cost is one extra release. The alternative, flipping both at once, is correct but noisy in exactly the deployment this backend exists to serve.
Nothing writes version 2 here.
decodeRecorddispatches on the version byte and version 2 is currently reached only from tests, deliberately.Version constants
The reader has to recognise exactly what the writer will later produce, so the constants are fixed here rather than in the second PR.
blobVersiongoes to 2, past the 1 the Redis record already uses, and is now written into the blob rather than left to the zero value a fresh allocation happens to carry. Both 0 and 1 are taken, and a record that either backend may read cannot be ambiguous about which layout it is in.Raw cache file
Writing
blobVersionmakes the raw file's records version 2, sorawCacheVersiongoes to 2 with them. A version 1 file holds the same layout underblobVersion0, and is now refused by the header check rather than losing every record one at a time to the per-record check, so it reports a cold start instead of silently coming up empty.Those two constants have to move together and nothing in the types can say so, so a test pins both values. It is the only thing that catches them drifting apart: a round trip writes and reads with the same constant, so it passes either way, and the damage only shows on a file written earlier. Confirmed by bumping
blobVersionalone and checking that this is the sole test that fails.That format is unreleased, no tag contains #614, so this costs nothing outside a dev build.
Tests
A version 2 record read back through the dispatch, over both prefetch values, which is the path the writer will exercise once it lands; a version 1 record still readable, which is what this backend writes today; the dispatch refusing an unknown version and an empty record; a version 1 raw file refused at the header, which had no coverage; and the constant pinning described above.