[larry-dalmeida]

Summary

1. Fix Dispatch/coalescing & follower bodies: Singleflight normalization was missing; coalesced requests didn't properly share bodies, forcing multiple origin fetches for identical concurrent requests.

2. Fix Vary/304 handling: Variant isolation failed to prevent same-Vary entries from shadowing fresh variants; 304 responses discarded entity representation instead of preserving it.

3. Fix Stale-if-error & refresh retention: SIE didn't properly retain last-good responses when backend returned 500/502/503/504 or disconnected; failed refreshes were lost.

4. Fix Backend phase timeouts: Root Options timeout fields weren't wired into the cache filter's transport; configured budgets were ignored.

5. Fix Expired L1 shadowing fresh L2: L1 freshness gate checked !IsStale(), which was true for entries in the stale-while-revalidate window; this bypassed fresh shared L2 and forced unnecessary origin calls.

How to verify

Review the five commits in order:

Commit	Purpose	Verification
78a9bf85	Normalise origin dispatch and preserve follower bodies.	Dispatch/body regression tests; coalescing requires one origin call for the synchronised burst.
d10650af	Isolate Vary variants and preserve 304 representations.	Vary/304 regression tests; the Vary-star oracle requires three origin calls.
b308eefa	Retain last-good responses across failed refreshes.	SIE covers 500/502/503/504, disconnect and expiry; queue saturation yields three drops and 257 successful conditional refreshes.
bdd9dc6b	Honour backend phase timeouts for cache-origin fetches.	Root cache_timeout_test.go checks option mapping, header-timeout stale preservation and recovery. These are phase budgets, not one end-to-end deadline.
cb8ac7e0	Consult shared L2 once L1 freshness expires.	Two-process reproduction: normal and cache-only probes return fresh B/HIT, with zero extra origin calls. Workload evictions match absent cache-only probes.

Want to run it yourself?

Setup and complete replay commands

After the linked harness/prerequisite setup, run from the harness root ($PROM is the native Prometheus binary configured there):

SHA=cb8ac7e00d29bf8c578e5578dc57c462af6689ac
git clone --filter=blob:none --no-checkout https://github.com/zalando/skipper.git .runtime/skipper-lower
git -C .runtime/skipper-lower fetch origin "$SHA"
git -C .runtime/skipper-lower checkout --detach "$SHA"
python3 verify_l1_freshness.py --checkout .runtime/skipper-lower --sha "$SHA" --output .runtime/lower-l1
python3 verify_candidate_behaviour.py --l1-result .runtime/lower-l1/result.json --output .runtime/lower-coalescing
python3 verify_cache_remaining.py --checkout .runtime/skipper-lower --sha "$SHA" --mode correctness --prometheus-binary "$PROM" --output .runtime/lower-remaining

Screenshots

These panels illustrate the pressure/queue workload. Queries, hashes and capture provenance.

Cache evictions: 196 actual evictions matched 196 absent cache-only probes in the workload.

Refresh errors / dropped jobs: the workload independently observed three queue-overflow drops.

Refresh queue depth: reached the configured capacity of 256, then drained.

References & Additional Context

• Supersedes cache: centralize dispatch and preserve response bodies #4370, cache: preserve representations across refresh and failure #4371 and cache: honor backend phase timeouts in origin fetches #4372.

[zalando-robot]

Docker image "registry-write.opensource.zalan.do/teapot/skipper:f17089f2ef5bab0e8ce46d2dfd23ed6049c817ec" is not based on an approved base image. Any production deployment relying on this image will be blocked.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Zalando Container Registry namespace library and use a recommended version as listed in the documentation.

[zalando-robot]

Docker image "registry-write.opensource.zalan.do/teapot/skipper:cb8ac7e00d29bf8c578e5578dc57c462af6689ac" is not based on an approved base image. Any production deployment relying on this image will be blocked.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Zalando Container Registry namespace library and use a recommended version as listed in the documentation.

[zalando-robot]

Docker image "registry-write.opensource.zalan.do/teapot/skipper:7f776e41eeb1e21b8ad0b275548473e127aca041" is not based on an approved base image. Any production deployment relying on this image will be blocked.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Zalando Container Registry namespace library and use a recommended version as listed in the documentation.

[zalando-robot]

Docker image "registry-write.opensource.zalan.do/teapot/skipper:2d787a53fbb78502dade261fa75fc8cbeee9591c" is not based on an approved base image. Any production deployment relying on this image will be blocked.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Zalando Container Registry namespace library and use a recommended version as listed in the documentation.