KAFKA-20499: Add isolation-level reads to in-memory stores#22681
Open
nicktelford wants to merge 1 commit into
Open
KAFKA-20499: Add isolation-level reads to in-memory stores#22681nicktelford wants to merge 1 commit into
nicktelford wants to merge 1 commit into
Conversation
The in-memory key-value, window, and session stores buffer uncommitted writes in a transaction buffer when transactional state stores are enabled. Their reads consulted that buffer unconditionally, so an interactive query had no way to ask for only committed data — a READ_COMMITTED query would observe writes still staged in the current transaction. Override readOnly(IsolationLevel) on each of the three in-memory stores to return a view that consults the transaction buffer under READ_UNCOMMITTED and bypasses it under READ_COMMITTED (reading the base map directly). For the window and session stores this extracts the buffer dispatch in each read path into private helpers that take the buffer as a parameter, collapsing the forward/backward pairs into shared helpers, so the view can opt in or out without duplicating the segment-map iteration logic. The key-value view synchronises on the store's monitor so concurrent IQ reads observe a consistent snapshot of the base map relative to processor-thread writes. Add semantic tests across the three store families asserting that READ_UNCOMMITTED observes staged writes (and deletes) while READ_COMMITTED sees only the last committed snapshot, covering single-key reads, prefix scans, findSessions, range fetch, and the all-iterators. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the KIP-892 interactive-query isolation-level series. With transactional state stores enabled, the in-memory key-value, window, and session stores buffer uncommitted writes in a transaction buffer, and their reads consulted that buffer unconditionally. An interactive query therefore had no way to request only committed data — a
READ_COMMITTEDquery would observe writes still staged in the open transaction.This overrides
readOnly(IsolationLevel)on each of the three in-memory stores to return a view that consults the transaction buffer underREAD_UNCOMMITTEDand bypasses it underREAD_COMMITTED, reading the base map directly. For the window and session stores the buffer dispatch in each read path is extracted into private helpers parameterised by the buffer (collapsing the forward/backward pairs into shared helpers), so the view can opt in or out without duplicating the iteration logic. The key-value view synchronises on the store's monitor so concurrent IQ reads observe a consistent snapshot relative to processor-thread writes.Semantic tests across the three store families assert that
READ_UNCOMMITTEDobserves staged writes (and deletes) whileREAD_COMMITTEDsees only the last committed snapshot.This branched off the now-merged in-memory transactional-store and IQ-isolation-framework work and now applies directly to trunk.
🤖 Generated with Claude Code