fix(memory): hold read lock while scanning sessions in SearchMemory#1139
Open
winklemad wants to merge 1 commit into
Open
fix(memory): hold read lock while scanning sessions in SearchMemory#1139winklemad wants to merge 1 commit into
winklemad wants to merge 1 commit into
Conversation
InMemoryService.SearchMemory read the per-user session map under the read lock but released it before iterating, while AddSessionToMemory writes into that same map under the write lock. Concurrent Search and Add on the same app/user therefore race and can panic with "concurrent map iteration and map write", even though the service is documented as thread-safe. Hold the read lock across the whole scan. AddSessionToMemory replaces session slices wholesale rather than mutating them in place, so a read lock over the iteration is sufficient and still lets searches run concurrently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
InMemoryService.SearchMemoryhas a data race: it reads the per-user session map under the read lock, but releases the lock before iterating.AddSessionToMemorywrites into that same map under the write lock, so a concurrentSearchandAddon the same app/user race — and can panic withfatal error: concurrent map iteration and map write. The service is documented as thread-safe.No linked issue — found while reviewing the in-memory services under the race detector.
Root cause
valuesis the shared per-usermap[sessionID][]value; iterating it after releasing the lock races withAddSessionToMemory, which doesv[sid] = valuesunder the write lock.Fix
Hold the read lock across the whole scan.
AddSessionToMemoryreplaces session slices wholesale rather than mutating them in place, so a read lock over the iteration is sufficient and still lets searches run concurrently. This mirrors the fix already applied to the session package'sstate.All()(#561).Tests
memory: addedTest_inMemoryService_SearchMemory_Concurrent, which runsSearchMemoryandAddSessionToMemoryon the same app/user in parallel. Under-race(as CI runs it) it reports aDATA RACEatinmemory.goonmainand passes with this change.Verified locally with the CI command
go test -race -mod=readonly -count=1 -shuffle=on ./...(all packages pass), plusgo vet,gofmt, andgolangci-lint runclean.