Repository navigation
Don't delete blobs of other instances when purging an instance whose id is their prefix - #1438
Open
Benjamin Cederholm (bcederholm) wants to merge 2 commits into
Open
Benjamin Cederholm (bcederholm) wants to merge 2 commits into
Benjamin Cederholm (bcederholm) wants to merge 2 commits into
Conversation
…id is their prefix
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Author
|
@microsoft-github-policy-service agree |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Purging can delete pending-message blobs belonging to slash-containing instance IDs and leave history blobs behind for IDs ending in a caret.
2 open issues
What changed in this PR
This PR addresses #1437 by narrowing blob matching during Azure Storage instance purges, protecting large payloads belonging to instances with similar IDs.
Changes:
- Adds a trailing
/to the blob-list prefix used during purge. - Adds a regression test verifying that purging one instance preserves an
_otherinstance’s blobs and readable history.
| File | Description |
|---|---|
test/DurableTask.AzureStorage.Tests/AzureStorageScenarioTests.cs |
Adds regression coverage for purging instances with overlapping ID prefixes. |
src/DurableTask.AzureStorage/MessageManager.cs |
Includes the separator in the blob prefix used for deletion. |
Review effort: Balanced
…ested ids when purging
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Fixes #1437.
Problem
MessageManager.DeleteLargeMessageBlobslisted blobs with the bare instance id as prefix. Every large-message and history blob of an instance is named{id}/..., so purging instanceabcalso deleted the blobs ofabc_other,abc:12,abc/otheror any other instance whose id starts withabc. That instance's history rows still reference the deleted blobs, so its next replay fails withBlobNotFoundon every attempt, and it can't be terminated either.Instance ids that
KeySanitation.EscapePartitionKeychanges had a second problem. Message blobs are named after the raw id and history blobs after the escaped id, but purge only listed the raw id. So purgingabc/nestedleft its history blobs (abc^0nested/history-...) behind.Change
DeleteLargeMessageBlobslists{id}/for message blobs and{escaped id}/for history blobs (a single listing when both are the same, which is the common case)./) and of the expected kind (message-orhistory-). Blobs of a nested id such asabc/other/message-..., and history blobs of an id whose escaped form equals another raw id, are no longer deleted.message-andhistory-name parts are now constants used by both the naming code and the purge.PurgeSingleInstance_DoesNotDeleteBlobsOfInstanceWithPrefixedId(_other,:12,/other): purging{guid}leaves all message and history blobs of{guid}{suffix}, and that instance's history still loads.PurgeSingleInstance_WithEscapedInstanceId_DeletesAllBlobs(^,/nested): purging an escaped id deletes both its message and history blobs.Testing
dotnet testonDurableTask.AzureStorage.Tests(net8.0, Azurite), filtered to the purge scenario tests: 18/18 pass. The new tests againstmain's purge code:_other,:12,/otherand/nestedfail. net48 builds; its tests were not run.