Skip to content

Don't delete blobs of other instances when purging an instance whose id is their prefix - #1438

Open
Benjamin Cederholm (bcederholm) wants to merge 2 commits into
Azure:mainfrom
bcederholm:fix/purge-blob-prefix-boundary
Open

Benjamin Cederholm (bcederholm) wants to merge 2 commits into
Azure:mainfrom
bcederholm:fix/purge-blob-prefix-boundary

Conversation

@bcederholm

@bcederholm Benjamin Cederholm (bcederholm) commented Nov 16, 2026 •

Copy link
Copy Markdown

Fixes #1437.

Problem

MessageManager.DeleteLargeMessageBlobs listed blobs with the bare instance id as prefix. Every large-message and history blob of an instance is named {id}/..., so purging instance abc also deleted the blobs of abc_other, abc:12, abc/other or any other instance whose id starts with abc. That instance's history rows still reference the deleted blobs, so its next replay fails with BlobNotFound on every attempt, and it can't be terminated either.

Instance ids that KeySanitation.EscapePartitionKey changes had a second problem. Message blobs are named after the raw id and history blobs after the escaped id, but purge only listed the raw id. So purging abc/nested left its history blobs (abc^0nested/history-...) behind.

Change

  • DeleteLargeMessageBlobs lists {id}/ for message blobs and {escaped id}/ for history blobs (a single listing when both are the same, which is the common case).
  • It deletes only blobs directly under that prefix (no further /) and of the expected kind (message- or history-). Blobs of a nested id such as abc/other/message-..., and history blobs of an id whose escaped form equals another raw id, are no longer deleted.
  • The message- and history- name parts are now constants used by both the naming code and the purge.
  • Tests:
    • PurgeSingleInstance_DoesNotDeleteBlobsOfInstanceWithPrefixedId (_other, :12, /other): purging {guid} leaves all message and history blobs of {guid}{suffix}, and that instance's history still loads.
    • PurgeSingleInstance_WithEscapedInstanceId_DeletesAllBlobs (^, /nested): purging an escaped id deletes both its message and history blobs.

Testing

dotnet test on DurableTask.AzureStorage.Tests (net8.0, Azurite), filtered to the purge scenario tests: 18/18 pass. The new tests against main's purge code: _other, :12, /other and /nested fail. net48 builds; its tests were not run.

Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:28
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@bcederholm

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Purging can delete pending-message blobs belonging to slash-containing instance IDs and leave history blobs behind for IDs ending in a caret.

2 open issues
What changed in this PR

This PR addresses #1437 by narrowing blob matching during Azure Storage instance purges, protecting large payloads belonging to instances with similar IDs.

Changes:

  • Adds a trailing / to the blob-list prefix used during purge.
  • Adds a regression test verifying that purging one instance preserves an _other instance’s blobs and readable history.
File Description
test/​DurableTask.AzureStorage.Tests/​AzureStorageScenarioTests.cs Adds regression coverage for purging instances with overlapping ID prefixes.
src/​DurableTask.AzureStorage/​MessageManager.cs Includes the separator in the blob prefix used for deletion.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the instance-scoped Azure Storage purge changes.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Purging an instance deletes large-message blobs of other instances whose id starts with the purged id

2 participants