Skip to content

Fix Durable client current-app detection - #3528

Open
wangbill (YunchuWang) wants to merge 10 commits into
devfrom
yunchuwang-fix-durable-client-current-app-detection
Open

wangbill (YunchuWang) wants to merge 10 commits into
devfrom
yunchuwang-fix-durable-client-current-app-detection

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Aug 20, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes #3523.
Fixes #1926.

This compatibility correction builds on merged PR #3514 (477a92915cbeb46423fe55ceb36c036b845b00b0). It keeps the single shared DurableClient.ClientReferencesCurrentApp gate introduced there, but compares a target client with the host's actual default durability provider instead of comparing the target provider with itself.

Before / after

Scenario Before After
Same exact TaskHub, different connection, missing orchestrator Rejected by local metadata; target provider not called Local validation skipped; target provider receives the start
Same exact TaskHub, different connection, missing entity Rejected by local metadata; message not sent Local validation skipped; entity signal is sent
Same exact TaskHub, different connection, locally startup-disabled orchestrator start/restart Rejected locally Local validation skipped; target backend receives start/restart
Azure Storage case-only TaskHub difference Classified as remote Classified as current app because Azure Storage hub identity is case-insensitive
Azure Managed / exact-case provider case-only TaskHub difference Classified as remote Remains classified as remote
ExternalClient = true with a matching internal cache entry Can reuse the internal client and incorrectly apply local validation External/internal clients have distinct cache identities; the explicit external opt-out is preserved
Unknown registry key / disabled null entry / active or deregistered listener #3514 behavior Unchanged; only current-app classification is corrected

Design rationale

  • ClientReferencesCurrentApp remains the only predicate used by missing-orchestrator, missing-entity, and startup-disabled-orchestrator validation.
  • Connection identity is evaluated by virtual dispatch on DurableTaskExtension.DefaultDurabilityProvider, passing the target provider. This preserves provider-specific connection matching direction.
  • TaskHub identity is an internal durability-provider semantic. The base implementation uses ordinal matching (including Azure Managed's shipped exact-case semantics), while Azure Storage overrides it with ordinal-ignore-case matching to reflect its lowercased storage resources.
  • An immutable client-cache key snapshots TaskHub, ConnectionName, ExternalClient, and DurableRequiresGrpc. Host and factory caches therefore preserve behavior-relevant client options instead of aliasing external and internal clients.
  • No provider-name string switches, public API additions, entity dispatch changes, or HTTP/gRPC error-mapping changes are introduced.

Compatibility

The intentional behavior changes are limited to client-cache identity and local metadata validation at corrected current/remote boundaries. Cross-connection operations now reach their configured backend; Azure Storage case variants now receive current-app validation. External and internal modes no longer share a cached client, preventing creation order from changing validation behavior. Exact-case provider semantics, the explicit external-client opt-out, replay, and deregistered listener drain behavior remain unchanged.

Tests

TDD RED evidence on net8.0 before production changes:

  • DurableClientBaseTests: 8 expected failures covering cross-connection missing/disabled orchestrator start, restart, entity signal, and Azure Storage case-only matching.
  • HTTP cross-connection disabled start: expected Accepted, observed BadRequest.
  • gRPC cross-connection disabled start: observed the expected pre-fix InvalidArgument local rejection.

GREEN results:

  • Focused DurableClientBaseTests: 50/50 passed on net8.0; 50/50 passed on net10.0.
  • Focused HTTP current/remote/missing/disabled tests: 4/4 passed on each TFM.
  • Focused gRPC current/remote/unknown/disabled/restart tests: 5/5 passed on each TFM.
  • Complete relevant classes (DurableClientBaseTests, DisabledFunctionDispatchTests, HttpApiHandlerTests, LocalGrpcListenerTests): 187/187 passed on net8.0; 187/187 passed on net10.0.
  • git -c core.whitespace=cr-at-eol diff --check: passed.

Automated Azure Storage E2E

CurrentAppDetectionTests runs the real FunctionsV2 Durable extension and Azure Storage provider against two isolated Azurite processes:

  • Same storage connection plus a case-only TaskHub variant uses a non-external in-host Durable client and proves a startup-disabled orchestrator is rejected and remains absent across repeated status reads.
  • Different storage connections plus the same TaskHub uses a source host with no target orchestration and a target ITestHost on a second Azurite backend. The source-host client dispatches to the target, the orchestration completes there, and repeated source-backend reads remain absent.
  • The public Azure Pipelines FunctionsV2E2E job starts independent Azurite data stores on ports 10000-10002 and 11000-11002 and probes all six service ports.

SecondaryStorage is strictly a second localhost Azurite process (127.0.0.1). Its account value is Microsoft’s documented, well-known public devstoreaccount1 emulator credential. It is not a live Azure Storage credential and cannot authenticate to an Azure Storage account.

Regression proof:

  • Exact base 334c7dd9aff4e6196421f2ae6e80651ee27450a7: both new tests fail behaviorally. The case-only client accepts and persists the disabled instance as Pending; the cross-account client is rejected by source metadata because TargetOnlyOrchestrator is not locally registered.
  • Current merged-dev branch: 2/2 passed on net8.0 and 2/2 passed on net10.0.
  • Complete relevant unit classes remain 187/187 passed on each TFM.

Hosted external-client cache regression (#1926)

Commit b31f039 adds AzureStorage_HostedClientCache_PreservesExternalClientMode to the existing FunctionsV2 E2E suite. It uses real indexed internal/external client bindings, a client-only source host, and a target-only orchestrator.

  • Internal-first: the later external binding must start the remote-only orchestrator successfully rather than inherit internal local-name validation.
  • External-first: the later internal binding must still reject an unknown local orchestrator rather than inherit the external opt-out.
  • Both orders observe the external orchestration completing on the target host; this is not only a cache-object identity assertion.

Behavioral RED with the hosted cache fix removed: both orders fail, respectively with the original No orchestrator functions are currently registered! exception and with an internal start incorrectly accepted. GREEN with the existing PR cache fix: the new theory passed 2/2 on net8.0 and 2/2 on net10.0, including a fresh publication-worktree run. The containing E2E class passed 4/4 on each TFM, and nearby external-client controls passed 3/3 on each TFM during implementation validation.

This commit adds only regression tests and fixture functions; no duplicate production fix is introduced. The broader #1926 investigation also exercised shared HTTP/gRPC behavior, but this automated regression specifically covers hosted client bindings and does not claim full isolated-worker-process E2E coverage.

Supplemental real Azure validation

A manual A/B validation deployed the exact base and production-fix builds to isolated Windows Consumption Function Apps backed by two real Azure Storage accounts. It reproduced the base behavior and verified the PR behavior for direct Durable clients and built-in Durable HTTP routes, including start, restart, cross-account entity signaling, account-specific persistence, runtime logs, and App Insights traces. This supplemental run covered Azure Storage only; it did not claim live gRPC or Durable Task Scheduler coverage. All temporary Azure resources were deleted after preserving non-secret evidence.

Principal-level reviews found no remaining critical or important findings after adding the second Azurite setup to every FunctionsV2 E2E CI surface.

Supersedes #3527, which was opened from a fork and could not receive the upstream repository secrets required by all CI workflows.

Compare durable client targets with the host default provider and let providers define TaskHub identity casing. Cover cross-connection routing and shared HTTP, gRPC, orchestrator, restart, and entity validation behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 20, 2026 18:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR corrects Durable client “current app” detection so local metadata validation (missing/disabled orchestrator/entity checks) only runs when the target client actually points at the host’s default backend, and so TaskHub identity respects provider-specific casing semantics (Azure Storage case-insensitive vs exact-case providers).

Changes:

  • Fix DurableClient.ClientReferencesCurrentApp to compare against DurableTaskExtension.DefaultDurabilityProvider (connection + TaskHub), not the target provider against itself.
  • Introduce provider-overridable TaskHub identity matching (DurabilityProvider.TaskHubNameMatches), with Azure Storage overriding to ordinal-ignore-case.
  • Add/adjust unit tests for cross-connection behavior and TaskHub case-variant behavior across direct, HTTP, and gRPC paths.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/FunctionsV2/Tests/Unit/LocalGrpcListenerTests.cs Adds gRPC coverage for “different connection” disabled-orchestrator start reaching the target provider and refines TaskHub casing scenarios.
test/FunctionsV2/Tests/Unit/HttpApiHandlerTests.cs Updates HTTP tests for different-connection behavior and adds Azure Storage case-variant TaskHub local rejection coverage for disabled orchestrators.
test/FunctionsV2/Tests/Unit/DurableClientBaseTests.cs Updates direct-client tests to validate corrected current-app gating across connections and provider-specific TaskHub casing semantics.
src/WebJobs.Extensions.DurableTask/DurabilityProvider.cs Adds TaskHubNameMatches as the provider-specific TaskHub identity predicate (base: ordinal).
src/WebJobs.Extensions.DurableTask/ContextImplementations/DurableClient.cs Uses host default durability provider for connection + TaskHub matching and gates current-app detection on config != null.
src/WebJobs.Extensions.DurableTask/AzureStorageDurabilityProvider.cs Overrides TaskHubNameMatches to use ordinal-ignore-case for Azure Storage TaskHub identity.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@YunchuWang
wangbill (YunchuWang) requested a balanced review from Copilot August 20, 2026 19:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 21, 2026 00:04
@YunchuWang
wangbill (YunchuWang) force-pushed the yunchuwang-fix-durable-client-current-app-detection branch from e9cd0a5 to 917a5e6 Compare August 21, 2026 00:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Exercise provider-aware TaskHub casing and cross-account routing with real Functions hosts against two isolated Azurite backends.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Mirror the GitHub FunctionsV2 E2E setup in the merged Azure Pipelines job while leaving analyzer CI unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Copilot AI review requested due to automatic review settings August 21, 2026 01:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Resolve the CI migration conflict by retaining dev's deleted GitHub workflow and preserving secondary Azurite setup in the replacement public FunctionsV2 E2E job.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Await task hub deletion before deterministically disposing the locally-created orchestration service.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Copilot AI review requested due to automatic review settings August 21, 2026 19:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Preserve exact-case task hub identity when explicit entity operations select a target client.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Copilot AI review requested due to automatic review settings August 21, 2026 20:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Use an internal ordinal cache key for provider-relevant configuration while restoring the public attribute equality/hash contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8dd495b5-0788-4ec5-bfdc-5dd9728bb509
Copilot AI review requested due to automatic review settings August 21, 2026 20:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Add hosted regression coverage for both internal/external client creation orders (#1926).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f73040c-3459-4728-8c53-8e481e672736
Copilot AI review requested due to automatic review settings September 11, 2026 19:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The explicit-client fast path can still alias case-distinct connections for exact-case providers.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 15/15 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment on lines +107 to 108
if (this.DurabilityProvider.TaskHubNameMatches(this.TaskHubName, taskHubName)
&& string.Equals(this.attribute.ConnectionName, connectionName, StringComparison.OrdinalIgnoreCase))
Copilot AI review requested due to automatic review settings September 11, 2026 20:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Explicit operations can still reuse the wrong client when connection names differ only by casing.

Review details

Suppressed comments (1)

src/WebJobs.Extensions.DurableTask/ContextImplementations/DurableClient.cs:107

  • The new provider-aware hub check can still return the wrong client when only the requested connection's casing differs. ConnectionNameMatches defines the base provider identity with case-sensitive equality (DurabilityProvider.cs:614-617), and the new cache key likewise preserves connection casing, but this fast path still treats Storage and storage as identical. Explicit-hub entity signals (and other callers of GetDurableClient) can therefore reuse the current provider instead of resolving the requested backend. Use an ordinal connection comparison here so a case-variant request reaches the provider factory and is classified consistently.
            if (this.DurabilityProvider.TaskHubNameMatches(this.TaskHubName, taskHubName)
  • Files reviewed: 15/15 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants