V2 Orchestrate: Ask an agent steps always fail with "This step's agent or action isn't available to you right now" #1699

Issue

In the V2 chat, any Orchestrate plan with an Ask an agent (agent_invoke) step fails before the agent runs. The step reports "This step's agent or action isn't available to you right now, so it didn't run. Check that it still exists, is enabled and is shared with you, then retry." The answer step then reports "A required dependency did not complete." It affects personal, group and global agents in both personal and shared conversations. The same agent works when Orchestrate is off.

This persists after 0.261.289 (the selected-agent-with-agents-turned-off fix). That fix removed an earlier refusal, so the step now reaches a later check, which fails for every agent.

Steps to Reproduce

1. In V2 chat, turn on Orchestrate and pick a personal agent, for example one with Microsoft 365 email actions.
2. Ask something only that agent can answer, such as "What are my latest emails?"
3. The plan's Ask an agent step fails with the message above.
4. Turn Orchestrate off, pick the same agent and ask again. The agent answers.

Expected Behavior

The Ask an agent step runs the picked agent, as plain chat does, after the provider rechecks the user's current access to that exact agent.

Actual Behavior

The executor logs [ORCHESTRATION_EXECUTOR] A dependency-bound step could not complete. with sc_capability_id=agent_invoke, sc_error_type=OrchestrationInvocationDeniedError, sc_failure_code=integration_unavailable and sc_authority_reason=result_external_source_unavailable. Before 0.261.289 the same symptom logged result_external_capability_unavailable.

Impact

Orchestrate can't use agents at all. The exact-type check has been in place since 0.261.127; until 0.261.270 and 0.261.289 removed earlier refusals, those refusals were hit first. Action steps are unaffected. There's no workaround except turning Orchestrate off.

Notes

Root cause

OrchestrationExternalSourceProvider._resolve_integration (application/single_app/functions_orchestration_external_sources.py) requires the agent returned by resolve_delegation_agent to be exactly a dict:

if type(resolved) is not dict or agent_reference(resolved, identity.user_id) != expected:
    raise ResultUnavailableError("result_external_source_unavailable")

resolve_delegation_agent returns _canonical_agent(record), a deepcopy of the Cosmos read_item result (functions_agent_delegation.py). With the pinned azure-cosmos==4.9.0, point reads return CosmosDict, a dict subclass, and deepcopy keeps the subclass, so the exact-type check always fails. The action branch of the same method uses isinstance and isn't affected. This is the same bug class fixed in ORCHESTRATION_COSMOS_RESPONSE_COMPATIBILITY_FIX.md (0.261.140) and ORCHESTRATION_SETTINGS_DOCUMENT_TYPE_FIX.md (0.261.237). The repo's existing remedies are app_settings_store._plain_document and functions_orchestration_bootstrap._document_response.

Evidence

• Telemetry for two failed runs, one shared and one personal conversation, shows the same Azure SDK call sequence from the executor:

  1. Identity, conversation and run reads.
  2. The agent catalog queries; seeded narrowing succeeded and logged no selected_agent_unavailable warning.
  3. One point read of the picked agent's document, which returned 200.
  4. The denial, logged about 50-70 ms later.

  That leaves only the checks after the read in resolve_delegation_agent and the provider's type check. Scope settings, governance and "not found" are excluded.

• Reproduced with the existing ExternalSourceWorld fixture, the unchanged provider and the unchanged resolve_delegation_agent. Only the fake container's read_item return type changes: plain dict → preflight passes; azure-cosmos 4.9.0 CosmosDict → preflight denied with result_external_source_unavailable after exactly one agent point read.

• Tests missed it because functional_tests/test_support/agent_delegation.py (DelegationServices.reader) returns deepcopy(record), a plain dict.

Plan

1. _canonical_agent returns a plain dict (deepcopy(dict(record))), fixing all 8 resolve_delegation_agent callers. _etag is a document field and is kept.
2. _resolve_integration accepts dict subclasses (isinstance) and keeps the exact reference comparison. This covers preflight, result admission and later reads.
3. DelegationServices.reader (the shared test fake) returns a dict subclass that stands in for CosmosDict, so exact-type checks on SDK results fail in tests. Fix or justify any new failures; never revert the fake to make a test pass.
4. Log which check refused a step: a new [ORCHESTRATION_EXTERNAL_SOURCES] refusal event with sc_stage, sc_authority_reason and an identifier-free sc_reason, for example integration_reference_mismatch, integration_not_found, integration_access_denied or selection_not_in_catalog. It correlates with the executor's failure event by run and step hash.
5. Regression test functional_tests/test_orchestration_agent_document_type_fix.py: the reported case passes; wrong owner, disabled, missing, governance-denied and mismatched agents are still denied with the right check; action steps are unchanged.
6. Fix doc ORCHESTRATION_AGENT_DOCUMENT_TYPE_FIX.md, a follow-up note on ORCHESTRATION_SELECTED_AGENT_DISABLED_PREFERENCE_FIX.md, an admin troubleshooting row, logging-tags updates and version 0.261.291.

After deployment, the next expected gate for Microsoft 365 agents is the existing Microsoft 365 sign-in, approval or read-only step check, which has its own message.

Related: #1661 (agents in Orchestrate; its expected behavior also needs this fix), #1660 (Microsoft 365 action steps), and the 0.261.289 selected-agent fix (#1696).