Cosmos: Update key-based live tests to fixed self-owned accounts. by aavasthy · Pull Request #5049 · Azure/azure-sdk-for-rust

Arooshi Avasthy (aavasthy) · 2026-08-11T21:33:18Z

PR Description

Our key-based Cosmos live tests currently deploy a brand-new Cosmos account into an ephemeral Microsoft test tenant on every CI run. That tenant is rotated frequently, and each rotation requires a new ADO service connection (which we can't create ourselves - it needs engineering systems). This makes the tests fragile and painful to keep alive.

Run the key-based live tests against fixed, self-owned Cosmos accounts in the subscription (sdk-ci resource group), with the endpoints and keys stored in a single Key Vault secret.

Test path	Behavior after this PR
Cosmos_live_test (6 key-based legs)	Uses 6 fixed accounts in `sdk-ci` via new `rust-ci` KV secret. Changed.
Cosmos_live_thinclient (GatewayV2, 2 legs)	Still ARM-deployed per run - moved to its own matrix file to keep it on the legacy path. Unchanged behavior.
Cosmos_live_test_aad	Untouched. Still ARM-deployed per run against the ephemeral tenant.
Emulator, benchmark, perf, release paths	Untouched.
Any Rust SDK source (`.rs`)	Not modified.

How it works

6 permanent Cosmos accounts live in the sdk-ci resource group. They
were created once (idempotent script under sdk/cosmos/pipeline/ account-provisioning/) and match the 6 previous per-run configurations
(Eventual/Session/Strong SingleWrite, Session MultiWrite, Session Split,
Session MR Failover).
Their endpoints and primary/secondary keys live in one Key Vault
secret (rust-ci, JSON blob keyed by logical account name), linked
into the existing Test Secrets for Cosmos Live Tests - user administered variable group.
Each Cosmos_live_test matrix leg carries an AccountSelector (e.g.
session-multiwrite). The new resolve-test-account-steps.yml pre-test
step runs resolve-cosmos-test-account.ps1, which picks the right slice
of the secret and exports the same env vars the ARM template used to
produce (AZURE_COSMOS_CONNECTION_STRING, ACCOUNT_HOST,
DATABASE_NAME, AZURE_COSMOS_DEFAULT_CONSISTENCY, COSMOS_RUSTFLAGS).
Because there is no ARM deploy, the leg also skips federated auth
entirely - it needs no tenant login.

Kevin Pilch (Pilchie) · 2026-08-11T21:41:59Z

Just reading the description, but isn't this the opposite of what we want? We need all key based auth to be in the ephemeral tenant, but can use MSI based EntraID auth in the normal tenant?

Copilot

Pull request overview

Moves key-based Cosmos live tests to permanent accounts while preserving ARM deployment for thin-client and AAD tests.

Changes:

Adds fixed-account provisioning, configuration, and secret resolution.
Splits test matrices by resource strategy.
Extends pipeline templates to support tests without ARM deployment.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`sdk/cosmos/pipeline/resolve-test-account-steps.yml`	Invokes account resolution.
`sdk/cosmos/pipeline/resolve-cosmos-test-account.tests.ps1`	Tests resolver validation.
`sdk/cosmos/pipeline/resolve-cosmos-test-account.ps1`	Exports selected account settings.
`sdk/cosmos/pipeline/README.md`	Documents fixed-account workflow.
`sdk/cosmos/pipeline/live-test-accounts.schema.json`	Defines secret schema.
`sdk/cosmos/pipeline/live-test-accounts.sample.json`	Provides sample account data.
`sdk/cosmos/pipeline/account-provisioning/README.md`	Documents provisioning and rotation.
`sdk/cosmos/pipeline/account-provisioning/New-CosmosLiveTestAccounts.ps1`	Provisions permanent accounts.
`sdk/cosmos/pipeline/account-provisioning/cosmos-live-test-accounts.definition.json`	Defines account topology.
`sdk/cosmos/live-thinclient-matrix.json`	Isolates ARM-backed thin-client tests.
`sdk/cosmos/live-platform-matrix.json`	Maps key-based legs to fixed accounts.
`sdk/cosmos/ci.yml`	Wires the new test paths.
`eng/pipelines/templates/stages/archetype-sdk-client.yml`	Generates fixed-account jobs.
`eng/pipelines/templates/jobs/live.tests.yml`	Supports skipping resource deployment.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:58Z

+                DefaultConsistencyLevel      = $acct.defaultConsistencyLevel
+                EnableAutomaticFailover      = $autoFailover
+                EnableMultipleWriteLocations = $multiWrite
+                ApiKind                      = 'GlobalDocumentDB'


Fixed — added both capabilities to the script to match the bicep. No live tests currently use vector or full-text search, so the already-provisioned accounts aren't blocking anything

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:55Z

+        }
+    }
+    else {
+        Write-Info "Cosmos account '$accountName' already exists (selector=$selector); leaving configuration as-is"


Intentional — reruns shouldn't silently mutate live accounts. Config changes go through a manual delete + rerun, per the runbook.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:01:32Z

+    FixedAccountMatrixConfigs:
      - Name: Cosmos_live_test
        Path: sdk/cosmos/live-platform-matrix.json
        Selection: sparse
        GenerateVMJobs: true


The scope of this PR is the move to fixed accounts; adding a sweeper is a separate concern with its own design questions (schedule, age threshold, safety against a concurrent run, which identity runs it).

I think it is not completely orthogonal - for example one step to take now might be to not just use UUID based names - but prefix them with some tiemstamp - so, that any async sweeper could delete let's say db/collections that were created >12 hours ago? Not necessarily blocking for this PR - but worth thinking about this because it will become a hot topic as soon as this PR is merged.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:48Z

+The script does **not** write to any ADO variable group itself - copy the
+JSON it prints (or the contents of `-OutputPath`) into the
+`rust-ci` secret variable in the
+`Test Secrets for Cosmos Live Tests - user administered` variable group (or
+whichever group `sdk/cosmos/pipeline/resolve-test-account-steps.yml` is
+pointed at). Treat the JSON as a secret at every step - it contains account
+keys.


Fixed — updated the runbook in sdk/cosmos/pipeline/account-provisioning/README.md to point rotation at the rust-ci Key Vault secret

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:45Z

+   accounts in the `sdk-ci` resource group and prints the JSON to store in
+   the ADO secret. See its README for the full runbook.
+4. That JSON is stored as the `rust-ci` secret
+   variable in the `Test Secrets for Cosmos Live Tests - user administered`
+   ADO variable group (already wired into `sdk/cosmos/ci.yml`).


Fixed — updated sdk/cosmos/pipeline/README.md to describe rust-ci as a Key Vault secret linked (read-through) into the variable group, and clarified that rotation happens on the KV secret itself since the variable-group value is not writable.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:42Z

@@ -0,0 +1,252 @@
+<#


Fixed — added the standard copyright/license header at the top of New-CosmosLiveTestAccounts.ps1 . Verified Get-Help still returns the synopsis, so the comment-based help block still parses correctly.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:36Z

+$endpoint = [string]$account.endpoint
+$key = [string]$account.key
+$secondaryKey = if ($account.PSObject.Properties.Name -contains 'secondaryKey') { [string]$account.secondaryKey } else { '' }
+$database = if ($account.PSObject.Properties.Name -contains 'database' -and $account.database) { [string]$account.database } else { 'shared-test-db' }
+$consistency = [string]$account.consistency
+$testCategory = [string]$account.testCategory


fixed. Added a Get-OptionalString helper and now read endpoint , key , secondaryKey , database , consistency , testCategory conditionally, so the targeted missing required '' messages surface instead of a StrictMode throw. Verified against payloads missing endpoint and consistency — both output the intended error.

Arooshi Avasthy (aavasthy) · 2026-08-11T23:13:29Z

+if (-not ($config.PSObject.Properties.Name -contains 'version') -or $config.version -ne 1) {
+    Fail "Unsupported or missing schema version '$($config.version)' (parser supports: 1)."
+}


fixed. Now reading version into a sentinel first, so the message formats cleanly instead of throwing under StrictMode. Verified with a payload missing version — outputs Unsupported or missing schema version '' as intended.

Arooshi Avasthy (aavasthy) · 2026-08-11T22:13:10Z

Just reading the description, but isn't this the opposite of what we want? We need all key based auth to be in the ephemeral tenant, but can use MSI based EntraID auth in the normal tenant?

Tests are still part of ephemeral tenant. In the existing process we would spin up a new account every time and run tests on it. With this new process, we have fixed accounts. We store the account + key in a secret vault and systems engineering team helped us create a new service connection for it just once. Now every time a tenant comes through we will just need to create new account and replace the account+key in vault but do not have to create a new service connection.

Kevin Pilch (Pilchie) · 2026-08-11T23:07:52Z

Ok, so the part of the description where it says:

No more ephemeral tenant,

is just wrong, and should be updated?
Yes removed it, we are using ephemeral tenant just removing the dependency of every time reaching out to eng sys for creating new service connection

Arooshi Avasthy (aavasthy) · 2026-08-11T23:26:20Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-11T23:26:45Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Arooshi Avasthy (aavasthy) · 2026-08-12T05:10:29Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-12T05:10:50Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Arooshi Avasthy (aavasthy) · 2026-08-12T19:17:42Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-12T19:18:08Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Heath Stewart (heaths) · 2026-08-12T19:18:31Z

Don't modify this file. Modify sdk/cosmos/.cspell.json. That's what it's there fore and doesn't require maintainer sign-off.

reverted the changes in sdk/cosmos/.cspell.json

Heath Stewart (heaths) · 2026-08-12T19:19:10Z

Daniel Jurek (@danieljurek) or Ben Broderick Phillips (@benbp) can you review this please? I think you helped them originally so I don't have as much context.

Fabian Meiswinkel (FabianMeiswinkel)

LGTM except few NITs and Heath's blocking comment

Arooshi Avasthy (aavasthy) · 2026-08-12T22:13:22Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-12T22:13:45Z

Azure Pipelines: Successfully started running 1 pipeline(s).

The build.rs comment referencing the emulator command-line switch `/enableaadauthentication` triggered a cspell 'Unknown word' failure on the Build Analyze step. Adding an inline `cspell:ignore` directive keeps the comment intact without touching the shared root cspell config, matching the existing pattern used in tests/emulator_tests/cosmos_aad.rs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Arooshi Avasthy (aavasthy) · 2026-08-12T23:22:59Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-12T23:23:23Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Arooshi Avasthy (aavasthy) · 2026-08-13T07:28:00Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-08-13T07:28:22Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Update key-based accounts to start using fixed test accounts.

f28292a

Arooshi Avasthy (aavasthy) self-assigned this Aug 11, 2026

Copilot AI balanced review requested due to automatic review settings August 11, 2026 21:33

Arooshi Avasthy (aavasthy) requested review from a team, Rick Winter (RickWinter), Daniel Jurek (danieljurek), Heath Stewart (heaths) and Jeff Fisher (xirzec) as code owners August 11, 2026 21:33

github-actions Bot added the Cosmos The azure_cosmos crate label Aug 11, 2026

Copilot started reviewing on behalf of Arooshi Avasthy (aavasthy) August 11, 2026 21:34 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

Arooshi Avasthy (aavasthy) and others added 2 commits August 11, 2026 15:27

Resolve merge conflicts

8e272f6

Merge branch 'main' into users/aavasthy/updatekeybasedtest

67b02b9

Arooshi Avasthy (aavasthy) and others added 3 commits August 11, 2026 16:13

Address review comments

8a9ab21

Merge branch 'main' into users/aavasthy/updatekeybasedtest

5c88035

Read me code cleanup

a088719

Arooshi Avasthy (aavasthy) and others added 2 commits August 11, 2026 21:56

Merge branch 'main' into users/aavasthy/updatekeybasedtest

83115ef

cspell update

5a1b948

Arooshi Avasthy (aavasthy) requested a review from Larry Osterman (LarryOsterman) as a code owner August 12, 2026 05:10

Continue to use same accounts for AAD tests

7b2a119

Heath Stewart (heaths) requested changes Aug 12, 2026

View reviewed changes

Fabian Meiswinkel (FabianMeiswinkel) approved these changes Aug 12, 2026

View reviewed changes

Arooshi Avasthy (aavasthy) added the Do Not Merge label Aug 12, 2026

Fix tests and revert cspell

13df811

Arooshi Avasthy (aavasthy) force-pushed the users/aavasthy/updatekeybasedtest branch from a5aea65 to 13df811 Compare August 12, 2026 22:12

Merge branch 'main' into users/aavasthy/updatekeybasedtest

0edf8b0

Fix emulator tests

f1de010

Arooshi Avasthy (aavasthy) force-pushed the users/aavasthy/updatekeybasedtest branch from 1e1ba4a to f1de010 Compare August 13, 2026 07:27

Merge branch 'main' into users/aavasthy/updatekeybasedtest

64c3117

Uh oh!

Conversation

Arooshi Avasthy (aavasthy) commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How it works

Uh oh!

Kevin Pilch (Pilchie) commented Aug 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 11, 2026

Uh oh!

Kevin Pilch (Pilchie) commented Aug 11, 2026 • edited by aavasthy Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 11, 2026

Uh oh!

azure-pipelines Bot commented Aug 11, 2026

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 12, 2026

Uh oh!

azure-pipelines Bot commented Aug 12, 2026

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 12, 2026

Uh oh!

azure-pipelines Bot commented Aug 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fabian Meiswinkel (FabianMeiswinkel) left a comment

Choose a reason for hiding this comment

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 12, 2026

Uh oh!

azure-pipelines Bot commented Aug 12, 2026

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 12, 2026

Uh oh!

azure-pipelines Bot commented Aug 12, 2026

Uh oh!

Arooshi Avasthy (aavasthy) commented Aug 13, 2026

Uh oh!

azure-pipelines Bot commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Arooshi Avasthy (aavasthy) commented Aug 11, 2026 •

edited

Loading

Kevin Pilch (Pilchie) commented Aug 11, 2026 •

edited by aavasthy

Loading