Cosmos: native driver distribution design + ADRs

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0000-index.md

@@ -0,0 +1,19 @@
+# Architecture Decision Records — native driver distribution
+
+ADRs (Architecture Decision Records) capture **what we decided** and a brief **why**, in a minimal, easy-to-reference form. They are **numbered and immutable**: once accepted, an ADR is not edited; a later ADR may **supersede** it. Detailed discussion and alternatives live in the design doc ([`../distribution-design.md`](../distribution-design.md)), not here — an ADR is the "regression test" that keeps us doing the same thing everywhere.
+
+**Format (template):** Context (2–4 sentences) · Decision (1–3 bullets) · Consequences (2–4 bullets) · Alternatives considered (1 line each) · Status.
+
+| # | Title | Status |
+|---|-------|--------|
+| [0001](0001-build-once-internal-handoff.md) | One build → internal-only hand-off artifact; no neutral consumer bundle | Accepted |
+| [0002](0002-per-language-feed-distribution.md) | Distribute as per-language packages on each language's existing feeds | Accepted |
+| [0003](0003-dotnet-nuget-nativeassets.md) | .NET consumes via per-RID NuGet NativeAssets + meta-package | Accepted |
+| [0004](0004-go-cgo-prebuilt.md) | Go consumes via cgo against a prebuilt header + lib from the Go feed | Accepted |
+| [0005](0005-abi-version-handshake.md) | Native lib exports an ABI version; hosts check it before use | Accepted |
+| [0006](0006-binding-owns-marshalling.md) | Each language binding owns marshalling and buffer copy-out | Accepted |
+| [0007](0007-native-is-opt-in.md) | Native transport is opt-in until GA, then default-with-fallback | Accepted |
+| [0008](0008-platform-matrix.md) | A defined platform matrix; unsupported platforms error clearly | Accepted |
+| [0009](0009-build-and-signing-pipeline.md) | One build, sign binaries once, fan-out; jobs never rebuild | Accepted |
+
+> These are **proposed** for the design review. "Accepted" is provisional until the review signs off.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0001-build-once-internal-handoff.md

@@ -0,0 +1,22 @@
+# ADR 0001 — One build → internal-only hand-off artifact; no neutral consumer bundle
+
+**Status:** Accepted (proposed for review)
+
+## Context
+The native driver must reach .NET, Go, and later Java, each with a different package format and its own feed. Two concerns are easily conflated: **provenance** (there must be exactly one build + binary-signing per release, or the languages drift onto different driver builds) and **distribution** (how a consumer pulls the bytes). A neutral consumer-facing bundle would force a Go user to download a package containing DLLs/JARs they cannot use, and would require standing up new consumer feed infrastructure.
+
+## Decision
+- For each release, **one** Rust build produces all platform binaries (cdylib + staticlib), the cbindgen C header, an `ABI_VERSION`, and checksums, and **signs the binaries**.
+- These are published as an **internal-only hand-off artifact** (e.g. an Azure Artifacts Universal Package or a pipeline artifact) that is the **single source of truth for provenance**.
+- This artifact is **not consumer-facing and not language-shaped.** Consumers never download it; per-language publish jobs consume it (distribution is ADR 0002; the pipeline that produces and fans it out is ADR 0009).
+- The Rust crate declares `crate-type = ["cdylib", "staticlib"]` so both link forms come from one build.
+
+## Consequences
+- Single provenance + SBOM anchor without forcing any cross-language or cross-platform download on consumers.
+- No new **consumer** feed to operate; only an internal hand-off artifact.
+- All per-language jobs must consume this artifact rather than rebuilding (enforced by ADR 0009).
+
+## Alternatives considered
+- Neutral consumer-facing bundle/feed — rejected: forces irrelevant bytes on consumers; new infra.
+- Per-language independent builds — rejected: drift risk, duplicated signing.
+

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0002-per-language-feed-distribution.md

@@ -0,0 +1,23 @@
+# ADR 0002 — Distribute as per-language packages on each language's existing feeds
+
+**Status:** Accepted (proposed for review)
+
+## Context
+Each language already has mature feed infrastructure with its own ACLs, signing (ESRP), and governance: .NET on nuget.org + an internal azure-sdk NuGet feed, Go on the azure-sdk-for-go feed, Java on the azure-sdk-for-java Maven feed + Maven Central. A neutral, cross-language consumer bundle would force consumers to download formats they cannot use (a Go user pulling DLLs/JARs) and would require new consumer feed infrastructure.
+
+## Decision
+- The native driver is distributed as **a normal dependency in each language's native package format on that language's existing internal + external feed** — not as a neutral cross-language bundle and not on any new consumer feed.
+  - .NET → NuGet NativeAssets + meta-package on nuget.org + internal NuGet feed (ADR 0003).
+  - Go → cgo-consumable header + lib via the azure-sdk-for-go feed (ADR 0004).
+  - Java → JAR on the azure-sdk-for-java Maven feed + Maven Central (future; not finalized).
+- A consumer only ever downloads its own language's package format.
+
+## Consequences
+- Reuses each SDK's mature feed, ACL, signing, and governance — **no new consumer feed to build**.
+- Idiomatic: matches how the Azure SDKs already ship dependencies.
+- Drift protection now depends on pipeline discipline (ADR 0009) rather than a single shared consumer artifact.
+
+## Alternatives considered
+- Single neutral consumer bundle/feed carrying all formats — rejected: forces irrelevant bytes on consumers; new infra; not idiomatic.
+
+> Pairs with ADR 0001: provenance is the internal hand-off (0001); distribution is per-language feeds (this ADR). The two are intentionally decoupled.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0003-dotnet-nuget-nativeassets.md

@@ -0,0 +1,21 @@
+# ADR 0003 — .NET consumes via per-RID NuGet NativeAssets + meta-package
+
+**Status:** Accepted (proposed for review)
+
+## Context
+.NET consumers expect NuGet. Native libraries are platform-specific, so a single package carrying every platform forces every consumer to download all of them. Mature native-backed SDKs (SkiaSharp, Microsoft.Data.SqlClient.SNI) solve this with a split package layout.
+
+## Decision
+- Ship per-RID **`Microsoft.Azure.Cosmos.NativeAssets.<rid>`** packages, each carrying one platform's dynamic lib under `runtimes/<rid>/native/`.
+- Front them with a thin **meta-package** whose `runtime.json` resolves the consumer's RID to the right per-RID package.
+- `Microsoft.Azure.Cosmos` takes an **opt-in** dependency on the meta-package (ADR 0007).
+- Publish to **nuget.org** and the **internal azure-sdk NuGet feed** (per ADR 0002).
+
+## Consequences
+- A consumer downloads only its platform's binary.
+- More package IDs to version and keep in lockstep (the meta `runtime.json` pins per-RID versions).
+- Reuses Azure's already-reserved `Microsoft.Azure.*` prefix — no new org-level construct.
+
+## Alternatives considered
+- Single "fat" NativeAssets package — kept only as an **interim** (Phase 1–2) for speed, not GA.
+- Embed natives directly in `Microsoft.Azure.Cosmos` — rejected: bloats the flagship for pure-managed users.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0004-go-cgo-prebuilt.md

@@ -0,0 +1,21 @@
+# ADR 0004 — Go consumes via cgo against a prebuilt header + lib from the Go feed
+
+**Status:** Accepted (proposed for review)
+
+## Context
+Go has no NuGet. Go links C libraries through cgo, which needs a C header and a library available at `go build` time. The Go SDK already implements the completion-queue receive loop, `cgo.Handle` correlation, and buffer copy-out; the only distribution question is how header+lib+ABI version reach the Go build.
+
+## Decision
+- Go consumes the prebuilt **`include/` header and `lib/` library via cgo** (`#cgo CFLAGS -I…` to parse the header into `C.*` symbols; `#cgo LDFLAGS -L… -lazurecosmosdriver` to link), **not NuGet**.
+- Prefer the **static `.a`** for a self-contained Go binary; dynamic linking is supported as an option.
+- The header + lib are delivered through the **azure-sdk-for-go feed** — an Azure Artifacts Universal Package fetched at build, or a vendored "binaries" Go module with per-OS build tags (delivery shape is open Q3). Either way it derives from the ADR 0001 hand-off artifact.
+
+## Consequences
+- Go reuses the exact same signed binaries as .NET — no Go-specific build of the driver.
+- cgo + static lib means `CGO_ENABLED=1` and a C toolchain on the Go build host; cross-compilation needs a cross C toolchain.
+- Everything resolves at `go build` — no runtime resolver / `runtime.json` / RID probing. The same `ABI_VERSION` feeds Go's handshake (ADR 0005).
+
+## Alternatives considered
+- Wrap the lib in NuGet for Go — rejected: Go can't consume NuGet.
+- A neutral consumer bundle Go downloads — rejected (ADR 0001/0002): pulls irrelevant formats.
+- Pure-Go reimplementation of the driver — rejected: defeats the shared-core goal.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0005-abi-version-handshake.md

@@ -0,0 +1,20 @@
+# ADR 0005 — Native lib exports an ABI version; hosts check it before use
+
+**Status:** Accepted (proposed for review)
+
+## Context
+The native library and each language SDK ship on independent cadences. A managed/native version mismatch at the byte level (the Application Binary Interface — ABI) corrupts memory or crashes deep in a call, with no clear cause.
+
+## Decision
+- The native library exports **`cosmos_abi_version() -> u32`**, and ships the same value in the hand-off's `ABI_VERSION`.
+- Every language host reads it **at load, before the first real call**, and fails fast with a versioned message when it is below the host's `MinSupported`.
+- A breaking C-ABI change is a **major bump**, coordinated with SDK releases; each SDK pins a compatible range.
+
+## Consequences
+- Mismatches surface as a clear, actionable error instead of a crash.
+- Adds a tiny, one-time check at load per process.
+- Creates an explicit compatibility contract across all three languages.
+
+## Alternatives considered
+- Rely on package version pins only — rejected: private deployment / manual overrides bypass the package graph.
+- No handshake — rejected: silent corruption is the worst failure mode here.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0006-binding-owns-marshalling.md

@@ -0,0 +1,20 @@
+# ADR 0006 — Each language binding owns marshalling and buffer copy-out
+
+**Status:** Accepted (proposed for review)
+
+## Context
+The driver core is schema-agnostic: the C-ABI passes request/response bodies as raw bytes (`const uint8_t*` + length), never parsed JSON ([`NATIVE_WRAPPER_SPEC.md`](../../NATIVE_WRAPPER_SPEC.md)). Pushing serialization into the wrapper would re-introduce the parse/re-serialize/re-parse waste the old `azure_data_cosmos_native` crate had.
+
+## Decision
+- The ABI stays **bytes-in / bytes-out**; the wrapper does no JSON parsing.
+- Each language binding **owns its own marshalling** (string encoding, structs) and **copies response buffers out of native memory** into host memory, then frees the native buffer.
+- Rust owns the buffer until the host copies it out; ownership transfer is explicit per the ABI spec.
+
+## Consequences
+- The native binaries are identical for every language; differences live in each binding.
+- No double-encoding cost across the FFI boundary.
+- Each language must implement copy-out correctly (lifetime + free) — a per-binding responsibility.
+
+## Alternatives considered
+- Serialize to JSON strings in the wrapper — rejected: redundant parsing, hides driver concepts.
+- Zero-copy borrow of native buffers into host memory — rejected for now: lifetime hazards across GC'd hosts; copy-out is the safe default.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0007-native-is-opt-in.md

@@ -0,0 +1,20 @@
+# ADR 0007 — Native transport is opt-in until GA, then default-with-fallback
+
+**Status:** Accepted (proposed for review)
+
+## Context
+`Microsoft.Azure.Cosmos` (pure managed) and the Go SDK run today on any platform with no native dependency. Introducing a platform-specific native library must not silently break that portability or force a migration on a version bump.
+
+## Decision
+- Native transport is **opt-in** per SDK (a client option / build flag / env switch) until it is GA on every supported platform.
+- Only after GA may a given SDK make native the **default**, and only with a **fallback** to the managed/pure path on an unsupported platform or a load failure.
+- No consumer is ever force-migrated to native by a routine version bump.
+
+## Consequences
+- The "runs anywhere" guarantee is preserved throughout the rollout.
+- The native path can mature behind a flag with real dogfooding before it is the default.
+- Each SDK needs a clean fallback path, not just an error, once native is the default.
+
+## Alternatives considered
+- Native-on by default from day one — rejected: breaks portability before the platform matrix is proven.
+- Separate "native-enabled" package flavor — kept as a possible packaging option, not mandated here.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0008-platform-matrix.md

@@ -0,0 +1,20 @@
+# ADR 0008 — A defined platform matrix; unsupported platforms error clearly
+
+**Status:** Accepted (proposed for review)
+
+## Context
+A native library must be built per platform (OS + architecture + libc). The support surface must be bounded and explicit so build, signing, and testing are tractable, and so consumers get a clear answer on an unsupported platform.
+
+## Decision
+- The GA matrix is: `win-x64`, `win-arm64`, `linux-x64` (glibc, low floor), `linux-musl-x64`, `linux-arm64`, `osx-x64`, `osx-arm64`.
+- `linux-musl-x64` is a **separate target** from `linux-x64` (a glibc build will not load on musl).
+- `wasm` is out of scope (no FFI story). An unsupported platform **fails with an actionable error** naming the supported set, never a silent or cryptic failure.
+
+## Consequences
+- Bounded, predictable build/sign/test surface.
+- Clear consumer experience on unsupported platforms.
+- Adding a platform later is an additive ADR + matrix row, not a redesign.
+
+## Alternatives considered
+- Build only the most common platforms and let others fail at link/load — rejected: poor experience, no clear message.
+- Include `win-x86` / mobile now — deferred (open Q): no demand yet.

sdk/cosmos/azure_data_cosmos_driver/docs/native-distribution/adr/0009-build-and-signing-pipeline.md

@@ -0,0 +1,20 @@
+# ADR 0009 — One build, sign binaries once, fan-out; jobs never rebuild
+
+**Status:** Accepted (proposed for review)
+
+## Context
+The binary is built in `azure-sdk-for-rust`, but the language packages live in other repos and publish to other feeds. The bytes that define the ABI must be signed at their source, and per-language packaging must not be able to alter, re-sign, or independently rebuild them — otherwise the languages drift onto different driver builds (the failure ADR 0001 exists to prevent).
+
+## Decision
+- A single pipeline next to the Rust build produces the per-platform binaries (ADR 0001), **signs each binary once** (Authenticode on Windows; codesign + notarization on macOS), checksums them, and publishes the internal hand-off artifact.
+- Per-language publish jobs **consume the already-signed hand-off** and emit NuGet / Go-consumable / JAR packages; they **never rebuild or re-sign the native binary** (they may sign their own package wrapper, e.g. the `.nupkg`/`.jar`, in that language's existing ESRP flow).
+- Build-once is enforced by discipline: all language jobs consume one hand-off from one Rust build.
+
+## Consequences
+- The ABI-defining bytes are signed once, at the source, identically for all languages.
+- Per-language jobs are simple repackagers; they can live in or near each SDK repo.
+- Requires the hand-off artifact (ADR 0001) and an SBOM / component-governance owner (open Q7).
+
+## Alternatives considered
+- Each language rebuilds + signs its own copy — rejected: drift and duplicated trust roots.
+- Sign only the language packages, not the native binary — rejected: leaves the actual loaded bytes unsigned.