Cosmos: native driver distribution design + ADRs #4651
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Architecture Decision Records — native driver distribution | ||
|
|
||
| ADRs (Architecture Decision Records) capture **what we decided** and a brief **why**, in a minimal, easy-to-reference form. They are **numbered and immutable**: once accepted, an ADR is not edited; a later ADR may **supersede** it. Detailed discussion and alternatives live in the design doc ([`../distribution-design.md`](../distribution-design.md)), not here — an ADR is the "regression test" that keeps us doing the same thing everywhere. | ||
|
|
||
| **Format (template):** Status (immediately under the title) · Context (2–4 sentences) · Decision (1–3 bullets) · Consequences (2–4 bullets) · Alternatives considered (1 line each). | ||
|
|
||
| | # | Title | Status | | ||
| |---|-------|--------| | ||
| | [0001](0001-build-once-internal-handoff.md) | One build → internal-only hand-off artifact; no neutral consumer bundle | Accepted | | ||
| | [0002](0002-per-language-feed-distribution.md) | Distribute as per-language packages on each language's existing feeds | Accepted | | ||
| | [0003](0003-dotnet-nuget-nativeassets.md) | .NET consumes via per-RID NuGet NativeAssets + meta-package | Accepted | | ||
| | [0004](0004-go-cgo-prebuilt.md) | Go consumes via cgo against a prebuilt header + lib from the Go feed | Proposed (delivery shape WIP) | | ||
| | [0005](0005-abi-version-handshake.md) | Native lib exports an ABI version; hosts check it before use | Accepted | | ||
| | [0006](0006-binding-owns-marshalling.md) | Each language binding owns marshalling and buffer copy-out | Accepted | | ||
| | [0007](0007-native-is-opt-in.md) | Native transport ships in a new SDK major; no parallel managed transport | Accepted | | ||
| | [0008](0008-platform-matrix.md) | A defined platform matrix; unsupported platforms error clearly | Accepted | | ||
| | [0009](0009-build-and-signing-pipeline.md) | One build, sign binaries once, fan-out; jobs never rebuild | Accepted | | ||
| | [0010](0010-native-version-fanout.md) | One native version fanned out to all feeds simultaneously; each SDK pins that exact version | Proposed | | ||
|
|
||
| > These are **proposed** for the design review. "Accepted" is provisional until the review signs off. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # ADR 0001 — One build → internal-only hand-off artifact; no neutral consumer bundle | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| The native driver must reach .NET, Go, and later Java, each with a different package format and its own feed. Two concerns are easily conflated: **provenance** (there must be exactly one build + binary-signing per release, or the languages drift onto different driver builds) and **distribution** (how a consumer pulls the bytes). A neutral consumer-facing bundle would force a Go user to download a package containing DLLs/JARs they cannot use, and would require standing up new consumer feed infrastructure. | ||
|
|
||
| ## Decision | ||
| - For each release, **one coordinated Rust release build/pipeline** produces all platform binaries (cdylib + staticlib), the cbindgen C header, an `ABI_VERSION`, and checksums, and **signs the binaries**. This may fan out across platform-specific jobs, but they join back into one hand-off governed as a single provenance event. | ||
| - These are published as an **internal-only hand-off artifact** (e.g. an Azure Artifacts Universal Package or a pipeline artifact) that is the **single source of truth for provenance**. | ||
| - This artifact is **not consumer-facing and not language-shaped.** Consumers never download it; per-language publish jobs consume it (distribution is ADR 0002; the pipeline that produces and fans it out is ADR 0009). | ||
| - The hand-off is **RID-keyed** (one subtree per `<os>-<arch>`/libc target) and carries **both link forms per RID** — the dynamic `.dll`/`.so`/`.dylib` *and* the static `.a` — plus a single **C-only header** (no C++ constructs). This layout is a hard requirement so any language, including a future Java JAR (ADR 0002), can repackage straight from the hand-off with **no redesign and no rebuild**. | ||
|
|
||
| - The Rust crate declares `crate-type = ["cdylib", "staticlib"]` so both link forms come from the same coordinated release build. | ||
|
|
||
| ## Consequences | ||
| - Single provenance + SBOM anchor without forcing any cross-language or cross-platform download on consumers. | ||
| - No new **consumer** feed to operate; only an internal hand-off artifact. | ||
| - All per-language jobs must consume this artifact rather than rebuilding (enforced by ADR 0009). | ||
|
|
||
| ## Alternatives considered | ||
| - Neutral consumer-facing bundle/feed — rejected: forces irrelevant bytes on consumers; new infra. | ||
| - Per-language independent builds — rejected: drift risk, duplicated signing. | ||
|
|
||
| > Pairs with ADR 0002: provenance is this internal hand-off (0001); distribution is per-language feeds (0002). The two are intentionally decoupled. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # ADR 0002 — Distribute as per-language packages on each language's existing feeds | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| Each language already has mature feed infrastructure with its own ACLs, signing (ESRP), and governance: .NET on nuget.org + an internal azure-sdk NuGet feed, Go on the azure-sdk-for-go feed, Java on the azure-sdk-for-java Maven feed + Maven Central. A neutral, cross-language consumer bundle would force consumers to download formats they cannot use (a Go user pulling DLLs/JARs) and would require new consumer feed infrastructure. | ||
|
|
||
| ## Decision | ||
| - The native driver is distributed as **a normal dependency in each language's native package format on that language's existing internal + external feed** — not as a neutral cross-language bundle and not on any new consumer feed. | ||
| - .NET → NuGet NativeAssets + meta-package on nuget.org + internal NuGet feed (ADR 0003). | ||
| - Go → cgo-consumable header + lib via the azure-sdk-for-go feed (ADR 0004). | ||
| - Java → JAR on the azure-sdk-for-java Maven feed + Maven Central (future; not finalized). | ||
| - A consumer only ever downloads its own language's package format. | ||
|
|
||
| ## Consequences | ||
| - Reuses each SDK's mature feed, ACL, signing, and governance — **no new consumer feed to build**. | ||
| - Idiomatic: matches how the Azure SDKs already ship dependencies. | ||
| - Drift protection now depends on the verification-backed fan-out pipeline (ADR 0009) rather than a single shared consumer artifact. | ||
|
|
||
| ## Alternatives considered | ||
| - Single neutral consumer bundle/feed carrying all formats — rejected: forces irrelevant bytes on consumers; new infra; not idiomatic. | ||
|
|
||
| > Pairs with ADR 0001: provenance is the internal hand-off (0001); distribution is per-language feeds (this ADR). The two are intentionally decoupled. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # ADR 0003 — .NET consumes via per-RID NuGet NativeAssets + meta-package | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| .NET consumers expect NuGet. Native libraries are platform-specific, so a single package carrying every platform forces every consumer to download all of them. Mature native-backed SDKs (SkiaSharp, Microsoft.Data.SqlClient.SNI) solve this with a split package layout. | ||
|
|
||
| ## Decision | ||
| - Ship per-RID **`Microsoft.Azure.Cosmos.NativeAssets.<rid>`** packages, each carrying one platform's dynamic lib under `runtimes/<rid>/native/`. | ||
| - Front them with a thin **meta-package** whose `runtime.json` resolves the consumer's RID to the right per-RID package. | ||
|
Comment on lines
+9
to
+10
There was a problem hiding this comment. We should run this by the .NET/NuGet teams and see if there are any unexpected consequences to this choice.
Member
Author
There was a problem hiding this comment. Makes sense. The ADR stays at the packaging-shape level, but we should validate the NuGet consequences with the .NET/NuGet folks before treating the layout as final. |
||
| - The **native-transport major** of `Microsoft.Azure.Cosmos` takes a **direct dependency** on the meta-package (ADR 0007); the per-RID native then restores transitively like any NuGet dependency. There is no opt-in toggle — taking that major *is* taking native. (A `<PackageReference>` always restores transitively, so an "opt-in dependency" would be a contradiction.) | ||
|
Comment on lines
+10
to
+11
|
||
| - Publish to **nuget.org** and the **internal azure-sdk NuGet feed** (per ADR 0002). | ||
|
|
||
| ## Consequences | ||
| - A consumer downloads only its platform's binary. | ||
| - More package IDs to version and keep in lockstep (the meta `runtime.json` pins per-RID versions). | ||
| - Reuses Azure's already-reserved `Microsoft.Azure.*` prefix — no new org-level construct. | ||
|
|
||
| ## Alternatives considered | ||
| - Single "fat" NativeAssets package — kept only as an **interim** (Phase 1–2) for speed, not GA. | ||
| - Embed every platform's native directly in the `Microsoft.Azure.Cosmos` package — rejected: forces every consumer to download all platforms' binaries (the per-RID split exists precisely to avoid that). Note this is *not* about sparing "pure-managed users" — in the native major (ADR 0007) there are none — it is about not shipping six platforms to every consumer. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # ADR 0004 — Go consumes via cgo against a prebuilt header + lib from the Go feed | ||
|
|
||
| **Status:** Proposed for review — core model firm, **delivery shape WIP** | ||
|
|
||
| > **WIP.** The *decision* — Go links a **packaged** prebuilt native library via cgo (`CGO_ENABLED=1`) using the `C.*` FFI stubs from the cbindgen header — is firm. Still being settled: the **delivery shape** (Universal Package vs vendored binaries module, Q3) and the static-vs-dynamic default (Q4). Explicitly **out of scope**: a pure-Go shim that downloads the native lib — the binary is packaged and linked, never fetched by a stub. The pure-Go-*port* alternative (re-implementing the driver in Go) was spiked and is discussed under *Alternatives considered* + the design doc. | ||
|
|
||
| ## Context | ||
|
|
||
| Go has no NuGet. Go links C libraries through cgo, which needs a C header and a library available at `go build` time. The Go SDK already implements the completion-queue receive loop, `cgo.Handle` correlation, and buffer copy-out; the only distribution question is how header+lib+ABI version reach the Go build. | ||
|
|
||
| ## Decision | ||
| - Go consumes the prebuilt **`include/` header and `lib/` library via cgo** with **`CGO_ENABLED=1`**: `#cgo CFLAGS -I…` parses the header into the `C.*` **FFI stubs**, and `#cgo LDFLAGS -L… -lazurecosmosdriver` links the library. **Not** NuGet, and **not** a pure-Go build. | ||
| - The native library is **packaged inside the delivered artifact** and linked at `go build`. It is **not** a pure-Go shim that downloads the library at build/run time — customers link a real binary through the FFI stubs; they are not handed a downloader stub. | ||
| - Prefer the **static `.a`** for a self-contained Go binary; dynamic linking is supported as an option. | ||
|
|
||
| - The header + lib are delivered through the **azure-sdk-for-go feed** — an Azure Artifacts Universal Package fetched at build, or a vendored "binaries" Go module with per-OS build tags (delivery *shape* is open Q3; in every shape the binary is **packaged**, not fetched by a shim). Either way it derives from the ADR 0001 hand-off artifact. | ||
|
|
||
|
|
||
| ## Consequences | ||
| - Go reuses the exact same signed binaries as .NET — no Go-specific build of the driver. | ||
| - cgo + static lib means `CGO_ENABLED=1` and a C toolchain on the Go build host; cross-compilation needs a cross C toolchain. | ||
|
Contributor
There was a problem hiding this comment. One |
||
| - Everything resolves at `go build` — no runtime resolver / `runtime.json` / RID probing. The same `ABI_VERSION` feeds Go's handshake (ADR 0005). | ||
|
|
||
| ## Alternatives considered | ||
| - Wrap the lib in NuGet for Go — rejected: Go can't consume NuGet. | ||
| - A neutral consumer bundle Go downloads — rejected (ADR 0001/0002): pulls irrelevant formats. | ||
| - A pure-Go **shim module** that downloads the native lib at build/run time — rejected: customers can't be handed a downloader stub; the binary must be packaged and linked via cgo. | ||
| - **Pure-Go reimplementation of the driver — rejected (for distribution).** A time-boxed spike built a working `CGO_ENABLED=0`, zero-dependency vertical slice (point read/create + retry, routing, and multi-region/transport failover) and validated it behavior-for-behavior against the real Rust driver with a differential harness. It still defeats build-once single provenance (ADR 0001): every language would re-port and re-verify, making "the driver" N hand-maintained translations that chase every Rust change. Valuable as a parity oracle / risk probe, not a shipping vehicle — see the design doc's *Alternative considered — a pure-Go port* section. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # ADR 0005 — Native lib exports an ABI version; hosts check it before use | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| The native library and each language SDK ship on independent cadences. A managed/native version mismatch at the byte level (the Application Binary Interface — ABI) corrupts memory or crashes deep in a call, with no clear cause. | ||
|
|
||
| ## Decision | ||
| - The native library exports **`cosmos_abi_version() -> u32`** — a **monotonically increasing integer ABI revision, not a packed SemVer** — and ships the same value in the hand-off's `ABI_VERSION`. | ||
| - Every language host reads it **at load, before the first real call**, and accepts only the **single ABI revision it was built against** (`Expected`): any mismatch — the native revision being **lower (too old)** *or* **higher (too new / unknown)** — fails fast with a versioned message, so a different native is never silently accepted. There is **no accepted range**: because every release is one coordinated native version fanned out to all SDKs (ADR 0010), each host targets exactly one ABI revision, which keeps the compatibility test matrix at a single point instead of a range. | ||
| - A breaking C-ABI change is a **major bump**, coordinated with SDK releases; each SDK pins the **exact** native version it targets (ADR 0010), not a range. | ||
|
|
||
| ## Consequences | ||
| - Mismatches surface as a clear, actionable error instead of a crash. | ||
| - Both too-old and too-new natives are rejected explicitly; a host never runs against an ABI revision it does not understand. | ||
| - Adds a tiny, one-time check at load per process. | ||
| - Creates an explicit compatibility contract across all three languages. | ||
|
|
||
| ## Alternatives considered | ||
| - Rely on package version pins only — rejected: private deployment / manual overrides bypass the package graph. | ||
| - No handshake — rejected: silent corruption is the worst failure mode here. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # ADR 0006 — Each language binding owns marshalling and buffer copy-out | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| The driver core is schema-agnostic: the C-ABI passes request/response bodies as raw bytes (`const uint8_t*` + length), never parsed JSON (see `NATIVE_WRAPPER_SPEC.md`, introduced in #4461). Pushing serialization into the wrapper would re-introduce the parse/re-serialize/re-parse waste the old `azure_data_cosmos_native` crate had. | ||
|
|
||
| ## Decision | ||
| - The ABI stays **bytes-in / bytes-out**; the wrapper does no JSON parsing. | ||
| - Each language binding **owns its own marshalling** (string encoding, structs) and **copies response buffers out of native memory** into host memory, then frees the native buffer. Where a binding turns host strings into bytes for the driver boundary, it uses **UTF-8**. | ||
| - Rust owns the buffer until the host copies it out; ownership transfer is explicit per the ABI spec. | ||
|
|
||
| ## Consequences | ||
| - The native binaries are identical for every language; differences live in each binding. | ||
| - No double-encoding cost across the FFI boundary. | ||
| - Each language must implement copy-out correctly (lifetime + free) — a per-binding responsibility. | ||
|
|
||
| ## Alternatives considered | ||
| - Serialize to JSON strings in the wrapper — rejected: redundant parsing, hides driver concepts. | ||
| - Zero-copy borrow of native buffers into host memory — rejected for now: lifetime hazards across GC'd hosts; copy-out is the safe default. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # ADR 0007 — Native transport ships in a new SDK major; no parallel managed transport | ||
|
|
||
| **Status:** Accepted (proposed for review) | ||
|
|
||
| ## Context | ||
|
|
||
| `Microsoft.Azure.Cosmos` (pure managed) and the Go SDK run today on any platform with no native dependency. Introducing a platform-specific native library changes the portability and trust profile of the SDK. The question is how consumers move onto native **without being force-migrated mid-stream**, and whether a managed and a native transport are maintained side by side. The switch to the Rust driver is a breaking change in how the SDK talks to the service — it lands at a **major version**, not as a per-call toggle. | ||
|
|
||
| ## Decision | ||
| - Native transport is introduced in a **new SDK major version**. Adopting that major **is** adopting native: there is no per-client / per-call opt-in switch, and no managed transport kept in parallel *inside the same major*. | ||
| - Existing majors keep their current managed / pure transport unchanged. No consumer is **force-migrated by a routine (minor/patch) bump** — they move to native deliberately, by choosing to take the new major. | ||
| - The supported surface is bounded by the platform matrix (ADR 0008). On an unsupported platform/RID the native major fails with a **clear, actionable error**; it does **not** silently fall back to a managed path. If a pure-managed / pure-Go audience must be served long-term, that is a **separate, explicitly limited package**, not a hidden fallback inside the flagship. | ||
|
Comment on lines
+9
to
+11
Comment on lines
+9
to
+11
|
||
| - **Integrity and contract failures are fail-loud:** a signature/checksum failure, a corrupt library, or an ABI mismatch (ADR 0005) on a library that *is* present surfaces a clear error and aborts — never a silent downgrade that could mask tampering or misconfiguration. | ||
|
|
||
| ## Consequences | ||
| - "Which transport am I on" has one answer per major: the native major is native, full stop — no per-deployment drift between managed and native, and no doubled transport matrix to test and support. | ||
| - The native path matures across the **prerelease line of the new major** (dogfood → preview → GA) rather than behind a runtime flag in an existing major. | ||
| - Migration is a deliberate major-version decision that goes through the usual major-version compatibility review — not a surprise on upgrade. | ||
| - Platforms outside the matrix are a clear error in the native major; serving them is a separate-package decision, not an in-SDK fallback. | ||
|
|
||
| ## Alternatives considered | ||
| - **Per-client / per-call opt-in toggle inside one major (native *or* managed)** — rejected: keeps two transports alive in parallel, doubles the test/support matrix, and does not match how the switch actually happens — moving to the Rust driver is a major-version event. | ||
| - **Silent runtime fallback to managed on unsupported platforms** — rejected: hides capability gaps and muddies the trust model; an explicit error (or a separate limited package) is clearer. | ||
| - **Native-on by default within an existing major** — rejected: that is a force-migration by upgrade. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.