feat(sdk): opt-in OTLP payload compression (gzip + zstd)

[igoramf]

Summary

Wires Content-Encoding on the three OTLP/HTTP exporters (logs, metrics, traces) behind DECO_OTEL_COMPRESSION=gzip|zstd. Default remains none — no wire behavior change for existing sites.

• gzip: native CompressionStream("gzip") — zero deps, works on every runtime we target (workerd, Node ≥18, Deno, Bun).
• zstd: node:zlib.zstdCompress. Requires nodejs_compat on workerd (landed via cloudflare/workerd#4013) and Node ≥22 on k8s. Silently falls back to gzip when unavailable; Content-Encoding always reflects the actual encoding used, so the collector doesn't care which path was taken.

Sizes on a synthetic OTLP JSON batch (50 log records):

encoding	bytes	vs raw
raw	6748	—
gzip	429	-94%
zstd	325	-95%

Zstd is ~24% smaller than gzip here and ~2–3x cheaper for the collector to decode. CompressionStream("zstd") is still non-standard (Bun-only as of Feb 2026) — when workerd + Node ship it, this helper collapses to new CompressionStream(compression).

Files touched

• src/sdk/otlpCompression.ts (new) — helper + lazy node:zlib probe.
• src/sdk/otelHttp{Log,Meter,Tracer}.ts — accept compression?: OtlpCompression, thread Content-Encoding header.
• src/sdk/otel.ts — bootObservability parses DECO_OTEL_COMPRESSION and passes it to all three adapters.

Test plan

• Local round-trip check on Node 24: gzip + zstd + none all correct, zstd decoded round-trips to identical JSON.
• Deploy a canary site with DECO_OTEL_COMPRESSION=gzip and verify Content-Encoding: gzip reaches deco-otel-ingest.
• Same with DECO_OTEL_COMPRESSION=zstd on a site with nodejs_compat enabled; verify Content-Encoding: zstd and no runtime error.
• Confirm silent fallback: set DECO_OTEL_COMPRESSION=zstd without nodejs_compat; expect Content-Encoding: gzip and no ingest errors.

🤖 Generated with Claude Code

---

Summary by cubic

Adds opt-in OTLP/HTTP payload compression for logs, metrics, and traces via DECO_OTEL_COMPRESSION, reducing bandwidth with no default behavior change.

• New Features

  • Supports DECO_OTEL_COMPRESSION=gzip|zstd|none in bootObservability; threaded to all HTTP exporters.
  • Gzip uses native CompressionStream("gzip"); sets Content-Encoding: gzip.
  • Zstd uses node:zlib.zstdCompress; silently falls back to gzip if unavailable; header matches actual encoding.
  • Typical savings: gzip ~-94%, zstd ~-95% on OTLP JSON batches.

• Migration

  • Enable with DECO_OTEL_COMPRESSION=gzip (safe everywhere) or zstd.
  • For zstd: require nodejs_compat on workerd and Node ≥22; otherwise auto-falls back to gzip.
  • Ensure your collector accepts Content-Encoding; when unsure, start with gzip.

^{Written for commit b2e9c98. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/sdk/otlpCompression.ts">

<violation number="1" location="src/sdk/otlpCompression.ts:55">
P2: When the probed zstd implementation is present but fails at runtime (e.g. memory pressure, runtime bugs, partially supported `node:zlib`), the catch block in `tryZstd` returns `null` without invalidating `zstdImpl`. That means every subsequent OTLP export request re-enters the failing path, throws again, and then falls back to gzip — creating repeated exception overhead on every payload instead of degrading once. After the first runtime failure, consider setting `zstdImpl = null` so the exporter skips straight to gzip on all subsequent calls.</violation>
</file>

<file name="src/sdk/otelHttpLog.ts">

<violation number="1" location="src/sdk/otelHttpLog.ts:312">
P1: The flush timeout timer now covers `compressJsonPayload` in addition to the network request. Before this PR, `JSON.stringify` was synchronous and effectively instant, so starting the timer before body preparation was fine. Now that `compressJsonPayload` is async — involving `CompressionStream` or `node:zlib.zstdCompress` — the timeout can fire before `fetchImpl` ever starts. This causes unnecessary aborts, which restore the snapshot to the buffer, and repeated failures can push the buffer to its cap and drop subsequent logs. Consider starting the `AbortController` and timer only after compression completes, so `flushTimeoutMs` bounds the network I/O alone.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: The flush timeout timer now covers compressJsonPayload in addition to the network request. Before this PR, JSON.stringify was synchronous and effectively instant, so starting the timer before body preparation was fine. Now that compressJsonPayload is async — involving CompressionStream or node:zlib.zstdCompress — the timeout can fire before fetchImpl ever starts. This causes unnecessary aborts, which restore the snapshot to the buffer, and repeated failures can push the buffer to its cap and drop subsequent logs. Consider starting the AbortController and timer only after compression completes, so flushTimeoutMs bounds the network I/O alone.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/sdk/otelHttpLog.ts, line 312:

<comment>The flush timeout timer now covers `compressJsonPayload` in addition to the network request. Before this PR, `JSON.stringify` was synchronous and effectively instant, so starting the timer before body preparation was fine. Now that `compressJsonPayload` is async — involving `CompressionStream` or `node:zlib.zstdCompress` — the timeout can fire before `fetchImpl` ever starts. This causes unnecessary aborts, which restore the snapshot to the buffer, and repeated failures can push the buffer to its cap and drop subsequent logs. Consider starting the `AbortController` and timer only after compression completes, so `flushTimeoutMs` bounds the network I/O alone.</comment>

<file context>
@@ -301,10 +309,16 @@ export function createOtlpHttpLogAdapter(options: OtlpHttpLogOptions): OtlpHttpL
     const controller = new AbortController();
     const timer = setTimeout(() => controller.abort(), flushTimeoutMs);
     try {
+      const { body, contentEncoding } = await compressJsonPayload(payload, compression);
+      const headers: Record<string, string> = {
+        ...extraHeaders,
</file context>

[cubic-dev-ai]

P2: When the probed zstd implementation is present but fails at runtime (e.g. memory pressure, runtime bugs, partially supported node:zlib), the catch block in tryZstd returns null without invalidating zstdImpl. That means every subsequent OTLP export request re-enters the failing path, throws again, and then falls back to gzip — creating repeated exception overhead on every payload instead of degrading once. After the first runtime failure, consider setting zstdImpl = null so the exporter skips straight to gzip on all subsequent calls.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/sdk/otlpCompression.ts, line 55:

<comment>When the probed zstd implementation is present but fails at runtime (e.g. memory pressure, runtime bugs, partially supported `node:zlib`), the catch block in `tryZstd` returns `null` without invalidating `zstdImpl`. That means every subsequent OTLP export request re-enters the failing path, throws again, and then falls back to gzip — creating repeated exception overhead on every payload instead of degrading once. After the first runtime failure, consider setting `zstdImpl = null` so the exporter skips straight to gzip on all subsequent calls.</comment>

<file context>
@@ -0,0 +1,82 @@
+  if (!zstdImpl) return null;
+  try {
+    return await zstdImpl(bytes);
+  } catch {
+    return null;
+  }
</file context>

Suggested change

	} catch {
	return null;
	} catch {
	zstdImpl = null;
	return null;
	}