Skip to content

[task] bump clickhouse to 0.15.3 and drop the network_compression_method=LZ4 stopgap #1164

Closed

Description

@MarcusKainth

What kind

None of the form's kinds apply cleanly. This is a dependency bump that removes a workaround, filed on the feature form because blank issues are disabled.

Which components

ClickHouse sink (spate-clickhouse)

What you are trying to do, and what stops you

Remove the network_compression_method=LZ4 setting the ClickHouse sink sends under compression: lz4.

ClickHouse 26.9 compresses query responses with ZSTD by default, and clickhouse 0.15.2 decodes them only as LZ4 under Compression::Lz4. The sink works around that by sending network_compression_method=LZ4 on every request (#796, which closed #458):

  • client_for in crates/spate-clickhouse/src/config.rs:795-800
  • the Compression::Lz4 rustdoc at config.rs:213-214
  • the unit test lz4_requests_lz4_response_frames at config.rs:1430
  • the test client in crates/spate-clickhouse/tests/coded_inserts.rs:92

clickhouse 0.15.3, released 2026-10-07, fixes this in the client. Its changelog: "Switched Lz4 compression to use HTTP standard Accept-Encoding negotiation, fixing a regression in ClickHouse 26.9" (ClickHouse/clickhouse-rs#482). Cargo.lock on main still has 0.15.2.

The same release stops sending enable_http_compression and http_zlib_compression_level on Query (ClickHouse/clickhouse-rs#487), and sends query parameters in a multipart/form-data body (ClickHouse/clickhouse-rs#455). The sink's schema fetch and readiness probe go through Query, so both need a run against 26.9 after the bump.

What the API might look like

No API change.

  1. Bump clickhouse to 0.15.3 in Cargo.lock, and raise the workspace floor in Cargo.toml from "0.15" to "0.15.3". Without the floor, a build that resolves 0.15.0–0.15.2 regresses against 26.9 once the setting is gone.
  2. Remove the setting from client_for and the test client, the sentence in the Compression::Lz4 rustdoc, and lz4_requests_lz4_response_frames.
  3. Run the container suite against ClickHouse 26.9 and 26.8, with compression: lz4 and format: native, so the schema fetch and the readiness probe are covered.

A fixed changelog fragment is not needed if the behaviour a user sees does not change. The raised floor is worth one line under Changed.

Semver impact, as best you can tell

Additive — nothing existing changes

Does this touch any of these?

  • INV-1 — delivery semantics, or anything that could commit a watermark past unacknowledged data
  • INV-2 — the source poll loop, which must never block on a channel send
  • INV-3 — the checkpoint tracker, which is synchronous and free of async runtimes
  • INV-5 — the sink intake path, which must not await outside its select!
  • INV-6 — adding a connector or 0.x dependency type to a public signature in spate-core
  • INV-8, INV-9 or INV-10 — metric naming, cardinality, or gauge ownership

Would you want to implement it?

Yes, if the design is agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions