Repository navigation
fix: use HTTP compression when requesting Lz4 for result sets - #482
Merged
Merged
Conversation
abonander
requested review from
joe-clickhouse,
kavirajk and
peter-leonov-ch
as code owners
September 28, 2026 14:11
5 tasks done
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
joe-clickhouse
approved these changes
Sep 30, 2026
mattsse
added a commit
to tempoxyz/tidx
that referenced
this pull request
Oct 6, 2026
ClickHouse 26.9 switched the codec of `compress=1` HTTP responses to ZSTD ([ClickHouse#108786](ClickHouse/ClickHouse#108786)), but the `clickhouse` crate 0.14 only decodes LZ4 native frames. Against `clickhouse-server:latest`, which all of our compose files use, every sink query fails with `decompression error: incorrect magic number`, starting with the version check in `ensure_schema`. The sink client now sets `network_compression_method=lz4`. Every 26.9 stable release honors it for `compress=1` responses ([ClickHouse#119825](ClickHouse/ClickHouse#119825)), and older servers already respond with LZ4. The upstream crate fix ([clickhouse-rs#482](ClickHouse/clickhouse-rs#482)) is unreleased; it avoids this setting because read-only users cannot change it, which doesn't matter for the sink since it needs write access. The query engine in `src/clickhouse.rs` uses plain reqwest without `compress=1` and was not affected. This also fixes `test_backfill_replays_blocks_with_extra_block_versions`, which failed on every server version. It wrote the stale and canonical versions of block 1 in one INSERT, which ReplacingMergeTree collapses on insert, so the two-version precondition never held. The product check is correct (`canonical_blocks_present` reads without `FINAL`). The test now writes the stale version separately into another monthly partition so it survives until the backfill repairs it. Stopping merges would not work here because it also blocks the repair's `ALTER TABLE ... DELETE` mutation. The ClickHouse integration tests pass against both 26.9.8.3 and 25.12.11.4. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This was referenced Oct 7, 2026
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ClickHouse/ClickHouse#108786 (released in 26.9) introduced a regression where
compress=1now means Zstd when existing client versions assumedcompress=1always meant Lz4 (#468, ClickHouse/clickhouse-java#3105).This was then partially addressed in ClickHouse/ClickHouse#119825 by letting us select Lz4 compression with
network_compression_method=lz4, theoretically letting us roll back to the previous behavior. However, it turns out thatnetwork_compression_methodcannot be set by read-only users (#476 (comment)), which is a known use-case for this crate so setting this unconditionally is unfortunately a non-starter.I started on a branch to change wholesale over to Zstd but it required enabling the
zstdfeature by default to fix this for all users. I think this would be better to do in a new major version and then we can droplz4as a default feature (https://github.com/ClickHouse/clickhouse-rs/compare/ab/zstd-native-framing).The only fix that doesn't require any user-facing changes is to switch to using HTTP compression exclusively (for result sets; native framing is still used for inserts for now). However,
lz4_flex's frame format implementation does not work in async code (it depends on the blockingReadtrait and is not re-entrant, so we can't just write an adapter that returnsWouldBlock), so it required hand-rolling our own.Using
async-compressionis also a non-starter because it links the nativeliblz4library which was previously dropped because a pure Rust implementation was desired (#107). Even though thezstdfeature ends up linkinglibzstd, I have to assume some users prefer thelz4feature simply because it does not pull in any native dependencies.Fixes #468
Fixes #483 (by a happy little accident!)
Checklist
Delete items not relevant to your PR: