Skip to content

Failure: clickhouse keeper / alter_partition_distributed / alter clear column in partition — SummingMergeTree + CLEAR COLUMN regression in ClickHouse >=25.8 #129

Description

@CarlosFelipeOR

Affected tests:

  • /clickhouse keeper/part 2/alter_partition_distributed/alter clear column in partition

Affected files:

  • clickhouse_keeper/tests/alter_partition_distributed.py
  • engines/tests/summing_merge_tree/summing_merge_tree.py (new)
  • engines/regression.py

Failing runs:

Description

The alter clear column in partition test in the clickhouse_keeper suite fails on ClickHouse versions < 25.8 (22.8, 25.3, 25.7). The test verifies that ALTER TABLE CLEAR COLUMN IN PARTITION replicates correctly across all nodes via ClickHouse Keeper.

The failure is caused by an interaction between SummingMergeTree, CLEAR COLUMN IN PARTITION, and OPTIMIZE TABLE FINAL. After clearing the only summed column to zero and forcing a merge, SummingMergeTree deletes the row entirely (documented behavior: "If the values were 0 in all of the columns for summation, the row is deleted"), making the SELECT return empty instead of 0.

Root Cause

Two commits introduced the issue:

  1. 2ca6d3e49 (Feb 24) — Changed ORDER BY tuple() to ORDER BY v to comply with ClickHouse 25.12+ restriction on ORDER BY tuple() in SummingMergeTree.

  2. dd664a7f1 (Feb 25) — Attempted to fix the test by adding column c UInt64, clearing c instead of v, and checking SELECT c WHERE p=1 expects "0". However, c became the only summed column (v is ORDER BY key, p is PARTITION BY key), and after CLEAR COLUMN c + OPTIMIZE TABLE FINAL, SummingMergeTree deleted the row because all summed columns were zero.

Version-dependent behavior

Version After CLEAR c + OPTIMIZE FINAL Test result
22.8, 25.3, 25.7 Row deleted (all summed cols = 0) FAIL (empty output)
25.8, 26.1 Row preserved PASS

Investigation revealed that this version-dependent behavior is itself a bug in ClickHouse >=25.8, likely introduced by upstream PR #83892 ("Don't delete default rows in coalescing merge tree"). CLEAR COLUMN sets the column to a "default" internal state that the SummingSortedAlgorithm no longer recognizes as zero, unlike ALTER TABLE UPDATE c = 0 which writes an explicit zero and correctly triggers row deletion in all versions.

Additional finding: validation inconsistency

CLEAR COLUMN on a summing column is blocked when columns_to_sum is explicitly declared (SummingMergeTree(c)), but allowed when auto-detected (SummingMergeTree without parameters) — even though c is the summing column in both cases.

Applied Solution

1. Fix the keeper test

Commit: 0441b01

Removed OPTIMIZE TABLE FINAL from the test. The test's purpose is to verify replication of CLEAR COLUMN via Keeper, not SummingMergeTree merge behavior. CLEAR COLUMN IN PARTITION applies directly to parts and replicates via the replication queue — no merge is needed. The retry(timeout=100) handles replication delay.

2. Add dedicated SummingMergeTree tests

PR: #128

Added engines/tests/summing_merge_tree/ test suite with three scenarios:

  • zero_row_deletion_with_update — verifies UPDATE c=0 + OPTIMIZE FINAL deletes the row (passes on all versions)
  • zero_row_deletion_with_clear_column — verifies CLEAR COLUMN + OPTIMIZE FINAL deletes the row (xfail on >=25.8, upstream issue)
  • clear_column_validation_consistency — verifies CLEAR COLUMN is blocked on auto-detected summing columns (xfail on all versions, upstream issue)

3. Upstream issue

Issue: ClickHouse/ClickHouse#101953

Reported two bugs:

  1. CLEAR COLUMN IN PARTITION does not trigger zero-row deletion after OPTIMIZE TABLE FINAL on SummingMergeTree (regression since 25.8)
  2. CLEAR COLUMN validation inconsistency between explicit and auto-detected summing columns

References

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions