Skip to content

Flaky tests in Altinity regression suite #94

Description

@CarlosFelipeOR

🧪 Description

This issue tracks flaky or unstable tests in the Altinity regression test suite.

These tests:

  • Are maintained by Altinity
  • Are not part of upstream ClickHouse tests
  • May fail intermittently but pass after reruns

⚠️ Flaky test List

  • Export Part

    • Test case: /s3/minio/export tests/export part/concurrent alter/during export/example=drop table
      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Intermittent
      • First observed: 2026-01-21
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • Swarms

    • Test case: /swarms/feature/swarm joins/join clause

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/feature/object storage cluster profile events with overloaded node

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/node failure/network failure

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • aggregate functions

    • Test case: /aggregate functions/part 3/merge/
      • Versions: 25.8.16.20002.altinityantalya
      • Flakiness: Rare
      • First observed: Mar 22, 2026
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:


📝 Instructions

Please follow the guidelines below when tracking new flaky tests in this issue.

➕ Adding a new flaky test

To add a new flaky test to the list, use the following template:

- **SUITE NAME**
  - Test case: `PATH/TO/TEST`
    - Versions: 
    - Flakiness: 
    - First observed: 
    - Assigned to: 
    - Status: Pending investigation
    - 🔗 Links:
      - CI run
      - Query in history database

📊 Flakiness levels

Use one of the following values for Flakiness:

  • Rare (< 5%)
  • Intermittent (< 30%)
  • Frequent (> 30%)

Choose the level based on how often the test fails in CI.

📌 Status values

Use one of the following Status values:

  • Pending investigation — flaky observed, not investigated yet
  • Under investigation — investigation in progress
  • Fixed (pending confirmation) — fix applied, awaiting CI stability
  • Fixed — confirmed stable after multiple CI runs

🔍 Investigation workflow

For deeper investigation:

  • Create a sub-issue under this issue for the flaky test
  • Use the sub-issue to track analysis, logs, discussions, and fixes

Once the flakiness is fixed and confirmed stable after multiple CI runs:

  • Update the sub-issue accordingly
  • Remove the test from the list in the description

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions