Skip to content

dbt Core 2.0 parity: bot that mirrors macro changes from this repo into the v2 macro tree #787

Open

Description

@BentsiLeviav

Problem

The ClickHouse adapter for dbt Core 2.0 vendors a copy of our macros in dbt-labs/dbt-core (crates/dbt-loader/src/dbt_macro_assets/dbt-clickhouse/macros). The goal is byte parity with dbt/include/clickhouse/macros here. Today nothing enforces it: a contributor fixes a macro in this repo, the PR merges, and the v2 copy stays behind until someone notices.

That already happened. Five PRs merged in Oct–Nov 2026 never reached v2, three of them bug fixes that v2 users hit today: #751 (list unique_key in snapshots), #755 (query_settings on refreshable MVs), #769 (distributed models on a Replicated database), plus #754 (temp view leak) and #753 (refactor).

Proposal

A GitHub Actions workflow in this repo, in the style of docs_drift_check.yml, that runs an agent when a PR touching dbt/include/clickhouse/macros/** is merged into main.

The agent:

  1. Diffs the merged change against the v2 macro tree on dbt-labs/dbt-core main. Known, intentional divergences (seed.sql insert path, check_schema_exists and get_columns_in_query in adapters.sql, should_on_cluster(relation=) in table.sql) are listed in a config file and ignored.
  2. Checks whether the contributor already handled it. Looks for an open or merged PR in dbt-labs/dbt that references this PR number or contains the same hunk. If found, it comments on our PR with the link and stops.
  3. Otherwise opens the equivalent PR in dbt-labs/dbt against the vendored macro files, with the ported tests where they exist, a body that links back to our PR, and [ClickHouse] in the title as the upstream convention requires. When the hunk does not apply cleanly, or the change touches a file with a known divergence, it opens an issue there instead of a PR and explains what needs a human.
  4. Reports back on our PR: link to the upstream PR or issue, or a note that no action was needed.

A second, scheduled mode runs the full tree diff nightly and opens one issue here listing any drift the per-PR run missed, so a workflow failure cannot leave a gap silently.

Out of scope

  • Changes that need Rust work in dbt-core (new adapter.* methods, config-key registration). The bot opens an issue for those, not a PR.
  • Merging on the upstream side. A human reviews and merges every PR the bot opens.
  • Python adapter changes in dbt/adapters/clickhouse. Those have no vendored copy and need a manual port decision.

Open questions

  • Token and permissions for opening PRs in dbt-labs/dbt from our CI.
  • Whether to run on PR open as an advisory comment ("this macro change will need a v2 port") in addition to the post-merge run.
  • Where the known-divergence list lives so both repos can read it.

Acceptance

  • A macro-only PR merged here results, within the same day, in either a linked upstream PR or a comment explaining why none was needed.
  • The five PRs above are detected by the nightly mode if the port PR has not merged yet.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions