[split 18/22] core: DDL schema-change waiter, source-schema integrity validator, replication freeze manager, error classifier (new classes, not yet wired) - #1373
Open
minguyen9988 wants to merge 1 commit into
Conversation
…plication freeze manager, error classifier (new classes, not yet wired) Adds four new core classes with their unit tests. NOT yet referenced by any existing code path - wiring happens in the executor PR - so this PR is behavior-neutral by construction. - DDLSchemaChangeWaiter: after a DDL, polls system.columns (bounded by ddl.schema.change.timeout.ms) until the destination schema reflects the change, so DML following a DDL never binds against a stale column set. Exempts the replication-history table (its audit schema never mirrors source columns; gating on it starved the shared system pool on the combined PR). - SourceSchemaIntegrityValidator + SourceSchemaColumns: compares source-event columns against the destination insert cache; genuinely missing columns block the batch (data-loss protection), while MySQL generated columns (MATERIALIZED on the destination, excluded from the insert cache by design) are filtered out - counting them as missing caused an 8,208-rebuild livelock on the combined PR. - TableReplicationFreezeManager: freezes replication for a table on unrecoverable schema mismatch instead of silently dropping records. - ClickHouseErrorClassifier: classifies ClickHouse exceptions into retryable vs fatal so the batch loop can stop retrying permanently-failing batches. Merge order: after the config PR (uses ddl.schema.change.* keys). Part of the split of #1353 into independently mergeable sub-PRs (each <= 10 files), so the 2.10.0 branch can absorb the fixes incrementally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR: based on the previous PR in the series (
omniwatcher/split-17-ddl-parser); this PR's own diff is only its listed files. Merge the series in order; after the predecessor merges, retarget this PR to2.10.0.Adds four new core classes with their unit tests. NOT yet referenced by any existing code path - wiring happens in the executor PR - so this PR is behavior-neutral by construction.
Merge order: after the config PR (uses ddl.schema.change.* keys).
Part of the split of #1353 into independently mergeable sub-PRs (each <= 10 files), so the 2.10.0 branch can absorb the fixes incrementally.
Split out of #1353, which this series replaces. Each sub-PR is <= 10 files; the union of all 22 reproduces the #1353 tree exactly (verified by tree SHA).