[SPARK-40259][SQL][FOLLOWUP] Add cross-table DSv2 scan merge regression test - #58202
Closed
cloud-fan wants to merge 1 commit into
Closed
[SPARK-40259][SQL][FOLLOWUP] Add cross-table DSv2 scan merge regression test#58202cloud-fan wants to merge 1 commit into
cloud-fan wants to merge 1 commit into
Conversation
Contributor
Author
peter-toth
approved these changes
Aug 21, 2026
uros-b
approved these changes
Aug 21, 2026
Member
|
Thank you @cloud-fan and @peter-toth! |
HyukjinKwon
approved these changes
Aug 24, 2026
Contributor
Author
|
thanks for review, merging to master/4.x |
cloud-fan
added a commit
that referenced
this pull request
Aug 24, 2026
…on test ### What changes were proposed in this pull request? Followup to #57360. Add an end-to-end regression test verifying that Data Source V2 scans of two different tables are not merged, even when both tables opt into scan merging and the query shapes are otherwise similar. ### Why are the changes needed? Merging scans from different tables could corrupt the optimized plan and return incorrect query results. The existing implementation compares canonical relations, and this test protects that table-identity safety boundary. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added coverage to `DSv2PlanMergingSuite` and ran the full suite. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) Closes #58202 from cloud-fan/SPARK-40259-followup-248101. Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit f791085) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Followup to #57360.
Add an end-to-end regression test verifying that Data Source V2 scans of two different tables are
not merged, even when both tables opt into scan merging and the query shapes are otherwise similar.
Why are the changes needed?
Merging scans from different tables could corrupt the optimized plan and return incorrect query
results. The existing implementation compares canonical relations, and this test protects that
table-identity safety boundary.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Added coverage to
DSv2PlanMergingSuiteand ran the full suite.Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-5)