Skip to content

Add ValidatesRunner cross-SDK test coverage comparison#39128

Draft
damccorm wants to merge 2 commits into
apache:masterfrom
damccorm:feature/compat-coverage-comparison
Draft

Add ValidatesRunner cross-SDK test coverage comparison#39128
damccorm wants to merge 2 commits into
apache:masterfrom
damccorm:feature/compat-coverage-comparison

Conversation

@damccorm

Copy link
Copy Markdown
Contributor

Summary

  • Adds .test-infra/compatCoverage/compat-test-comparison.md comparing the validates-runner test suite coverage between the Java and Python SDKs
  • Table covers 100+ test cases across 14 feature categories: ParDo, State & Timers, GroupByKey, Flatten, Combine, CoGroupByKey, Side Inputs, Windowing, Metrics, TestStream, Impulse, Splittable DoFn, Reshuffle, Redistribute, DoFn Lifecycle, Timestamps, Per-key Ordering, and Cross-Language (XLang) transforms
  • Each row links directly to the relevant test in Java (@Category(ValidatesRunner.class)) and the closest Python equivalent (in ptransform_test.py, sideinputs_test.py, combiners_test.py, fn_runner_test.py, metric_test.py, dofn_lifecycle_test.py, and validate_runner_xlang_test.py)
  • Includes an Observations section summarizing key coverage gaps (e.g., Java-only: TestStream, Redistribute, per-key ordering, set/map state; near-parity: XLang transforms)

Test plan

  • Verify all Java SDK links resolve to the correct test method and line number in the repo
  • Verify all Python SDK links resolve to the correct test method and line number in the repo
  • Confirm the Observations section accurately reflects the gap analysis

🤖 Generated with Claude Code

damccorm and others added 2 commits June 26, 2026 18:15
…DK test mapping

Replaces the initial stub with a full comparison table covering 100+ test cases
across 14 feature categories: ParDo, State & Timers, GroupByKey, Flatten, Combine,
CoGroupByKey, Side Inputs, Windowing, Metrics, TestStream, Impulse, SDF, Reshuffle,
Redistribute, DoFn Lifecycle, Timestamps, Per-key Ordering, and XLang transforms.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
@github-actions github-actions Bot added the infra label Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant