Skip to content

[python] Introduce merge_map_with_keytime aggregator function - #8734

Merged
JingsongLi merged 3 commits into
apache:masterfrom
PyRSA:feature/pypaimon-merge-map-with-keytime-impl
Jul 21, 2026
Merged

[python] Introduce merge_map_with_keytime aggregator function#8734
JingsongLi merged 3 commits into
apache:masterfrom
PyRSA:feature/pypaimon-merge-map-with-keytime-impl

Conversation

@PyRSA

@PyRSA PyRSA commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Purpose

This PR introduces the Python FieldMergeMapWithKeyTimeAgg implementation and aligns the Python merge_map_with_keytime aggregation behavior with the Java implementation.

Tests

  • python -m pytest pypaimon/tests/test_field_aggregators.py::FieldMergeMapWithKeyTimeAggTest -q
  • python -m pytest pypaimon/tests/test_aggregation_e2e.py -q
  • python -m pytest pypaimon/tests/test_aggregation_merge_function.py -q
  • git diff --check

@JingsongLi

Copy link
Copy Markdown
Contributor
  • When fields.<field>.ts-field is not configured, the Java semantics use the last field in the ROW.
  • In Python, when calling input_row.get(None) on a PyArrow/dict ROW, the timestamp returned is always empty, causing the new value to be ignored.

@PyRSA

PyRSA commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author
  • When fields.<field>.ts-field is not configured, the Java semantics use the last field in the ROW.
  • In Python, when calling input_row.get(None) on a PyArrow/dict ROW, the timestamp returned is always empty, causing the new value to be ignored.

Thanks! Fixed. The Python implementation now matches the Java behavior: if fields.<field>.ts-field is not configured, it resolves the last field of the ROW as the timestamp field during initialization instead of looking up None at runtime. I've also added a test to cover this default timestamp field behavior.

@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 87aeb13 into apache:master Jul 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants