Skip to content

[FLINK-39995][json] Add ISO-8601 timestamp format with explicit timezone offset support#28546

Open
ddebowczyk92 wants to merge 1 commit into
apache:masterfrom
ddebowczyk92:FLINK-39995
Open

[FLINK-39995][json] Add ISO-8601 timestamp format with explicit timezone offset support#28546
ddebowczyk92 wants to merge 1 commit into
apache:masterfrom
ddebowczyk92:FLINK-39995

Conversation

@ddebowczyk92

@ddebowczyk92 ddebowczyk92 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

The current ISO-8601 timestamp format option for JSON is misleading as it does not support timestamps with explicit timezone offsets (e.g., 2026-06-25T09:15:00+00:00).

The existing ISO-8601 format only handles timestamps with UTC 'Z' suffix: yyyy-MM-ddTHH:mm:ss.s{precision}Z

It cannot parse valid ISO-8601 timestamps with explicit timezone offsets like +02:00, -05:00, etc.

This change adds a new timestamp format option ISO-8601-WITH-OFFSET that properly
handles timestamps with explicit timezone offsets in the format:
yyyy-MM-ddTHH:mm:ss.s{precision}±HH:mm

Brief change log

  • Added TimestampFormat.ISO_8601_WITH_OFFSET enum value
  • Added ISO8601_TIMESTAMP_WITH_OFFSET_FORMAT DateTimeFormatter
  • Updated JSON format options to support ISO-8601-WITH-OFFSET

Verifying this change

Added unit tests covering various offset formats and precisions

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? not documented

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

@flinkbot

flinkbot commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@ddebowczyk92 ddebowczyk92 force-pushed the FLINK-39995 branch 2 times, most recently from 24e3845 to f5b83f2 Compare June 25, 2026 15:46
@hiSandog

Copy link
Copy Markdown

The new format parses explicit offsets, but serialization currently emits the resulting instant with +00:00. That may be the right internal semantics, but the option name could lead users to expect offset preservation. It would be worth documenting or testing this explicitly: parsing 1990-10-14T12:12:43.123+02:00 will round-trip as the UTC instant, not the original local time plus original offset.

* Formatter for ISO8601 string representation of a timestamp value (with explicit timezone
* offset).
*/
public static final DateTimeFormatter ISO8601_TIMESTAMP_WITH_OFFSET_FORMAT =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please could you update the docs around this change.

case ISO_8601:
parsedTimestamp = ISO8601_TIMESTAMP_FORMAT.parse(jp.getText());
break;
case ISO_8601_WITH_OFFSET:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my AI says
For DST: Document that this format preserves the explicit offset only, not the timezone. Users needing DST-aware handling should use TIMESTAMP_WITH_LOCAL_TIMEZONE with a proper timezone ID (e.g., America/New_York), not just offsets.

For Half-Hour Timezones: Add comprehensive tests:

root.put("timestamp_india", "1990-10-14T12:12:43.123+05:30");
root.put("timestamp_nepal", "1990-10-14T12:12:43.123+05:45");

Consider: Whether output should preserve the original offset instead of always converting to UTC. Current behavior may surprise users expecting offset preservation.

Documentation: Clearly state that this format converts all timestamps to UTC offset (+00:00) on output, regardless of input offset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants