Skip to content

Fix DateTime64: throw instead of wrapping when a scale 8/9 value does not fit in Int64 - #712

Open
polyglotAI-bot wants to merge 1 commit into
mainfrom
polyglot/datetime64-scale-overflow
Open

polyglotAI-bot wants to merge 1 commit into
mainfrom
polyglot/datetime64-scale-overflow

Conversation

@polyglotAI-bot

@polyglotAI-bot polyglotAI-bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #711.

The HTTP client writes a DateTime64 as an Int64 count of 10^-scale seconds since the epoch. DateTime64Type.ToClickHouseTicks gets that count from the .NET tick count (100 ns) with MathUtils.ShiftDecimalPlaces, which multiplies by 10^(scale-7) with no overflow check. Scales 8 and 9 are finer than a tick, so they reach the Int64 limit inside the DateTime range: DateTime64(9) before 1677-09-21 or after 2262-04-11 UTC, and DateTime64(8) after 4892-10-07 UTC. For those values the multiply wrapped, and the server stored a different instant with no error (for example, 1500-06-01 was stored as 2084-12-18 at scale 9).

DateTime64Type.WriteChecked now checks the tick count against long.MaxValue / factor and long.MinValue / factor before it scales up. Out-of-range values throw ArgumentOutOfRangeException. The DateTime and Date32 writers and the TCP codec (DateTime64ColumnCodec.CountFromDateTimeOffset) already do this. Scales 0–7 are not changed.

Changes

  • ClickHouse.Driver/Types/DateTime64Type.cs: range check in WriteChecked for scale > 7. The exception reports the caller's original value (ActualValue), the column type and the range that can be written. The old comment ("No range check: any coerced instant is representable") was wrong and is removed. For scale > 7 the count is computed once (ticks * factor) after the check.
  • ClickHouse.Driver.Tests/Types/DateTimeBinaryRangeTests.cs: DateTime64 cases next to the existing Date/Date32/DateTime range tests.
  • docs/http.mdx: the write type map and the "Out-of-range values" note now include DateTime64(8) and DateTime64(9).
  • changelog.d/711-datetime64-overflow.fixes.md.

Test

The tests insert through InsertBinaryAsync into a real server:

  • At the Int64 limit, the value round-trips. Covers the DateTime64(9) min and max, the DateTime64(8) max, the max as a DateTimeOffset at +01:00, and the max as an Unspecified DateTime in an Asia/Tokyo column. The test asserts the stored count with reinterpretAsInt64(t). It does not use toString(t), because 26.3 renders an instant before 1900 as 1900-01-01 (checked on 26.3.28.5).
  • One tick past the limit, the write throws ArgumentOutOfRangeException. Covers DateTime (Utc, Local, Unspecified in a non-UTC column), DateTimeOffset with an offset, DateOnly and NodaTime Instant. The test checks the type name, the range in the message and ActualValue.

The expected counts and boundary dates come from the Int64 range (±9223372036854775800 at tick precision), not from client output. All past-limit cases fail on main (no exception); all cases pass with the fix. The full ClickHouse.Driver.Tests suite passes on net10.0 against 26.9. DateTimeBinaryRangeTests also passes against 26.3. The HTTP client builds for net6.0, net8.0, net9.0 and net10.0. changelog.cs --check passes.

Every binary write entry point (Write(object), the typed WriteValue overloads used by POCO inserts, and the Nullable/Array/Map/Tuple/Variant/JSON wrappers) goes through WriteChecked, so the one check covers all of them. The HTTP parameter path sends an ISO string and is not affected.

Checklist

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided to include in CHANGELOG

Pre-PR validation gate

  • Deterministic repro confirmed on main
  • Root cause documented above
  • Fix targets the root cause (bound check before the scale-up multiply)
  • Tests fail without the fix and pass with it
  • No existing tests changed or broken
  • Convention compliance verified per AGENTS.md and ClickHouse.Driver/AGENTS.md

Related: ClickHouse/clickhouse-java#3292 (the report this issue was relayed from).

🤖 Generated with Claude Code


Note

Medium Risk
Touches core binary serialization for high-precision DateTime64 inserts; behavior changes from silent wrong timestamps to exceptions for out-of-range values at scales 8–9.

Overview
Fixes silent corruption when binary-inserting fine-scale DateTime64 values whose scaled epoch count would not fit in Int64. For scales 8 and 9, scaling ticks before write could wrap and persist the wrong instant; those values now throw ArgumentOutOfRangeException from DateTime64Type.WriteChecked, with a message that names the column type and the storable range (scales ≤7 are unchanged).

Adds end-to-end InsertBinaryAsync tests for DateTime64(8/9) at the Int64 boundary (including timezone/offset/Instant/DateOnly cases) and just past it. Docs and a changelog entry document the accepted UTC ranges and the new validation on the binary write path.

Reviewed by Cursor Bugbot for commit 232d110. Bugbot is set up for automated code reviews on this repo. Configure here.

… not fit in Int64

The binary write path scaled the 100 ns tick count up with an unchecked
multiply. For DateTime64(9) (and DateTime64(8) after 4892-10-07) a value
outside the Int64 range at that scale wrapped, and a different instant was
stored with no error. WriteChecked now checks the bound before scaling up
and throws ArgumentOutOfRangeException with the value, the column type and
the range, like the Date/Date32/DateTime writers and the TCP codec.

Fixes: #711

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 7, 2026 03:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the DateTime64 overflow-validation change.

No open issues 🎉

What changed in this PR

Fixes #711 in the HTTP client by rejecting binary DateTime64(8/9) writes whose scaled timestamp exceeds Int64 limits, preventing silent timestamp corruption. Scales 0–7 remain unchanged.

Changes:

  • Adds overflow validation before scaling, with errors reporting the original value, column type, and writable range.
  • Adds server-backed tests for boundary values and out-of-range inputs, including timezone and offset conversions.
  • Documents the limits and records the behavior change in a changelog fragment.
File Description
docs/​http.mdx Documents DateTime64 write limits and out-of-range exceptions.
ClickHouse.Driver/​Types/​DateTime64Type.cs Checks Int64 bounds before scaling and formats range errors.
ClickHouse.Driver.Tests/​Types/​DateTimeBinaryRangeTests.cs Adds boundary round-trip and rejection tests for scales 8–9.
changelog.d/​711-datetime64-overflow.fixes.md Records the overflow fix and new exception behavior.

Review effort: Balanced

@codecov

codecov Bot commented Nov 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP client: DateTime64(9) silently stores a wrapped (wrong) value for dates outside 1677-2262

2 participants