Skip to content

Normalize timestamps to millisecond (3-digit) precision across all databases#94

Closed
vmatt wants to merge 12 commits into
erezsh:masterfrom
vmatt:2026-02-fixes
Closed

Normalize timestamps to millisecond (3-digit) precision across all databases#94
vmatt wants to merge 12 commits into
erezsh:masterfrom
vmatt:2026-02-fixes

Conversation

@vmatt

@vmatt vmatt commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

Standardizes timestamp normalization to consistently use 3 fractional digits (milliseconds) across all database backends, instead of the previous behavior which used up to 6 digits (microseconds) or varied per-database.

Problem

When diffing rows between different databases, timestamp comparisons could fail due to inconsistent fractional-second precision in the normalized string representations. Different databases handled timestamp precision differently:

  • Some databases would pad to 6 fractional digits
  • Others would use the column's native precision
  • Rounding vs. truncation behavior was inconsistent across backends

This led to false positives (phantom diffs) when comparing timestamps across heterogeneous database pairs, particularly when one database stored microseconds and another only stored milliseconds.

Changes

All database backends are updated to normalize timestamp strings to 3 fractional digits (YYYY-MM-DD HH:MI:SS.fff):

  • DuckDB: Fixed substring length to use TIMESTAMP_PRECISION_POS + 3 instead of + coltype.precision
  • MySQL: Rounding path now truncates to 3 digits via LEFT(); non-rounding path pads to TIMESTAMP_PRECISION_POS + 3 instead of + 6
  • Oracle: FF6FF3 for rounding path; RPAD target length reduced from + 6 to + 3
  • PostgreSQL: LEFT() and RPAD() both capped at 3 fractional digits using min(coltype.precision, 3)
  • Snowflake: Removed convert_timezone('UTC', ...) wrapping, switched to timestamp_ntz types, and formats output with FF3 instead of FF6

Rationale

  • Millisecond precision is the practical common denominator across most database engines (e.g., MySQL's DATETIME defaults to second precision, many systems don't reliably store beyond ms)
  • Reduces cross-database diff noise without sacrificing meaningful comparison fidelity
  • Snowflake changes also fix timezone conversion issues by using timestamp_ntz consistently

Version Bump

reladiff version: 0.6.00.6.6

@vmatt vmatt closed this Mar 6, 2026
@vmatt
vmatt deleted the 2026-02-fixes branch March 6, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant