Skip to content

HTTP parameters: DateTime64 values are formatted with the current culture, so a non-Gregorian calendar sends a wrong date #714

Open

Description

@polyglotAI-bot

Describe the bug

The HTTP client formats DateTime64 query parameters with the current culture, not the invariant culture. When the current culture uses a non-Gregorian default calendar, the client sends a different date, and the server stores or compares that wrong date with no error.

For 2024-01-15 12:30:45.123 bound to {p:DateTime64(3)}:

CultureInfo.CurrentCulture Calendar String sent as param_p Value on the server
invariant / en-US Gregorian 2024-01-15 12:30:45.1230000 2024-01-15 12:30:45.123
th-TH ThaiBuddhistCalendar 2567-01-15 12:30:45.1230000 2567-01-15 12:30:45.123
ar-SA UmAlQuraCalendar 1445-07-03 12:30:45.1230000 1445-07-03 12:30:45.123
fa-IR PersianCalendar 1402-10-25 12:30:45.1230000 1402-10-25 12:30:45.123
fi-FI, da-DK Gregorian, time separator . 2024-01-15 12.30.45.1230000 2024-01-15 12:30:45.123 (this server accepts it)

All DateTime64 arms are affected: DateTime of any Kind, DateTimeOffset, and wrapped types (Nullable(DateTime64(3)), Array(DateTime64(3))). The same values bound to DateTime, DateTime('UTC') or Date are correct in every culture, because those arms already use CultureInfo.InvariantCulture. The native TCP client is not affected: ClickHouse.Driver.Tcp/Parameters/TcpParameterFormatter.cs formats DateTime64 with CultureInfo.InvariantCulture.

Steps to reproduce

  1. Set CultureInfo.CurrentCulture = new CultureInfo("th-TH") (or ar-SA, fa-IR).
  2. Insert a DateTime/DateTimeOffset through a {p:DateTime64(...)} parameter, or select it back.
  3. Read the stored value. The year (and for ar-SA/fa-IR also the month and day) is from the culture's calendar, not the Gregorian one.

Expected behaviour

The parameter string is culture-invariant, so the server receives the Gregorian date the caller passed. The server parses the parameter text literally:

$ curl -G http://clickhouse:8123/ --data-urlencode "query=SELECT toString({p:DateTime64(3, 'UTC')})" \
    --data-urlencode "param_p=2024-01-15 12:30:45.1230000"
2024-01-15 12:30:45.123
$ curl -G http://clickhouse:8123/ --data-urlencode "query=SELECT toString({p:DateTime64(3, 'UTC')})" \
    --data-urlencode "param_p=2567-01-15 12:30:45.1230000"
2567-01-15 12:30:45.123

So the client must send the first string. ClickHouse.Driver/AGENTS.md also states that HttpParameterFormatter.Format "does culture-invariant value formatting for all 60+ types".

Code example

var table = "dt64_culture";
await connection.ExecuteStatementAsync($"CREATE TABLE {table} (c String, v DateTime64(3, 'UTC')) ENGINE = Memory");
var value = new DateTimeOffset(2024, 1, 15, 12, 30, 45, 123, TimeSpan.Zero);

foreach (var name in new[] { "en-US", "th-TH", "ar-SA", "fa-IR" })
{
    CultureInfo.CurrentCulture = new CultureInfo(name);
    using var cmd = connection.CreateCommand();
    cmd.CommandText = $"INSERT INTO {table} VALUES ({{c:String}}, {{p:DateTime64(3, 'UTC')}})";
    cmd.AddParameter("c", name);
    cmd.AddParameter("p", value);
    await cmd.ExecuteNonQueryAsync();
}
CultureInfo.CurrentCulture = CultureInfo.InvariantCulture;

using var reader = await connection.ExecuteReaderAsync($"SELECT c, toString(v) FROM {table}");
while (reader.Read())
    Console.WriteLine($"{reader.GetString(0)}: {reader.GetString(1)}");

Output:

en-US: 2024-01-15 12:30:45.123
th-TH: 2567-01-15 12:30:45.123
ar-SA: 1445-07-03 12:30:45.123
fa-IR: 1402-10-25 12:30:45.123

Error log

None. The wrong value is stored silently.

Root cause

ClickHouse.Driver/Formats/HttpParameterFormatter.cs (at c8a72abf) uses interpolated strings for the two DateTime64 formats. An interpolated string formats with CultureInfo.CurrentCulture, so yyyy/MM/dd use that culture's calendar and : becomes its time separator:

  • line 140, DateTime with Kind == Unspecified: $"{dtv:yyyy-MM-dd HH:mm:ss.fffffff}"
  • line 282, FormatDateTime64InTargetTimezone (UTC/Local DateTime and DateTimeOffset): $"{zonedDateTime.ToDateTimeUnspecified():yyyy-MM-dd HH:mm:ss.fffffff}"

Suggested fix

Format both values with the invariant culture, the same as the Date/DateTime arms and the TCP formatter, for example:

dtv.ToString("yyyy-MM-dd HH:mm:ss.fffffff", CultureInfo.InvariantCulture)
zonedDateTime.ToDateTimeUnspecified().ToString("yyyy-MM-dd HH:mm:ss.fffffff", CultureInfo.InvariantCulture)

A regression test can set CultureInfo.CurrentCulture to th-TH and ar-SA, round-trip a DateTime64 parameter through the server, and restore the culture afterwards. The output for the invariant culture must not change.

Configuration

Environment

  • Client version: main at c8a72abf (after 1.5.0)
  • Language version: C# (repository default)
  • .NET version: .NET SDK 10.0.403, net10.0
  • OS: Linux (Ubuntu 24.04 container, ICU 74)

ClickHouse server

  • ClickHouse Server version: 26.9.1.1629
  • ClickHouse Server non-default settings, if any: none
  • CREATE TABLE statements for tables involved: in the code example

Found by automated analysis (Polyglot AI) while working on #711, and verified against a live ClickHouse server, not only by inspection.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions