[polyglotAI-bot]

Summary

Fixes #636. Follow-up to #467.

A collection written as one Dynamic value (decimal[], List<ClickHouseDecimal>, Dictionary<string, decimal>, a tuple, nested forms) took its type from the cache keyed by the .NET type. That type has a fixed Decimal(38, 9) element. ArrayType/MapType/TupleType write every element with it, so ScaleMantissa(value, 9) truncated each element with more than 9 fractional digits. Reproduced on main (server 26.9): new[] { 0.0123456789012345m, 0.0000000001m } was stored as Array(Decimal(38, 9)).

The fix follows the policy in the issue. For each decimal position in the type, the driver applies the #467 scalar policy to all the non-null values at that position together: the largest integer-digit count and the largest scale select the narrowest width with precision P, at scale P - integerDigits. The example above is now Array(Decimal(18, 18)), and [1.2m], [1.23m] and [1.2345m] are all Array(Decimal(9, 8)).

• A single element gets the same type as the scalar value.
• A position with no non-null value (an empty or all-null array) keeps Decimal(38, 9), the type it had before.
• If no Decimal256 holds all the values at a position, the write throws ArgumentOutOfRangeException, wrapped by InsertBinaryAsync. It does not truncate.
• The value-dependent type is never cached. Types that contain no decimal come from the cache as before: the same instance, with one extra cached bool.

Changes

• Types/DynamicType.cs: the cache also stores whether the type contains a decimal. If it does, InferDecimalTypes rebuilds the type by walking Nullable/Array/Map/Tuple in parallel with the value. It covers jagged and multi-dimensional arrays, IDictionary and List<KeyValuePair<,>> maps, Tuple and ValueTuple, and decimal map keys.
• Types/TypeConverter.cs:
  • New InferCommonDecimalType. It shares the width selection with InferDecimalType.
  • The 76-digit check now ignores trailing fractional zeros, which a smaller scale holds exactly. Without that, {10^28, 1.000…0 (scale 48)} would be rejected, although main writes it exactly and it fits Decimal(76, 47). The scalar check has the same change, so [x] and x always agree. This only affects values that Fix Dynamic: store decimals with the client-v2 scale policy instead of truncating to scale 9 #467 (not yet released) rejected.
• Unhinted JSON paths in JsonWriteMode.Binary write through DynamicType.Write, so they are fixed too.
• docs/http.mdx describes the collection rule, the fallback and the failure. A changelog fragment is added.

No public API change. The TCP client is not affected: it rejects decimal inside Dynamic composites and does not truncate them. Query parameters with decimal[] take a different path, which #637 tracks.

Test

New tests use InsertBinaryAsync against the real server and assert dynamicType() plus the numeric round trip:

• InsertBinaryAsync_DecimalCollectionToDynamic_StoresCommonTypeAndPreservesValues, which is parametrized:
  • arrays and lists of decimal and ClickHouseDecimal, including negative and tiny high-scale values, and the largest scale first or last;
  • the width boundaries 9/18/38/76, reached only by combining elements, including Decimal256 and values beyond System.Decimal;
  • zeros and trailing zeros;
  • nullable elements, and the empty and all-null fallback;
  • nested arrays (null inner array, rank-2), maps (null value, decimal key, key-value list), Tuple/ValueTuple, and an array of tuples.
• InsertBinaryAsync_DecimalArraysOfOneClrTypeToDynamic_InfersTypeForEachCell: successive cells of one .NET type with different types, which catches cache contamination.
• Read_DecimalCollectionFromDynamicWithoutCustomDecimals_ReturnsSystemDecimals: UseCustomDecimals=false reads stored scales above 28 and mantissas above 96 bits.
• InsertBinaryAsync_DecimalCollectionWithNoCommonDecimal256TypeToDynamic_Throws: each value fits alone, but together they need 77 digits. Nothing is inserted.
• InsertBinaryAsync_WithUnhintedDecimalArrayScaleAbove9_ShouldPreserveAllFractionalDigits: the JSON path.
• One scalar case is added to the Fix Dynamic: store decimals with the client-v2 scale policy instead of truncating to scale 9 #467 source: 1 at scale 77 is stored as Decimal(76, 75).
• BoxFreeWriteAllocationTests.Write_DecimalArrayToDynamic_AllocatesNothingPerElementToInferType: writes a 1000-element decimal[] and ClickHouseDecimal[] to Dynamic, and subtracts the allocations of writing the same array as its inferred type. Before the second commit: 112.5 and 88.5 B per element. Now: a constant ~0.5 KB per array.
• TypeMappingTests.InferDecimalType_IntegerAtDigitBoundary_KeepsEveryIntegerDigit and InferCommonDecimalType_DecimalAtDigitBoundary_MatchesClickHouseDecimal: the digit count at 10^n - 1, 10^n, 2^n - 1 and 2^n (up to 76 digits), and the decimal overload against the ClickHouseDecimal one.

On main, all the new real-server cases fail except the two fallback cases, which pin the unchanged Decimal(38, 9). With the fix, the full HTTP suite passes on net10.0: 11121 passed, 0 failed, 142 skipped by version gates.

Performance

Array(Decimal) writes a decimal[] or ClickHouseDecimal[] without boxing its elements, so the inference reads them the same way. A position that holds only decimal[] reads scale and mantissa from the decimal bits, so no BigInteger is built. Digits are counted with a binary search over cached powers of ten, not ToString().Length. Other collections (List<T>, maps, tuples, decimal?[], multi-dimensional arrays) already box on their write paths, and they keep the generic path.

BenchmarkDotNet, InsertBinaryAsync of 100 rows × 10,000-element arrays (1M elements per operation) into a Null-engine table. The values have 5 integer and 16 fractional digits. The typed-column rows insert the same arrays into Array(Decimal(38, 33)), the type that Dynamic now infers, so no inference runs:

Code	Shape	Allocated (Dynamic)	Allocated (typed column)
main (truncates to scale 9)	decimal[]	68.69 MB	—
main (truncates to scale 9)	ClickHouseDecimal[]	30.55 MB	—
first commit (7b663f2)	decimal[]	236.59 MB	106.84 MB
first commit (7b663f2)	ClickHouseDecimal[]	167.93 MB	68.69 MB
this PR (2b6fa5f)	decimal[]	108.20 MB	106.84 MB
this PR (2b6fa5f)	ClickHouseDecimal[]	68.73 MB	68.69 MB

The difference between main and the typed column is the write at the correct scale: ScaleMantissa multiplies by 10^17 instead of dividing by 10^7. In-process, the inference allocates 416–440 B per 10,000-element array. Its CPU cost is 2–3% of the write (best of 9: 7.00 → 7.23 ms for decimal[] and 6.80 → 6.92 ms for ClickHouseDecimal[] per array). The end-to-end mean times on the shared host had a StdDev of 75–260 ms, so I do not quote them.

Benchmark source (not committed)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using BenchmarkDotNet.Attributes;
using ClickHouse.Driver;
using ClickHouse.Driver.Numerics;

namespace ClickHouse.Driver.Benchmark;

/// <summary>
/// Measures large decimal arrays inserted into a <c>Dynamic</c> column (issue #636). Dynamic infers one
/// Decimal type from all the elements of an array before it writes them, so that no element loses digits.
/// <c>Array(Decimal)</c> writes a <c>decimal[]</c> or a <c>ClickHouseDecimal[]</c> without boxing the
/// elements, and the inference must not box them either:
/// <list type="bullet">
/// <item><description><see cref="DecimalArray"/>: <c>decimal[]</c>.</description></item>
/// <item><description><see cref="ClickHouseDecimalArray"/>: <c>ClickHouseDecimal[]</c>.</description></item>
/// </list>
/// The elements have 16 fractional digits, more than the scale of 9 that the type inferred from the .NET type
/// alone has. Inserts into a <c>Null</c>-engine table to isolate client serialization.
/// </summary>
[BenchmarkCategory(BenchmarkCategories.HttpInvestigation)]
[Config(typeof(ComparisonConfig))]
[MemoryDiagnoser(true)]
public class DynamicDecimalArrayInsert
{
    private const string Table = "test.benchmark_dynamic_decimal_array";

    private ClickHouseClient client;
    private List<object[]> decimalRows;
    private List<object[]> clickHouseDecimalRows;

    [Params(100)]
    public int Rows { get; set; }

    [Params(10_000)]
    public int Elements { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        var connectionString = Environment.GetEnvironmentVariable("CLICKHOUSE_CONNECTION")
            ?? "Host=localhost";
        client = new ClickHouseClient(connectionString);

        client.ExecuteNonQueryAsync("CREATE DATABASE IF NOT EXISTS test").GetAwaiter().GetResult();
        client.ExecuteNonQueryAsync($"CREATE TABLE IF NOT EXISTS {Table} (value Dynamic) ENGINE Null").GetAwaiter().GetResult();

        decimalRows = new List<object[]>(Rows);
        clickHouseDecimalRows = new List<object[]>(Rows);
        for (var n = 0; n < Rows; n++)
        {
            var decimals = new decimal[Elements];
            for (var i = 0; i < Elements; i++)
                decimals[i] = 12345.6789012345678901m + (((n * Elements) + i) * 0.0000000000000001m);

            decimalRows.Add(new object[] { decimals });
            clickHouseDecimalRows.Add(new object[] { decimals.Select(d => new ClickHouseDecimal(d)).ToArray() });
        }
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        client?.Dispose();
    }

    [Benchmark]
    public async Task<long> DecimalArray() =>
        await client.InsertBinaryAsync(Table, new[] { "value" }, decimalRows);

    [Benchmark]
    public async Task<long> ClickHouseDecimalArray() =>
        await client.InsertBinaryAsync(Table, new[] { "value" }, clickHouseDecimalRows);
}

The typed-column rows used two extra methods that insert the same row lists into test.benchmark_typed_decimal_array (value Array(Decimal(38, 33))) ENGINE Null.

Checklist

• Unit and integration tests covering the common scenarios were added
• A human-readable description of the changes was provided to include in CHANGELOG (changelog.d/636-dynamic-decimal-collections.fixes.md)
• Docs updated (docs/http.mdx)

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

Decimal array inference introduces per-element boxing allocations in the binary-insert hot path.

Review effort: Balanced
Findings: 1

Open (1)

• Decimal array inference boxes every element, regressing insert performance · New

What changed in this PR

Fixes decimal precision loss in HTTP-client Dynamic collections and unhinted binary JSON paths.

Changes:

• Adds value-aware decimal type inference for nested collections.
• Expands real-server regression coverage.
• Updates documentation and changelog.

File	Description
ClickHouse.Driver/​Types/​TypeConverter.cs	Infers common lossless decimal types.
ClickHouse.Driver/​Types/​DynamicType.cs	Applies inference across composite values.
ClickHouse.Driver.Tests/​Types/​DynamicTests.cs	Adds collection regression coverage.
ClickHouse.Driver.Tests/​Types/​JsonTypeTests.cs	Covers unhinted JSON decimal arrays.
docs/​http.mdx	Documents inference behavior.
changelog.d/​636-dynamic-decimal-collections.fixes.md	Records the fix.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 96.21212% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
ClickHouse.Driver/Types/DynamicType.cs	92.95%	3 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!