[alex-clickhouse]

This is part 6c of the stack of pull requests for ClickHouse/integrations#801, a converter layer for the native TCP
client. It is stacked on #677 (part 6b), which is on #674 (part 6a), #673, #672, #671, #670, #668, #667, #666 and #665.
It refuses, in every read and write tier, the .NET array casts that give each element of an array another meaning.

What changes for a user

• An array cast that gives each element another meaning is refused, in QueryAsync<T>, Block.ReadAs,
  InsertAsync, InsertRowsAsync<T>, InsertRowsAsync with object[] rows, ClickHouseTcpTypes.CanRead and
  ClickHouseTcpTypes.CanWrite:
  • a cast between integer types of the other sign: Array(UInt32) does not read as int[] (3000000000 read as
    -1294967296), a uint[] is not written into Array(Int32), and the same for byte/sbyte, ushort/short and
    ulong/long;
  • an enum array over an integer type of the other sign: Array(Int32) does not read as an array of an enum over
    uint, and an array of an enum over sbyte is not written into Array(UInt8) or String;
  • the same casts in jagged arrays and in the generic collection interfaces of an array (IReadOnlyList<int> for
    Array(UInt32)).
• InsertAsync refuses these casts in each way that it writes a column: a column of your own, an array that
  ClickHouseTcpColumn.CreateArray builds, and a column that a query read, also when the insert writes such a column
  part by part (for example an Array(UInt32) part next to a DateTime64(3) part, into a
  Tuple(DateTime64(6), Array(Int32)) column).
• An enum array and an array of its underlying type still cast both ways: Array(Enum8(...)) reads as MyEnum[]
  for an enum MyEnum : sbyte, through POCO mapping and ReadAs, and a MyEnum[] is written into Array(Int8) or
  Array(Enum8(...)). Each element keeps its integer value. Casts to object, IEnumerable and other non-generic
  types are not affected. A uint[] value in a Dynamic column is not a cast: it is written as Array(UInt32), as
  before.
• The refusal comes before a value is read or written, with the message of each tier, which names the type to use:
  for example "Column 'value' (Array(UInt32)) maps to property 'CountsRow.Value' of type System.Int32[], which it cannot
  be read as. It reads as System.UInt32[]. ..." for POCO mapping, and "... It accepts System.Int32[] ..." for the
  inserts.
• This is a breaking change for code that relied on these casts, which the released POCO reads, POCO inserts and
  untyped inserts accepted: change the property or value type to the one that the message names (uint[] for
  Array(UInt32)).

How

The read rules and the write rules share one cast rule, ReadRules.IsAssignable: the casts that the CLR allows,
except an array cast that gives the elements another meaning (ReadRules.ReinterpretsElements). For two value-type
element types of one size, which are the only ones that the CLR casts between, the cast keeps the meaning only between
an enum and its exact underlying type. Reference elements are checked one level down, so jagged arrays follow the same
rule. The write rules take their cast targets through the same check, so every tier refuses the same set. Every insert
route that writes a column through the converter tree of its CLR type gets the rule from there: a column of your own,
an array that CreateArray builds, and a column that a query read (whole, or a part of another type whose values keep
their meaning). The
routes that write the stored bytes of a column, and Dynamic and Variant, which choose a type from the runtime type of
each value, make no cast. No read or write loop changes: the rule runs when a converter is derived, and the derivation
is cached.

Why

A .NET array cast between uint[] and int[] keeps the array and reads its bits as the other type, so a value above
int.MaxValue silently becomes a negative number, and back. The client accepted such casts because the CLR allows
them. Refusing them in the one rule makes every tier give the same answer, and the message points to the type that
keeps the values.

Tests

• The tables of what each column type reads as and is written from change by the refused casts only (for example
  Array(UInt32) loses int[], IntEnum[], IEnumerable<int> and IReadOnlyList<int>, and the string types lose
  sbyte[] writes).
• Every array cast that the CLR allows between the client's canonical arrays and arrays of integers or enums of the same
  size is listed: 12 accepted (an enum array from an array of its underlying type) and 36 refused. The write cast
  targets of the matrix sources keep 4 element casts (an enum array as an array of its underlying type).
• uint[] into Array(Int32) is refused by the columnar insert, the POCO insert and the untyped insert, and
  CanWrite says false; Array(UInt32) as int[] is refused by ReadAs and CanRead, with the messages.
• The insert routes: an array that CreateArray builds over a uint column (also over a column that a query read, an
  enum over uint, and a jagged array) is refused into Array(Int32); a column that a query read with an
  Array(UInt32) is refused into Array(Int32) as a whole column and part by part (next to a part of another scale,
  and next to a Nullable), and each test checks which of these ways the insert takes. With the rule of part 6b, each
  of these cases writes the bits of the uint values. A byte pin shows a uint[] and an int[] in a Dynamic column,
  written as Array(UInt32) and Array(Int32) with their values.
• Against the server: Array(Enum8(...)) into an enum array through POCO mapping and ReadAs; Array(UInt32) as
  int[] refused by POCO mapping and ReadAs; uint[] into Array(Int32) refused by the three inserts and by a
  CreateArray insert before a block goes out; a column that a query read, Array(UInt32) and
  Tuple(DateTime64(3, 'UTC'), Array(UInt32)), refused into the Int32 forms with nothing stored. Each message is
  pinned.
• Tests changed on purpose: the tests that used a sign cast as their example of a cast now use an enum array (the
  untyped write of a cast target, the cast of references over segments, the AssignWriter derivation, the InsertAsync
  write rule against the server), or assert the refusal (the read and write examples of the reinterpretation, one
  CanRead case, the two lists of array casts, the two tables). No test of part 6b changes its expected outcome; this
  part adds tests to three of its test files.
• net9.0, full TCP suite with a server: 6614 passed on part 6b, 6642 passed here (+28), 0 failed, 0 skipped. net8.0 and
  net10.0, the offline suite (no server): 4514 passed each. The public API does not
  change.

Benchmarks

No benchmark change is expected: the rule runs only when a converter is derived, and the derivation is cached. No read
or write loop changes.

Where the speedup of the stack comes from

This section is about the whole stack (parts 0 to 6c). The ratios were measured on part 6a; part 6b made some LowCardinality writes faster again.

The converter layer does not convert values faster. Both the old and the new code call the same conversion code
for each value (for example DateTimeToOffset, with the timezone resolved once when the codec is built, and the
same UTF-8 encode and decode). The gains come from work that the old structure did around each value:

1. Indirection on each row. In the old code, a conversion was a hook on a codec: TryProjectColumnRead gave a
   view, and TryProjectRead gave an expression that was compiled into a delegate for each value. A composite codec
   put the view of its child inside its own view. So one converted value cost a delegate call, an interface cast,
   and interface calls into the column (each of which built a span again), and, for a composite, the same again
   for each element. The POCO scatter used its fast tier only for columns that implement IStoredValuesColumn;
   String, Nullable(...) and LowCardinality(...) columns do not, so each row went through the column's
   indexer. The writes read the caller's column one element at a time through IColumn<T>[i], and Nullable put a
   substitute view around it. The new code derives one tree for the whole type. Bind does the casts and builds
   the spans once for each column (once for each window of 256 rows in a POCO read). The bulk Fill loops are
   generics closed over struct conversions, so the conversion is inlined into the loop. A POCO read splices the
   per-row expression of the whole tree into one compiled loop for each column. For example:
   • LowCardinality(String) read as byte[]: a delegate call, an interface cast, two interface calls and a
     type-checked array store for each row, against two loads and a store.
   • Array(String) read as byte[][]: about 120,000 dispatches for each block of 10,000 rows, against 30,000.
   • Nullable(DateTime) written from DateTimeOffset?: three interface calls for each row, against none.
2. Allocation and rebuilding for each block. The old ReadAs<T> allocated a new array for the whole converted
   column in each block. At 10,000 rows a DateTimeOffset[] (160 KB) or DateTimeOffset?[] (240 KB) is on the large
   object heap, and a reader that keeps going runs a garbage collection about once in 20 blocks. The new view uses a
   pooled array that the block owns (about 300 bytes allocated for each block). The old LowCardinality writer
   built a new Dictionary with no capacity in each block: for 10,000 distinct values that is 11 resizes, 941 KB
   for each block (719 KB on the large object heap) and a generation 2 collection about once in 10 blocks, and it
   hashed each new value two times. The new interners keep their tables in pooled buffers that grow with the
   number of distinct values, and hash each value one time (1.8 KB for each block for FixedString).
3. One writer call for each fixed-width value. The old leaf writes made one ClickHouseBinaryWriter call for
   each null byte, each UInt32 and each dictionary key. The new writes put the null map in one copy, convert into
   4 KB chunks on the stack, and write the keys in chunks.

The work that both versions share limits the gain. A read that converts a String leaf only (String as
string, DateTime in a POCO) does the same work for each row in both versions. A LowCardinality dictionary is
converted one time for each block in both. A repeated string into LowCardinality(String) costs one dictionary
lookup for each row in both.

Ratios in the default runtime mode (tiered compilation with dynamic PGO, the steady state), new time / old time
for a block of 10,000 rows, measured with the old and the new code side by side:

Shape	new / old
Block.ReadAs: DateTime('UTC') as DateTimeOffset	0.53
Block.ReadAs: Nullable(DateTime('UTC')) as DateTimeOffset?	0.47 to 0.52
Block.ReadAs: LowCardinality(String) as byte[]	0.45
Block.ReadAs: Array(String) as byte[][]	0.76 to 0.79
POCO read: LowCardinality(Nullable(String)) as string	0.80
POCO read: LowCardinality(String) as byte[]	0.72
POCO read: Array(String) as byte[][]	0.79
POCO read: a row of six columns	0.74
POCO read: Nullable(DateTime('UTC')) as DateTimeOffset?	0.87
POCO read: String as string, DateTime('UTC') as DateTimeOffset	0.93, 1.00
InsertAsync: Nullable(DateTime('UTC')) from DateTimeOffset?	0.68 to 0.74
InsertAsync: LowCardinality(FixedString(16)) from byte[], all distinct	0.63 to 0.66
InsertAsync: LowCardinality(String) from string, all distinct	0.90 to 0.93
InsertAsync: LowCardinality(String) from string, 100 distinct	0.97 to 1.05
InsertRowsAsync<T>: Nullable(DateTime('UTC')) from DateTimeOffset?	0.82 to 0.88

The gate of each part runs with tiered compilation off, so that the code does not change during a run.
Its ratios differ from the ones above: in that mode the JIT cannot remove the old per-row interface calls of
ordinary methods, and it compiles the new loops without the inlining that tier 1 adds.

Changes after review

• An untyped row insert applies the sign-cast rule to each value, not only to the value that selects the write type: a uint[] after an int[] in Array(Int32) is refused with its row, also across blocks (Polyglot). docs/tcp.mdx says so.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Copilot]

🟢 Approval recommended

The centralized rule is correct, documented, comprehensively tested across all TCP tiers, and current CI and coverage checks pass.

0 open findings

What changed in this PR

This TCP-client change prevents CLR array casts that reinterpret signedness while preserving enum/underlying-type casts.

Changes:

• Rejects unsafe array reinterpretations in shared read/write rules.
• Adds comprehensive unit, integration, and insert-route coverage.
• Updates public documentation and adds a breaking-change fragment.

File	Description
ReadRules.cs	Filters unsafe array casts.
WriteRules.cs	Applies the shared cast filter to writes.
ClickHouseTcpTypes.cs	Documents CanRead/CanWrite behavior.
ReadRulesTests.cs	Covers accepted and rejected read casts.
WriteRulesTests.cs	Covers all write tiers and cast targets.
WriteRuleWriterTests.cs	Verifies retained enum-array writes.
DerivedColumnTests.cs	Updates CanRead expectations.
ArrayElementCastIntegrationTests.cs	Tests all public read/write tiers.
DecodedColumnInsertIntegrationTests.cs	Covers decoded-column refusals.
WriteRulesIntegrationTests.cs	Retains enum-array integration coverage.
DecodedColumnConversionTests.cs	Tests decoded composite routes.
InsertWritePinTests.cs	Pins dynamic and dense insert behavior.
docs/​tcp.mdx	Documents the breaking cast rules.
converter-6c-array-sign-casts.breaking.md	Records the breaking change.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[polyglotAI-bot]

Summary
The TCP client rejects sign-changing array casts while preserving casts between enum arrays and their exact underlying types. The previously reported untyped-row bypass is fixed, with regression tests covering mixed arrays within one block and across block boundaries. Static review found no remaining actionable findings. Builds and tests were not run in this sandbox. ✅ Approve.

Final verdict

✅ Approve

_{Automated review by Polyglot AI (gpt-6-astra) of d5b9839. Replies on the inline threads are read on the next review. A maintainer can ask for a new review by mentioning @polyglotAI-bot or requesting a review from it.}