Repository navigation
Fix UseFormDataParameters: send SQL over 128 KiB as several query form fields - #701
Open
polyglotAI-bot wants to merge 1 commit into
Open
polyglotAI-bot wants to merge 1 commit into
polyglotAI-bot wants to merge 1 commit into
Conversation
…ataParameters With UseFormDataParameters=true the whole SQL text went into one multipart `query` field, which the server caps at http_max_field_value_size (128 KiB by default), so larger queries failed with "Field value too long". The server joins all `query` fields of a multipart request in order, so the SQL is now split into fields of at most 128 KiB, never inside a multi-byte UTF-8 character. Smaller queries produce the same request as before. Fixes: #700 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
polyglotAI-bot
requested review from
alex-clickhouse and
mzitnik
as code owners
October 25, 2026 22:52
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The multipart transport change needs human confirmation of compatibility across supported frameworks, server versions, and Cloud beyond the reported test runs.
No open issues 🎉
What changed in this PR
Fixes #700 in the HTTP client by splitting large SQL text into repeated multipart query fields when UseFormDataParameters is enabled. The native TCP client is unchanged.
Changes:
- Limits each SQL field to 128 KiB without splitting UTF-8 characters.
- Adds regression tests for large literals, UTF-8 boundaries, and multi-field inline inserts.
- Documents the remaining per-parameter size limit and adds a changelog fragment.
| File | Description |
|---|---|
docs/http.mdx |
Clarifies parameter-value limits in both transport modes. |
ClickHouse.Driver/ClickHouseClient.cs |
Splits SQL into size-limited multipart fields. |
ClickHouse.Driver.Tests/SQL/FormDataLargeQueryTests.cs |
Tests field sizes, UTF-8 integrity, and large inserts. |
changelog.d/700-form-data-large-sql.fixes.md |
Records the large-SQL fix for #700. |
Review effort: Balanced
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Draft
2 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #700.
With
UseFormDataParameters=true,BuildHttpRequestMessageWithFormDatasent the whole SQL text as one multipart form field namedquery. The server limits each form field tohttp_max_field_value_size(128 KiB by default), so any query with SQL text over 128 KiB failed withHTML Form Exception: Field value too long. This happened with or without parameters, on every entry point (ClickHouseClientandClickHouseCommandboth go throughPostSqlQueryAsync).For a multipart request, the server joins the values of all
queryfields in order (DynamicQueryHandler::getQueryinsrc/Server/HTTPHandler.cpp:full_query += it.secondfor each field with the query parameter name). The fix uses this: the SQL text is now sent as one or morequeryfields of at most 128 KiB each. A chunk boundary never splits a multi-byte UTF-8 character. A query under 128 KiB still produces onequeryfield with the sametext/plain; charset=utf-8part header as before, so its request does not change.Moving the SQL to the URL instead would not help: the server applies the same per-value limit to URL query-string values (
HTMLForm::readQuery), and proxies often set a lower URL limit. Form-data mode exists to keep large values out of the URL (#334).Changes
ClickHouse.Driver/ClickHouseClient.cs: newAddQueryFormFieldssplits the UTF-8 SQL text intoqueryfields of at most 128 KiB (the server default ofhttp_max_field_value_size).ClickHouse.Driver.Tests/SQL/FormDataLargeQueryTests.cs: regression tests.docs/http.mdx: theUseFormDataParametersrow states the per-parameter limit (see below).changelog.d/700-form-data-large-sql.fixes.md: changelog fragment.Not changed: one parameter value over 128 KiB
The issue also notes that a single parameter value over 128 KiB hits the same limit. This is a server limit that applies in both modes, so the client cannot avoid it:
param_<name>field the server keeps the first value, so a value cannot be split across fields;http_max_field_value_sizeto URL query-string values too (HTMLForm::readQuery), so moving the value to the URL does not help: a 131,073-byteparam_pURL value fails with the sameField value too long;http_max_field_value_sizedoes not raise it, because the server parses the form with its default settings.All three verified on 26.9. The
UseFormDataParametersrow indocs/http.mdxnow states this limit.Test
FormDataLargeQueryTests. All 6 cases fail onmainwithField value too longand pass with the fix, on ClickHouse 26.9 and 26.3:ExecuteReaderAsync_FormDataWithSqlOverFieldLimit_SendsValidFieldsAndReturnsLiteralUnchanged: a parameterizedSELECTwith a ~200 KB string literal, sent to the real server through a forwardingHttpMessageHandlerthat records thequeryparts. It asserts:querypart is at most 128 KiB. This holds on any server configuration, including one that raised the limit;The cases are ASCII, plus a 4-byte character with ASCII padding that puts the field boundary at each byte position inside the character. If the no-split loop is removed, the three cases with a boundary inside the character fail. The server alone cannot detect this, because it joins raw bytes.
ExecuteNonQueryAsync_FormDataInsertWithInlineRowsOverFieldLimit_InsertsAllRows: anINSERT ... FORMAT JSONEachRowwith ~760 KiB of inline rows (six fields). It asserts the row count,sum(id), and that each row'ssmatches itsid, so a lost or reordered field fails it.The full
ClickHouse.Driver.Testssuite (net10.0) passes against 26.9: 14,824 passed, 0 failed.Pre-PR validation gate
main)AGENTS.md/ClickHouse.Driver/AGENTS.md(changelog fragment, docs,CreateTableName)Related: ClickHouse/clickhouse-rs#501 (same bug in the Rust client), #334.
🤖 Generated with Claude Code
Note
Medium Risk
Changes how multipart HTTP requests encode SQL for
UseFormDataParameters, affecting large queries and bulk inline inserts; behavior for SQL under 128 KiB is intended to stay identical.Overview
Fixes
UseFormDataParametersfailures when SQL text exceeds ClickHouse’s defaulthttp_max_field_value_size(128 KiB), which previously surfaced asHTML Form Exception: Field value too long.ClickHouseClientnow sends the SQL as one or more repeatedquerymultipart fields (≤128 KiB UTF-8 bytes each), relying on the server concatenating them in order. Chunk boundaries never split a multi-byte UTF-8 character; queries under the limit still use a single field with the same content type as before.Adds
FormDataLargeQueryTests(largeSELECTwith captured form parts, plus a multi-fieldINSERT ... FORMAT JSONEachRow), a changelog note for #700, and docs clarifying the per-value 128 KiB limit for parameters in both URL and form-data modes (individual parameter values are not split by this change).Reviewed by Cursor Bugbot for commit fc24565. Bugbot is set up for automated code reviews on this repo. Configure here.