Skip to content

feat: add QueryResult::data_str to read results without copying - #59

Merged
ShawnChen-Sirius merged 1 commit into
chdb-io:mainfrom
hyperbyte-cloud:feat/query-result-data-str
Sep 30, 2026
Merged

ShawnChen-Sirius merged 1 commit into
chdb-io:mainfrom
hyperbyte-cloud:feat/query-result-data-str

Conversation

@austin-barrington

@austin-barrington austin-barrington commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Right now if you want a query result as text you've got data_utf8(), which copies the whole result buffer into a new String, or data_utf8_lossy(). There's no way to just look at it as a &str.

This adds QueryResult::data_str() -> Result<&str>. It checks the bytes are valid UTF-8 in place and borrows straight from the result buffer, so nothing gets allocated.

On a 92MB JSONEachRow result (3M rows):

  • data_utf8(): ~45ms, plus a 92MB allocation
  • data_str(): ~6ms, no allocation

Also:

  • New Error::InvalidUtf8 for when the data isn't valid UTF-8 (binary formats like RowBinary or Parquet). Error is #[non_exhaustive] so it's not a breaking change.
  • data_utf8() is unchanged, its docs now point at data_str() if you don't need an owned string.
  • examples/03_query_results.rs and docs/examples.md show it.

Tests cover a normal result (and check it points at the same memory as data_ref()), a binary result returning InvalidUtf8, and an empty result.

Note

Add QueryResult::data_str to return result text as a borrowed &str

  • Adds a public data_str method that validates the result buffer in place with std::str::from_utf8 and returns a borrowed string slice without allocating. See query_result.rs.
  • Adds an Error::InvalidUtf8 variant carrying std::str::Utf8Error for results that are not valid UTF-8, such as binary output. See error.rs.
  • Updates docs and the main example to show the borrowed read alongside the existing owned, lossy, and byte-slice accessors.
  • Adds tests for zero-copy borrowing, binary rejection, and empty results in query_result.rs.

Macroscope summarized 085dcd2.

The only ways to get a result as text were data_utf8(), which copies the
whole buffer into a new String, or data_utf8_lossy(). data_str() checks
the bytes in place and hands back a &str into the result buffer.

On a 92MB JSONEachRow result it takes ~6ms vs ~45ms for data_utf8, and
skips the 92MB allocation. Adds Error::InvalidUtf8 for when the data
isn't valid UTF-8.
@ShawnChen-Sirius
ShawnChen-Sirius merged commit 9d83326 into chdb-io:main Sep 30, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants