perf: batch sink flushes and cache cluster group records when extending RNTuples

[ariostas]

🤖 AI text below 🤖

Addresses finding 2 of "PR 2" in #1688.

Flush per blob → flush per commit

add_rblob flushed the sink for every blob it wrote, so a single extend flushed once per column, once for the page list envelope and once for the footer, before the explicit sink.flush() already at the end of extend. That is especially expensive for remote sinks. Flushing now happens only at the commit boundary; write() flushes explicitly since it relied on add_rblob for that.

Data is still on disk at the end of every extend — there is a test for that.

Cached cluster group records

Every extension rewrites the whole footer, which re-serialized every cluster group record written so far, even though a record is never modified once appended. At 800 extends that was ~320k calls to NTuple_ClusterGroupRecord.serialize and roughly a third of the wall time. Caching the bytes removes the quadratic term.

Measured

Repeated extend of an int64 RNTuple, local file sink:

	before	after
5 columns, 1600 extends	2.00 s, 12803 flushes	1.24 s, 1602 flushes
50 columns, 100 extends	0.48 s, 5303 flushes	0.46 s, 102 flushes

The 50-column row shows the flush reduction rather than a wall-clock win, because a local flush is cheap; that is the case remote sinks care about.

File contents are byte-for-byte unchanged.

Not addressed

The bytes written for footers still grow quadratically (the footer is rewritten in full on every extension, and it grows by one cluster group record each time — ~38 kB after 800 extends). That is inherent to keeping a valid footer on disk after every extension; deferring it to close() would leave an unclosed file unreadable, so it seems like a design decision rather than something to slip into a perf PR.

I also left sink.set_file_length per blob rather than hoisting it, since skipping it mid-extend relies on writes past EOF zero-filling, which is guaranteed locally but not obviously so for every fsspec backend.

Tests

tests/test_1688_rntuple_extend_flushing.py: flush count per extend pinned at 1 for 1, 8 and 32 columns; a 20-step round trip; the record cache; and readability of the file between extends. 4 of the 6 fail on main.

Full suite passes locally (1029 passed, 90 skipped).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.88%. Comparing base (e598b61) to head (e74d621).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

Files with missing lines	Coverage Δ
src/uproot/writing/_cascadentuple.py	89.01% <100.00%> (+0.05%)	⬆️