pwiz-sharp: added mzPeak format support (read, write, msconvert)

[chambm]

Summary

• Adds mzPeak (Apache Parquet tables packed in a ZIP) as a first-class MSData format in pwiz-sharp — reader, writer, IReader/format plumbing, and msconvert --mzpeak output.
• Full-document round-trip parity with mzMLb (metadata + binary) across all 8 vendor readers, gated in the vendor reader test harness (TestMzPeakRoundTrip).
• Cross-stack read support for mzPeak files written by the independent mzPeak.NET (Apache-Arrow) stack: type/width-adaptive column reads (uint8 ms_level, float32 scalars, int32/uint32/uint64 ints), Parquet-statistics row-group discovery with multi-group merge, and the separate wavelength_spectra_* (UV/DAD) entries appended after the MS spectra.
• Lazy per-row-group reads with bounded memory; parquet read in place from the ZIP (no extraction) when entries are Stored.

Test plan

• MsData.Tests - 97/97 (mzPeak column/IReader/round-trip + cross-stack read tests against committed mzPeak.NET fixtures)
• Thermo/Agilent/Bruker/Sciex/Waters/Mobilion/UIMF/UNIFI vendor reader tests - 112/112 (each includes the mzPeak round-trip)

See ai/todos/active/TODO-20260616_pwiz_sharp_mzpeak.md

Co-Authored-By: Claude noreply@anthropic.com

[chambm]

@mobiusklein I vibe coded this partly based on MzPeak.NET. Want to try it out?

[mobiusklein]

And I had just reinstalled Visual Studio too. This looks structurally correct, but I haven't run it yet. I know you're only prototyping so the null marking, chunked layout and so on aren't there yet, which is where a lot of the complexity lives. I saw the comments say something about no async runtime. I had to absorb the async dependency from ParquetSharp's Arrow interop layer. Is this a no-go scenario for you?

…

On Wed, Jun 17, 2026 at 4:34 PM Matt Chambers ***@***.***> wrote: *chambm* left a comment (ProteoWizard/pwiz#4311) <#4311 (comment)> @mobiusklein <https://github.com/mobiusklein> I vibe coded this partly based on MzPeak.NET. Want to try it out? — Reply to this email directly, view it on GitHub <#4311?email_source=notifications&email_token=AAK4E6NUAMQSMOSH6QKHRCT5AL6GXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINZTGUYDONJUGEY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4735075411>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAK4E6NVI5OADKNCVH5OOL35AL6GXAVCNFSNUABFKJSXA33TNF2G64TZHMYTENBZGUZDKNZXHNEXG43VMU5TINRYGY4DENZSHE22C5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AAK4E6NQGGVOJWPW3BHVZNT5AL6GXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINZTGUYDONJUGEY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android <https://github.com/notifications/mobile/android/AAK4E6LKCHKE42SSTAXJW5T5AL6GXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINZTGUYDONJUGEY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>

[chambm]

And I had just reinstalled Visual Studio too. This looks structurally correct, but I haven't run it yet. I know you're only prototyping so the null marking, chunked layout and so on aren't there yet, which is where a lot of the complexity lives. I saw the comments say something about no async runtime. I had to absorb the async dependency from ParquetSharp's Arrow interop layer. Is this a no-go scenario for you?

Right, we don't use async/await in Skyline as a rule due to its tendency to use the system thread pool which makes non-instrumented handle and leak detection annoyingly tricky. I did defer the chunked implementation but I'll kick it off now. This format seems really complicated to me: it'll be interesting to see how much traction it gets in the community.

[mobiusklein]

Thank you for explaining. If the async was isolated to be entirely within a SpectrumList implementation and/or run on a same thread-TaskScheduler, would that be acceptable? The same thread scheduler seems like it would be safer for leak avoidance, but I have not dealt with this issue myself.

The only reason the original code used async anywhere was because ParquetSharp only lets you read Arrow arrays via an async enumerator interface they implement from the .NET Apache Arrow library, and that leaks into everything else. My attempts to write nesting depth-independent column readers against ParquetSharp weren't successful, but it was comparatively easy using Arrow's Visitor API. The Arrow API doesn't directly use the .NET Arrow library to do the conversion, that gets done in the C++ library bindings, but getting the Arrow arrays out is more complicated.

Which part is most complicated seeming to you?