Skip to content

skyline: Port the native waters_connect/UNIFI readers to IdentityModel 7 - #4637

Merged
chambm merged 7 commits into
masterfrom
Skyline/work/20260902_identitymodel7
Sep 9, 2026
Merged

skyline: Port the native waters_connect/UNIFI readers to IdentityModel 7#4637
chambm merged 7 commits into
masterfrom
Skyline/work/20260902_identitymodel7

Conversation

@chambm

@chambm chambm commented Sep 3, 2026

Copy link
Copy Markdown
Member

Why

Updating Shared/Lib/IdentityModel.dll to 7.0.0.0 broke the native build. IdentityModel 7 removed the TokenClient constructor overloads and the AuthenticationStyle enum, and only the managed callers had been ported — UnifiData.cpp and WatersConnectData.ipp still built their token request the 3.9 way, so pwiz_vendor_api_unifi failed to compile. That cascades: pwiz_data_cli.dll, msconvert.exe and BlibBuild.exe all link that library, so none of them built either.

Sharing the request instead of restating it

OAuthPasswordGrantClient moves from CommonMsData down into CommonUtil, and both native readers now call it. This keeps the POST, the Basic authorization header and the protocol-error-vs-HTTP-error classification in exactly one place rather than growing a second C++/CLI copy.

CommonUtil is the right home because it has no ProjectReference of its own. Referencing CommonMsData from a native target would have closed a cycle back through ProteowizardWrapperpwiz_data_cli → this very library.

UnifiAccount/UnifiSession also move onto HttpClientWithProgress, matching the shape #4613 established for waters_connect, and pick up using disposal along the way.

Build plumbing

pwiz.CommonUtil.dll is declared as a Jamfile target and referenced by target ID, not by path — <assembly> is a dependency feature whose values are virtual targets, so bjam builds it on demand.

It builds with MSBuild, not dotnet build: the SDK's MSBuild resolves cultures through ICU, which doesn't recognize the legacy zh-CHS of CommonUtil's satellite .resx files and so gives them the same manifest name as the neutral resources (MSB3577). Framework/VS MSBuild still uses NLS.

System.Text.Json.dll joins the vendored assemblies in Shared/Lib. IdentityModel 7 dropped Newtonsoft and parses the token response with it, so it's a plain runtime dependency for every consumer of the UNIFI reader — it sits alongside the other runtime-only entries already in .shared-assemblies (System.Buffers, Microsoft.Bcl.AsyncInterfaces, and so on).

msvc.jam

<assembly> was declared a dependency feature but never behaved like one. Every value had always been a prebuilt file, so nothing noticed — until one was generated by the build: set-assemblies put the paths into /FU without declaring an edge, and the compile raced ahead of the DLL (fatal error C1192: #using failed). copy-assemblies in the same file already declares that edge for the copy it makes; this adds the matching one for the compile.

Test fix

Skyline opens one GraphChromatogram per replicate. FindOpenForm deliberately asserts the form is unique, so the existing code only ever worked for a single-file import and failed TestWatersConnect with Multiple GraphChromatogram forms open simultaneously as soon as a second file was imported.

Each replicate's graph is now looked up by name, and the graphs are tiled first — a chromatogram graph that isn't showing draws no curves, so without tiling the second replicate reports CurveCount 0. Replicate names come from the document rather than _filenames, because ImportResultsNameDlg strips the common prefix and suffix (ID33140_03a_…/ID33141_03a_… become replicates named 0 and 1).

The curve count is now an explicit per-dataset expectation. The old CurveCount == _filenames.Length compared two unrelated quantities that happened to coincide.

Testing

Full native build green, from this branch's own source rather than borrowed binaries: pwiz_data_cli.dll, msconvert.exe, BlibBuild.exe, Skyline-daily.exe.

TestWatersConnect passes, verified over repeated runs against the live server. Authentication was confirmed end to end — no auth errors, tokens issued by the real identity server.

TestUnifi still fails, both before and after this change: it reaches the curve check while the live import is still running (AllChromatogramsGraph at 10%, No final cache), because WaitForDocumentLoaded() isn't covering the remote import. TestWatersConnect survives the same race only because its import finishes inside the 5s wait. Left alone here as a pre-existing, separate issue.

Also includes the stale RemoteApi path fix for the VCS trigger config (cherry-picked from #4632's 48dff2020d), without which changes to these very files don't trigger the TestConnected/Container CI.

🤖 Generated with Claude Code

https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr

…del 7

IdentityModel 7 removed the TokenClient constructor overloads and the
AuthenticationStyle enum, so pwiz_vendor_api_unifi no longer compiled once
Shared/Lib/IdentityModel.dll was updated: only the managed callers had been
ported, and the C++/CLI readers still built their token request the 3.9 way.

Rather than restate the request shape a second time in C++/CLI, move
OAuthPasswordGrantClient down from CommonMsData into CommonUtil and call it
from both native readers. CommonUtil has no ProjectReference of its own, so a
native target can reference it without the cycle that CommonMsData would
create back through ProteowizardWrapper -> pwiz_data_cli -> this library.
UnifiAccount/UnifiSession also move onto HttpClientWithProgress, matching the
shape #4613 already established for waters_connect.

pwiz.CommonUtil.dll is declared as a Jamfile target and referenced by target
ID: <assembly> is a dependency feature whose values are virtual targets, so
bjam builds it on demand. It is built with MSBuild rather than "dotnet build"
because the SDK's MSBuild resolves cultures through ICU, which does not know
the legacy zh-CHS of CommonUtil's satellite .resx files and fails with
MSB3577; Framework MSBuild still uses NLS. System.Text.Json.dll joins the
vendored assemblies in Shared/Lib as a plain runtime dependency of
IdentityModel 7, which parses the token response with it.

msvc.jam: <assembly> declared a dependency but never acted like one. Every
value had always been a prebuilt file, so nothing noticed until one was
generated: set-assemblies put the paths in /FU without declaring an edge, and
the compile raced ahead of the DLL (C1192). copy-assemblies already declares
the same edge for its copy.

UnifiFunctionalTest: Skyline opens one GraphChromatogram per replicate, so
FindOpenForm, which asserts the form is unique, only ever worked for a
single-file import and failed TestWatersConnect once a second file was
imported. Look each replicate's graph up by name instead, and tile the graphs
first, since a chromatogram graph that is not showing draws no curves. The
curve count becomes an explicit per-dataset expectation; the old
_filenames.Length matched only by coincidence.

TestWatersConnect passes. TestUnifi still fails, before and after this change,
waiting on a live import that has not finished when the curve check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
Copilot AI lite review requested due to automatic review settings September 3, 2026 20:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The updated UNIFI HTTP call sites should dispose HttpResponseMessage instances returned by SendRequest(), and there’s also a concrete XML documentation mismatch in the newly added OAuth helper.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR restores the native UNIFI / waters_connect reader build after upgrading the vendored IdentityModel assembly to v7 by centralizing the OAuth password-grant token request logic in a shared managed helper that both managed and C++/CLI callers can use. It also updates related UNIFI managed code to use HttpClientWithProgress consistently, fixes a functional test that assumed only one chromatogram graph form, and adjusts build/CI plumbing to correctly build and stage the new managed dependency.

Changes:

  • Introduces OAuthPasswordGrantClient in CommonUtil and ports UNIFI/waters_connect native readers + managed account authentication to use it (IdentityModel 7-compatible).
  • Updates UNIFI managed session/account HTTP calls to use HttpClientWithProgress and improves WatersConnect auth error message fallback behavior.
  • Fixes UnifiFunctionalTest replicate graph handling; updates VCS trigger path mappings; fixes Boost.Build MSVC assembly dependency edges and adds a Jamfile target to build pwiz.CommonUtil.dll.
File summaries
File Description
scripts/misc/vcs_trigger_and_paths_config.py Updates path→target mappings so WatersConnect/Unifi/RemoteApi changes trigger the intended CI targets.
pwiz_tools/Skyline/TestConnected/UnifiFunctionalTest.cs Fixes multi-replicate chromatogram graph lookup and curve-count expectations; guards waters_connect-only test assertions.
pwiz_tools/Shared/CommonUtil/SystemUtil/OAuthPasswordGrantClient.cs Adds a shared OAuth password-grant token request helper that returns IdentityModel TokenResponse under IdentityModel 7 constraints.
pwiz_tools/Shared/CommonUtil/CommonUtil.csproj Adds IdentityModel reference and includes the new OAuth helper source file.
pwiz_tools/Shared/CommonMsData/RemoteApi/WatersConnect/WatersConnectAccount.cs Uses shared OAuth helper for token requests and improves error message fallback for certain OAuth error types.
pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiSession.cs Switches UNIFI folder/file listing to HttpClientWithProgress.SendRequest with explicit request objects.
pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiAccount.cs Replaces IdentityModel TokenClient usage with shared OAuth helper and returns an authenticated HttpClientWithProgress.
pwiz_aux/msrc/utility/vendor_api/UNIFI/WatersConnectData.ipp Ports native waters_connect token acquisition to shared OAuth helper (removes TokenClient).
pwiz_aux/msrc/utility/vendor_api/UNIFI/UnifiData.cpp Ports native UNIFI token acquisition to shared OAuth helper (removes TokenClient / AuthenticationStyle).
pwiz_aux/msrc/utility/vendor_api/UNIFI/Jamfile.jam Adds build target for pwiz.CommonUtil.dll via MSBuild and wires it into <assembly> dependencies.
libraries/boost-build/src/tools/msvc.jam Ensures /FU assembly references depend on generated assembly targets to prevent compile races.
Review details

Suppressed comments (2)

pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiSession.cs:68

  • The HttpResponseMessage returned by SendRequest() is IDisposable; it should be disposed to promptly release the underlying response content/connection resources. Wrap the response in a using declaration before reading Content.
            var response = httpClient.SendRequest(request);

pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiAccount.cs:160

  • The HttpResponseMessage from SendRequest() should be disposed (IDisposable) to avoid holding response resources longer than necessary. Use a using declaration for the response before reading Content.
            var response = httpClient.SendRequest(request);
  • Files reviewed: 11/13 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiAccount.cs Outdated
Comment thread pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiSession.cs Outdated
Comment thread pwiz_tools/Shared/CommonUtil/SystemUtil/OAuthPasswordGrantClient.cs Outdated
Comment thread pwiz_tools/Shared/CommonUtil/SystemUtil/OAuthPasswordGrantClient.cs Fixed
chambm and others added 2 commits September 3, 2026 17:50
* Dispose the HttpResponseMessage returned by SendRequest at the four UNIFI
  call sites, which already disposed the client and the request but not the
  response.
* Moved the orphaned <summary> off PasswordGrantForm and onto RequestToken,
  which it describes and which had none. Reordering the methods had left two
  summaries stacked on one of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
Referencing pwiz.CommonUtil.dll and System.Text.Json.dll as <assembly> put both
into INSTALLER_VENDOR_FILES.txt, which every installer expands into Components.
Both were already declared explicitly, so each ended up declared twice and the
WiX link failed with LGHT0091 (duplicate symbol), taking down Core Windows,
Bumbershoot Windows and Skyline Windows; the Docker container failure was a
snapshot dependency on the first. The no-vendor-DLLs build passed throughout,
which is the tell: the UNIFI reader only builds when vendor DLLs are present.

pwiz.CommonUtil.dll is ours rather than a vendor redistributable and all three
installers name it directly, so it is excluded from the enumeration.
System.Text.Json.dll keeps coming from the vendor list, because the pwiz
installer has no declaration of its own and msconvert needs it at run time to
authenticate; instead it is removed from Skyline's WXS template, which is the
convention the generator already documents ("DO NOT ADD TO THE WXS TEMPLATE!")
and that IdentityModel.dll already follows.

Also the three ReSharper warnings in code this branch touched: two using
directives left unused once the token request moved to OAuthPasswordGrantClient,
and an object initializer inside a using declaration, where a throw constructing
the content would have leaked the response it was initializing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
Comment thread pwiz_tools/Shared/CommonUtil/SystemUtil/OAuthPasswordGrantClient.cs Dismissed
The vendored copy was 8.0.0.0 (package 8.0.0) while Skyline resolves 8.0.0.5
(package 8.0.5) through NuGet, so once the installer actually linked and the
admin-installer tests ran against an installed Skyline, binding failed both ways
- "the located assembly's manifest definition does not match the assembly
reference" for 8.0.0.5 and for 8.0.0.0 - and every test that reflects over the
Skyline assembly threw ReflectionTypeLoadException. Vendoring 8.0.5 leaves one
version in the tree.

ArdiaLoginDlg: IdentityModel 7 annotates DeviceAuthorizationResponse.DeviceCode
as nullable, which makes assigning it to DeviceTokenRequest.DeviceCode the one
inspection warning this branch added over master (master inspects clean at 0).
Passed on unchanged so a missing device code still surfaces as the server's own
error rather than a different one from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
@chambm
chambm requested a review from rita-gwen September 4, 2026 14:05
@chambm

chambm commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

@rita-gwen See what you think about this. It should supersede #4632 and will make things a bit easier when merging into the .NET port I think.

Comment thread pwiz_tools/Shared/CommonUtil/SystemUtil/OAuthPasswordGrantClient.cs
Comment thread pwiz_tools/Shared/CommonMsData/RemoteApi/Unifi/UnifiAccount.cs
@rita-gwen

Copy link
Copy Markdown
Contributor

Looks good. I tested it against the existing WatersConnect server, works fine. Did you consider adding auth token caching to UNIFI same way it is done for w_c? It would save some unnecessary authentication calls.

@chambm

chambm commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

Looks good. I tested it against the existing WatersConnect server, works fine. Did you consider adding auth token caching to UNIFI same way it is done for w_c? It would save some unnecessary authentication calls.

Not really, I consider UNIFI to be legacy code that won't see much maintenance going forward unless Waters or a user asks for it. Thanks for your review comments!

chambm and others added 2 commits September 8, 2026 15:51
* Guarded the token-response rebuild in OAuthPasswordGrantClient.RequestToken, so a
  failure there returns an error TokenResponse instead of escaping the method
* Made UnifiAccount.GetAuthenticatedHttpClient report the identity server's own error
  when the token request fails, instead of sending an empty bearer token

See TODO-20260902_identitymodel7_master_backport.md in pwiz-ai/todos

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Gave each MSConvertGUI MSBuild invocation its own intermediate directory, so csc
  stops compiling CommonUtil onto the path copy-assemblies hardlinks
  pwiz.CommonUtil.dll into - the intermittent CS2012 failing bt83
* Pinned PwizBuildPath, which MSConvertGUI.csproj derives from BaseIntermediateOutputPath

See TODO-20260902_identitymodel7_master_backport.md in pwiz-ai/todos

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@chambm
chambm merged commit 79454aa into master Sep 9, 2026
23 checks passed
@chambm
chambm deleted the Skyline/work/20260902_identitymodel7 branch September 9, 2026 16:18
chambm added a commit that referenced this pull request Sep 9, 2026
Master now carries the IdentityModel 7 work (#4637, merged as 79454aa), so
where both sides had independently solved the same problem this takes master's
shared implementation and keeps the branch's net8-specific knowledge.

Auth: both sides replaced the TokenClient constructors IdentityModel 7 removed,
the branch with the RequestPasswordTokenAsync/RequestRefreshTokenAsync
extensions, master with OAuthPasswordGrantClient. Master's wins because the
native C++/CLI UNIFI/waters_connect readers call the same helper, and it drops
the IHttpClientFactory pipeline and the two mock handler classes master deleted.
The branch's Task.Run existed to keep the async token call off the UI thread;
the shared helper's request is synchronous, so nothing regresses. Kept the
branch's comments on why IdentityModel 7 populating TokenResponse.Raw makes the
invalid_scope and invalid_grant branches reachable, minus the net472-versus-net8
contrast that no longer holds now master is on IdentityModel 7 too.

UnifiFunctionalTest: took master's graph handling, which tiles the graphs and
asserts an explicit per-replicate curve count, over the branch's
SelectedGraphChromName approach; the tiling call had already merged into the
surrounding code. The branch's socket-derived ConnectionRefusedMessage and
DnsResolutionFailedMessage are outside the conflicts and survive - they are
required, since .NET 8 surfaces the raw Winsock text where net472 raised a
WebException. Master also guards the invalid-password assertion behind the
waters_connect check, because that message is Waters wire text a Unifi server
need not match.

csproj: kept the branch's SDK-style projects, whose globs cover master's
explicit Compile and EmbeddedResource items, and ported the one semantic
addition - the IdentityModel reference CommonUtil now needs for
OAuthPasswordGrantClient. Master's Microsoft.Bcl.AsyncInterfaces pin is not
carried: it exists for net472 app.config binding redirects and CommonMsData is
net10.0-windows only here. .gitignore keeps the branch's unanchored entries and
adds master's rolled-log pattern.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants