skyline: Fixed the small-molecule conversion tests racing the library loader - #4631
Conversation
… loader
RefineConvertToSmallMoleculesTest and RefineConvertToSmallMoleculeMassesAndNamesTest
fail whichever one runs first in a TestRunner process, and pass on every later run:
TransitionGroupChromInfo results differ:
YLGAYLLATLGGNASPSAQDVLK Charge 2 vs pep_YLGAYLLATLGGNASPSAQDVLK Charge [M+2H]
[0] [0] LibraryDotProduct 0.72051907 vs (null)
ConvertToSmallMolecules only converts libraries when
Settings.PeptideSettings.Libraries.IsLoaded, which is the right call for a document
that never loaded one. But AsSmallMoleculeTestUtil gets its document from
ResultsTestDocumentContainer.AssertComplete, and that only inspects the chromatogram
loader's progress - it happily returns while the library manager is still working. On
a cold cache the conversion therefore reads IsLoaded as false, skips the library
branch, and produces a document with no library dot product while the original has
one. On a warm cache the library is already there and everything matches.
So these tests were not just flaky, they were silently not testing library conversion:
in the runs that failed, no worm.1.1.converted_to_small_molecules.blib was written at
all and the converted precursors had no bibliospec_spectrum_info.
Wait for the libraries before converting. A document with no libraries reports itself
loaded, so this is a no-op everywhere else.
Verified by reproducing the order dependence and then re-running both orderings: before
the change, whichever test ran first failed and only the second wrote a converted .blib;
after it, all orderings pass and both modes write one (masses_only correctly still does
not, since the mode is excluded by design).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
There was a problem hiding this comment.
Pull request overview
This PR addresses an order-dependent flake in Skyline’s small-molecule conversion tests by ensuring spectral libraries have finished loading before conversion/comparison logic runs, so library conversion paths are exercised consistently.
Changes:
- Added a
ResultsTestDocumentContainer.WaitForLibrariesLoaded()helper with a longer timeout tailored to library loading. - Updated
AsSmallMoleculeTestUtil.ConvertToSmallMolecules()to wait for libraries to load after document load completion, closing the race with the library loader.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| pwiz_tools/Skyline/TestUtil/ResultsUtil.cs | Adds a dedicated wait helper for spectral library load completion in test document containers. |
| pwiz_tools/Skyline/TestUtil/AsSmallMoleculeTestUtil.cs | Uses the new wait to ensure libraries are loaded before running small-molecule conversion comparisons. |
Suppressed comments (2)
pwiz_tools/Skyline/TestUtil/ResultsUtil.cs:261
GetWaitCycles()can return 0 whenmillis < SLEEP_INTERVAL, which causes callers to skip the condition check loop and immediatelyAssert.Fail()even if the condition is already satisfied. Since this helper is used by the new library-load wait, it should be robust for all inputs.
This issue also appears on line 293 of the same file.
public const int WAIT_TIME_LIBRARIES = 60 * 1000; // 60 seconds
private static int GetWaitCycles(int millis = WAIT_TIME)
{
return millis / SLEEP_INTERVAL;
pwiz_tools/Skyline/TestUtil/ResultsUtil.cs:297
WaitForLibrariesLoadednever re-checksLibraries.IsLoadedafter the finalSleep. If the libraries finish loading during the last sleep interval, the method can still fail even though the condition is true by the time it reaches theAssert.Fail(). Add one last check after the loop before failing.
{
if (Document.Settings.PeptideSettings.Libraries.IsLoaded)
return;
Thread.Sleep(SLEEP_INTERVAL);
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
* WaitForLibrariesLoaded re-checks IsLoaded once after the wait loop before failing. The loop sleeps after its final check, so libraries finishing during that last interval produced a spurious failure - the same shape of race the method exists to close. The post-loop check also covers a timeout shorter than SLEEP_INTERVAL, which yields zero cycles and would otherwise fail without ever evaluating the condition. Left GetWaitCycles alone: hardening it there would change behaviour for the pre-existing WaitForProcessing caller, and no caller passes a sub-interval timeout today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr
|
Addressed both suppressed suggestions from the Copilot review in 214fc5f.
I deliberately did not harden Re-verified both orderings after the change — |
|
Not needed with #4629 fixes. |
|
It seems this race still exists even with 4629. This fixes it according to my testing. |
The failure
RefineConvertToSmallMoleculesTestandRefineConvertToSmallMoleculeMassesAndNamesTestfail whichever one runs first in a TestRunner process, and pass on every later run in that process:Reversing the order moves the failure to the other test, so it is ordering, not mode:
.blibwrittenformulas, thenmasses_and_namesmasses_and_namesformulasmasses_and_names, thenformulasformulasmasses_and_namesCause
RefinementSettings.ConvertToSmallMoleculesconverts libraries only when the document reports them loaded:That is the right call for a document that never loaded a library. The problem is on the test side:
AsSmallMoleculeTestUtil.ConvertToSmallMoleculestakes its document fromResultsTestDocumentContainer.AssertComplete(), which only inspects the chromatogram loader's progress and returns while the library manager is still working.So on a cold cache the conversion reads
IsLoadedas false, skips the library branch, and yields a converted document with no library dot product while the original has one. On a warm cache the library is already present and the two agree.This was a coverage hole, not only a flake
In the runs that failed, no
worm.1.1.converted_to_small_molecules.blibwas written at all and the converted precursors carried no<bibliospec_spectrum_info>. Library conversion was not being exercised in those runs — the test failed only because the two sides then disagreed about a dot product.Fix
Wait for the libraries before converting. A document with no libraries reports itself loaded, so this is a no-op everywhere else.
Verification
Reproduced the order dependence, then re-ran both orderings after the change: all orderings pass, and both
formulasandmasses_and_namesnow write a converted.blibregardless of order.masses_onlystill writes none, which is correct — that mode is excluded by themode != masses_onlyterm.Built and run on the .NET 10 port branch (#4619), where all three touched files are byte-identical to
master; this PR exists to get it exercised onmasteritself. Every API it uses is already used in the same file onmaster.🤖 Generated with Claude Code
https://claude.ai/code/session_0182oRXbeKQGEpF1kTkdQAsr