Version: v1.17 (1331)
Steps
- Run tests repeatedly in Xcode Cloud
Expected results
ProfilesManagerTests should pass every single time.
Actual results
Sometimes tests fail with
Test Failure
ProfilesManagerTests.testStreamProfileYieldsCachedProfileByDefault()
XCTAssertNotNil failed - Profile should be cached in NDB
ProfilesManagerTests.swift:46
Test Failure
ProfilesManagerTests.testStreamProfileYieldsCachedProfileByDefault()
XCTAssertEqual failed: ("nil") is not equal to ("Optional("testuser")")
ProfilesManagerTests.swift:47
Analysis
I attempted to reproduce this issue on a local machine, by running the relevant tests 100 times, and those have passed all 100 times, indicating that there is flakiness on the test (rather than an issue with the feature itself). Either:
- Something about the CI environment causes the issue
- Some interaction between two concurrently running tests is causing the failure (i.e. an issue with test isolation)
Version: v1.17 (1331)
Steps
Expected results
ProfilesManagerTestsshould pass every single time.Actual results
Sometimes tests fail with
Analysis
I attempted to reproduce this issue on a local machine, by running the relevant tests 100 times, and those have passed all 100 times, indicating that there is flakiness on the test (rather than an issue with the feature itself). Either: