Skip to content

testStreamProfileYieldsCachedProfileByDefault flakiness in CI #3791

Description

@danieldaquino

Version: v1.17 (1331)

Steps

  1. Run tests repeatedly in Xcode Cloud

Expected results

ProfilesManagerTests should pass every single time.

Actual results

Sometimes tests fail with

Test Failure
ProfilesManagerTests.testStreamProfileYieldsCachedProfileByDefault()
XCTAssertNotNil failed - Profile should be cached in NDB
ProfilesManagerTests.swift:46
Test Failure
ProfilesManagerTests.testStreamProfileYieldsCachedProfileByDefault()
XCTAssertEqual failed: ("nil") is not equal to ("Optional("testuser")")
ProfilesManagerTests.swift:47

Analysis

I attempted to reproduce this issue on a local machine, by running the relevant tests 100 times, and those have passed all 100 times, indicating that there is flakiness on the test (rather than an issue with the feature itself). Either:

  • Something about the CI environment causes the issue
  • Some interaction between two concurrently running tests is causing the failure (i.e. an issue with test isolation)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions