Reduce test flakiness - #10809
Merged
MarkEWaite merged 4 commits intoJul 8, 2025
Merged
Conversation
https://app.launchableinc.com/organizations/jenkins/workspaces/jenkins/data/test-paths/class%3Dhudson.model.QueueTest%23%23%23testcase%3DinQueueTaskLookupByAPI?dateFilter=90d reports that inQueueTaskLookupByAPI has been flaky in th epast 90 days. The flakiness seems to be specific to Windows and seems to be related to the job not be in the queue quickly enough after the call toe schedule the build. If the job is not queued, sleep for 3 seonds to allow it time to queue. Testing done: Confirmed that the job queues consistently on my Linux computer, so the sleep statement is not executed.
Per https://developer.mozilla.org/en-US/docs/Web/CSS/resolution , `x` is an alias for `dppx`, so let's use `dppx` to avoid warnings from HTMLUnit.
https://app.launchableinc.com/organizations/jenkins/workspaces/jenkins/data/test-paths/class%3Dhudson.cli.ComputerStateTest%23%23%23testcase%3DtestUiForConnected?dateFilter=90d reports that there have been multiple failures of ComputerStateTest#testUiForConnected in the last 90 days. The failures seem to be related to a race condition that the agent is not yet disconnected when the state of the agent is checked. Wait for 1 second to allow the agent to disconnect before checking agent status. The average time to run the test is 34 seconds. This will increase that average time by 1 second. Not a significant increase compared to the benefit of avoiding a flaky test.
Contributor
Author
|
Since the first test run showed another flaky test, I added a The SCSS media query was causing HTMLUnit to report a warning that If that change causes objections because it is bundled inside this pull request, I'm happy to move it to another pull request. @janfaracik and @scherler, is it OK if I leave it in this pull request? |
https://app.launchableinc.com/organizations/jenkins/workspaces/jenkins/data/test-paths/class%3Dhudson.model.RunTest%23%23%23testcase%3DpreventXssInBadgeTooltip?dateFilter=90d shows there have been random test failures of preventXssInBadgeTooltip in the last 90 days. Reeduce failures by sleeping before the key assertions in order to allow Jenkins to complete the initialization of the page.
MarkEWaite
changed the title
scherler
approved these changes
Jul 7, 2025
jonesbusy
approved these changes
Jul 7, 2025
Contributor
Author
|
This PR is now ready for merge. We will merge it after approximately 24 hours if there is no negative feedback. /label ready-for-merge |
comment-ops-bot
Bot
added
the
ready-for-merge
krisstern
approved these changes
Jul 7, 2025
NotMyFault
approved these changes
Jul 7, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduce test flakiness
Launchable reports that inQueueTaskLookupByAPI has been flaky in the past 90 days. The flakiness seems to be specific to Windows and seems to be related to the job not being in the queue quickly enough after the call to schedule the build.
If the job is not queued, sleep for 3 seconds to allow it time to queue.
Since the first test run showed another flaky test, I added a Thread.sleep() to that flaky test, along with an explanation of the details. Refer to d5a6e2c for the details.
The SCSS media query was causing HTMLUnit to report a warning that
1xis not the right unit of measure for the media query. https://developer.mozilla.org/en-US/docs/Web/CSS/resolution says that x is an alias for dppx so I replaced 1x with 1dppx in the one place where it occurs. The HTMLUnit warning no longer appears.Detected while processing the 2.516.1 backporting pull request:
Testing done
Confirmed that the job queues consistently on my Linux computer, so the sleep statement is not executed.
Proposed changelog entries
Proposed changelog category
/label skip-changelog
/label internal
Proposed upgrade guidelines
N/A
Submitter checklist
@Restrictedor have@since TODOJavadocs, as appropriate.@Deprecated(since = "TODO")or@Deprecated(forRemoval = true, since = "TODO"), if applicable.evalto ease future introduction of Content Security Policy (CSP) directives (see documentation).Desired reviewers
N/A
Before the changes are marked as
ready-for-merge:Maintainer checklist
upgrade-guide-neededlabel is set and there is a Proposed upgrade guidelines section in the pull request title (see example).lts-candidateto be considered (see query).