Fix the two flaky CI tests: stale torus readback and random landscape parameters - #287
Merged
Merged
Conversation
… parameters TorusRenderIntegrationTest read the atlas back with glGetTexImage straight after drawing it. Under Mesa llvmpipe, which renders on worker threads, the readback sometimes returned the previous frame, so the capture with a worker assigned matched the capture without one (503 -> 503) and the marker check failed about one run in five at interface scale 3. glFinish() before the readback makes it wait for the frame. CustomGameSetupHarness failed about one run in twenty at three points: - After "Randomize parameters" it pressed Regenerate, which keeps the random draw, not Reset, which restores each landscape's own parameters, so the final resize checks ran on a random ridge layout that could fail validation (screen.validMap). - It applied the picked landscape without the picker's request, unlike the lobby's Use, so the lobby kept an earlier random draw and could roll a different map from the same seed (start positions differ). - It asserted one corner pixel was not black, but a colony start near the map corner puts its black number label on that pixel. It now checks the whole first row and column, skipping pixels under markers. Map generation itself is deterministic: the same request generated on the main thread, on a new thread and again after other maps matched for 300 requests across every landscape with random parameters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MN62g3sFzcFWiMzums3qge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two flaky tests have been failing CI runs on master and PRs whatever their code:
CustomGameSetupHarness(from #240) andTorusRenderIntegrationTest. Both are test bugs, not game bugs. This PR changes only the two test files.How it was reproduced
In a Docker container with CI's Ubuntu 24.04 packages, software GL and xvfb, running CI's exact commands in a loop on master 4ba685c:
torus-render-test(scale 1, then scale 3 in the same profile)CustomGameSetupHarness(visual)The full CI custom-game step also passes with the fix.
TorusRenderIntegrationTest: stale readbackThe failing check reads the torus atlas back with
glGetTexImageright after drawing it. Mesa's llvmpipe renders on worker threads, and the readback sometimes returned the previous frame:503 -> 503; passing runs are503 -> 639).LP_NUM_THREADS=0), the unchanged test passed 16 of 16.Fix:
glFinish()before the readback.CustomGameSetupHarness: random parameters leaking into later checksIt failed at three different assertions:
screen.validMap(line 1174 on master). After "Randomize parameters", the test pressed Regenerate, which keeps the random draw. It meant Reset, as its own comment says ("back to the landscapes' own parameters"). So the final resize ran on a randomrain-shadowridge layout, which could fail the lobby's validation ("The valleys are too narrow for the homes…"). Instrumented, the tile's parameters were non-default at that point in 6 of 6 runs. Fix: press Reset.CustomGameScreen.cpp:322) passes it, so without it the lobby kept the random draw it was given earlier. Fix: passpicker.chosenRequest(), and assert the options arrived.To rule out a real engine problem behind the start-position mismatch, I also generated 300 requests across every landscape with random parameters three ways: on the main thread, on a new thread, and again after other maps. All matched, so map generation is deterministic.
Limits
glReadPixelson the framebuffer being drawn, not a texture readback, but I didn't test whether it has a similar timing issue under llvmpipe.🤖 Generated with Claude Code
https://claude.ai/code/session_01MN62g3sFzcFWiMzums3qge