[ci] Windows test support#4885
Draft
gamesh411 wants to merge 10 commits into
Draft
Conversation
gamesh411
changed the title
gamesh411
force-pushed
the
windows-test-support
branch
14 times, most recently
from
8081c64 to
7219423
Compare
8 tasks
gamesh411
force-pushed
the
windows-test-support
branch
14 times, most recently
from
332b945 to
d69c7ab
Compare
gamesh411
force-pushed
the
windows-test-support
branch
10 times, most recently
from
6211588 to
3d9de0e
Compare
gamesh411
force-pushed
the
windows-test-support
branch
9 times, most recently
from
ed482b6 to
5250e76
Compare
Replace the custom compatibility.multiprocessing shim with direct imports from the multiprocess package. On macOS and Windows, the default fork start method is unsafe (Obj-C runtime crashes) or unavailable, so set spawn explicitly in cli.py:main(). Spawn workers do not inherit parent state, so analysis_manager and pre_analysis_manager now explicitly set up logging in spawned workers, and log_parser's _init_log_parser_worker propagates the shared compiler_info dict to children. Add mypy override for multiprocess (no type stubs available).
On macOS/Windows (spawn), worker processes cannot inherit the HTTPServer instance or shared Queue/Value objects from the parent. Reconstruct the server in spawned API workers from serializable config (_build_worker_server) and use SyncManager.Queue() for the task queue so it can be pickled to spawned task workers. Spawned workers detect shutdown via SIGINT (not a shared flag), and the cwd is validated before spawning to avoid os.getcwd() failures if the inherited cwd was deleted.
The JSON compilation database spec allows entries to use either 'command' (string) or 'arguments' (list). The intercept-build wrapper used on macOS (bear) produces the 'arguments' form. Normalize 'arguments' to 'command' in tu_collector wherever the build action command is accessed, so both formats work on all platforms.
Add CC_TEST_API_WORKERS and CC_TEST_TASK_WORKERS env vars that map to --api-handler-processes and --task-worker-processes server flags. On macOS CI, spawn workers are expensive (each re-imports the full Python stack), so constraining to 1 worker reduces resource pressure. Also improve wait_for_server_start with an HTTP fallback check (port-based readiness) and fail-fast diagnostics for common startup errors.
Analyzer tests:
- Use os.path.realpath('/tmp') to handle macOS /tmp -> /private/tmp
symlink in log_parser and option_parser tests
- Skip LD_PRELOAD/LD_LIBRARY_PATH tests on macOS (not applicable)
- Skip gcc analyzer test when g++ is an Apple clang shim
- Skip -m32 test on macOS (not available)
- Use -fsyntax-only in buildcmd escaping tests (linking may fail
with Homebrew clang on macOS)
- Normalize intercept-build vs ld-logger output strings
- Normalize gcc/infer diagnostic quoting and type annotations
across versions for cross-platform comparison
- Add self.fail() in test_skip for tu_collector errors
- Guard 'command' key access for arguments-format entries
Web tests:
- OAuth mock server: bind to 127.0.0.1, poll for readiness
instead of fixed sleep, track and terminate the process in
teardown, use sys.executable instead of 'python3'
- blame tests: restore cwd in finally block to prevent test
pollution on assertion failure
- task_management tests: poll for state transitions instead of
fixed sleeps (spawn workers take ~42s to start on macOS),
use 5s tasks to reliably observe RUNNING state
- Makefile: use $(ROOT)/build for workspace and CodeChecker cmd
paths (consistent with __init__.py's hardcoded PKG_ROOT)
- env.py: no functional change
Add install-deps-macos.sh that installs llvm@14, gcc@13, cppcheck, bear, and Infer (from GitHub releases, no Homebrew formula). The script creates an intercept-build wrapper around bear (LLVM's libear.dylib is broken on macOS ARM64 with SIP), gcc/g++ symlinks matching Linux naming, and sets SDKROOT/MACOSX_DEPLOYMENT_TARGET/ CC/CXX to ensure correct compilation and SARIF output. Add macOS matrix entries to the analyzer and tools jobs, and a separate web-macos job (sqlite only, with reduced worker counts via CC_TEST_API_WORKERS/CC_TEST_TASK_WORKERS). All macOS jobs use continue-on-error as they are experimental.
Skip clang-diagnostic checker tests when diagtool is unavailable (the Windows LLVM package ships no diagtool), fix an infinite loop in the compilation-database directory walk on Windows path roots, and make the remaining analyzer unit tests platform-portable.
Guard the Unix-only signals used for graceful shutdown and the forced synchronous store: fall back when signal.SIGHUP / signal.SIGUSR1 are unavailable so the background task worker and the store watchdog do not crash on Windows. Also fix the store client's ZIP member paths for drive-absolute paths so stored reports resolve to their source files.
Build logging (CodeChecker log / check -b) is unsupported on Windows, so generate a compile_commands.json and run CodeChecker analyze directly. Invoke the CodeChecker entry point via the Python interpreter, pass stdin=DEVNULL so subprocesses cannot hang on an interactive prompt, and redirect the credential/session files into the workspace on Windows (where expanduser ignores HOME). Skip the LDAP unit tests when python-ldap is not installed.
Run the analyzer unit suite, the web unit suites, and the Windows- compatible web functional suites on Windows (continue-on-error). Build logging (CodeChecker log / check -b) is unsupported on Windows, so only functional tests that store results through check_and_store() (which uses CodeChecker analyze) are enabled. LLVM and Cppcheck are cached and pytest timeouts guard against hangs. install-deps-windows.sh installs the toolchain.
gamesh411
force-pushed
the
windows-test-support
branch
from
5250e76 to
cb6240a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add analyzer and web (sqlite) test jobs on windows-latest. Both use continue-on-error since Windows support is exploratory. Uses bash shell and choco for dependency installation.
This is an umbrella PR to test the Windows runners (like #4883). The first 3 commits in this PR are the same as those in #4883, and the other 4 are the Windows-related changes. I think I will try to create 3 PRs from these 4 commits to make the review easier.