This repository is a monorepo: the shared GrumPHP configuration in the root and several packages in packages/. CONTRIBUTING.md describes the layout, the checks and the release process. Read it first.
- Run
vendor/bin/grumphp runin the root and in every package you changed, see Running the checks. - Commit subjects use the TYPO3 prefixes
[FEATURE],[BUGFIX]and[TASK].
When you change code or tests in packages/rector-p:
- Run the tests:
vendor/bin/phpunitinpackages/rector-p(they also run as part ofvendor/bin/grumphp runthere). - Run Infection as described in Mutation testing:
- with
--only-covering-test-cases, - with the default settings, never with
--with-uncovered.
- with
- The score must stay at or above
minMsiinpackages/rector-p/infection.json5. If it went up, raiseminMsito the new value, rounded down. Never lower it. - Read the escaped mutants in
var/infection/infection.log. Write or sharpen tests for real gaps. Equivalent or purely cosmetic mutants do not need to be killed; if they are cosmetic output, add a narrow ignore with a comment toinfection.json5(see below). - Do not call
exit()insrc/; throwStopException. The tests runPartialCommandin the test process. - Every test class needs
#[CoversClass]for the class it tests (#[CoversNothing]for tests that only start the binary). This keeps Infection fast. - New tests of
PartialCommanduse the fake Rector intests/Fixture/bin/rector(the default inPartialCommandTest). CalluseRealRector()only when the test is about working with the real Rector.
Lessons from setting up mutation testing for rector-p:
#[CoversClass]plus--only-covering-test-casesmake the run fast. Without#[CoversClass]PHPUnit credits every line a test executes to that test.PartialCommandTeststarts real Rector runs and usesCache,ChunkConfigandTimeHelper, so every mutation in those small classes also ran the slow command tests. Without--only-covering-test-casesInfection runs the whole test class instead of the covering test methods. With both, a full run took about 20 minutes on 4 cores (363 mutations) while every test still used the real Rector; without them it was on track for about 40 minutes.- Only code that runs in the PHPUnit process can kill mutations. A test that starts
rector-pas its own process (RectorPBinaryTest) is useful, but it never sees mutated code. That is whyPartialCommandTestruns the command in-process and whysrc/must not callexit(). If a behaviour depends on what another process left behind, prepare that state in the test and run rector-p in-process afterwards:testListsAllFilesWithoutTouchingTheResultCacheOfRectorlets the real Rector fill its result cache first.RectorPBinaryTestchecks the same with separate processes, like a user. - Check
var/infection/summary.logforSkipped. Infection skips a mutation if even the fastest covering test takes longer thantimeoutininfection.json5, and skipped mutations do not count in the score. A high score with many skipped mutations means nothing. - Coverage is limited to
src/by the<source>element inphpunit.xml.dist. Without it PCOV also measures Rector and PHPStan, and every test becomes much slower. - Infection writes its logs only at the end. If you stop a run early, you get no list of escaped mutations. Run it in the background and wait.
- A fake Rector makes the command tests fast. A real Rector run takes about a second, and
PartialCommandTestneeds several per test.tests/Fixture/bin/rectoranswers in milliseconds: a file withlist($a, $b)has changes,function broken(fails,fail on applyfails when the changes are applied. It logs every call torector-calls.login the project,ProjectFixture::rectorCalls()reads it. Only the threetestRealRector…tests,testListsAllFilesWithoutTouchingTheResultCacheOfRectorandRectorPBinaryTeststart the real Rector. Together with the ignores below, a full Infection run went from about 21 to about 6 minutes on 4 cores (384 mutations). - Ignored mutations are not run at all.
infection.json5ignores cosmetic mutations (style tags, the time estimate, the duration printed with-vvv) and a few that can not change the behaviour, each with a comment. A regex inignoreSourceCodeByRegexmatches a whole line of the original code, so it hides every mutation of that mutator on the line, also useful ones. Keep entries narrow (one mutator, one line or one method) and prefer a test where one is possible, like the regex for the time estimate intestFindsRectorDistPhpAndPassesItToEveryRectorRun. Never ignore whole classes or mutators to raise the score. The escaped mutants are left on purpose: style tag mutants (a removed</error>or</>) on lines that also have useful mutations,$matches[1]→$matches[0]inparseChunkConfig(), which is equivalent ((int)'2/3'is 2), and removingmkdir()of the temporary directory infindFiles(): the installed Rector creates it as well, only Rector 2.2 with its in-memory cache in CI does not. - Mutations run real code. A mutated path can create directories anywhere: mutations of the temporary directory name created directories in
/. Ignore mutations of such names. - Infection runs several test processes at once. Tests must not compare or clean shared places like
/tmp. Once, one test removed the directory of another, so a mutation escaped by chance and others were killed by chance. Use the directories ofProjectFixture, every test has its own. - A test that fails for its own reason kills mutations by mistake. Infection replaces PHP's
filestream wrapper, so a file that PHP opens is no file descriptor forproc_open(), and a test that started Rector that way failed for every mutation. Start helper processes through the shell (exec()with redirection) and check their exit code. After adding tests, run Infection once with--log-verbosity=alland check invar/infection/infection.logthat killed mutants fail because of the mutation.