Repository navigation
Lead the nightly mutation issue with its status and name it a report - #1244
Merged
Merged
Conversation
The nightly rewrote its issue with the survivor backlog alone, so a reader could not tell whether anything was wrong, and its title read like a defect list. gate.py gains --status, one line per suite with the score against its floor, baseline progress, the critical-module count (naming any below floor, unmeasured or still to measure) and the files whose score fell. The report job writes those lines under a Status heading, after a preamble that says the issue is a report and not a bug list, and before the survivors. The issue is retitled "Mutation nightly: survivors by module"; an open issue under the old title is renamed in place so no second issue is ever opened. The breach flag travels through a file rather than step output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014eZ98kaESE7YBa84fT1msc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The nightly rewrites its issue with the survivor backlog alone, so after run 28 (every floor holding) #1184 still read like an open defect list with nothing saying whether anything was wrong. This PR makes the issue a report: it opens with one status line per suite (score against its floor, baseline progress, critical-module count, files whose score fell), says in its preamble that it is not a bug list, and carries a title that says so. The workflow renames the existing #1184 in place, so no second issue is ever opened. Tooling only: no VERSION bump and no CHANGELOG entry, since CI tooling has no effect on the running product.
Type
Changes
scripts/mutation/gate.py: new--status PATH(written for--scope suiteonly) appends one Markdown bullet per suite fromstatus(): the score sentence (78.9% of 59,697 mutants killed; floor 69% — pass/**FAIL**; for a partial baselinebaseline 99% measured (383 mutants left for the next run); 67.7% of those measured killed; the 0% floor applies once complete;**incomplete run**when records are pending without--allow-pending), thenCritical modules: k of n at or above floor, naming any below its floor with score and floor, any floored module with nothing measured (check), and the number still to measure, then the files whose score fell since the previous run (before → now) or a line saying none did. The "which files fell" computation moved out ofregressions()intodropped_files(); theregressions()table is unchanged..github/workflows/mutation-nightly.yml: the score step passes--status status.mdfor each suite and writes the breach flag to afloors-breachedfile, which the issue step reads instead of step output. The issue body is now:harness.mdfirst when present (unchanged), a preamble (a nightly mutation report for the linked run; not a bug list, each entry is a change no test notices, a confirmed bug is filed as its own issue; the triage and floor rules; every run rewrites the body, so a note that must persist goes in a comment, a pragma or a floor),## Status(all floors holding, or a floor breached / records missing, then the suite lines),## Survivors by module(the existing list).ISSUE_TITLEbecomesMutation nightly: survivors by module;LEGACY_ISSUE_TITLEkeepsMutation survivors (nightly), and an open issue under it is edited with--title, so Mutation survivors (nightly) #1184 is renamed rather than duplicated.status.mdjoins themutation-survivorsartifact.scripts/mutation/README.mdandCLAUDE.md: the sentence that names the issue.Test Plan
backend/tests/core/test_mutation_scripts.py::TestGate(new cases):status()on a complete suite gives the pass line,1 of 1critical modules ok and no dropped-file line; a partial baseline reports the mutants left and a module still to measure; a breached module and a dropped file are both named with their numbers; a floored module with nothing measured is flagged; an unfinished run without--allow-pendingreads "incomplete run";regressions()renders the same table throughdropped_files();main --statuswrites the file for suite scope and not for diff scope.backend/tests/services/test_ci_workflow_mutation.py(new and updated pins): the new title and the legacy title; the issue step edits with--titleunder the legacy title and creates only when neither is open;--status status.mdis passed; the body carries the preamble,## Status, the breach-flag file,cat status.mdbeforecat survivors.md, andharness.mdbefore the preamble; no${{ steps.interpolation in the issue step.Locally:
ruff format/ruff checkclean; the two test files above pass (26 tests). The nightly workflow's pull-request smoke run exercises the report job's plumbing on this PR (informational, it never touches the issue). The issue change itself is proven by the first scheduled run onmainafter merge: Mutation survivors (nightly) #1184 renamed, its body opening with the three suite lines matching the job summary, the survivor list unchanged below, and no second issue.All CI checks pass (backend lint, backend tests, frontend lint, frontend build, frontend tests, mutation tests)
Manually tested the affected feature (the scheduled nightly cannot be run locally; see above)
Added/updated tests for new or changed behavior
Mutation gate: my tests kill the mutants on the lines I changed (
make mutation-diff). Score: n/a. No product lines changed:scripts/mutation/is outside what mutmut (backend/app) and Stryker (frontend/src) mutate, so the PR mutation jobs have nothing in scope. No surviving mutant kept, no pragma added.Checklist
CLAUDE.mdasync deffor all new route handlers and DB operations/VERSIONand added aCHANGELOG.mdentry (if user-facing change)docs/(if UI or feature change)🤖 Generated with Claude Code
https://claude.ai/code/session_014eZ98kaESE7YBa84fT1msc
Generated by Claude Code