Skip to content

Lead the nightly mutation issue with its status and name it a report - #1244

Merged
vincentmakes merged 1 commit into
mainfrom
claude/mutation-nightly-issue-status
Oct 10, 2026
Merged

vincentmakes merged 1 commit into
mainfrom
claude/mutation-nightly-issue-status

Conversation

@vincentmakes

Copy link
Copy Markdown
Owner

Summary

The nightly rewrites its issue with the survivor backlog alone, so after run 28 (every floor holding) #1184 still read like an open defect list with nothing saying whether anything was wrong. This PR makes the issue a report: it opens with one status line per suite (score against its floor, baseline progress, critical-module count, files whose score fell), says in its preamble that it is not a bug list, and carries a title that says so. The workflow renames the existing #1184 in place, so no second issue is ever opened. Tooling only: no VERSION bump and no CHANGELOG entry, since CI tooling has no effect on the running product.

Type

  • feature
  • fix
  • refactor
  • docs
  • chore (CI / build / dependency / housekeeping)
  • security

Changes

  • scripts/mutation/gate.py: new --status PATH (written for --scope suite only) appends one Markdown bullet per suite from status(): the score sentence (78.9% of 59,697 mutants killed; floor 69% — pass / **FAIL**; for a partial baseline baseline 99% measured (383 mutants left for the next run); 67.7% of those measured killed; the 0% floor applies once complete; **incomplete run** when records are pending without --allow-pending), then Critical modules: k of n at or above floor, naming any below its floor with score and floor, any floored module with nothing measured (check), and the number still to measure, then the files whose score fell since the previous run (before → now) or a line saying none did. The "which files fell" computation moved out of regressions() into dropped_files(); the regressions() table is unchanged.
  • .github/workflows/mutation-nightly.yml: the score step passes --status status.md for each suite and writes the breach flag to a floors-breached file, which the issue step reads instead of step output. The issue body is now: harness.md first when present (unchanged), a preamble (a nightly mutation report for the linked run; not a bug list, each entry is a change no test notices, a confirmed bug is filed as its own issue; the triage and floor rules; every run rewrites the body, so a note that must persist goes in a comment, a pragma or a floor), ## Status (all floors holding, or a floor breached / records missing, then the suite lines), ## Survivors by module (the existing list). ISSUE_TITLE becomes Mutation nightly: survivors by module; LEGACY_ISSUE_TITLE keeps Mutation survivors (nightly), and an open issue under it is edited with --title, so Mutation survivors (nightly) #1184 is renamed rather than duplicated. status.md joins the mutation-survivors artifact.
  • scripts/mutation/README.md and CLAUDE.md: the sentence that names the issue.

Test Plan

  • backend/tests/core/test_mutation_scripts.py::TestGate (new cases): status() on a complete suite gives the pass line, 1 of 1 critical modules ok and no dropped-file line; a partial baseline reports the mutants left and a module still to measure; a breached module and a dropped file are both named with their numbers; a floored module with nothing measured is flagged; an unfinished run without --allow-pending reads "incomplete run"; regressions() renders the same table through dropped_files(); main --status writes the file for suite scope and not for diff scope.

  • backend/tests/services/test_ci_workflow_mutation.py (new and updated pins): the new title and the legacy title; the issue step edits with --title under the legacy title and creates only when neither is open; --status status.md is passed; the body carries the preamble, ## Status, the breach-flag file, cat status.md before cat survivors.md, and harness.md before the preamble; no ${{ steps. interpolation in the issue step.

  • Locally: ruff format / ruff check clean; the two test files above pass (26 tests). The nightly workflow's pull-request smoke run exercises the report job's plumbing on this PR (informational, it never touches the issue). The issue change itself is proven by the first scheduled run on main after merge: Mutation survivors (nightly) #1184 renamed, its body opening with the three suite lines matching the job summary, the survivor list unchanged below, and no second issue.

  • All CI checks pass (backend lint, backend tests, frontend lint, frontend build, frontend tests, mutation tests)

  • Manually tested the affected feature (the scheduled nightly cannot be run locally; see above)

  • Added/updated tests for new or changed behavior

  • Mutation gate: my tests kill the mutants on the lines I changed (make mutation-diff). Score: n/a. No product lines changed: scripts/mutation/ is outside what mutmut (backend/app) and Stryker (frontend/src) mutate, so the PR mutation jobs have nothing in scope. No surviving mutant kept, no pragma added.

Checklist

  • My changes follow the conventions in CLAUDE.md
  • I added permission checks to any new mutating endpoints
  • I created an Alembic migration for any schema changes
  • I did not introduce hardcoded card types or fields (metamodel is data-driven)
  • I used async def for all new route handlers and DB operations
  • I did not expose sensitive fields (password hashes, encrypted secrets) in API responses
  • I bumped /VERSION and added a CHANGELOG.md entry (if user-facing change)
  • I added translations for new UI strings in all 10 locales (if applicable)
  • I updated user documentation in docs/ (if UI or feature change)
  • Screenshots attached for UI changes (if applicable)

🤖 Generated with Claude Code

https://claude.ai/code/session_014eZ98kaESE7YBa84fT1msc


Generated by Claude Code

The nightly rewrote its issue with the survivor backlog alone, so a
reader could not tell whether anything was wrong, and its title read
like a defect list. gate.py gains --status, one line per suite with the
score against its floor, baseline progress, the critical-module count
(naming any below floor, unmeasured or still to measure) and the files
whose score fell. The report job writes those lines under a Status
heading, after a preamble that says the issue is a report and not a bug
list, and before the survivors. The issue is retitled "Mutation nightly:
survivors by module"; an open issue under the old title is renamed in
place so no second issue is ever opened. The breach flag travels through
a file rather than step output.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014eZ98kaESE7YBa84fT1msc
@vincentmakes
vincentmakes merged commit 3dfe2ce into main Oct 10, 2026
58 checks passed
@vincentmakes
vincentmakes deleted the claude/mutation-nightly-issue-status branch October 10, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants