Skip to content

fix(test-insights): make the closing CTA one partial, not eleven copies - #12417

Merged
mergify[bot] merged 1 commit into
mainfrom
devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d
Aug 24, 2026
Merged

fix(test-insights): make the closing CTA one partial, not eleven copies#12417
mergify[bot] merged 1 commit into
mainfrom
devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d

Conversation

@jd

@jd jd commented Aug 17, 2026

Copy link
Copy Markdown
Member

Every test framework recipe ends by sending the reader to the Test Insights
dashboard, and each one carried its own copy of that sentence. Ten of the copies
had drifted onto https://dashboard.mergify.com/test-insights/jobs, which is not
a page — Test Insights has Prevention, Detection and Mitigation — and an
eleventh named the dashboard with no link at all. #12490 has since corrected all
eleven in place, so the links work today; what it could not fix is why one stale
URL became ten broken links, which is that the sentence exists eleven times.

So the paragraph becomes _review-in-test-insights.mdx, included by the
recipes. The rendered text is byte-identical to what #12490 shipped; the next
time that link moves it is one edit rather than eleven, and a recipe cannot
quietly drift out of step with its siblings again. Shared partials are already
how these files handle the Buildkite quarantine setup and the upload steps.

Separately, two inline GitHub Actions examples were missing the
continue-on-error: true that the quarantine partial they include requires:
the Gradle example in the JUnit recipe and the rake test example in the
Minitest one. Without it a failing test step ends the job before the upload step
runs, so the run that most needs a report produces none — and in both files the
sibling example directly above already had the line, so this was drift rather
than a deliberate difference.

Part of MRGFY-8720

@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 fix(test-insights): point the framework recipes at a dashboard page that exists #12417 👈
2 fix(test-insights): stop pinning the Buildkite plugin to its first release #12426

@mergify

mergify Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 6 merge protections satisfied — ready to merge.

Show 6 satisfied protections

🟢 🤖 Continuous Integration

  • all of:
    • check-success = build
    • check-success = lint
    • check-success = test
    • any of:
      • check-success = test-broken-links
      • label = ignore-broken-links
    • any of:
      • check-success=Cloudflare Pages
      • -head-repo-full-name~=^Mergifyio/

🟢 👀 Review Requirements

  • any of:
    • #approved-reviews-by >= 2
    • author = dependabot[bot]
    • all of:
      • author = mergify-ci-bot
      • -head ~= ^docs-agent/

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|internal|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?!?:

🟢 🔎 Reviews

  • #changes-requested-reviews-by = 0
  • #review-requested = 0
  • #review-threads-unresolved = 0

🟢 📕 PR description

  • body ~= (?ms:.{48,})

🟢 🚦 Auto-queue

When all merge protections are satisfied, this pull request will be queued automatically.

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 17, 2026 17:19 Failure
@mergify
mergify Bot requested a review from a team August 17, 2026 17:21
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

This stack answers three of the four docs asks filed on 2026-08-17 (Mergifyio/ci-bot#369, #370, #372). The fourth, Mergifyio/ci-bot#366, turned out not to be a docs change and is Mergifyio/monorepo#38775 instead.

#366 asked whether this repo should grow a section on how a ruleset bypass mode interacts with condition injection, taking the #bypass-actors-and-injection anchor that five engine messages already link to, or stay silent and have those links repointed at the section that exists.

I read the flag that decides it. FORCE_EXEMPTED_RULESET_CONDITIONS_INJECTION landed six days ago, is opt-in per org with no grandfathering, and turns off the skip one org at a time; its own note says it is temporary and disappears once every org is enabled. So the destination is known but almost nobody is there yet. Writing "exempt no longer skips injection" would be wrong for nearly every reader today, and writing the current behavior would push customers onto something that reverses under them without warning. The reversible half is to point the links at the section that does exist, which is also the section those messages are actually about, so that is what Mergifyio/monorepo#38775 does. When the rollout finishes, the docs get a section on injection and the links can move again.

One thing that fix deliberately does not settle, noted on that PR: the branch_protection_injection_mode field description still tells customers that exempt is how you skip injection for one ruleset, which is the behavior being removed.

Copilot AI lite review requested due to automatic review settings August 17, 2026 19:00
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from c604f0e to 10ece46 Compare August 17, 2026 19:00
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Revision history

# Type Changes Reason Date
1 initial c604f0e 2026-08-17 19:00 UTC
2 rebase c604f0e → 10ece46 (rebase only) 2026-08-17 19:00 UTC
3 rebase 10ece46 → 0ec2b78 (rebase only) 2026-08-18 07:18 UTC
4 content 0ec2b78 → 3d4d468 Rebased onto main after #12439 landed the id: tests half of this change; resolved the resulting conflicts in junit.mdx and minitest.mdx by keeping the two continue-on-error lines, which #12439 did no… 2026-08-21 15:09 UTC
5 content 3d4d468 → a2f4cc7 rebased onto main after #12490 landed the same link fix inline; resolved the eleven conflicts to the partial and rewrote the message to describe what is left (the de-duplication and the two continue-… 2026-08-24 08:33 UTC

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 17, 2026 19:01 Failure
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Restacked only — c604f0e10ece46 (compare). The content of this commit is unchanged; it moved because #12415 at the bottom of the stack was amended to address a review comment.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes broken/absent “review results in Test Insights” links across the Test Framework recipes by centralizing the CTA into a shared MDX partial that points to the existing Test Insights → Detection dashboard page. It also updates the GitHub Actions recipe snippets to reliably propagate the test step outcome into the quarantine/upload step by adding id: tests (and ensuring continue-on-error: true where required).

Changes:

  • Replace per-recipe “review in Test Insights” copy/links with a shared <ReviewInTestInsights /> partial targeting /test-insights/detection.
  • Add id: tests to the test-runner step in GitHub Actions examples so ${{ steps.tests.outcome }} resolves correctly.
  • Add missing continue-on-error: true in the affected workflow examples to match the quarantine setup requirements.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/content/docs/test-insights/test-frameworks/testng.mdx Adds id: tests to test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/rust.mdx Adds id: tests to both alternative test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/phpunit.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/pest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/nunit.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/mstest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/minitest.mdx Adds id: tests (and missing continue-on-error: true) to the test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/junit.mdx Adds id: tests (and missing continue-on-error: true for Gradle) and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/jest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/golang.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/cypress.mdx Adds the shared review partial where the dashboard was previously mentioned without a link.
src/content/docs/test-insights/test-frameworks/_review-in-test-insights.mdx Introduces a reusable “review results” partial linking to the Test Insights Detection page.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jd
jd marked this pull request as ready for review August 18, 2026 06:49
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from 10ece46 to 0ec2b78 Compare August 18, 2026 07:18
@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 18, 2026 07:18 Failure
@jd

jd commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

Restacked only — 10ece460ec2b78 (compare). The content of this commit is unchanged; it moved because the base commit (#12415) was amended and the stack rebased on main.

Base automatically changed from devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/drop-removed-gradle-report-props--24b7f2d8 to main August 21, 2026 14:45
@mergify

mergify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@jd this pull request is now in conflict 😩

@mergify mergify Bot added the conflict label Aug 21, 2026
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from 0ec2b78 to 3d4d468 Compare August 21, 2026 15:09
@jd

jd commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main and resolved the conflict 0ec2b783d4d468 (compare).

#12439 landed the id: tests half of this change across the same recipes while this sat open, which is what conflicted. I kept the two continue-on-error: true lines it did not add — junit.mdx (Gradle) and minitest.mdx (minitest-reporters) — because the quarantine partial these recipes include needs the test step to keep the job alive, otherwise a failing test run ends before the upload step and produces no report at all.

What is left in this PR is therefore the dashboard link fix (all eleven recipes, via the new _review-in-test-insights.mdx partial) plus those two lines. The commit message also credited #12411 for the overlap; it was #12439, and that is corrected.

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 21, 2026 15:10 Failure
@mergify mergify Bot removed the conflict label Aug 21, 2026
Every test framework recipe ends by sending the reader to the Test Insights
dashboard, and each one carried its own copy of that sentence. Ten of the copies
had drifted onto `https://dashboard.mergify.com/test-insights/jobs`, which is not
a page — Test Insights has Prevention, Detection and Mitigation — and an
eleventh named the dashboard with no link at all. #12490 has since corrected all
eleven in place, so the links work today; what it could not fix is why one stale
URL became ten broken links, which is that the sentence exists eleven times.

So the paragraph becomes `_review-in-test-insights.mdx`, included by the
recipes. The rendered text is byte-identical to what #12490 shipped; the next
time that link moves it is one edit rather than eleven, and a recipe cannot
quietly drift out of step with its siblings again. Shared partials are already
how these files handle the Buildkite quarantine setup and the upload steps.

Separately, two inline GitHub Actions examples were missing the
`continue-on-error: true` that the quarantine partial they include requires:
the Gradle example in the JUnit recipe and the `rake test` example in the
Minitest one. Without it a failing test step ends the job before the upload step
runs, so the run that most needs a report produces none — and in both files the
sibling example directly above already had the line, so this was drift rather
than a deliberate difference.

Part of MRGFY-8720

Change-Id: I482ec81d7710f37d83dcffdf65abc029b73437fc
@jd jd changed the title fix(test-insights): point the framework recipes at a dashboard page that exists fix(test-insights): make the closing CTA one partial, not eleven copies Aug 24, 2026
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from 3d4d468 to a2f4cc7 Compare August 24, 2026 08:33
@jd

jd commented Aug 24, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main and rewrote this one, because #12490 landed the same link fix while this sat here — 3d4d468a2f4cc7 (compare).

#12490 corrected all eleven closing links in place, to the same page this PR was pointing them at (/test-insights/detection). So the dead link is fixed on main already and I dropped the Fixes keyword — I'll close the ask against #12490 rather than against this.

What is left here is the part #12490 could not do, and I think it is still worth landing:

  • The sentence becomes one partial. _review-in-test-insights.mdx, included by the recipes. The rendered text is byte-identical to what docs(agent): point the Test Insights dashboard link at a page that exists #12490 shipped — I resolved every conflict to the partial and checked the built HTML. The reason to keep it is what just happened: one stale URL became ten broken links precisely because the paragraph existed eleven times, and an eleventh copy had already drifted to having no link at all. Next time that link moves it is one edit. This is also how these files already handle the Buildkite quarantine setup and the upload steps, so it is not a new pattern.
  • Two continue-on-error: true lines that are still missing on main — the Gradle example in the JUnit recipe and the rake test example in the Minitest one. Both include the quarantine partial, which needs it: without it a failing test step ends the job before the upload runs, so the run that most needs a report produces none. In both files the sibling example directly above already has the line, which is what makes me read it as drift rather than intent.

If you would rather not carry the partial, the two continue-on-error lines are the only part that has to land and I'll cut it down to those.

@mergify
mergify Bot deployed to Mergify Merge Protections August 24, 2026 08:34 Active
@mergify
mergify Bot requested a review from a team August 24, 2026 08:41
@mergify

mergify Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 3 minutes 24 seconds in the queue, including 2 minutes 51 seconds running CI.

Required conditions to merge

@mergify mergify Bot added the queued label Aug 24, 2026
@mergify
mergify Bot merged commit 5c55c68 into main Aug 24, 2026
10 checks passed
@mergify
mergify Bot deleted the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch August 24, 2026 11:55
@mergify mergify Bot removed the queued label Aug 24, 2026
mergify Bot pushed a commit that referenced this pull request Aug 24, 2026
…lease (#12426)

The two Buildkite upload snippets hardcode `mergifyio/mergify-ci#v1` in their
`.astro` source. `v1` is the oldest of six releases — `v1` through `v6` are
distinct tags, not a floating major — so every reader who copies one of these
snippets pins the first version ever cut and never receives anything shipped
since.

Nothing else in the docs does this. Every Buildkite page writes
`@@BUILDKITE_PLUGIN_VERSION@@`, which `plugins/remark-buildkite-version.ts`
substitutes from `src/data/buildkite-plugin-version.json` at build time. That
plugin only visits markdown nodes, so the sentinel does nothing inside an
`.astro` component and these two were left pinning by hand. The fix is the one
the GitHub Actions counterpart already uses: import the data file and
interpolate it, exactly as `MergifyCIUploadStepMatrix.astro` does with
`gha-mergify-ci-version.json`.

Bumping `v1` to `v6` would have recreated the same problem one release later.

This matters now because the commit at the bottom of this stack recommends the
plugin's `job_name` property again. A reader who copies the matrix snippet and
pins `v1` gets a plugin where that property predates the code reading it — the
docs would recommend a knob that is dead in the version the snippet installs.

The non-matrix component has the same pin and is the more widely used of the
two: eleven test-framework recipes render it. Both are fixed here, since it is
one bug with one cause.

Verified on the built output: all 78 rendered occurrences of the plugin
reference now read `v6`, none read `v1`, and no template literal leaked into
the HTML. `pnpm check` 0 errors, `pnpm build` 384 pages, `pnpm test` 144
passed, `pnpm check:internal-leaks` clean.

Refs MRGFY-8720

Depends-On: #12417
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants