Skip to content

fix(test-insights): point the framework recipes at a dashboard page that exists - #12417

Open
jd wants to merge 1 commit into
mainfrom
devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d
Open

fix(test-insights): point the framework recipes at a dashboard page that exists#12417
jd wants to merge 1 commit into
mainfrom
devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d

Conversation

@jd

@jd jd commented Aug 17, 2026

Copy link
Copy Markdown
Member

Eleven test framework recipes closed by sending the reader to the Test Insights
dashboard. Ten of them linked
https://dashboard.mergify.com/test-insights/jobs, which is not a page: Test
Insights has Prevention, Detection and Mitigation, so that URL matched the
router and rendered nothing. jobs is a CI Insights route, a different and
org-scoped surface, which is where the copy-paste came from. The eleventh
(Cypress) named the dashboard in the same sentence with no link at all.

The paragraph promises "test results, including any failures or flaky tests",
and Detection is the page listing flaky and broken tests, so that is where all
eleven now point.

It is also a partial now, included by the recipes rather than pasted into each.
One stale URL became ten broken links precisely because it was ten copies of
one sentence, and an eleventh copy had already drifted into a dead end.

Separately, two of the recipes' inline GitHub Actions examples were missing the
continue-on-error: true that the quarantine partial they include requires:
without it a failing test step ends the job before the upload step runs, so the
run that most needs a report produces none. The id: tests half of that fix
landed meanwhile in #12439 across the same recipes, so what is left here is the
two continue-on-error lines.

All eleven recipes are fixed here rather than the six the ask named: the
six-versus-four split was an artifact of how that audit run was scoped, and
leaving four behind keeps the corpus inconsistent.

Fixes https://github.com/Mergifyio/ci-bot/issues/372
Part of MRGFY-8720

@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 fix(test-insights): point the framework recipes at a dashboard page that exists #12417 👈
2 fix(test-insights): stop pinning the Buildkite plugin to its first release #12426

@mergify

mergify Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 2 of 6 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 👀 Review Requirements 👀 reviews
🔴 🔎 Reviews 👀 reviews
🟢 🤖 Continuous Integration
🟢 Enforce conventional commit
🟢 📕 PR description
🟢 🚦 Auto-queue

🔴 👀 Review Requirements

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.
  • any of:
    • #approved-reviews-by >= 2
    • author = dependabot[bot]
    • all of:
      • author = mergify-ci-bot
      • -head ~= ^docs-agent/

🔴 🔎 Reviews

Waiting for

  • #review-requested = 0
This rule is failing.
  • #review-requested = 0
  • #changes-requested-reviews-by = 0
  • #review-threads-unresolved = 0

Show 4 satisfied protections

🟢 🤖 Continuous Integration

  • all of:
    • check-success = build
    • check-success = lint
    • check-success = test
    • any of:
      • check-success = test-broken-links
      • label = ignore-broken-links
    • any of:
      • check-success=Cloudflare Pages
      • -head-repo-full-name~=^Mergifyio/

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|internal|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?!?:

🟢 📕 PR description

  • body ~= (?ms:.{48,})

🟢 🚦 Auto-queue

When all merge protections are satisfied, this pull request will be queued automatically.

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 17, 2026 17:19 Failure
@mergify
mergify Bot requested a review from a team August 17, 2026 17:21
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

This stack answers three of the four docs asks filed on 2026-08-17 (Mergifyio/ci-bot#369, #370, #372). The fourth, Mergifyio/ci-bot#366, turned out not to be a docs change and is Mergifyio/monorepo#38775 instead.

#366 asked whether this repo should grow a section on how a ruleset bypass mode interacts with condition injection, taking the #bypass-actors-and-injection anchor that five engine messages already link to, or stay silent and have those links repointed at the section that exists.

I read the flag that decides it. FORCE_EXEMPTED_RULESET_CONDITIONS_INJECTION landed six days ago, is opt-in per org with no grandfathering, and turns off the skip one org at a time; its own note says it is temporary and disappears once every org is enabled. So the destination is known but almost nobody is there yet. Writing "exempt no longer skips injection" would be wrong for nearly every reader today, and writing the current behavior would push customers onto something that reverses under them without warning. The reversible half is to point the links at the section that does exist, which is also the section those messages are actually about, so that is what Mergifyio/monorepo#38775 does. When the rollout finishes, the docs get a section on injection and the links can move again.

One thing that fix deliberately does not settle, noted on that PR: the branch_protection_injection_mode field description still tells customers that exempt is how you skip injection for one ruleset, which is the behavior being removed.

Copilot AI lite review requested due to automatic review settings August 17, 2026 19:00
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from c604f0e to 10ece46 Compare August 17, 2026 19:00
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Revision history

# Type Changes Reason Date
1 initial c604f0e 2026-08-17 19:00 UTC
2 rebase c604f0e → 10ece46 (rebase only) 2026-08-17 19:00 UTC
3 rebase 10ece46 → 0ec2b78 (rebase only) 2026-08-18 07:18 UTC
4 content 0ec2b78 → 3d4d468 Rebased onto main after #12439 landed the id: tests half of this change; resolved the resulting conflicts in junit.mdx and minitest.mdx by keeping the two continue-on-error lines, which #12439 did no… 2026-08-21 15:09 UTC

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 17, 2026 19:01 Failure
@jd

jd commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Restacked only — c604f0e10ece46 (compare). The content of this commit is unchanged; it moved because #12415 at the bottom of the stack was amended to address a review comment.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes broken/absent “review results in Test Insights” links across the Test Framework recipes by centralizing the CTA into a shared MDX partial that points to the existing Test Insights → Detection dashboard page. It also updates the GitHub Actions recipe snippets to reliably propagate the test step outcome into the quarantine/upload step by adding id: tests (and ensuring continue-on-error: true where required).

Changes:

  • Replace per-recipe “review in Test Insights” copy/links with a shared <ReviewInTestInsights /> partial targeting /test-insights/detection.
  • Add id: tests to the test-runner step in GitHub Actions examples so ${{ steps.tests.outcome }} resolves correctly.
  • Add missing continue-on-error: true in the affected workflow examples to match the quarantine setup requirements.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/content/docs/test-insights/test-frameworks/testng.mdx Adds id: tests to test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/rust.mdx Adds id: tests to both alternative test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/phpunit.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/pest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/nunit.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/mstest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/minitest.mdx Adds id: tests (and missing continue-on-error: true) to the test steps and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/junit.mdx Adds id: tests (and missing continue-on-error: true for Gradle) and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/jest.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/golang.mdx Adds id: tests to the test step and replaces the broken dashboard link with the shared review partial.
src/content/docs/test-insights/test-frameworks/cypress.mdx Adds the shared review partial where the dashboard was previously mentioned without a link.
src/content/docs/test-insights/test-frameworks/_review-in-test-insights.mdx Introduces a reusable “review results” partial linking to the Test Insights Detection page.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jd
jd marked this pull request as ready for review August 18, 2026 06:49
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from 10ece46 to 0ec2b78 Compare August 18, 2026 07:18
@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 18, 2026 07:18 Failure
@jd

jd commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

Restacked only — 10ece460ec2b78 (compare). The content of this commit is unchanged; it moved because the base commit (#12415) was amended and the stack rebased on main.

Base automatically changed from devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/drop-removed-gradle-report-props--24b7f2d8 to main August 21, 2026 14:45
@mergify

mergify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@jd this pull request is now in conflict 😩

@mergify mergify Bot added the conflict label Aug 21, 2026
…hat exists

Eleven test framework recipes closed by sending the reader to the Test Insights
dashboard. Ten of them linked
`https://dashboard.mergify.com/test-insights/jobs`, which is not a page: Test
Insights has Prevention, Detection and Mitigation, so that URL matched the
router and rendered nothing. `jobs` is a CI Insights route, a different and
org-scoped surface, which is where the copy-paste came from. The eleventh
(Cypress) named the dashboard in the same sentence with no link at all.

The paragraph promises "test results, including any failures or flaky tests",
and Detection is the page listing flaky and broken tests, so that is where all
eleven now point.

It is also a partial now, included by the recipes rather than pasted into each.
One stale URL became ten broken links precisely because it was ten copies of
one sentence, and an eleventh copy had already drifted into a dead end.

Separately, two of the recipes' inline GitHub Actions examples were missing the
`continue-on-error: true` that the quarantine partial they include requires:
without it a failing test step ends the job before the upload step runs, so the
run that most needs a report produces none. The `id: tests` half of that fix
landed meanwhile in #12439 across the same recipes, so what is left here is the
two `continue-on-error` lines.

All eleven recipes are fixed here rather than the six the ask named: the
six-versus-four split was an artifact of how that audit run was scoped, and
leaving four behind keeps the corpus inconsistent.

Fixes Mergifyio/ci-bot#372
Part of MRGFY-8720

Change-Id: I482ec81d7710f37d83dcffdf65abc029b73437fc
@jd
jd force-pushed the devs/jd/jd/mrgfy-8720-settle-four-docs-asks-the-agent-could-not-bypass-anchor-ci/point-framework-recipes-dashboard-page-exists--482ec81d branch from 0ec2b78 to 3d4d468 Compare August 21, 2026 15:09
@jd

jd commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main and resolved the conflict 0ec2b783d4d468 (compare).

#12439 landed the id: tests half of this change across the same recipes while this sat open, which is what conflicted. I kept the two continue-on-error: true lines it did not add — junit.mdx (Gradle) and minitest.mdx (minitest-reporters) — because the quarantine partial these recipes include needs the test step to keep the job alive, otherwise a failing test run ends before the upload step and produces no report at all.

What is left in this PR is therefore the dashboard link fix (all eleven recipes, via the new _review-in-test-insights.mdx partial) plus those two lines. The commit message also credited #12411 for the overlap; it was #12439, and that is corrected.

@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 21, 2026 15:10 Failure
@mergify mergify Bot removed the conflict label Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants