Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .qwen/skills/verify-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,13 @@ secondary claims. Budget by value:
1. **A/B load-bearing proof of the central claim** (always, ~half the budget).
2. **One or two wire-oracle harnesses** on the changed surface.
3. **Targeted gates**: tests/typecheck of the affected workspace(s) only.
4. **Capture the A/B and the matrix as they print** (~5 minutes, whenever
`QWEN_VERIFY_CHROMIUM=1`). This is a budget line, not an afterthought:
two live runs with the browser installed and working produced **zero**
images, because the instruction lived in the artifact contract while the
plan the agent follows is this list. Decide here how many captures the
round needs — normally two, at most a handful — and reserve the time.
See the artifact contract for the mechanics and the naming rule.

Everything else is explicitly out of scope — and is **listed as not covered**
in the report. Never let breadth eat the A/B: one proven load-bearing claim
Expand Down Expand Up @@ -572,6 +579,8 @@ central claim from being tested — say why.
them. Cite the tables below by name instead of restating their numbers in
prose: a number written twice is a number that can disagree with itself.
3. **Central claim + A/B table** (cells, oracles, head vs control counts).
Reference the capture of those cells here by its filename — a table with
no witness beside it is the shape every report has had so far.
4. **Corrections**, when an earlier review round or bot comment described
the code inaccurately (a wrong ARIA role, a wrong mechanism, a
misattributed cause). State the correct fact with its evidence and label
Expand Down
30 changes: 30 additions & 0 deletions scripts/tests/qwen-triage-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -3382,6 +3382,36 @@ describe('qwen-triage verify round-3 hardening', () => {
expect(soft).toBeGreaterThanOrEqual(Math.floor(agentMinutes * 0.8));
});

// Third instance of one structural bug: an instruction placed outside the
// flow the agent actually follows. #7917 buried the /verify recommendation
// in a "local invocation ONLY" section; #8016 marked captures "Optionally";
// and captures still came out ZERO on two live runs (#7975, #8066) where
// the browser installed fine — because the plan the agent executes is the
// Scope-selection budget list, and that list had no capture line at all.
it('budgets evidence capture in scope selection, not only in the contract', () => {
const scope = verifySkill.slice(
verifySkill.indexOf('## Scope selection'),
verifySkill.indexOf('## Method'),
);
// A numbered budget item alongside the A/B, harnesses and gates.
expect(scope).toMatch(/^4\. \*\*Capture/m);
expect(scope).toContain('QWEN_VERIFY_CHROMIUM=1');
// Time reserved, and the failure that motivated it named — a rule
// stated without its failure reads as advice and gets skipped.
expect(scope).toContain('~5 minutes');
expect(scope).toContain('produced **zero**');
// Bounded: the cap exists so "budget it" does not become eight images.
expect(scope).toMatch(/normally two, at most a handful/);

// And the report has somewhere to put it, or a produced capture has no
// referent and the naming rule means nothing.
const structure = verifySkill.slice(
verifySkill.indexOf('### report.md structure'),
verifySkill.indexOf('## Hard rules'),
);
expect(structure).toContain('Reference the capture of those cells here');
});

// Cleanups must never descend through a PR-writable parent, and an
// outward-resolving hooks entry must be removed rather than reported.
it('survives symlink escapes in the workspace cleanup', () => {
Expand Down
Loading