Skip to content
Merged
51 changes: 36 additions & 15 deletions .qwen/skills/verify-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,15 @@ secondary claims. Budget by value:
1. **A/B load-bearing proof of the central claim** (always, ~half the budget).
2. **One or two wire-oracle harnesses** on the changed surface.
3. **Targeted gates**: tests/typecheck of the affected workspace(s) only.
4. **Capture the A/B and the matrix as they print** (~5 minutes, whenever
`QWEN_VERIFY_CHROMIUM=1`). This is a budget line, not an afterthought:
two live runs with the browser installed and working produced **zero**
images, because the instruction lived in the artifact contract while the
plan the agent follows is this list. Decide here how many captures the
round needs — normally two, at most a handful — and reserve the time.
See the artifact contract for the mechanics and the naming rule.
4. **Capture the A/B and the matrix as they print** — one command each,
`node scripts/verify-capture.mjs --out …/01-ab.png -- <cmd>`, so budget
~2 minutes, not the ~5 an ad-hoc pipeline would need. This is a budget
line, not an afterthought: **four live runs produced zero images**, first
because the instruction was worded as optional, then because it lived in
the artifact contract while the plan the agent follows is this list, and
underneath both because the pipeline it named did not exist. Decide here
how many captures the round needs — normally two, at most a handful — and
reserve the time. Mechanics and the naming rule: artifact contract.

Everything else is explicitly out of scope — and is **listed as not covered**
in the report. Never let breadth eat the A/B: one proven load-bearing claim
Expand Down Expand Up @@ -541,15 +543,34 @@ workflow globs). It must contain:
headline number. One capture of the terminal showing `2999 → 0` is worth
more than the sentence asserting it.

**Chromium is pre-installed for you** when `QWEN_VERIFY_CHROMIUM=1` is set;
`PLAYWRIGHT_BROWSERS_PATH` already points at it. Do **not** run
`playwright install` — you run as `node` with a fresh `HOME` and no apt
rights, so it downloads ~170 MB and then fails on system deps. If
`QWEN_VERIFY_CHROMIUM` is unset the capability is unavailable in this run:
ship the text-only report and note it under _Not covered_ in one line, do
not spend budget working around it.
**One command, already wired — do not build a capture pipeline.**

```bash
node scripts/verify-capture.mjs --out tmp/pr<n>-verify-<ts>/evidence/01-ab.png \
--title 'A/B: the gate flips on noisy data' -- node my-harness.mjs
# or pipe: my-harness | node scripts/verify-capture.mjs --out …/02-matrix.png
```

It runs the command, parses its ANSI through `@xterm/headless`, and
rasterises the cell grid with `sharp` — the 16 base ANSI colours and bold
preserved (256-colour and truecolor fall back to the default grey), **no
browser and no pseudo-terminal**. A non-zero exit from the captured command
is fine and often the point: capturing a failing base arm is normal. Options
that matter: `--cols` (default 100) to stop wrapping, `--title` for the
caption, `--rows` to cap height (output taller than `--rows` keeps the tail
and warns on stderr that the top was dropped).

This helper covers flat command output only: it gives the captured command
no TTY, so it cannot render an ink TUI or a browser page; for a TUI or
web-UI capture, see the `terminal-capture` skill. Earlier versions of this
section sent you to build that browser pipeline yourself. Its dependencies
do resolve from this repo, but it needs a browser, is slower, and is wired
fragilely (integration-tests/terminal-capture is not a root workspace, so
its package.json is never installed as a unit), and four live runs produced
zero images. Prefer this one command. If `verify-capture.mjs` is missing or
fails, say so under _Not covered_ in one line and ship the text-only report;
do not reconstruct the pipeline by hand.

Route: `terminal-capture` skill (node-pty → xterm.js → Playwright PNG).
The publish job hosts what you produce on a per-PR branch
(`pr-assets/<N>-verify`) and appends it below the report, capped at
**8 images, 2 MB each**; anything
Expand Down
20 changes: 13 additions & 7 deletions scripts/tests/qwen-triage-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -3164,13 +3164,17 @@ describe('qwen-triage verify round-3 hardening', () => {
'QWEN_VERIFY_CHROMIUM',
);

// And the skill must stop calling captures optional, must name the
// gate, and must forbid the install the agent cannot complete.
// And the skill must stop calling captures optional.
const flat = verifySkill.replace(/\s+/g, ' ');
expect(flat).toContain('Produce these whenever you ran a harness');
expect(flat).toContain('QWEN_VERIFY_CHROMIUM=1');
expect(flat).toContain('Do **not** run `playwright install`');
expect(flat).not.toContain('Optionally `evidence/*.png`');
// The TERMINAL capture route no longer depends on this browser at all —
// it is @xterm/headless + sharp, so the skill must not gate captures on
// QWEN_VERIFY_CHROMIUM or the agent skips when the browser is absent.
// The variable and its install stay for a future web-UI capture; see
// scripts/verify-capture.mjs for why the terminal route needs neither.
expect(flat).toContain('node scripts/verify-capture.mjs --out');
expect(flat).not.toContain('Route: `terminal-capture` skill');
});

// The browser step's require.resolve + cli.js join is otherwise guarded
Expand Down Expand Up @@ -3395,11 +3399,13 @@ describe('qwen-triage verify round-3 hardening', () => {
);
// A numbered budget item alongside the A/B, harnesses and gates.
expect(scope).toMatch(/^4\. \*\*Capture/m);
expect(scope).toContain('QWEN_VERIFY_CHROMIUM=1');
// The command, so budgeting does not mean budgeting for pipeline
// authoring — that is what made this cost ~5 minutes and never happen.
expect(scope).toContain('node scripts/verify-capture.mjs');
expect(scope).toContain('~2 minutes');
// Time reserved, and the failure that motivated it named — a rule
// stated without its failure reads as advice and gets skipped.
expect(scope).toContain('~5 minutes');
expect(scope).toContain('produced **zero**');
expect(scope).toContain('four live runs produced zero images');
// Bounded: the cap exists so "budget it" does not become eight images.
expect(scope).toMatch(/normally two, at most a handful/);

Expand Down
Loading
Loading