[Yiminnn]

Summary

Scope: make the full integration test workflow actually runnable end-to-end, plus add an automatic trigger for the codex final review. Two parts:

1. Fix the plan-job credential-filter install (the blocker)

The first live dispatch of L3 integration-final-review (against #803) failed at the plan job:

ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

The L2/L3 plan jobs sparse-checkout only .github + tests/integration, then pip install . to make benchflow.agents.env importable for the credential filter — but the sparse checkout omits pyproject.toml/src, so the build fails and the whole plan job goes red (→ run-matrix + review-pack skipped). It slipped past #802's own CI because that path was never exercised (L1 has no filter; L2 plan only runs on the integration:medium label; L3 is dispatch-only).

• Add pyproject.toml + README.md + src to the plan job's sparse-checkout so hatchling can build benchflow.
• Switch pip install . → uv pip install --system (astral-sh/setup-uv) — consistent with the rest of the workflows, no pip.
• Make the install non-fatal (|| ::warning) so the fail-open credential filter is never blocked by an install hiccup.

(The only remaining pip install is release-notes text in internal-preview-release.yml, not a CI step.)

2. Auto-trigger the codex final review on ready-to-merge

Today the codex equivalence reviewer runs only in L3, which is workflow_dispatch-only (manual). This adds a pull_request: [labeled] trigger so that applying the ready-to-merge label to an internal PR runs the full final review (plan → matrix → deterministic review pack + codex) automatically.

• plan job gated if: workflow_dispatch OR (pull_request && label == 'ready-to-merge'); any other label is a no-op (plan + dependent jobs skip, workflow stays green — no rollout spun up).
• PR resolved from inputs.pr_number || pull_request.number (resolve / plan / comment steps); concurrency keyed the same way (cancel-in-progress: false).
• Security: plain pull_request (never pull_request_target) → fork PRs get no secrets; the resolve step still refuses any non-internal head repo; only write-access users can apply labels. Codex stays fail-closed as the L3 gate, but L3 is not a required status check, so a red final-review informs without blocking the merge (mitigates the dead-OPENAI_API_KEY red-block concern).

The ready-to-merge label has been created in the repo.

Test plan

• YAML valid (both workflows); planner/filter unit suites pass
• End-to-end: benchflow builds from {pyproject.toml, README.md, src} via uv pip install . and resolve_agent_env imports with real deps
• Trigger wiring verified: on = {workflow_dispatch, pull_request[labeled]}, plan if gates on the label, PR resolved from both event shapes
• After merge: label a real internal PR ready-to-merge → L3 auto-runs (plan reaches the credential filter, matrix runs, review pack + codex verdict post)
• After merge: re-dispatch L3 on Preserve pi-acp model metadata through LiteLLM proxy #803 to confirm the plan-install fix end-to-end

[greptile-apps]

Greptile Summary

Fixes the plan-job crash introduced in #802 by adding pyproject.toml, README.md, and src to the sparse checkout so hatchling can build benchflow, and switches the install from pip to uv pip install --system (consistent with the rest of the workflows). A || echo "::warning::" guard makes the install non-fatal, which correctly matches filter_credentialed_cells.py's own fail-open design (confirmed: the script catches all import exceptions and always exits 0).

• Sparse checkout expansion (pyproject.toml, README.md, src): gives hatchling everything it needs to build the package in the plan job without pulling unrelated sources.
• uv install + fail-open guard: uv pip install --system --quiet . || echo "::warning::" applied identically in both L2 and L3 plan jobs; a failed install produces a workflow warning but never blocks the plan.
• New pull_request: labeled trigger (L3 only): triggers the full final review when ready-to-merge is added; guarded by the plan job's if expression, an internal-repo assertion in the resolve step, and the pypi-internal-preview environment gate — unrelated labels produce a skipped run only.

Confidence Score: 5/5

Safe to merge — the fix is targeted, the fail-open path is verified in filter_credentialed_cells.py, and the new label trigger is properly gated.

The sparse-checkout expansion and uv switch directly address the reported crash with no side effects on other jobs. The fail-open concern raised previously is confirmed to be a non-issue: filter_credentialed_cells.py catches all exceptions on import and always exits 0, so the plan job cannot be blocked by an install failure. The new pull_request:labeled trigger is layered with three independent guards (plan if-condition, internal-repo assertion in resolve, and the pypi-internal-preview environment gate), making accidental or unauthorized triggering unlikely.

No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/integration-final-review.yml	Adds pyproject.toml/README.md/src to sparse checkout, switches install to uv pip install --system with fail-open, and adds a pull_request:labeled trigger for the ready-to-merge auto path; all plumbing is correct and security guards are in place.
.github/workflows/integration-scope.yml	Same sparse-checkout and uv install fix as integration-final-review.yml; no trigger changes, straightforward parity fix.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant GH as GitHub Event
    participant Plan as plan job
    participant Filter as filter_credentialed_cells.py
    participant Matrix as run-matrix jobs
    participant Pack as review-pack job

    GH->>Plan: "workflow_dispatch OR pull_request[labeled=ready-to-merge]"
    Plan->>Plan: resolve + assert internal head repo
    Plan->>Plan: sparse checkout (.github, tests/integration, pyproject.toml, README.md, src)
    Plan->>Plan: uv pip install --system . (fail-open)
    Plan->>Filter: python filter_credentialed_cells.py --matrix matrix.json
    alt benchflow import fails
        Filter-->>Plan: exception caught, matrix unchanged, exit 0
    else import succeeds
        Filter-->>Plan: filtered matrix, uncredentialed cells dropped, exit 0
    end
    Plan->>Matrix: emit matrix + head_sha
    Matrix->>Pack: cell artifacts
    Pack->>Pack: build review pack + codex review
    Pack->>GH: upsert PR comment + publish check-run

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant GH as GitHub Event
    participant Plan as plan job
    participant Filter as filter_credentialed_cells.py
    participant Matrix as run-matrix jobs
    participant Pack as review-pack job

    GH->>Plan: "workflow_dispatch OR pull_request[labeled=ready-to-merge]"
    Plan->>Plan: resolve + assert internal head repo
    Plan->>Plan: sparse checkout (.github, tests/integration, pyproject.toml, README.md, src)
    Plan->>Plan: uv pip install --system . (fail-open)
    Plan->>Filter: python filter_credentialed_cells.py --matrix matrix.json
    alt benchflow import fails
        Filter-->>Plan: exception caught, matrix unchanged, exit 0
    else import succeeds
        Filter-->>Plan: filtered matrix, uncredentialed cells dropped, exit 0
    end
    Plan->>Matrix: emit matrix + head_sha
    Matrix->>Pack: cell artifacts
    Pack->>Pack: build review pack + codex review
    Pack->>GH: upsert PR comment + publish check-run

Loading

_{Reviews (2): Last reviewed commit: "feat(integration): auto-trigger the L3 c..." | Re-trigger Greptile}

[greptile-apps]

Fail-open guarantee depends on filter_credentialed_cells.py exit code

The || echo "::warning::" makes the install non-fatal, but the very next step (Filter uncredentialed cells) runs under set -euo pipefail and calls filter_credentialed_cells.py directly. If that script raises an unhandled ImportError when benchflow is missing and exits non-zero, the plan job fails — negating the fail-open intent here. The comments assert "Fails OPEN: if benchflow import fails the matrix passes through unchanged", but this only holds if filter_credentialed_cells.py catches the ImportError, passes the matrix through unmodified, and exits 0. This is the same concern in both integration-final-review.yml (line 178) and integration-scope.yml (line 230).