feat: publish benchmark results to GitHub

[DennisYu07]

What this PR does

Completes the shortest release-to-score POC:

GitHub Release/manual dispatch → self-hosted ECS runner → local dispatcher → SQLite → systemd worker → Harbor SWE-bench trial → GitHub Release benchmark summary

The runner resolves the requested release tag to an immutable commit and invokes the root-owned local dispatcher. The dispatcher validates the request and inserts it into SQLite. The worker executes the pinned Qwen Code npm release through Harbor, grades it, writes checksummed artifacts, and replaces the benchmark section on the triggering Release.

Successful runs publish dataset/revision, suite, evaluation method, completed/resolved/unresolved/infrastructure-error counts, score, exact Qwen Code version and commit, and ECS run ID. Failed terminal runs publish status and counts without a numerical score.

This refactor removes the unused public FastAPI service, shared webhook token, Nginx template, release poller, supervisor, and commit Check Run. Release is now the only public result surface.

Final end-to-end verification

• Test prerelease: v0.20.0-nightly.20260722.b98306b7e
• Immutable Qwen Code commit: 77115af
• PR implementation commit: ed1c33baeb535adf025df378cc2f61bcde05aecd
• Actions run: 29984285727, success
• ECS run: qwen-bench-5b351919ee5b4cf3, one attempt, no recovery
• Dataset/case: swe-bench/swe-bench-verified@2, sympy__sympy-20590
• Result: 1/1 completed, 1 resolved, 0 unresolved, 0 infrastructure errors, 100.00%
• Harbor verifier reward: 1.0; exception_info: null
• Runtime: approximately 4m16s from queue creation to terminal state
• Artifact verification: 22 checksum entries passed; no publisher-error.json
• GitHub publication: Release GET 200, PATCH 200; no Check Run request
• Tests: 13 passed locally and 13 passed on ECS
• Worker FD check: 3 immediately after the run and 3 after another 20 seconds of polling

Reviewer test plan

1. Dispatch .github/workflows/stale.yml from this branch with benchmark_poc=true, a nightly prerelease tag, suite swebench_verified_harbor_smoke, and its Release ID.
2. Confirm the Action resolves the tag to an immutable commit and returns a new ECS run ID.
3. Confirm the ECS run reaches SUCCEEDED in one attempt.
4. Verify summary.json, Harbor trial result, verifier reward, and artifact checksums.
5. Confirm the triggering nightly Release contains the updated Qwen Code benchmark table and the new run ID.
6. Run PYTHONPATH=. pytest -q from benchmark-service; expected result: 13 passed.

Risk and scope

• Release publishing is best-effort; API failures are recorded in publisher-error.json. A durable outbox is outside this POC.
• Failed-run formatting is unit-tested, but no deliberately failed result was written to the public nightly Release.
• Multi-case terminal failure thresholds remain a later policy decision.
• Harbor's Docker backend can expose the model secret in child process arguments. The POC assumes a restricted single-tenant host; production should use credential injection that does not expand secrets into process arguments.
• The test updates an existing nightly prerelease only. It does not create or publish a stable Release.

中文说明

本 PR 将链路收敛为：GitHub Release/人工触发 → ECS self-hosted runner → 本地 dispatcher → SQLite → systemd worker → Harbor SWE-bench → GitHub Release 分数回写。不再需要公网 FastAPI，也不再创建 commit Check Run，Release 是唯一公开结果入口。

最终验收只使用现有 nightly 测试版本 v0.20.0-nightly.20260722.b98306b7e，没有创建或发布正式版本。Actions run 29984285727 创建 ECS run qwen-bench-5b351919ee5b4cf3；sympy__sympy-20590 单次成功，reward 1.0，Release 已展示 1/1 resolved 和 100.00%。22 个产物 checksum 全部通过，没有 publisher-error.json；发布日志只有 Release GET/PATCH 200。

[gwinthis]

Architecture Review — PR #7487

Verdict: ✅ Approve — clean async benchmark result publication.

Design pattern: Async result callback

The dispatch job finishes when ECS accepts the request, but the benchmark continues asynchronously. A terminal callback publishes the result back to the GitHub Release + Check Run.

Key decisions:

• Tag commit resolution via API (no full checkout on self-hosted runner) — minimizes runner I/O
• Best-effort publication with publisher-error.json on API failure — publication failure doesn't block the benchmark
• Failed runs reported as failed without a numerical score — distinguishes infrastructure failure from low performance
• Immutable metadata: commit SHA, Qwen Code version, ECS run ID — full provenance chain

[DennisYu07]

Superseded by #7583, which consolidates the trigger, ECS/Harbor worker, failure handling, and Release-only result publication into one PR based on the current main. The consolidated branch has also completed a fresh nightly prerelease end-to-end run.