Skip to content

Commit a8cd2cf

Browse files
Merge branch 'v5' into workflowmodel/v5-p4-editor
2 parents 4eb1456 + 8fe262e commit a8cd2cf

37 files changed

Lines changed: 2869 additions & 52 deletions
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Benchmarks: every hive now contributes **anonymized** per-PR benchmark records to the public leaderboard at <https://hivecommons.dev/benchmarks/> (data repo `hivecommons/benchmarks`), built from the review evidence bundles it already seals. A record carries a salted hash of the hive id, the agent, harness, model, sandbox runtime, task kind, ACMM level, review verdicts and merge/CI/revert outcome — never the hive id, PR numbers, SHAs, prompts, diffs, logins or costs. Publication is **on by default**; `benchmarks.publish: false` opts a hive out, and `benchmarks.repo_consent` (default on) names repositories only when GitHub reports them public, so no new information is disclosed. Cadence is hub-driven and hourly: the spoke pushes new records to the hub's `POST /api/benchmarks/ingest` with its heartbeat bearer, and the hub commits month files to the data repo through its App identity; `GET /api/benchmarks/export?since=` and the `benchmarks` block in `/api/status` expose the same data and last-push state on the spoke, `GET /api/hub/benchmarks` on the hub, and Settings → Features → Benchmarks shows the toggles and the hive's hash. New bundles are stamped with the agent's sandbox runtime and task kind. See `docs/benchmarks.md` (#17694).

‎dashboard/openapi.json‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25993,6 +25993,67 @@
2599325993
}
2599425994
}
2599525995
},
25996+
"/api/benchmarks/export": {
25997+
"get": {
25998+
"tags": [
25999+
"System"
26000+
],
26001+
"summary": "Export this hive's anonymized benchmark records",
26002+
"description": "Owner session. Folds the review evidence bundles under /data/evidence into hivecommons/benchmarks schema records (one per PR and agent role): model, harness, sandbox runtime, outcome, time-to-merge, ACMM level, keyed by this hive's salted hash. Repository and org names appear only for public repositories unless `benchmarks.repo_consent: false`. Returns `records: []` when `benchmarks.publish: false`. The same records are pushed hourly to the hub (hivecommons/hive#17694).",
26003+
"parameters": [
26004+
{
26005+
"name": "since",
26006+
"in": "query",
26007+
"required": false,
26008+
"schema": {
26009+
"type": "string"
26010+
},
26011+
"description": "RFC3339 or YYYY-MM-DD; PRs whose newest bundle is older are skipped. Default: 90 days ago."
26012+
}
26013+
],
26014+
"responses": {
26015+
"200": {
26016+
"description": "Benchmark records",
26017+
"content": {
26018+
"application/json": {
26019+
"schema": {
26020+
"type": "object",
26021+
"properties": {
26022+
"hive_hash": {
26023+
"type": "string"
26024+
},
26025+
"generated_at": {
26026+
"type": "string",
26027+
"format": "date-time"
26028+
},
26029+
"publish": {
26030+
"type": "boolean"
26031+
},
26032+
"since": {
26033+
"type": "string",
26034+
"format": "date-time"
26035+
},
26036+
"count": {
26037+
"type": "integer"
26038+
},
26039+
"records": {
26040+
"type": "array",
26041+
"items": {
26042+
"type": "object",
26043+
"description": "hivecommons/benchmarks schema/result.schema.json record"
26044+
}
26045+
}
26046+
}
26047+
}
26048+
}
26049+
}
26050+
},
26051+
"400": {
26052+
"description": "Bad since"
26053+
}
26054+
}
26055+
}
26056+
},
2599626057
"/api/evidence/summary": {
2599726058
"get": {
2599826059
"tags": [

‎src/cmd/hive/main_sweeps.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import (
1919
"github.com/hivecommons/hive/pkg/config"
2020
"github.com/hivecommons/hive/pkg/dashboard"
2121
"github.com/hivecommons/hive/pkg/escalation"
22+
"github.com/hivecommons/hive/pkg/evidence/benchmarks"
2223
"github.com/hivecommons/hive/pkg/forge"
2324
"github.com/hivecommons/hive/pkg/github"
2425
"github.com/hivecommons/hive/pkg/github/automerge"
@@ -2015,6 +2016,11 @@ func installReviewRelaySettings(client *github.Client, cfg *config.Config, logge
20152016
client.SetReviewEvidence(func(repo string) github.ReviewEvidenceSettings {
20162017
return reviewEvidenceSettings(cfg, repo)
20172018
})
2019+
// hivecommons/hive#17694: stamp the sandbox runtime and benchmark task
2020+
// kind on agent-authored bundles so the exporter need not guess.
2021+
github.EvidenceAgentContext = func(agent string) (string, string) {
2022+
return cfg.BenchmarkSandboxRuntime(agent), benchmarks.TaskKindFor(agent)
2023+
}
20182024
// hivecommons/hive#14581: issue-author credit on every merge path, read
20192025
// live so a reload of auto_merge.credit_issue_authors applies.
20202026
client.SetCreditIssueAuthors(func(string) bool { return cfg.AutoMerge.CreditIssueAuthorsEnabled() })

‎src/docs/api-reference/agents-and-controls.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f
3232
| `GET` | `/api/review/evidence/list` | Merger or owner | Every retained evidence bundle id and head for one PR (`repo`, `number`), oldest first, with the newest marked latest; `400` for a bad `repo`/`number` | `pkg/dashboard/api.go:528` |
3333
| `GET` | `/api/evidence/write-counters` | Merger or owner | Durable attempted vs blocked agent GitHub write counters ([#12069](https://github.com/hivecommons/hive/issues/12069)): `totals` by intent tier (ACMM mode) with `blocked_by_reason`, and `weeks` keyed by the UTC Monday of each ISO week with per-tier and per-reason counts. Persisted across restarts; observation only, never gates a write or merge. Empty `totals`/`weeks` before anything is recorded | `pkg/dashboard/api.go:530` |
3434
| `GET` | `/api/evidence/summary` | Dashboard auth/session | Read-only aggregate over `/data/evidence` bundles (schema v1) plus reviewer accuracy. Query `hours` (default 24, capped like `/api/pr-throughput`; `0` = all time) or a custom `from`/`to` range (RFC3339 or `YYYY-MM-DD`, date-only `to` inclusive, `to` defaults to now; `400` when `to` precedes `from`, the span exceeds 3 × 366 days, or `to` is given without `from`; the response echoes `from`/`to`), `repo` (`owner/name`) and `author_kind=human\|agent\|bot`; `400` for a bad value. A PR falls in the window by its open time (earliest bundle `created_at` or `merge.opened_at`), never by when a bundle was written; all bundles of a PR are folded so the newest supplies verdicts and the merge / `reverted_by` / backfilled close come from whichever record knows them. Returns `available_windows` (`{"24","168","720","all"}` → PR count under the same filters), `outcomes_by_author_kind` (merged/closed/open; `closed` comes from backfilled records, `reverted_14d` is null until bundles record it), `reviewer_calibration` buckets per perspective, `reviewer_accuracy_by_model` split by author kind (blocks split into `enforced_blocks` / `advisory_blocks` by `policy.require_approval`; advisory blocks that preceded a merge are `dissent_before_merge`, not false blocks), `sentinel_hits_per_week`, `shadow_gate` (`n`, `would_have_held`, `would_have_blocked`, and per gate `review_gate` / `intent` the evaluated `n` with `held` / `passed` outcomes `{n, merged, closed, open, reverted}`, plus `review_gate_by_lane` `{merge_eligible, self_merge}`), `write_counters` (per intent tier `{attempted, blocked, blocked_by_reason}` from the proxy write counter file; omitted when absent or unreadable), `write_activity` (blocked ÷ every agent write in the window: `total_writes`, `blocked`, `direct` `{recorded, since, partial_coverage, attempted, blocked, by_tier, blocked_by_reason}` from the proxy's weekly counter buckets under `/data/hive-metrics/write-counters.json`, `mediated` `{n, refused, by_action}` from the audit log), `claim_vs_verdict` (per ISO week of agent-authored PRs: `n`, `claimed`, `ci_captured`, `ci_not_green`, `verdict_not_approve`, `both`; records with empty CI are excluded from the CI column and disclosed under `missing.records_empty_ci`), `ci_recovery` (`policy` `{self_authored_automerge, required_checks}`, `n`, `at_claim` `{green, red, pending, none}`, `red` `{n, recovered_merged, green_open, abandoned, pending, outcome_unknown, enriched, not_enriched, median_pushes_to_green, median_hours_to_green, same_head_recovered}`, `gate` `{merged_n, with_checks, green, green_from_bundles, non_required_red, required_unknown, breaches, breach_prs[], suspected, suspected_prs[], pending, unknown, green_to_merge_n, median_green_to_merge_hours}` over every merged PR by GitHub-confirmed state of the *required* checks on the merged head, as the merge gate judges (`breach_prs[]` carry `red`, `merged_by`; a red non-required check is disclosed, not a breach; a red review-time snapshot, a required check with no run, or a red read without a resolvable required set is only suspected), and `convergence` `{n, passes {"1","2","3","4+","never"}, settled, in_flight, unknown, median_passes, from_github, from_bundles, by_week[]}` over agent PRs by CI passes to first green, from the durable `/data/hive-metrics/ci-recovery.json` GitHub check-run trail rollup), `work_balance` (per ISO week: `n`, `opened` by actor kind `{human, agent, bot}`, `closed`, `prs_opened`, `prs_merged`, `prs_closed`, `queue_issues`, `queue_prs`, `issue_source` `github|audit`, `throttled`) with `work_balance_sources` (`issues_github_since`, `issues_fetched_at`, `issues_known`, `audit_only_weeks`, `issue_series_weeks`, `queue_samples`, `queue_retention_days`; issues from the durable GitHub rollup `/data/hive-metrics/issue-activity.json` for weeks at or after `issues_github_since` and, for earlier weeks, from the backfilled issue series `<evidence-root>/<owner>/<repo>/issues/<YYYY-Www>.json` written by the issue backfill ([#12527](https://github.com/hivecommons/hive/pull/12527)) when its coverage spans the requested portion of the week (`issue_series_weeks` counts the repo-weeks it supplied), else the audit log (`audit_only_weeks`), PRs from folded bundles, queue depth from governor snapshots; `missing.issues_human_opens_unobserved` is true while no GitHub rollup exists unless the issue series covers every row, and whenever any row fell back to the audit log, `missing.throttle_windows_unavailable` discloses that no governor throttle windows are reported), `intent_counters` (per intent tier `{attempted, blocked}` from `/data/hive-metrics/intent-counters.json` — evaluations and the subset the intent gate would have blocked with enforcement on, [#12362](https://github.com/hivecommons/hive/issues/12362); omitted when absent or unreadable), `complexity_by_repo` (per managed repo the newest 12 monthly complexity snapshots, oldest first, each `{month, ref, captured_at, files, functions, mean_cyclomatic, median_cyclomatic, p90_cyclomatic, max_cyclomatic, functions_over_threshold, threshold, static_warnings, static_warnings_by_tool}`; omitted when no repo has a snapshot), the `/api/reviewer/accuracy` summary, and `n` plus `missing` counts (unknown-model verdicts, empty CI, unreadable records). Never consulted by merge decisions. | `pkg/dashboard/api.go:533` |
35+
| `GET` | `/api/benchmarks/export` | Owner only | Anonymized per-PR [benchmark records](../benchmarks.md) (schema `hivecommons/benchmarks` `result.schema.json`) built from the evidence bundles updated since `since` (RFC3339 or `YYYY-MM-DD`; default 90 days; `400` when malformed) plus `hive_hash`, `publish`, `generated_at` and `count`. `records` is empty when `benchmarks.publish: false`; repo/org names appear only for GitHub-public repos when `benchmarks.repo_consent` is on | `pkg/dashboard/api.go:534` |
3536
| `GET` | `/api/agents` | Dashboard auth/session | Agents List | `pkg/dashboard/api.go:320` |
3637
| `POST` | `/api/agents` | Owner only | Agent Create | `pkg/dashboard/api.go:321` |
3738
| `POST` | `/api/agents/import` | Owner only | Agent Import | `pkg/dashboard/api.go:322` |

‎src/docs/api-reference/auth.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,6 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f
2424
| `GET` | `/api/openrouter/models` | Dashboard auth/session | Open Router Models | `pkg/dashboard/openrouter.go:47` |
2525
| `GET` | `/api/openrouter/credit` | Dashboard auth/session | Open Router Credit | `pkg/dashboard/openrouter.go:48` |
2626
| `GET` | `/openrouter/callback` | Public | Open Router Callback | `pkg/dashboard/openrouter.go:49` |
27-
| `POST` | `/api/github-app/recheck` | Dashboard auth/session | GitHub App Recheck | `pkg/dashboard/server.go:1675` |
28-
| `POST` | `/api/github-app/install-clicked` | Dashboard auth/session | GitHub App Install Clicked | `pkg/dashboard/server.go:1676` |
29-
| `GET` | `/gh-setup` | Public | GitHub App Setup Callback | `pkg/dashboard/server.go:1677` |
27+
| `POST` | `/api/github-app/recheck` | Dashboard auth/session | GitHub App Recheck | `pkg/dashboard/server.go:1681` |
28+
| `POST` | `/api/github-app/install-clicked` | Dashboard auth/session | GitHub App Install Clicked | `pkg/dashboard/server.go:1682` |
29+
| `GET` | `/gh-setup` | Public | GitHub App Setup Callback | `pkg/dashboard/server.go:1683` |

‎src/docs/api-reference/configuration.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f
8585
| `GET` | `/api/compliance/export` | Owner only | Evidence download for `?since=`/`?until=`: `kind=controls` (`json`/`md`), `posture` (`json`/`csv`), `audit` (`json`), `config` (redacted effective config + sha256, `json`), `attestations` (`json`/`csv`) or `bundle` (`json`). Every format includes source, period, generator, selected profile versions, attesters, retention and a per-section SHA-256 manifest; audited as `compliance_export` with the SHA-256 of the delivered bytes ([#11081](https://github.com/hivecommons/hive/issues/11081), [#13582](https://github.com/hivecommons/hive/issues/13582)) | `pkg/dashboard/api.go:369` |
8686
| `GET` | `/api/compliance/attestations` | Owner only | Owner attestations, newest first (`?framework=` filters) ([#11081](https://github.com/hivecommons/hive/issues/11081)) | `pkg/dashboard/api.go:370` |
8787
| `POST` | `/api/compliance/attestations` | Owner only | Record `{"framework", "reviewed_on", "note"}` for the signed-in owner; audited as `compliance_attestation`; `201` with the attestation, `400` for an unknown framework, a bad or future date, or a note over 2000 characters ([#11081](https://github.com/hivecommons/hive/issues/11081)) | `pkg/dashboard/api.go:371` |
88-
| `PUT` | `/api/config/governor/compliance` | Owner only | Settings → Compliance framework picker: `{"frameworks": [...]}` sets `compliance.frameworks` (empty list deselects all, absent key leaves it unchanged; unknown/duplicate ids are a `400` before any change); audited as `config_compliance`; returns the re-evaluated `compliance.Report` ([#11080](https://github.com/hivecommons/hive/issues/11080)) | `pkg/dashboard/api.go:540` |
88+
| `PUT` | `/api/config/governor/compliance` | Owner only | Settings → Compliance framework picker: `{"frameworks": [...]}` sets `compliance.frameworks` (empty list deselects all, absent key leaves it unchanged; unknown/duplicate ids are a `400` before any change); audited as `config_compliance`; returns the re-evaluated `compliance.Report` ([#11080](https://github.com/hivecommons/hive/issues/11080)) | `pkg/dashboard/api.go:541` |
8989

9090
| `GET` | `/api/config/governor/threshold-scaling` | Owner only | Governor Threshold Scaling Get | `pkg/dashboard/api.go:220` |
9191
| `PUT` | `/api/config/governor/threshold-scaling` | Owner only | Governor Threshold Scaling Set | `pkg/dashboard/api.go:221` |
@@ -94,8 +94,8 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f
9494
| `PUT` | `/api/config/governor/watchdog` | Owner only | Governor Watchdog | `pkg/dashboard/api.go:230` |
9595
| `GET` | `/api/config/escalation` | Owner only | Escalation Config Get | `pkg/dashboard/api.go:233` |
9696
| `PUT` | `/api/config/escalation` | Owner only | Escalation Config Set | `pkg/dashboard/api.go:234` |
97-
| `GET` | `/api/config/write-surface` | Owner only | Lane write allowlist (`write_surface.allowlist`, [#9587](https://github.com/hivecommons/hive/issues/9587)): `allowlist`, `ops`, `warnings` | `pkg/dashboard/api.go:536` |
98-
| `PUT` | `/api/config/write-surface` | Owner only | Replace the lane write allowlist; `{"allowlist": {}}` clears it; unknown operation or bad lane name is a 400 with nothing changed | `pkg/dashboard/api.go:537` |
97+
| `GET` | `/api/config/write-surface` | Owner only | Lane write allowlist (`write_surface.allowlist`, [#9587](https://github.com/hivecommons/hive/issues/9587)): `allowlist`, `ops`, `warnings` | `pkg/dashboard/api.go:537` |
98+
| `PUT` | `/api/config/write-surface` | Owner only | Replace the lane write allowlist; `{"allowlist": {}}` clears it; unknown operation or bad lane name is a 400 with nothing changed | `pkg/dashboard/api.go:538` |
9999
| `GET` | `/api/config/review` | Dashboard auth/session | Review Config Get | `pkg/dashboard/api.go:237` |
100100
| `PUT` | `/api/config/review` | Owner only | Review Config Set | `pkg/dashboard/api.go:240` |
101101
| `PUT` | `/api/config/governor/features` | Owner only | Governor Features | `pkg/dashboard/api.go:246` |

0 commit comments

Comments
 (0)