Repository navigation
π fix(review): A review the GitHub proxy cut short at the agent cap is no verdict and is asked again, whatever its wording - #17376
Conversation
β¦ver its wording When the reviewer reached its agents.github_api_hourly_cap, the proxy refused its reads and the session finished with requires_human in its own words: "GitHub reads stopped at the user's request after the hourly cap; ... could not be verified" (projectbluefin/utah-packages#959), "not assessed: GitHub API cap prevented reading" (projectbluefin/utah#1161). review.UnreadableReviewReason (hivecommons#16994) only knows "could not read"-style wording, so these were recorded and parked the PRs at 085. The proxy now remembers, per agent, when it last refused a request for the hourly cap or the reserve floor (AgentGitHubRefusedSince). The review relay asks it: a requires_human with no in-scope finding, no high-risk marker and no already-landed PR, from an agent refused at or after the perspective's dispatch time, is not recorded. The dispatch row stays open and the perspective is re-asked after review.pending_ttl, as in the hivecommons#16994 case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Read against v6 at the merge-base, including the callers. The change looks correct to me; one open question and one behavioural note, neither a defect.
Verified
src/crux/proxy/agent_github_budget.goβgithubBudgetRefusalis called only fromgithub_proxy_mitm.go:294, and the newdefermarks the agent refused on both thecapandreservereturns.markRefused/refusedSinceguard a nil receiver, so the reserve-floor path withp.githubBudget == nilis safe (it just records nothing).agentCapSessionLookback(1h) matchesagentGitHubBudgetWindowatagent_github_budget.go:19.src/pkg/github/review_verdict_agent_cap.goβreviewPerspectiveDispatchedAtreadsPendingReview.Repo/Number/Perspective/Dispatchedas they exist inpkg/review/dispatch.go:398-429;LoadDispatchState("")is the existing default-path entry atdispatch.go:631. Thecontinueinreview_verdict_relay.gomirrors the unreadable-wording branch directly above it, so the pending row survives and is re-asked on the same TTL path.review.CapStoppedReviewReasonusesInScopeFinding(review.go:1415) and the existingDecisionRiskHigh/RedundantOffields; it never fires forapprove/changes_requested.cmd/hive/main.go:4333installs the hook right afterSetAgentGitHubBudgetConfig, so the proxy exists when it is wired.- Body and diff agree, including the "Not in this PR" items.
Open question (not verified either way)
capStoppedReview keys the lookup by req.Agent (the name hive-review writes, or the sandbox manifest's agent at review_request_watcher.go:288), while the proxy keys refused by the MITM connection's agentName. If those two names ever differ for the same lane (e.g. a sandbox spool), the check is silently inert β it would never set a verdict aside, which fails safe but defeats the fix. Worth a one-line confirmation from whoever knows the proxy's agent identity path that they are the same string.
Behavioural note
The dispatch-row lookup does not filter by PendingReview.Agent, and a refusal that lands after the agent posts its verdict but before the watcher tick also counts. Both err toward re-asking, as the body says, and each costs one max_redispatch slot; fine as designed, just naming it.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from The code does what the body says, and I found no defect in the mechanism. The question I cannot settle is a policy one about which What I verified
The policy question (correctness/intent, medium)
Two cheap ways to narrow it if you agree it matters: keep the report when I have not closed or changed anything. Evidence bundle `#1737681bcae9` sha256:087abe065092684ad3ddc4ddf2071bc0273334389de80735ba135487b7089705 β download from the dashboard or `hivectl review evidence #17376`. |
β¦ max_redispatch The previous commit sets aside a requires_human from a session the proxy refused at the agent cap, and the perspective is asked again after review.pending_ttl. But the expiry that follows still counted against review.max_redispatch (3), so a reviewer capped for about 2h15m on one head (three 45m TTLs) still parked it as review-dispatch-exhausted and a person was asked about a PR nobody had read. The proxy now also remembers when each refusal said the agent could read again (its Retry-After: the sliding hour freeing a slot, or the shared bucket's reset), and exposes it as GitHubProxy.AgentGitHubRefusal. The same process-wide hook feeds the relay and, through review.DispatchOptions.AgentGitHubRefusal, the planner. When every row of an expiry went to an agent the proxy refused after the row was sent, the planner waits for the refusal to clear (at most an hour past the TTL), then re-asks and counts the expiry in the new cap_redispatches[] instead of redispatches[]. That count is bounded separately (DefaultMaxCapRedispatch = 8) and ends in the same review-dispatch-exhausted hold, so a reviewer that can never read still reaches a person instead of looping. Any other expiry counts as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether the relay may set aside a requires_human that carries a hive-decision-question (no findings) on proxy-refusal evidence alone, including the 1-hour fallback lookback used when no dispatch row exists for the perspective.
The mechanics read correctly; the open point is a routing policy choice, not a defect.
Verified
src/crux/proxy/agent_github_budget.go(PR head): thedeferingithubBudgetRefusalruns after both thereason=capandreason=reservereturns, so both refusals are recorded;markRefusedis nil-receiver-safe, so a proxy built without a budget (p.githubBudget == nil) does not panic.refusedSincecompareslast.at.Before(since)under the same mutex as the other maps.src/pkg/review/dispatch.goexpirePending: the cap-caused branch is taken beforeRedispatchesis touched, so cap-caused expiries neither increment nor resetredispatches[], as the doc claims.pruneStatedropsCapRedispatchesfor closed heads alongsideRedispatches.src/pkg/review/dispatch_agent_cap.gocapCausedExpiry: one expired row whose agent was not refused makes it an ordinary expiry (returnsfalse,falseon the first miss), and the wait is bounded atearliest + ttl + 1h, so a refusal record with a far-futureclearscannot park a head indefinitely.src/pkg/github/review_verdict_relay.go:126-137mirrors the existing unreadable set-aside: the comment is still posted, only the verdict write is skipped.review-swarm.mdand the changelog fragment describe exactly this behaviour, including the restart caveat.
The policy question
src/pkg/review/unreadable.go CapStoppedReviewReason keeps a report only on DecisionRiskHigh, RedundantOf or an in-scope finding. A requires_human whose whole content is a hive-decision-question β the normal shape for "a maintainer must choose intent/scope/ownership" β is set aside whenever the posting agent had any refusal after since. And src/pkg/github/review_verdict_agent_cap.go falls back to since = now - 1h when no pending row matches, which is the case for reviewer kicks that did not come from the planner. Reviewer lanes that sit near their 300/h cap most of the day would then have their needs-decision verdicts deferred up to MaxCapRedispatch (8) Γ pending_ttl before converging to a hold, with a fresh comment on each re-ask. The PR body states this trade-off ("errs toward asking again"); I am not saying it is wrong, only that it is a routing policy decision an approver should make knowingly, because it changes when 085-needs-decision gets reached on a capped hive.
Two cheaper alternatives if the answer is "too broad": (a) keep a report whose DecisionQuestion is non-empty unless the summary also matches rateLimitCauseRe; or (b) drop the 1-hour fallback and only set aside when a dispatch row pins since.
One nit, not blocking: reviewPerspectiveDispatchedAt picks the latest row across heads for the PR+perspective, so when an older head's row is still open and a newer head was dispatched later, a refusal between the two dispatches is missed. Conservative direction (records the verdict), so fine.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Summary
When the reviewer hits its own GitHub API cap (
agents.github_api_hourly_cap, 300/h), the proxy refuses its reads and the review comes backrequires_humanin whatever words the model picks. The existing guard (Refs #16994, #17082) only recognises "could not read β¦ 429"-style wording, so these reviews were recorded as verdicts and parked PRs at 085 on 2026-11-09. This PR makes the decision on the cap event itself: the proxy remembers when it last refused each agent, and the review relay sets aside arequires_humanfrom an agent that was refused after the perspective was dispatched. The perspective is asked again afterreview.pending_ttl, the same path as the 429 case. A follow-up commit makes sure those retries don't use upreview.max_redispatch, so a reviewer capped for a few hours no longer ends with a person being asked anyway.What went wrong
The live log shows
agent GitHub API hourly cap reached agent=reviewer count=300 cap=300at 01:33, 03:32 and 05:37Z. The warning is logged once per agent per hour, so the reviewer was capped for longer than those three moments. Reviews that ran into the cap came backrequires_humanand asked for a person:hive: agent GitHub API hourly cap reached (300/300).None of these contains a phrase from
unreadableReviewTerms, soreview.UnreadableReviewReasonpasses them through. The "at the user's request" wording comes from the proxy's own nudge, which tells the agent to "stop all GitHub reads for this session".What changes
crux/proxy/agent_github_budget.go): every refusal fromgithubBudgetRefusal(the hourly cap or the reserve floor) records the time against the agent. NewGitHubProxy.AgentGitHubRefusedSince(agent, since)answers whether the agent was refused at or aftersince. The record is in memory only, like the counts.pkg/github/review_verdict_agent_cap.go,review_verdict_relay.go): after the existing wording check,recordReviewVerdictasks the proxy whether the posting agent was refused since the open dispatch row for that PR and perspective was sent. If no row is found, it looks back one hour. If the agent was refused, a qualifyingrequires_humanis not recorded and the watcher logs why.review.CapStoppedReviewReason):requires_humanwith no in-scope finding, nohive-decision-risk: highmarker and no already-landed PR. Anapproveorchanges_requestedis still recorded. A decision question does not keep the report, because a cut-short session asks for "a review with access to the changes" (finpilot#659). A real question comes back from the re-review that can read the PR. This differs on purpose from the wording check, which keeps decision questions because it has no other evidence.cmd/hive/main.go):github.SetAgentGitHubRefusalFunc(b.githubProxy.AgentGitHubRefusal)is installed where the proxy's budget config is set. It is a process-wide hook, likeSetIssueReopenRecorder, because the proxy is built after the GitHub client. The review planner reads the same hook (github.AgentGitHubRefusalFunc(),cmd/hive/main_sweeps.go).The test errs toward asking again. If the cap hit the lane's previous session after this row was dispatched, the report is still set aside, which costs one extra review and never records a verdict that wasn't read.
Follow-up commit: cap retries don't use up
max_redispatchThe first commit set the report aside, but the expiry after it still counted as one of the three
review.max_redispatchtries. Three 45-minute TTLs is about 2h15m, so a reviewer capped that long on one head still ended in areview-dispatch-exhaustedhold.Retry-After. That is when the sliding hour frees a slot, or when the shared bucket resets.GitHubProxy.AgentGitHubRefusal(agent, since)returns that time.AgentGitHubRefusedSinceis now a thin wrapper around it.pkg/review/dispatch_agent_cap.go): when the pending rows expire, the planner checks whether the proxy refused the agent of every expired row after that row was sent. If so, the cap caused the expiry.Retry-Afterhas passed, for at most one hour pastpending_ttl. Asking sooner would just be refused again. The rows stay in place while it waits, so the perspective is not asked twice.cap_redispatches[]count instead ofredispatches[]. The governor'sreview pending expiredline now carriescap_caused=true.DefaultMaxCapRedispatch= 8. That's about a working day at the default TTL, and it isn't a config key. When it is reached, the planner sets the samereview-dispatch-exhaustedhold, with a reason that names the cap. So a reviewer whose cap is too low to ever finish a review still reaches a person and does not loop forever.pending_ttl.max_redispatchif any expired row's agent was not refused, if the refusal came before the row was sent, if the refusal hit another agent, or if no proxy record exists. A restart forgets the proxy record, so an expiry right after a restart counts as ordinary.Files
src/crux/proxy/agent_github_budget.go,agent_github_budget_test.gosrc/pkg/review/unreadable.go,unreadable_test.gosrc/pkg/review/dispatch.go,dispatch_agent_cap.go,dispatch_agent_cap_test.go(follow-up)src/pkg/github/review_verdict_agent_cap.go,review_verdict_agent_cap_test.go,review_verdict_relay.gosrc/cmd/hive/main.go,main_sweeps.gosrc/docs/review-swarm.md,changelog.d/fixed-review-agent-cap-no-verdict.mdTests
None were run locally (AGENTS.md); CI is the check.
TestAgentGitHubRefusedSinceRecordsCapAndReserveRefusals: a cap refusal and a reserve-floor refusal are both recorded, scoped by agent and time. Under the cap nothing is recorded. The hive's own calls and a nil proxy never read as refused.TestCapStoppedReviewReasonDoesNotDependOnWording/TestCapStoppedReviewReasonKeepsJudgements: the utah-packages#959 and utah#1161 summaries are set aside, and the test asserts that the wording check misses them. The finpilot#659 decision question is set aside too. Approve, changes_requested, an in-scope finding, the high-risk marker and an already-landed PR are all kept.TestRecordReviewVerdictCapStoppedSessionIsNoVerdict: the three utah-packages#959 reports are posted after a refusal that falls inside the session. Nothing is written, the refusal window starts at the dispatch time, and after the pending TTL the planner re-asks without asking a person.TestRecordReviewVerdictKeepsRequiresHumanWithoutARefusalInTheSession: the same reports are recorded when no proxy is wired, when the refusal came before the dispatch, when another agent was refused, and when the verdict is approve.Follow-up commit:
TestPlanDispatchCapCausedExpiriesDoNotSpendRedispatchBudget: eight cap-caused expiries in a row. After the third, and up to the seventh, the head is not held and is asked again, andredispatchesstays at 0. The eighth sets thereview-dispatch-exhaustedhold, once, with a reason that names the cap.TestPlanDispatchOrdinaryExpiriesStillExhaust: no proxy record, a refusal before the row was sent, another agent refused, or only one expired row's agent refused. In each case the head is still held after three expiries, as before.TestPlanDispatchCapCausedExpiryWaitsForTheRefusalToClear: no re-ask and no count before the refusal clears. One re-ask after it clears. No wait more than an hour past the TTL.TestPlanDispatchCapRedispatchesResetOnNewHead: a new push drops the cap count.TestCapStoppedSessionsInARowDoNotExhaustTheRedispatchBudget: the full path, relay plus planner. Four utah-packages#959 sessions in a row, each refused by the proxy, never set a hold. The testsuite#1116 "could not read β¦ 429" wording with no proxy refusal still sets the hold after the third session.TestAgentGitHubRefusedSinceRecordsCapAndReserveRefusalsnow also checks the clear time: one hour after the oldest counted request for the cap, the bucket reset for the reserve floor.On the v6 tip and on the first commit, the follow-up tests don't compile, because
AgentGitHubRefusal,CapRedispatchCountForandSetAgentGitHubRefusalFuncare new. Without the planner change, the cap loop sets the hold on its third cycle and fails.On the v6 tip the first three tests don't compile (
AgentGitHubRefusedSince,CapStoppedReviewReasonandSetAgentGitHubRefusedSinceFuncdon't exist). Without the relay change, the utah-packages#959 reports would be recorded and the first relay test would fail onrecorded 3 verdicts.Not in this PR
review.review_headson, a head dispatch that expires still counts towardreview.head_max_retrieswhatever the cause. The PR-levelmax_redispatchfallback after that does get the cap rule.redispatch_countshows only the ordinary count. Cap-caused expiries are in the state file'scap_redispatches[]and the governor log, not on the dashboard.π€ Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1