Skip to content

Commit af42053

Browse files
Danatharclaudehivecommons-hive[bot]
authored
🐛 [v6] contributor-first: a relay at its task limit does not cover a lane (#17268)
RoleCoverage counted a relay as covering its role even when task selection refused it work because its identity had used up its tier's max_per_hour or max_per_day. The relay stays connected and is refused every ready with hourly_limit / daily_limit until the window rolls, so the governor kept the resident agent's kicks withheld while nobody worked the lane. Same class as the standby / quota-hold gaps #17233 handled. - Factor the hourly/daily rule out of selectTaskPass into tierRateLimitReached (hub.tier_limits[tier], caps <= 0 unlimited, hourly checked first). Selection keeps its log lines, decision text and refusal reasons. - RoleCoverage skips a candidate whose identity is at either cap, using the same function, after h.mu is released (rateMu only; no selectMu). - Docs: RoleCoverage comment, governor contributor-first header, and the contributor-first coverage list in agent-configuration.md. Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
1 parent 74620f6 commit af42053

6 files changed

Lines changed: 235 additions & 32 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Contributor-first lanes are no longer left unworked by a relay that has used up its hourly or daily task allowance. A relay at its tier's `max_per_hour` or `max_per_day` is refused work until the window rolls, but it still counted as covering its role, so the governor kept the resident agent's kicks withheld while nobody worked the lane (for the daily cap, that can be most of a day). The hub now leaves such a relay out of coverage, using the same limit check that refuses it work.

‎src/docs/agent-configuration.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1384,9 +1384,16 @@ A relay counts as covering the role only when all of these hold:
13841384
tells the hub (`quota_hold_v1`, see
13851385
[the contributor relay guide](contributor-relay.md#a-held-relay-tells-the-hub-quota_hold_v1));
13861386
an older relay is still counted while held;
1387+
- it has not used up its hourly or daily task allowance
1388+
(`hub.tier_limits.<tier>.max_per_hour` / `max_per_day` for the
1389+
contributor's trust tier). The hub refuses such a relay work
1390+
(`hourly_limit` / `daily_limit`) until the window rolls, so it covers
1391+
nothing in the meantime. Coverage and task assignment use the same check,
1392+
so they always agree on which limits apply and how they are counted;
13871393
- contributions are not suspended (`hub.contribute_suspended`).
13881394

1389-
A relay that is busy on a task still counts.
1395+
A relay that is busy on a task still counts, including one holding its
1396+
tier's `max_concurrent` tasks.
13901397

13911398
```yaml
13921399
agents:

‎src/pkg/dashboard/contribute_coverage.go‎

Lines changed: 22 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -31,23 +31,30 @@ import (
3131
// Observed on 2026-11-08: a relay held at 15% of its weekly window kept
3232
// the scanner lane "covered" while nobody worked it. A relay too old to
3333
// send quota_hold is still counted, as before;
34+
// - its identity has not used up its hourly or daily task allowance
35+
// (hub.tier_limits max_per_hour / max_per_day for its trust tier). Task
36+
// selection refuses such a relay with hourly_limit / daily_limit until the
37+
// window rolls, so it takes nothing in the meantime. The check is
38+
// tierRateLimitReached, the same function selection refuses with, so the
39+
// two can never disagree about which limits apply or how they are counted;
3440
// - the operator has not suspended contributions (hub.contribute_suspended).
3541
// The suspend switch refuses work to every relay while their sockets stay
3642
// up, so a connected relay is then covering nothing; reporting it as
3743
// coverage would turn "pause contributors" into "pause the lane".
3844
//
3945
// A relay that is busy on a task still counts — it is draining the lane,
40-
// which is the point. Standby and quota hold do not wait for the task to end:
41-
// both mean the relay will not take the lane's NEXT item, which is what the
42-
// governor is deciding whether to withhold. Whether the relay is ALLOWED the role (trust tier,
46+
// which is the point. Standby, quota hold and a used-up allowance do not wait
47+
// for the task to end: each means the relay will not take the lane's NEXT
48+
// item, which is what the governor is deciding whether to withhold. (Holding
49+
// max_concurrent tasks is different: that relay is busy, and counts.) Whether the relay is ALLOWED the role (trust tier,
4350
// grants, the delegatable list) was settled when the hub set c.role; nothing
4451
// is re-derived here.
4552
func (h *ContributeWSHub) RoleCoverage(now time.Time) map[string]governor.ContributorCoverage {
4653
if h == nil || h.contributionsSuspended() {
4754
return nil
4855
}
4956
type candidate struct {
50-
role, login, identity string
57+
role, login, identity, tier string
5158
}
5259
var candidates []candidate
5360
h.mu.RLock()
@@ -58,26 +65,33 @@ func (h *ContributeWSHub) RoleCoverage(now time.Time) map[string]governor.Contri
5865
c.mu.Lock()
5966
role := normalizeAgentRole(c.role)
6067
fresh := now.Sub(c.lastPong) <= wsHeartbeatTimeout
61-
login := ""
68+
login, tier := "", ""
6269
if c.profile != nil {
6370
login = strings.TrimSpace(c.profile.GitHubUsername)
71+
tier = c.profile.TrustTier
6472
}
6573
notTakingWork := c.standbyDeclared || len(c.standby) > 0 || c.quotaHeld
6674
c.mu.Unlock()
6775
if role == "" || !fresh || login == "" || notTakingWork {
6876
continue
6977
}
70-
candidates = append(candidates, candidate{role: role, login: login, identity: identityOf(c)})
78+
candidates = append(candidates, candidate{role: role, login: login, identity: identityOf(c), tier: tier})
7179
}
7280
h.mu.RUnlock()
7381

74-
// The failure-streak ledger has its own lock; consult it with h.mu
75-
// released so the two are never nested.
82+
// The failure-streak ledger and the assignment-rate windows each have
83+
// their own lock (failureStreakMu, rateMu); consult them with h.mu
84+
// released so neither is ever nested inside it. The rate check is a scan
85+
// of at most a day of assignment stamps for one identity, once per
86+
// candidate per governor evaluation, and never takes selectMu.
7687
byRole := make(map[string]map[string]struct{})
7788
for _, cand := range candidates {
7889
if paused, _, _ := h.contributorFailureStreakActive(cand.identity, now); paused {
7990
continue
8091
}
92+
if h.tierRateLimitReached(cand.identity, cand.tier, now).Reason != "" {
93+
continue
94+
}
8195
if byRole[cand.role] == nil {
8296
byRole[cand.role] = make(map[string]struct{})
8397
}

‎src/pkg/dashboard/contribute_coverage_test.go‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import (
66
"testing"
77
"time"
88

9+
"github.com/hivecommons/hive/pkg/config"
910
"github.com/hivecommons/hive/pkg/governor"
1011
standbypkg "github.com/hivecommons/hive/pkg/standby"
1112
)
@@ -214,3 +215,142 @@ func TestRoleCoverage_RejectedStandbyDeclarationDoesNotCover(t *testing.T) {
214215
t.Fatalf("rejected standby relay counted as cover: RoleCoverage = %+v, want nil", got)
215216
}
216217
}
218+
219+
// rateLimitedCoverageHub builds a hub whose "contributor" tier has the given
220+
// limits and seeds identityOf(relay) with one assignment stamp per age, as
221+
// recordAssignment does when selection hands a task out.
222+
func rateLimitedCoverageHub(t *testing.T, limits config.TierRate, relay *ContributorConnection, now time.Time, ages ...time.Duration) (*ContributeWSHub, *Server) {
223+
t.Helper()
224+
hub, s := covK2Hub(t)
225+
s.deps.Config.Hub.TierLimits = map[string]config.TierRate{"contributor": limits}
226+
for _, age := range ages {
227+
hub.recordAssignment(identityOf(relay), now.Add(-age))
228+
}
229+
hub.mu.Lock()
230+
hub.connections[relay.profile.GitHubUsername] = relay
231+
hub.mu.Unlock()
232+
return hub, s
233+
}
234+
235+
// selectionRefusal asks real task selection for work on behalf of relay's
236+
// identity and returns the task_unavailable reason. The probe connection has
237+
// no role, so the answer is about the identity's limits, not role policy.
238+
func selectionRefusal(t *testing.T, hub *ContributeWSHub, s *Server, relay *ContributorConnection) string {
239+
t.Helper()
240+
oneActionableIssueN(s, 0)
241+
msg := hub.selectTask(&ContributorConnection{profile: relay.profile, lastPong: time.Now()})
242+
if msg == nil || msg.Type != "task_unavailable" {
243+
t.Fatalf("setup: selection must refuse this identity, got %+v", msg)
244+
}
245+
return msg.Reason
246+
}
247+
248+
// A relay that has been handed its tier's max_per_hour tasks in the last hour
249+
// is refused work with hourly_limit until the window rolls. It is connected
250+
// and acting as the role, but it will not take the lane's next item, so it
251+
// must not keep the resident agent withheld.
252+
func TestRoleCoverage_HourlyLimitReachedDoesNotCover(t *testing.T) {
253+
now := time.Now()
254+
relay := coverageConn("alice", "quality", now)
255+
hub, s := rateLimitedCoverageHub(t, config.TierRate{MaxPerHour: 2, MaxPerDay: 50, MaxConcurrent: 5},
256+
relay, now, 10*time.Minute, 20*time.Minute)
257+
258+
if got := selectionRefusal(t, hub, s, relay); got != taskUnavailableHourlyLimit {
259+
t.Fatalf("setup: selection refused with %q, want %q", got, taskUnavailableHourlyLimit)
260+
}
261+
if got := hub.RoleCoverage(now); got != nil {
262+
t.Fatalf("relay at its hourly limit counted as cover: RoleCoverage = %+v, want nil", got)
263+
}
264+
}
265+
266+
// The same for the daily cap: no assignment in the last hour, but the tier's
267+
// max_per_day in the last 24 hours, so selection refuses with daily_limit.
268+
func TestRoleCoverage_DailyLimitReachedDoesNotCover(t *testing.T) {
269+
now := time.Now()
270+
relay := coverageConn("alice", "scanner", now)
271+
hub, s := rateLimitedCoverageHub(t, config.TierRate{MaxPerHour: 10, MaxPerDay: 3, MaxConcurrent: 5},
272+
relay, now, 2*time.Hour, 5*time.Hour, 20*time.Hour)
273+
274+
if got := selectionRefusal(t, hub, s, relay); got != taskUnavailableDailyLimit {
275+
t.Fatalf("setup: selection refused with %q, want %q", got, taskUnavailableDailyLimit)
276+
}
277+
if got := hub.RoleCoverage(now); got != nil {
278+
t.Fatalf("relay at its daily limit counted as cover: RoleCoverage = %+v, want nil", got)
279+
}
280+
}
281+
282+
// A relay one task short of both caps still covers: only a used-up allowance
283+
// takes a relay out. Stamps older than the day window do not count. Bob, in
284+
// the same tier and lane, is at the hourly cap and is the only one dropped.
285+
func TestRoleCoverage_UnderRateLimitsCovers(t *testing.T) {
286+
now := time.Now()
287+
alice := coverageConn("alice", "quality", now)
288+
bob := coverageConn("bob", "quality", now)
289+
// alice: 1 of 2 this hour, 2 of 3 today (the 25h-old stamp has rolled off).
290+
hub, _ := rateLimitedCoverageHub(t, config.TierRate{MaxPerHour: 2, MaxPerDay: 3, MaxConcurrent: 5},
291+
alice, now, 10*time.Minute, 3*time.Hour, 25*time.Hour)
292+
hub.recordAssignment(identityOf(bob), now.Add(-5*time.Minute))
293+
hub.recordAssignment(identityOf(bob), now.Add(-15*time.Minute))
294+
hub.mu.Lock()
295+
hub.connections["bob"] = bob
296+
hub.mu.Unlock()
297+
298+
want := map[string]governor.ContributorCoverage{"quality": {Contributors: []string{"alice"}}}
299+
if got := hub.RoleCoverage(now); !reflect.DeepEqual(got, want) {
300+
t.Fatalf("RoleCoverage = %+v, want %+v (alice under both caps, bob at the hourly cap)", got, want)
301+
}
302+
}
303+
304+
// tierRateLimitReached is the single rule selection and RoleCoverage share.
305+
func TestTierRateLimitReached(t *testing.T) {
306+
now := time.Date(2026, 11, 8, 12, 0, 0, 0, time.UTC)
307+
cases := []struct {
308+
name string
309+
limits map[string]config.TierRate
310+
tier string
311+
ages []time.Duration
312+
want tierRateLimitHit
313+
}{
314+
{name: "under both caps", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 2, MaxPerDay: 3}},
315+
tier: "contributor", ages: []time.Duration{time.Minute, 2 * time.Hour}},
316+
{name: "at the hourly cap", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 2, MaxPerDay: 50}},
317+
tier: "contributor", ages: []time.Duration{time.Minute, 30 * time.Minute},
318+
want: tierRateLimitHit{Reason: taskUnavailableHourlyLimit, Count: 2, Max: 2}},
319+
{name: "at the daily cap", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 10, MaxPerDay: 2}},
320+
tier: "contributor", ages: []time.Duration{2 * time.Hour, 23 * time.Hour},
321+
want: tierRateLimitHit{Reason: taskUnavailableDailyLimit, Count: 2, Max: 2}},
322+
{name: "both caps reports hourly first", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 1, MaxPerDay: 1}},
323+
tier: "contributor", ages: []time.Duration{time.Minute},
324+
want: tierRateLimitHit{Reason: taskUnavailableHourlyLimit, Count: 1, Max: 1}},
325+
{name: "stamps past the day window do not count", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 1, MaxPerDay: 1}},
326+
tier: "contributor", ages: []time.Duration{25 * time.Hour}},
327+
{name: "zero caps are unlimited", limits: map[string]config.TierRate{"advisor": {}},
328+
tier: "advisor", ages: []time.Duration{time.Minute, 2 * time.Minute, 3 * time.Minute}},
329+
{name: "a tier with no row is unlimited", limits: map[string]config.TierRate{"contributor": {MaxPerHour: 1, MaxPerDay: 1}},
330+
tier: "custom", ages: []time.Duration{time.Minute, 2 * time.Minute}},
331+
}
332+
for _, tc := range cases {
333+
t.Run(tc.name, func(t *testing.T) {
334+
hub, s := covK2Hub(t)
335+
s.deps.Config.Hub.TierLimits = tc.limits
336+
for _, age := range tc.ages {
337+
hub.recordAssignment("c-alice", now.Add(-age))
338+
}
339+
if got := hub.tierRateLimitReached("c-alice", tc.tier, now); got != tc.want {
340+
t.Fatalf("tierRateLimitReached = %+v, want %+v", got, tc.want)
341+
}
342+
})
343+
}
344+
345+
// No server or config behind the hub: nothing to enforce, never a panic.
346+
bare := NewContributeWSHub(slog.Default(), nil)
347+
t.Cleanup(bare.Close)
348+
bare.recordAssignment("c-alice", now)
349+
if got := bare.tierRateLimitReached("c-alice", "contributor", now); got != (tierRateLimitHit{}) {
350+
t.Fatalf("hub without a server: tierRateLimitReached = %+v, want zero", got)
351+
}
352+
var none *ContributeWSHub
353+
if got := none.tierRateLimitReached("c-alice", "contributor", now); got != (tierRateLimitHit{}) {
354+
t.Fatalf("nil hub: tierRateLimitReached = %+v, want zero", got)
355+
}
356+
}

‎src/pkg/dashboard/contribute_select.go‎

Lines changed: 62 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,48 @@ func (h *ContributeWSHub) rateWindowCounts(identity string, now time.Time) (hour
162162
return hour, day
163163
}
164164

165+
// tierRateLimitHit says which assignment-rate cap an identity has used up:
166+
// Reason is taskUnavailableHourlyLimit or taskUnavailableDailyLimit, Count is
167+
// the assignments counted in that window and Max the cap. A zero value means
168+
// the identity may be handed another task.
169+
type tierRateLimitHit struct {
170+
Reason string
171+
Count, Max int
172+
}
173+
174+
// tierRateLimitReached is the one rule for "this identity has used up its
175+
// hourly or daily task allowance" (#2566). selectTaskPass refuses work with it,
176+
// and RoleCoverage uses it so a relay that selection refuses for this reason
177+
// does not count as contributor-first cover.
178+
//
179+
// The limits are hub.tier_limits[tier] for the connection's trust tier, and
180+
// nothing else: config load fills in the shipped row for each built-in tier
181+
// the operator did not set (config/defaults.go), there is no per-contributor
182+
// override (ContributorProfile.RateLimits is stored but never enforced), and a
183+
// tier with no row is unlimited. A cap of 0 or less is unlimited. The hourly
184+
// cap is checked first, so a relay at both caps reports hourly_limit, as
185+
// selection always has.
186+
//
187+
// It takes rateMu (through rateWindowCounts) and reads the config without a
188+
// lock, as selection does. The caller must not hold rateMu.
189+
func (h *ContributeWSHub) tierRateLimitReached(identity, tier string, now time.Time) tierRateLimitHit {
190+
if h == nil || h.server == nil || h.server.deps == nil || h.server.deps.Config == nil {
191+
return tierRateLimitHit{}
192+
}
193+
limits, ok := h.server.deps.Config.Hub.TierLimits[tier]
194+
if !ok || (limits.MaxPerHour <= 0 && limits.MaxPerDay <= 0) {
195+
return tierRateLimitHit{}
196+
}
197+
hourCount, dayCount := h.rateWindowCounts(identity, now)
198+
if limits.MaxPerHour > 0 && hourCount >= limits.MaxPerHour {
199+
return tierRateLimitHit{Reason: taskUnavailableHourlyLimit, Count: hourCount, Max: limits.MaxPerHour}
200+
}
201+
if limits.MaxPerDay > 0 && dayCount >= limits.MaxPerDay {
202+
return tierRateLimitHit{Reason: taskUnavailableDailyLimit, Count: dayCount, Max: limits.MaxPerDay}
203+
}
204+
return tierRateLimitHit{}
205+
}
206+
165207
// recordAssignment appends an assignment timestamp for the identity. Called once
166208
// per task_assign actually shipped, so the rate windows count tasks HANDED OUT
167209
// (matching max_concurrent's semantics and the max_tasks_per_hour/day naming), not
@@ -590,29 +632,28 @@ func (h *ContributeWSHub) selectTaskPass(c *ContributorConnection, skippedUnmint
590632
strconv.Itoa(limits.MaxConcurrent))
591633
return h.taskUnavailable(taskUnavailableConcurrencyLimit)
592634
}
593-
if limits.MaxPerHour > 0 || limits.MaxPerDay > 0 {
594-
hourCount, dayCount := h.rateWindowCounts(identityOf(c), time.Now())
595-
if limits.MaxPerHour > 0 && hourCount >= limits.MaxPerHour {
596-
h.logger.Warn("[contribute-ws] refusing task: hourly rate limit reached",
597-
"username", identityOf(c), "tier", tier,
598-
"assigned_last_hour", hourCount, "max_per_hour", limits.MaxPerHour)
599-
h.recordDecision(decisionUsername(c), decisionRefused, "", "", 0,
600-
"hourly rate limit for tier "+tier+": "+strconv.Itoa(hourCount)+
601-
" assigned in the last hour, max "+strconv.Itoa(limits.MaxPerHour))
602-
return h.taskUnavailable(taskUnavailableHourlyLimit)
603-
}
604-
if limits.MaxPerDay > 0 && dayCount >= limits.MaxPerDay {
605-
h.logger.Warn("[contribute-ws] refusing task: daily rate limit reached",
606-
"username", identityOf(c), "tier", tier,
607-
"assigned_last_day", dayCount, "max_per_day", limits.MaxPerDay)
608-
h.recordDecision(decisionUsername(c), decisionRefused, "", "", 0,
609-
"daily rate limit for tier "+tier+": "+strconv.Itoa(dayCount)+
610-
" assigned in the last day, max "+strconv.Itoa(limits.MaxPerDay))
611-
return h.taskUnavailable(taskUnavailableDailyLimit)
612-
}
613-
}
614635
}
615636
}
637+
// The hourly/daily caps live in tierRateLimitReached, which RoleCoverage
638+
// shares, so a relay refused here is never reported as covering its role.
639+
switch hit := h.tierRateLimitReached(identityOf(c), tier, time.Now()); hit.Reason {
640+
case taskUnavailableHourlyLimit:
641+
h.logger.Warn("[contribute-ws] refusing task: hourly rate limit reached",
642+
"username", identityOf(c), "tier", tier,
643+
"assigned_last_hour", hit.Count, "max_per_hour", hit.Max)
644+
h.recordDecision(decisionUsername(c), decisionRefused, "", "", 0,
645+
"hourly rate limit for tier "+tier+": "+strconv.Itoa(hit.Count)+
646+
" assigned in the last hour, max "+strconv.Itoa(hit.Max))
647+
return h.taskUnavailable(taskUnavailableHourlyLimit)
648+
case taskUnavailableDailyLimit:
649+
h.logger.Warn("[contribute-ws] refusing task: daily rate limit reached",
650+
"username", identityOf(c), "tier", tier,
651+
"assigned_last_day", hit.Count, "max_per_day", hit.Max)
652+
h.recordDecision(decisionUsername(c), decisionRefused, "", "", 0,
653+
"daily rate limit for tier "+tier+": "+strconv.Itoa(hit.Count)+
654+
" assigned in the last day, max "+strconv.Itoa(hit.Max))
655+
return h.taskUnavailable(taskUnavailableDailyLimit)
656+
}
616657

617658
// #6450: pause claims for an identity whose recent assignments all died
618659
// within seconds — a dying agent runtime. Checked after the operator gates

‎src/pkg/governor/contributor_first.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ import (
2121
//
2222
// contributor_first inverts the order for one lane. While at least one
2323
// connected, non-stale, non-failure-paused relay is acting as the role and
24-
// taking work (not in standby, not quota-held; see RoleCoverage in
25-
// pkg/dashboard), and the operator has not suspended contributions, the
24+
// taking work (not in standby, not quota-held, not at its hourly or daily
25+
// task limit; see RoleCoverage in pkg/dashboard), and the operator has not suspended contributions, the
2626
// governor WITHHOLDS the resident agent's kicks — at exactly the gates where it
2727
// withholds them for an exhausted budget. The relay is the primary worker; the
2828
// resident agent is the fallback, and it resumes on its normal cadence once no

0 commit comments

Comments
 (0)