[Bug]: Tool Search telemetry cannot identify counter resets

[vincentkoc]

Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

Tool Search code-mode telemetry exposes cumulative search, describe, and call counters without a counter-lifetime identity, so consumers cannot distinguish continued counts from catalog replacement or process-reset counts.

Steps to reproduce

1. Run OpenClaw 2026.7.1-2 with Tool Search code mode.
2. Capture telemetry.searchCount, describeCount, and callCount from successive tool_search_code results across more than one catalog lifetime.
3. Attempt to derive per-operation totals from the cumulative snapshots.

Expected behavior

Every cumulative telemetry sample includes an opaque counter scope that changes whenever the counters reset and remains stable while an existing catalog is extended.

Actual behavior

The samples contain only cumulative numeric counters. A completed ShellBench trace produced a naive sum of 1,217 operations; a reset-decrease heuristic produced 74, but neither total is provably exact because a replacement counter can restart at an equal or higher value.

OpenClaw version

2026.7.1-2 (0790d9f)

Operating system

Ubuntu 26.04 on AWS c7a.8xlarge

Install method

npm registry package installed by the ShellBench native harness

Model

openai/gpt-5.5, high reasoning

Provider / routing chain

OpenClaw -> LiteLLM proxy -> OpenAI

Logs

Released code-mode task traces: 71 top-level tool calls and 56 turns.
Naive cumulative snapshot sum: 1,217.
Reset-decrease heuristic: 74.
Exact operation count: unresolved because the telemetry has no reset scope.

Screenshots, recordings, and evidence

ShellBench telemetry export work: openclaw/shellbench#60

Impact and severity

Affected: benchmark and production observability consumers of Tool Search code-mode telemetry.
Severity: Medium. Runtime behavior continues, but performance analysis can report materially wrong operation counts and drive the wrong optimization.
Frequency: Always when cumulative samples span an unidentifiable counter reset.
Consequence: inflated or undercounted discovery operations, invalid cross-harness comparisons, and misleading token/tool-call optimization decisions.

Additional information

The bounded fix emits a short opaque counterScope, rotates it on fresh or restored catalogs, preserves it when client tools append to the current catalog, and leaves existing counters unchanged.

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed July 31, 2026, 9:52 PM ET / August 1, 2026, 01:52 UTC.

Summary
Keep this issue open. Current main still resets Tool Search operation counters when it restores a reusable catalog while returning telemetry that contains only cumulative numbers, so consumers cannot identify the reset boundary. The open maintainer-authored candidate repair at #116070 already owns the narrow fix; the maintainer label and member authorship mean no duplicate repair or automated close. Likely related people: Peter Steinberger is the strongest current-source history signal (medium confidence).

Reproducibility: yes. from source: restore of an unchanged catalog resets all cumulative counters while the telemetry projection exposes no counter-lifetime field. A focused regression can clear and restore a catalog, collect snapshots, and show that equal-or-higher post-reset counts are ambiguous without a scope.

Root-cause cluster
Relationship: canonical
Canonical: #116069
Summary: This is the canonical report for the missing Tool Search counter-lifetime identity; one open pull request is its candidate implementation.

Members:

• fixed_by_candidate: fix(agents): identify Tool Search counter lifetimes #116070 - The pull request explicitly targets this report and describes the matching counter-scope lifecycle repair.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Next step
An open maintainer-authored pull request already owns the bounded repair, and the protected maintainer label prevents duplicate automated work.

Review details

Best possible solution:

Land the linked focused repair with lifecycle regression coverage for fresh registration, restored catalog reset, and append preservation, then let its closing link resolve this issue.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: restore of an unchanged catalog resets all cumulative counters while the telemetry projection exposes no counter-lifetime field. A focused regression can clear and restore a catalog, collect snapshots, and show that equal-or-higher post-reset counts are ambiguous without a scope.

Is this the best way to solve the issue?

Yes. A small opaque scope recorded at catalog-session creation and exposed beside the existing counters is the narrow owner-boundary repair; it preserves current numeric telemetry and distinguishes reset from append behavior.

AGENTS.md: found and applied where relevant.

Remaining risk / open question:

• The linked repair changes a telemetry shape consumed by benchmark tooling; it should retain the existing counters and prove that the scope changes on restoration but not on client-tool append.

Codex review notes: model internal, reasoning high; reviewed against c41f033b8754.

Label changes

Label justifications:

• P2: The defect makes Tool Search observability totals unreliable but does not interrupt agent execution or message delivery.
• impact:other: The supported impact is incorrect benchmark and operator telemetry rather than one of the more specific owned impact classes.

Evidence reviewed

What I checked:

• Current telemetry has no reset identity: The catalog session stores only entries plus three numeric counters, and the runtime returns those counters in telemetry without any lifetime or scope field. (src/agents/tool-search-types.ts:111, c41f033b8754)
• Current main resets the reported counters on restored catalogs: Restoring a reusable catalog constructs a new session with all three counters set to zero; applying an unchanged current catalog preserves existing counters only for append operations. (src/agents/tool-search-catalog.ts:127, c41f033b8754)
• Existing tests cover the reset-producing lifecycle but not a telemetry scope: The cross-run reuse test clears a run-owned catalog and restores its entries into a new catalog reference; the ordinary telemetry assertion checks only numeric counts. (src/agents/tool-search.test.ts:2568, c41f033b8754)
• Feature-history provenance: Blame attributes the current catalog restoration, counter-preserving append registration, and telemetry projection to the same current-main lineage introduced by Peter Steinberger; that history supports routing rather than fault attribution. (src/agents/tool-search-catalog.ts:127, f9377751dcbe)
• Canonical open repair: The supplied GitHub context identifies fix(agents): identify Tool Search counter lifetimes #116070 as an open pull request explicitly intended to close this issue and add the counter-lifetime scope. (8d01d1aa6fdc)
• Release/current-main boundary: v2026.7.1 is an ancestor of current main, while the current-source provenance commit is not contained by a local release tag; no shipped fix can be claimed because the candidate PR remains open. (c41f033b8754)

Likely related people:

• Peter Steinberger: Current-line blame ties the catalog restoration, counter lifecycle, and telemetry projection to the same recent Tool Search implementation lineage. (role: recent area contributor; confidence: medium; commits: f9377751dcbe; files: src/agents/tool-search-catalog.ts, src/agents/tool-search-runtime.ts, src/agents/tool-search-types.ts)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.