Skip to content

Commit 463df67

Browse files
Merge pull request #60 from StepenkoAnatoli/fix/collected-is-not-spent
A run said it collected 8 pages while its artifact carried 2
2 parents 4fc1e04 + 18f75d9 commit 463df67

5 files changed

Lines changed: 21 additions & 2 deletions

File tree

‎docs/ARCHITECTURE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -426,6 +426,7 @@ each line — is in the ADR named beside it and in
426426
| A finding a reviewer has **considered and overruled** is recorded, not repeated forever: `[single-witness: …]` in an unknown's Evidence cell and `[render-reviewed: …]` in an evidence row's Finding cell turn a true-but-permanent warning into a pass **that prints the reason**. A reason under 30 characters is named (`single-witness-unreasoned`) rather than obeyed, and a note left on an unknown that has since been corroborated is reported (`single-witness-stale`) so the acknowledgement cannot rot. The render note lives on the ROW because `verifyLedger` hashes the whole capture file — a capture is evidence and stays byte-immutable, a judgement about it is corpus prose | [ADR-0038](adr/0038-a-judgement-the-gate-can-read.md) |
427427
| The collector lock is exclusive **on a local filesystem, and not promised on a network one**: Node states "the exclusive flag might not work with network file systems", a condition ADR-0020 never carried. Measured 2026-09-22 - 24 racers, 1 winner, 23 EEXIST on win32 - so the guarantee holds where it was tested and nothing detects where it is standing. Evidence: `docs/decisions/2026-09-22-wx-lock/` | ADR-0020 |
428428
| The SEARCH side is **selectable at dispatch**: collect.yml takes a search_transport input (auto | serpapi | firecrawl-cli) and collect-remote.mjs a matching --search-transport, passed as RESEARCH_KIT_SEARCH_TRANSPORT. ADR-0027 built the ladder and the collector never exposed it, so a dispatched run always took whatever auto-detect picked and two providers could not be compared on one topic through the kit at all. auto becomes the EMPTY STRING, falsy in the ladder, so the default is the previous behaviour exactly | ADR-0027 |
429+
| A run reports **what landed separately from what it cost**: `collected` is captures on disk, `spent` is budget consumed (collected + failed). They were one number under the label `collected`, so a run that fetched 2 pages and lost 6 to a Firecrawl rate limit reported `collected 8` while its artifact carried 2. The free tier is 10 requests per minute and the collector does not throttle, so a paired run started inside the same minute sheds most of its fetches | ADR-0005 |
429430
| An entrypoint **refuses a flag it does not know** rather than ignoring it: `refuseUnknownFlags` in `lib/core.mjs`, called immediately after `parseFlags` in all eleven bins. `parseFlags` knows no flag names by design, so an entrypoint that skips the check gets silence — the flag is parsed, never read, and the command proceeds to its default. On 2026-09-22 that default was **a real collection**: `research.mjs --totally-made-up-flag` spent 26 credits against this repository's corpus, and `audit.mjs` wrote twelve files. The same survey found a dead `--max-scrapes` in two workflows, where the page bound people believed it set was really `plan.maxScrapes` — real in `collect.yml`, and **absent** in `live-collection.yml`, whose `max_pages` was validated and then ignored | ADR-0020 |
430431
| A capture that **says it failed to render** is reported, though it arrived whole: `completeness` grades the transport and cannot see a page built by script that arrives complete and empty. The rule matches sentences a page PRINTS, never a guess at whether its text looks substantial — emptiness itself was measured and found mechanically undetectable, in both directions ([ADR-0037](adr/0037-an-empty-capture-is-not-mechanically-detectable.md)) | [ADR-0037](adr/0037-an-empty-capture-is-not-mechanically-detectable.md) |
431432
| A privacy claim is **measured, not asserted**: `bin/disclosure.mjs` re-runs the probe, and a report never lets an unchecked thing read as a clean one | [ADR-0035](adr/0035-a-privacy-claim-is-measured-not-asserted.md) |

‎research-kit/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -395,7 +395,7 @@ node research-kit/bin/selftest.mjs # all of it
395395
node research-kit/bin/selftest.mjs gate hook # just these files
396396
```
397397

398-
855 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
398+
856 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
399399
the assertions settled (ADR-0021), and each test is raced against a watchdog
400400
(`RESEARCH_KIT_TEST_TIMEOUT`, default 60s).
401401

‎research-kit/bin/research.mjs‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,10 @@ const run = runResearch(root, {
112112

113113
process.stdout.write(`${heading('run')}
114114
depth ${run.depth} (budget ${run.budget} scrapes)
115-
collected ${run.spent}
115+
collected ${run.collected}
116116
cached ${run.cached}
117117
failed ${run.failed}
118+
spent ${run.spent} (budget consumed: collected + failed)
118119
`);
119120
// The second meter reports separately, and a degradation is never silent: it means the
120121
// run quietly moved spend back onto the fetch budget.

‎research-kit/lib/research-run.mjs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,10 @@ ${compatibility.remedy}`);
268268
transport: adapter.name,
269269
searchTransport: searchName,
270270
depth: tier, budget, attempts, spent, cached, failed,
271+
// What actually landed on disk, as distinct from what the run cost. `spent` is
272+
// collected + failed, because a failed fetch can still consume budget; reporting it as
273+
// "collected" told the operator they had pages they did not have.
274+
collected: spent - failed,
271275
searchesUsed, searchFailures, degraded,
272276
results, discovered,
273277
};

‎research-kit/test/cli.test.mjs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,3 +338,16 @@ test('no workflow passes a flag its entrypoint does not accept', () => {
338338
}
339339
assert.deepEqual(offenders, [], 'a workflow passes a flag the entrypoint does not have');
340340
});
341+
342+
test('the run summary distinguishes what landed from what it cost', () => {
343+
// `spent` is collected + failed, because a failed fetch still consumes budget. The
344+
// summary printed that number under the label "collected", so a run that fetched two
345+
// pages and lost six to a Firecrawl rate limit reported `collected 8` while its artifact
346+
// carried two captures. Found 2026-09-22 by comparing a run log against its own ZIP.
347+
const r = run('research.mjs', ['--dry-run'], { root: project() });
348+
assert.equal(r.status, 0, r.err);
349+
assert.match(r.out, /^collected\s+\d+/m);
350+
assert.match(r.out, /^failed\s+\d+/m);
351+
assert.match(r.out, /^spent\s+\d+ \(budget consumed: collected \+ failed\)/m,
352+
'the cost must be reported separately from what arrived');
353+
});

0 commit comments

Comments
 (0)