Skip to content

Commit 5f54d98

Browse files
Merge pull request #58 from StepenkoAnatoli/feat/search-transport-input
A dispatch input for the search side, so two providers can be compared on one topic
2 parents 96aa972 + d5de082 commit 5f54d98

3 files changed

Lines changed: 15 additions & 0 deletions

File tree

‎.github/workflows/collect.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,12 @@ on:
9595
description: 'PUBLIC on a public repository - it becomes the run name and the artifact name. Use an opaque id (job-0417), never the subject. Optional: correlation does not need it - pin X-GitHub-Api-Version and read workflow_run_id. Shape [A-Za-z0-9][A-Za-z0-9._-]{0,63}.'
9696
required: false
9797
default: ''
98+
search_transport:
99+
description: 'Which provider performs the SEARCH. auto follows the ladder - a SerpApi key wins, else the fetch provider. Pin one to compare them on the same topic.'
100+
required: true
101+
default: 'auto'
102+
type: choice
103+
options: [auto, serpapi, firecrawl-cli]
98104
runner:
99105
description: 'Which runner to collect on'
100106
required: true
@@ -396,6 +402,10 @@ jobs:
396402
# than failing, because the ladder's last rung is the fetch provider.
397403
SERPAPI_API_KEY: ${{ secrets.SERPAPI_API_KEY }}
398404
RESEARCH_KIT_TRANSPORT: firecrawl-cli
405+
# 'auto' becomes the empty string, which is falsy in the ladder and therefore
406+
# means "no explicit choice" rather than "a provider called auto". Pinning a
407+
# value is what makes two runs of the same topic comparable.
408+
RESEARCH_KIT_SEARCH_TRANSPORT: ${{ inputs.search_transport == 'auto' && '' || inputs.search_transport }}
399409
MAX_PAGES: ${{ inputs.max_pages }}
400410
DEPTH: ${{ inputs.depth }}
401411
run: |

‎docs/ARCHITECTURE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,6 +425,7 @@ each line — is in the ADR named beside it and in
425425
| `corroboration` **reports the closest pair it measured** on an `independent` verdict, and does not act on it. The threshold has two measured blind spots, and containment - the obvious repair for the second - ranks a genuinely different pair (0.506) ABOVE a real mirror (0.380) over 2404 pairs, so trying harder would report second sources as copies. The number informs a reviewer; the grade does not move | ADR-0036 |
426426
| A finding a reviewer has **considered and overruled** is recorded, not repeated forever: `[single-witness: …]` in an unknown's Evidence cell and `[render-reviewed: …]` in an evidence row's Finding cell turn a true-but-permanent warning into a pass **that prints the reason**. A reason under 30 characters is named (`single-witness-unreasoned`) rather than obeyed, and a note left on an unknown that has since been corroborated is reported (`single-witness-stale`) so the acknowledgement cannot rot. The render note lives on the ROW because `verifyLedger` hashes the whole capture file — a capture is evidence and stays byte-immutable, a judgement about it is corpus prose | [ADR-0038](adr/0038-a-judgement-the-gate-can-read.md) |
427427
| The collector lock is exclusive **on a local filesystem, and not promised on a network one**: Node states "the exclusive flag might not work with network file systems", a condition ADR-0020 never carried. Measured 2026-09-22 - 24 racers, 1 winner, 23 EEXIST on win32 - so the guarantee holds where it was tested and nothing detects where it is standing. Evidence: `docs/decisions/2026-09-22-wx-lock/` | ADR-0020 |
428+
| The SEARCH side is **selectable at dispatch**: collect.yml takes a search_transport input (auto | serpapi | firecrawl-cli) and collect-remote.mjs a matching --search-transport, passed as RESEARCH_KIT_SEARCH_TRANSPORT. ADR-0027 built the ladder and the collector never exposed it, so a dispatched run always took whatever auto-detect picked and two providers could not be compared on one topic through the kit at all. auto becomes the EMPTY STRING, falsy in the ladder, so the default is the previous behaviour exactly | ADR-0027 |
428429
| An entrypoint **refuses a flag it does not know** rather than ignoring it: `refuseUnknownFlags` in `lib/core.mjs`, called immediately after `parseFlags` in all eleven bins. `parseFlags` knows no flag names by design, so an entrypoint that skips the check gets silence — the flag is parsed, never read, and the command proceeds to its default. On 2026-09-22 that default was **a real collection**: `research.mjs --totally-made-up-flag` spent 26 credits against this repository's corpus, and `audit.mjs` wrote twelve files. The same survey found a dead `--max-scrapes` in two workflows, where the page bound people believed it set was really `plan.maxScrapes` — real in `collect.yml`, and **absent** in `live-collection.yml`, whose `max_pages` was validated and then ignored | ADR-0020 |
429430
| A capture that **says it failed to render** is reported, though it arrived whole: `completeness` grades the transport and cannot see a page built by script that arrives complete and empty. The rule matches sentences a page PRINTS, never a guess at whether its text looks substantial — emptiness itself was measured and found mechanically undetectable, in both directions ([ADR-0037](adr/0037-an-empty-capture-is-not-mechanically-detectable.md)) | [ADR-0037](adr/0037-an-empty-capture-is-not-mechanically-detectable.md) |
430431
| A privacy claim is **measured, not asserted**: `bin/disclosure.mjs` re-runs the probe, and a report never lets an unchecked thing read as a clean one | [ADR-0035](adr/0035-a-privacy-claim-is-measured-not-asserted.md) |

‎research-kit/bin/collect-remote.mjs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ const HELP = `collect-remote - run the collector on GitHub Actions and bring the
3737
--repository OWNER/REPO required
3838
--topic "<text>" required. Visible to anyone who can read the repository.
3939
--max-pages <1-25> default 8. Each page costs at least one credit.
40+
--search-transport <name> auto | serpapi | firecrawl-cli. Default auto: a SerpApi key
41+
wins, else the fetch provider. Pin one to compare them.
4042
--depth probe|quick|normal default quick
4143
--client-ref <id> optional. PUBLIC: it becomes the run and artifact name.
4244
--runner ubuntu-latest|windows-latest default ubuntu-latest
@@ -63,6 +65,7 @@ requireRuntime({ node: true });
6365

6466
const KNOWN = new Set([
6567
'repository', 'topic', 'max-pages', 'depth', 'client-ref', 'runner', 'workflow', 'ref',
68+
'search-transport',
6669
'out', 'timeout', 'no-wait', 'json', 'help',
6770
]);
6871
const unknown = Object.keys(flags).filter((f) => !KNOWN.has(f));
@@ -113,6 +116,7 @@ const inputs = {
113116
max_pages: String(flags['max-pages'] ?? 8),
114117
depth: String(flags.depth ?? 'quick'),
115118
runner: String(flags.runner ?? 'ubuntu-latest'),
119+
search_transport: String(flags['search-transport'] ?? 'auto'),
116120
};
117121
if (flags['client-ref'] !== undefined && flags['client-ref'] !== true) inputs.client_ref = String(flags['client-ref']);
118122

0 commit comments

Comments
 (0)