Skip to content

Commit c340855

Browse files
Merge pull request #37 from StepenkoAnatoli/feat/corroboration-check
A thirteenth check: the gate can see single-sourcing (ADR-0036)
2 parents 606e86e + f24a199 commit c340855

9 files changed

Lines changed: 301 additions & 11 deletions

File tree

‎CONTEXT.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,3 +151,4 @@ Added 2026-09-21 under ADR-0032. The vocabulary of the ZIP that leaves this kit.
151151
| **unrewritten finding** | An `EVIDENCE.md` `Finding` cell byte-identical to what the extractor re-produces from the cached capture — so it was demonstrably never rewritten by a human. The negative half of a question with no positive answer: ADR-0013 refuses to grade whether an agent reasoned properly, and this does not try. It detects the unreviewed case only. |
152152
| **structural refusal** | A ZIP fault decided from the **central directory**, before one byte is inflated: traversal, absolute path, drive letter, backslash, NUL, duplicate, case collision, symlink, entry count, declared size, compression ratio. The directory is attacker-controlled too, so `inflateRawSync` also runs with `maxOutputLength` — the ceiling belongs to the runtime rather than to a field the attacker wrote. |
153153
| **transport, not storage** | What an artifact is. Workflow artifacts last at most 90 days on a public repository and vanish with the run that produced them, so anything that must persist is committed instead. |
154+
| **corroboration** | How many INDEPENDENT readings a closed unknown rests on, and the thirteenth check. Three shapes: one row (`single-source`), several rows from one host (`one-voice`), several hosts (`independent`). The middle one is the distinction a count cannot make — two pages from one vendor catch a misreading and cannot catch the vendor being wrong about itself. It reports and never fails on its own: a vendor's own reference is the authority on that vendor, and demanding a second opinion about it would be ceremony. `evidencePolicy=strict` is where an operator says otherwise. |

‎docs/ARCHITECTURE.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,7 @@ each line — is in the ADR named beside it and in
415415
| A ZIP's structural faults are refused from the **central directory, before inflation**; the inflation ceiling is `maxOutputLength`, not a field the archive declared about itself | ADR-0032 |
416416
| **No dispatch input reaches a shell**: `${{ inputs.x }}` inside a `run:` block is substitution *before* the shell parses, so inputs arrive through `env:` and then an argv array. Enforced across **every** workflow, not only the newest | [ADR-0033](adr/0033-the-collector-refuses-rather-than-degrades.md), ADR-0020 |
417417
| The collector's spend gate is a **protected environment whose existence is proven**, because GitHub silently creates an *unprotected* one when a workflow names a missing environment; a missing credential **refuses** rather than falling back to a keyless route | ADR-0033 |
418+
| A claim's **support is weighed, not just checked**: `corroboration` reports whether a closed unknown rests on one reading, several readings of one source, or several hosts — it never fails a corpus on its own, because single-sourcing is often the right answer | [ADR-0036](adr/0036-the-gate-can-see-single-sourcing.md) |
418419
| A privacy claim is **measured, not asserted**: `bin/disclosure.mjs` re-runs the probe, and a report never lets an unchecked thing read as a clean one | [ADR-0035](adr/0035-a-privacy-claim-is-measured-not-asserted.md) |
419420
| A credential that resolves in a job declaring **no** environment is **not behind the approval gate**, and fails the run: the scope question GitHub will not answer directly is answered by asking it from a context that can see only one scope | ADR-0033 |
420421
| The vendor **package name** lives in `lib/firecrawl.mjs` (`CLI_PACKAGE`, `cliInstallSpec`), never in a workflow: the CLI is **`firecrawl-cli`**, and the npm package called `firecrawl` is the SDK and ships no binary. Both workflows installed the wrong one for weeks, and only a real dispatch could find it | ADR-0005, ADR-0033 |
@@ -436,7 +437,7 @@ Counted against the filesystem, not estimated:
436437
| `bin/` entrypoints | **29**, of which **4 are Python**: 3 conformance runners and the module they share |
437438
| `schemas/` | **12** |
438439
| `conformance/` vector packets | **3** |
439-
| checks in the registry | **12** |
440+
| checks in the registry | **13** |
440441
| supported CI platforms | **2** |
441442

442443
The `schemas/` count became 12 on 2026-09-21 when `artifact-manifest.schema.json` arrived
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# ADR-0036 — The gate can see single-sourcing, and says so without blocking
2+
3+
- **Date:** 2026-09-21
4+
- **Status:** accepted
5+
- **Area:** contract checks, evidence policy
6+
- **Depends on:** ADR-0004 (the checks are an ordered registry), ADR-0013 (a check that judged reasoning would be wrong)
7+
8+
## Context
9+
10+
Twelve checks asked whether a claim was **backed**: is there a row, is there a capture,
11+
does the chain verify, is the capture whole, is it fresh, is it primary, has it been
12+
superseded. Not one asked whether it was backed **more than once**.
13+
14+
So a corpus in which every unknown rested on a single page passed `preflight` and
15+
`--strict` with nothing to say about it. That is not hypothetical — it is this
16+
repository's own delivery-architecture corpus, on the day it was used to justify ADR-0031
17+
and ADR-0032. Nine rows, all primary, all `full`, all fresh, and eight unknowns each
18+
resting on exactly one of them. Three of those eight carried architecture decisions.
19+
20+
**From inside such a corpus, a single correct source and a single lucky one are
21+
indistinguishable.** Every other check would report green either way.
22+
23+
## Decision
24+
25+
A thirteenth check, `corroboration`, reporting the **shape** of an unknown's support:
26+
27+
| rule | when | severity |
28+
|---|---|---|
29+
| `single-source` | one row | `warn` |
30+
| `one-voice` | several rows, one host | `warn` |
31+
| `independent` | several rows, several hosts | `pass` |
32+
33+
### It never fails a corpus on its own
34+
35+
Single-sourcing is frequently the right answer. A vendor's own API reference *is* the
36+
authority on that API, and demanding a second opinion about it is ceremony. This check
37+
declines to make that judgement — it describes the support and leaves the call to a
38+
reviewer, which is the same line ADR-0013 drew when it refused to let a check grade
39+
reasoning.
40+
41+
`corroboration` joins `transport-provenance` and `capture-completeness` in `POLICY_CHECKS`,
42+
so `evidencePolicy=strict` is where an operator says they want the harder rule. All three
43+
are questions an *evidence policy* should answer rather than the kit: how it was fetched,
44+
how much arrived, how many times it was read.
45+
46+
Every existing corpus stays green. Verified on all three in this repository — the
47+
repository's own, and both nested decision projects — each still `PASS` under the default
48+
policy, now with warnings that say something true.
49+
50+
### The host distinction is the point, not a refinement
51+
52+
Counting rows would have been easier and would have missed the case that matters. Two
53+
pages from one vendor are **one witness read twice**: that catches a misreading, and
54+
cannot catch a vendor being wrong about itself. The remedies differ — one wants another
55+
page, the other wants another party — so the finding names which it found.
56+
57+
## What it immediately said about work finished an hour earlier
58+
59+
The corroboration pass that preceded this ADR added a second source to the three
60+
delivery-architecture unknowns that carry decisions, and the commit called them
61+
*"independent"*. The check disagrees: all three second sources are `docs.github.com`, so
62+
all three report `one-voice` rather than `independent`.
63+
64+
**The check is right and the commit message was loose.** They are independent *pages* —
65+
different URLs, written by different teams, and they did catch facts the originals omitted,
66+
including that an unpinned API client silently moves to a different version on a date. They
67+
are not independent *sources*. The corpus's own contract already said this in prose —
68+
"a second reading of the same vendor" — and the commit summarising it reached for the
69+
stronger word anyway.
70+
71+
That is the argument for the check in miniature: the distinction is easy to state, easy to
72+
believe you have honoured, and invisible unless something counts hosts.
73+
74+
## Alternatives considered
75+
76+
**Fail on single-sourcing.** Would have broken every corpus in this repository and most
77+
that anyone writes, to enforce a rule that is wrong about half the time.
78+
79+
**Count rows and ignore hosts.** Simpler, and it would have graded the delivery corpus as
80+
corroborated when it is not.
81+
82+
**Require a second source only for unknowns marked load-bearing.** No such marking exists,
83+
and inventing one moves the judgement into the contract without making it checkable.
84+
85+
**Leave it to review.** That is what was happening. Review missed it for as long as the
86+
corpus existed, and the corpus was used to make two architecture decisions in the meantime.
87+
88+
## Consequences
89+
90+
- The registry is thirteen checks; the pinned-order test and the inventory move with it.
91+
- One preflight test asserted "every check reports a **pass**" on the fixture project. That
92+
fixture is single-sourced, so it now carries one warning. The assertion was changed to
93+
"every check **reports**", with the expected warning named — rather than padding the
94+
fixture with a second source to preserve the old wording, which would have been tuning
95+
the world to fit the assertion.
96+
- The policy-membership test stopped inferring "hygiene is not hardened" from a whole
97+
verdict passing, and now isolates the claim. That proxy only ever worked while nothing
98+
else in the policy set fired.
99+
100+
## What this ADR does not claim
101+
102+
That two hosts make a claim true. Two vendors can repeat one another, and a specification
103+
and its own tutorial are not independent whatever their hostnames say. `independent` means
104+
*this check found no reason to flag the support*, which is weaker than correct and is the
105+
strongest thing a mechanical check can honestly report.

‎docs/adr/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,5 +51,6 @@ directory and are treated as ADR-equivalent.
5151
| [0033](0033-the-collector-refuses-rather-than-degrades.md) | **The collector refuses rather than degrades, and no dispatch input reaches a shell.** `collect.yml` implements ADR-0031 against ADR-0032's contract, separate from `live-collection.yml` because "does the adapter work" and "collect this, for a person, now" deserve different approvers. The environment is the spend gate — and is **checked for existence**, because GitHub silently *creates* an unprotected environment when a workflow names one that is missing (E-06), which would remove the approval gate without failing anything. The honest limit is written in the file: GitHub exposes no scope information, so the marker proves the environment was configured and **cannot** prove a repository-level value did not serve the secret. No `http-keyless` fallback: a missing credential refuses rather than shipping a quietly worse corpus under the same artifact name. Every input arrives through `env:` and then an argv array — `${{ inputs.x }}` in a `run:` block is substitution *before* the shell parses, so a backtick in a topic executes. The test enforcing that scans every workflow and found `live-collection.yml` had been doing it since it was written; fixed here **Amended the same day**: the required reviewer is dropped and the environment kept. A reviewer approving a *dispatch* is a spend gate, not a review — nothing is collected yet to judge — and it makes autonomous dispatch impossible. The environment stays because it was never about approval: it is the only place the credential is unreadable by every other workflow. Spend is bounded by `max_pages`, the depth tier and the vendor's 402-at-zero cap. The control that replaces the reviewer is **token scope**: a fine-grained PAT with `Actions: read and write` on one repository, driven by `bin/collect-remote.mjs`, which takes **no `--token` flag** and redacts by pattern rather than by remembering its own value |
5252
| [0034](0034-the-collector-speaks-mcp-over-stdio.md) | **The collector speaks MCP over stdio** — and this is the first module in the repository built from research that PASSED THE GATE rather than from a plan; every decision cites an `E-` row in [`docs/decisions/2026-09-21-agent-interface/`](../decisions/2026-09-21-agent-interface/research/BRIEF.md). stdio because semantics are identical on both transports, so the difference is operational, and remote costs the entire OAuth 2.1 chain. Two tools, not one, because a server cannot initiate a request — so the alternative is a call held open for minutes and an agent that loses a paid run when its process dies. A `resource_link`, never embedded bytes: the spec defines embedding and **states no maximum size**, so "permitted" is not "known to work". The credential stays in the environment, which for stdio is what the spec itself prescribes. **The gate refused the first draft of that brief**, naming a load-bearing GAP; two more pages went into the same ledger to close it, and `resource_link` is what came back |
5353
| [0035](0035-a-privacy-claim-is-measured-not-asserted.md) | **A privacy claim is measured, not asserted — and this repository stays public.** The collector's own header said dispatch inputs are visible to anyone with read access. I wrote that from reasoning and never tested it; unauthenticated probing found the topic in **none** of the readable responses — run metadata, step names, timing and the artifact *listing* are `200`, while the artifact **download** is `401` and the job **logs** are `403`. A stranger learns a run's shape, not its subject. Wrong in the direction that feels safe is still wrong: a reader who finds one warning false discounts the next. Staying public because this account is on the **free** plan (checked, not inferred) where going private makes environment secrets *ignored* — a plan change, not a toggle. The claim is now executable: `bin/disclosure.mjs`, unauthenticated by construction, exit codes as the finding, and `needleChecked` so an **unchecked** thing can never print as a clean one |
54+
| [0036](0036-the-gate-can-see-single-sourcing.md) | **The gate can see single-sourcing, and says so without blocking.** Twelve checks asked whether a claim was *backed*; none asked whether it was backed more than once — so a corpus where every unknown rested on one page passed `preflight` AND `--strict`, and that corpus was this repository's own, on the day it justified ADR-0031 and ADR-0032. `corroboration` reports the SHAPE of support: `single-source`, `one-voice` (several rows, one host), or `independent` (several hosts). It **never fails on its own** — a vendor's own reference is the authority on that vendor — and joins the `POLICY_CHECKS` so `evidencePolicy=strict` is where an operator asks for the harder rule. The host distinction is the point: two pages from one vendor are one witness read twice. It immediately contradicted a commit written an hour earlier that called three `docs.github.com` pages "independent"; they are independent *pages*, not independent *sources*, and the check was right |
5455

5556
Domain vocabulary lives in [`CONTEXT.md`](../../CONTEXT.md).

‎research-kit/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -395,7 +395,7 @@ node research-kit/bin/selftest.mjs # all of it
395395
node research-kit/bin/selftest.mjs gate hook # just these files
396396
```
397397

398-
824 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
398+
829 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
399399
the assertions settled (ADR-0021), and each test is raced against a watchdog
400400
(`RESEARCH_KIT_TEST_TIMEOUT`, default 60s).
401401

‎research-kit/lib/checks.mjs‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
// A check emits findings. It never decides what a finding MEANS for the build - that is
88
// the verdict's single judgement, in lib/preflight.mjs.
99

10-
import { PATHS, resolve, exists, ageInDays } from './core.mjs';
10+
import { hostOf, PATHS, resolve, exists, ageInDays } from './core.mjs';
1111
import { captureOf, traceOf, citedIds } from './corpus.mjs';
1212
import { coverageOfUniversals } from './dimensions.mjs';
1313
import { verifyLedger } from './provenance.mjs';
@@ -477,6 +477,72 @@ function corpusShape(corpus) {
477477
return out;
478478
}
479479

480+
481+
// ---------------------------------------------------------------- 13. corroboration
482+
483+
/**
484+
* How many independent readings does a closed unknown rest on?
485+
*
486+
* WHY THIS EXISTS. Every check before it asks whether a claim is BACKED - is there a row,
487+
* is there a capture, does the chain verify, is it fresh, is it primary. None of them asks
488+
* whether it is backed more than once, so a corpus in which every unknown rested on a
489+
* single page passed preflight and `--strict` with nothing to say about it. From inside
490+
* such a corpus, a single CORRECT source and a single LUCKY one are indistinguishable.
491+
*
492+
* Found in this repository's own delivery-architecture corpus on 2026-09-21: nine rows,
493+
* all primary, all complete, all fresh - and eight unknowns each resting on exactly one of
494+
* them. Three of those eight carried architecture decisions.
495+
*
496+
* WHY IT IS A WARNING. Single-sourcing is often the right answer: a vendor's own API
497+
* reference is the authority on that API, and demanding a second opinion about it would be
498+
* ceremony. This check refuses to make that judgement - it reports the shape of the
499+
* support so a reviewer can make it. `evidencePolicy=strict` or `--strict` is where an
500+
* operator says they want the harder rule.
501+
*
502+
* WHY THE HOST MATTERS. Two pages from one vendor are not two witnesses. They are one
503+
* witness read twice, which catches a misreading and cannot catch a vendor being wrong
504+
* about itself. The finding says which of those it found, because the remedies differ:
505+
* one wants another page, the other wants another party.
506+
*/
507+
function corroboration(corpus) {
508+
const out = [];
509+
const byId = new Map(corpus.evidence.map((row) => [row.id.toUpperCase(), row]));
510+
511+
for (const unknown of corpus.unknowns) {
512+
// A KNOWN-UNKNOWN rests on nothing by definition, and unknown-closure already owns
513+
// the case of a CLOSED one citing no row at all.
514+
if (unknown.status !== 'CLOSED') continue;
515+
516+
const rows = unknown.cites
517+
.filter((id) => /^E-\d+$/i.test(id))
518+
.map((id) => byId.get(id.toUpperCase()))
519+
.filter(Boolean);
520+
if (!rows.length) continue;
521+
522+
if (rows.length === 1) {
523+
out.push(finding('warn', 'corroboration', 'single-source',
524+
`${unknown.id} rests on ${rows[0].id} alone - one reading, so a correct source and a lucky one look the same`,
525+
{ row: unknown.id, line: unknown.line }));
526+
continue;
527+
}
528+
529+
const hosts = new Set(rows.map((row) => hostOf(row.url)).filter(Boolean));
530+
if (hosts.size === 1) {
531+
out.push(finding('warn', 'corroboration', 'one-voice',
532+
`${unknown.id} cites ${rows.length} rows and all are ${[...hosts][0]} - a second reading of one source, which catches a misreading and not a source that is wrong about itself`,
533+
{ row: unknown.id, line: unknown.line }));
534+
continue;
535+
}
536+
537+
out.push(finding('pass', 'corroboration', 'independent',
538+
`${unknown.id} rests on ${rows.length} rows across ${hosts.size} hosts`,
539+
{ row: unknown.id, line: unknown.line }));
540+
}
541+
542+
if (!out.length) out.push(finding('pass', 'corroboration', 'support', 'no closed unknown to weigh'));
543+
return out;
544+
}
545+
480546
// ---------------------------------------------------------------- the registry
481547

482548
/** Order is part of the interface: a failure a person must act on precedes a warning. */
@@ -493,6 +559,7 @@ export const CHECKS = Object.freeze([
493559
{ name: 'collection-attempts', about: 'a known-unknown was reached for before it was declared unreachable', run: collectionAttempts },
494560
{ name: 'hygiene', about: 'duplicate rows, uncited captures, unparseable dates', run: hygiene },
495561
{ name: 'corpus-shape', about: 'the corpus parses and the shape is there', run: corpusShape },
562+
{ name: 'corroboration', about: 'how many independent readings a closed unknown rests on', run: corroboration },
496563
]);
497564

498565
export const CHECK_NAMES = Object.freeze(CHECKS.map((c) => c.name));

0 commit comments

Comments
 (0)