Skip to content

Commit f24a199

Browse files
A thirteenth check: the gate can see single-sourcing, and says so without blocking
Twelve checks asked whether a claim was BACKED - is there a row, a capture, a verifying chain, a whole page, a fresh date, a primary type, an unsuperseded row. None asked whether it was backed MORE THAN ONCE. So a corpus in which every unknown rested on a single page passed preflight AND --strict with nothing to say about it. Not hypothetical: that corpus is this repository's own delivery-architecture project, on the day it was used to justify ADR-0031 and ADR-0032. Nine rows, all primary, all full, all fresh, eight unknowns each resting on exactly one. Three of those eight carried architecture decisions. From inside such a corpus a single CORRECT source and a single LUCKY one are indistinguishable, and every other check reports green either way. WHAT IT REPORTS, AND WHY IT NEVER FAILS ALONE single-source one row one-voice several rows, one host independent several rows, several hosts Single-sourcing is often the right answer - a vendor's own API reference IS the authority on that API, and demanding a second opinion about it is ceremony. So the check describes the support and leaves the judgement to a reviewer, the same line ADR-0013 drew when it refused to let a check grade reasoning. It joins transport-provenance and capture-completeness in POLICY_CHECKS, so evidencePolicy=strict is where an operator asks for the harder rule. Every existing corpus stays green: all three here still PASS under the default policy, now with warnings that say something true. THE HOST DISTINCTION IS THE POINT, NOT A REFINEMENT Counting rows was easier and would have missed the case that matters. Two pages from one vendor are ONE WITNESS READ TWICE: that catches a misreading, and cannot catch a vendor being wrong about itself. The remedies differ - one wants another page, the other wants another party - so the finding names which it found. AND IT IMMEDIATELY CONTRADICTED ME The corroboration pass an hour earlier added a second source to the three delivery-architecture unknowns that carry decisions, and that commit called them "independent". The check disagrees: all three are docs.github.com, so all three report one-voice. The check is right and the commit message was loose. They are independent PAGES - written by different teams, and they did catch facts the originals omitted - and they are not independent SOURCES. The corpus's own contract already said so in prose, "a second reading of the same vendor", and the commit summarising it reached for the stronger word anyway. That is the argument for this check in miniature: the distinction is easy to state, easy to believe you have honoured, and invisible unless something counts hosts. TWO TESTS CHANGED RATHER THAN BEING MADE TO PASS One asserted "every check reports a PASS" on the fixture project. That fixture is single-sourced, so it now carries one warning. The assertion became "every check REPORTS", with the expected warning named - rather than padding the fixture with a second source to preserve the old wording, which would have been tuning the world to fit the assertion. The other inferred "hygiene is not policy-hardened" from a whole verdict passing. That proxy worked only while nothing else in the policy set fired; it now isolates the claim, and asserts the mirror - that a check which IS in the set hardens, with the promotion attributed. 829 passed, 0 failed. All three corpora PASS. Repository preflight PASS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 606e86e commit f24a199

9 files changed

Lines changed: 301 additions & 11 deletions

File tree

‎CONTEXT.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,3 +151,4 @@ Added 2026-09-21 under ADR-0032. The vocabulary of the ZIP that leaves this kit.
151151
| **unrewritten finding** | An `EVIDENCE.md` `Finding` cell byte-identical to what the extractor re-produces from the cached capture — so it was demonstrably never rewritten by a human. The negative half of a question with no positive answer: ADR-0013 refuses to grade whether an agent reasoned properly, and this does not try. It detects the unreviewed case only. |
152152
| **structural refusal** | A ZIP fault decided from the **central directory**, before one byte is inflated: traversal, absolute path, drive letter, backslash, NUL, duplicate, case collision, symlink, entry count, declared size, compression ratio. The directory is attacker-controlled too, so `inflateRawSync` also runs with `maxOutputLength` — the ceiling belongs to the runtime rather than to a field the attacker wrote. |
153153
| **transport, not storage** | What an artifact is. Workflow artifacts last at most 90 days on a public repository and vanish with the run that produced them, so anything that must persist is committed instead. |
154+
| **corroboration** | How many INDEPENDENT readings a closed unknown rests on, and the thirteenth check. Three shapes: one row (`single-source`), several rows from one host (`one-voice`), several hosts (`independent`). The middle one is the distinction a count cannot make — two pages from one vendor catch a misreading and cannot catch the vendor being wrong about itself. It reports and never fails on its own: a vendor's own reference is the authority on that vendor, and demanding a second opinion about it would be ceremony. `evidencePolicy=strict` is where an operator says otherwise. |

‎docs/ARCHITECTURE.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,7 @@ each line — is in the ADR named beside it and in
415415
| A ZIP's structural faults are refused from the **central directory, before inflation**; the inflation ceiling is `maxOutputLength`, not a field the archive declared about itself | ADR-0032 |
416416
| **No dispatch input reaches a shell**: `${{ inputs.x }}` inside a `run:` block is substitution *before* the shell parses, so inputs arrive through `env:` and then an argv array. Enforced across **every** workflow, not only the newest | [ADR-0033](adr/0033-the-collector-refuses-rather-than-degrades.md), ADR-0020 |
417417
| The collector's spend gate is a **protected environment whose existence is proven**, because GitHub silently creates an *unprotected* one when a workflow names a missing environment; a missing credential **refuses** rather than falling back to a keyless route | ADR-0033 |
418+
| A claim's **support is weighed, not just checked**: `corroboration` reports whether a closed unknown rests on one reading, several readings of one source, or several hosts — it never fails a corpus on its own, because single-sourcing is often the right answer | [ADR-0036](adr/0036-the-gate-can-see-single-sourcing.md) |
418419
| A privacy claim is **measured, not asserted**: `bin/disclosure.mjs` re-runs the probe, and a report never lets an unchecked thing read as a clean one | [ADR-0035](adr/0035-a-privacy-claim-is-measured-not-asserted.md) |
419420
| A credential that resolves in a job declaring **no** environment is **not behind the approval gate**, and fails the run: the scope question GitHub will not answer directly is answered by asking it from a context that can see only one scope | ADR-0033 |
420421
| The vendor **package name** lives in `lib/firecrawl.mjs` (`CLI_PACKAGE`, `cliInstallSpec`), never in a workflow: the CLI is **`firecrawl-cli`**, and the npm package called `firecrawl` is the SDK and ships no binary. Both workflows installed the wrong one for weeks, and only a real dispatch could find it | ADR-0005, ADR-0033 |
@@ -436,7 +437,7 @@ Counted against the filesystem, not estimated:
436437
| `bin/` entrypoints | **29**, of which **4 are Python**: 3 conformance runners and the module they share |
437438
| `schemas/` | **12** |
438439
| `conformance/` vector packets | **3** |
439-
| checks in the registry | **12** |
440+
| checks in the registry | **13** |
440441
| supported CI platforms | **2** |
441442

442443
The `schemas/` count became 12 on 2026-09-21 when `artifact-manifest.schema.json` arrived
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# ADR-0036 — The gate can see single-sourcing, and says so without blocking
2+
3+
- **Date:** 2026-09-21
4+
- **Status:** accepted
5+
- **Area:** contract checks, evidence policy
6+
- **Depends on:** ADR-0004 (the checks are an ordered registry), ADR-0013 (a check that judged reasoning would be wrong)
7+
8+
## Context
9+
10+
Twelve checks asked whether a claim was **backed**: is there a row, is there a capture,
11+
does the chain verify, is the capture whole, is it fresh, is it primary, has it been
12+
superseded. Not one asked whether it was backed **more than once**.
13+
14+
So a corpus in which every unknown rested on a single page passed `preflight` and
15+
`--strict` with nothing to say about it. That is not hypothetical — it is this
16+
repository's own delivery-architecture corpus, on the day it was used to justify ADR-0031
17+
and ADR-0032. Nine rows, all primary, all `full`, all fresh, and eight unknowns each
18+
resting on exactly one of them. Three of those eight carried architecture decisions.
19+
20+
**From inside such a corpus, a single correct source and a single lucky one are
21+
indistinguishable.** Every other check would report green either way.
22+
23+
## Decision
24+
25+
A thirteenth check, `corroboration`, reporting the **shape** of an unknown's support:
26+
27+
| rule | when | severity |
28+
|---|---|---|
29+
| `single-source` | one row | `warn` |
30+
| `one-voice` | several rows, one host | `warn` |
31+
| `independent` | several rows, several hosts | `pass` |
32+
33+
### It never fails a corpus on its own
34+
35+
Single-sourcing is frequently the right answer. A vendor's own API reference *is* the
36+
authority on that API, and demanding a second opinion about it is ceremony. This check
37+
declines to make that judgement — it describes the support and leaves the call to a
38+
reviewer, which is the same line ADR-0013 drew when it refused to let a check grade
39+
reasoning.
40+
41+
`corroboration` joins `transport-provenance` and `capture-completeness` in `POLICY_CHECKS`,
42+
so `evidencePolicy=strict` is where an operator says they want the harder rule. All three
43+
are questions an *evidence policy* should answer rather than the kit: how it was fetched,
44+
how much arrived, how many times it was read.
45+
46+
Every existing corpus stays green. Verified on all three in this repository — the
47+
repository's own, and both nested decision projects — each still `PASS` under the default
48+
policy, now with warnings that say something true.
49+
50+
### The host distinction is the point, not a refinement
51+
52+
Counting rows would have been easier and would have missed the case that matters. Two
53+
pages from one vendor are **one witness read twice**: that catches a misreading, and
54+
cannot catch a vendor being wrong about itself. The remedies differ — one wants another
55+
page, the other wants another party — so the finding names which it found.
56+
57+
## What it immediately said about work finished an hour earlier
58+
59+
The corroboration pass that preceded this ADR added a second source to the three
60+
delivery-architecture unknowns that carry decisions, and the commit called them
61+
*"independent"*. The check disagrees: all three second sources are `docs.github.com`, so
62+
all three report `one-voice` rather than `independent`.
63+
64+
**The check is right and the commit message was loose.** They are independent *pages* —
65+
different URLs, written by different teams, and they did catch facts the originals omitted,
66+
including that an unpinned API client silently moves to a different version on a date. They
67+
are not independent *sources*. The corpus's own contract already said this in prose —
68+
"a second reading of the same vendor" — and the commit summarising it reached for the
69+
stronger word anyway.
70+
71+
That is the argument for the check in miniature: the distinction is easy to state, easy to
72+
believe you have honoured, and invisible unless something counts hosts.
73+
74+
## Alternatives considered
75+
76+
**Fail on single-sourcing.** Would have broken every corpus in this repository and most
77+
that anyone writes, to enforce a rule that is wrong about half the time.
78+
79+
**Count rows and ignore hosts.** Simpler, and it would have graded the delivery corpus as
80+
corroborated when it is not.
81+
82+
**Require a second source only for unknowns marked load-bearing.** No such marking exists,
83+
and inventing one moves the judgement into the contract without making it checkable.
84+
85+
**Leave it to review.** That is what was happening. Review missed it for as long as the
86+
corpus existed, and the corpus was used to make two architecture decisions in the meantime.
87+
88+
## Consequences
89+
90+
- The registry is thirteen checks; the pinned-order test and the inventory move with it.
91+
- One preflight test asserted "every check reports a **pass**" on the fixture project. That
92+
fixture is single-sourced, so it now carries one warning. The assertion was changed to
93+
"every check **reports**", with the expected warning named — rather than padding the
94+
fixture with a second source to preserve the old wording, which would have been tuning
95+
the world to fit the assertion.
96+
- The policy-membership test stopped inferring "hygiene is not hardened" from a whole
97+
verdict passing, and now isolates the claim. That proxy only ever worked while nothing
98+
else in the policy set fired.
99+
100+
## What this ADR does not claim
101+
102+
That two hosts make a claim true. Two vendors can repeat one another, and a specification
103+
and its own tutorial are not independent whatever their hostnames say. `independent` means
104+
*this check found no reason to flag the support*, which is weaker than correct and is the
105+
strongest thing a mechanical check can honestly report.

‎docs/adr/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,5 +51,6 @@ directory and are treated as ADR-equivalent.
5151
| [0033](0033-the-collector-refuses-rather-than-degrades.md) | **The collector refuses rather than degrades, and no dispatch input reaches a shell.** `collect.yml` implements ADR-0031 against ADR-0032's contract, separate from `live-collection.yml` because "does the adapter work" and "collect this, for a person, now" deserve different approvers. The environment is the spend gate — and is **checked for existence**, because GitHub silently *creates* an unprotected environment when a workflow names one that is missing (E-06), which would remove the approval gate without failing anything. The honest limit is written in the file: GitHub exposes no scope information, so the marker proves the environment was configured and **cannot** prove a repository-level value did not serve the secret. No `http-keyless` fallback: a missing credential refuses rather than shipping a quietly worse corpus under the same artifact name. Every input arrives through `env:` and then an argv array — `${{ inputs.x }}` in a `run:` block is substitution *before* the shell parses, so a backtick in a topic executes. The test enforcing that scans every workflow and found `live-collection.yml` had been doing it since it was written; fixed here **Amended the same day**: the required reviewer is dropped and the environment kept. A reviewer approving a *dispatch* is a spend gate, not a review — nothing is collected yet to judge — and it makes autonomous dispatch impossible. The environment stays because it was never about approval: it is the only place the credential is unreadable by every other workflow. Spend is bounded by `max_pages`, the depth tier and the vendor's 402-at-zero cap. The control that replaces the reviewer is **token scope**: a fine-grained PAT with `Actions: read and write` on one repository, driven by `bin/collect-remote.mjs`, which takes **no `--token` flag** and redacts by pattern rather than by remembering its own value |
5252
| [0034](0034-the-collector-speaks-mcp-over-stdio.md) | **The collector speaks MCP over stdio** — and this is the first module in the repository built from research that PASSED THE GATE rather than from a plan; every decision cites an `E-` row in [`docs/decisions/2026-09-21-agent-interface/`](../decisions/2026-09-21-agent-interface/research/BRIEF.md). stdio because semantics are identical on both transports, so the difference is operational, and remote costs the entire OAuth 2.1 chain. Two tools, not one, because a server cannot initiate a request — so the alternative is a call held open for minutes and an agent that loses a paid run when its process dies. A `resource_link`, never embedded bytes: the spec defines embedding and **states no maximum size**, so "permitted" is not "known to work". The credential stays in the environment, which for stdio is what the spec itself prescribes. **The gate refused the first draft of that brief**, naming a load-bearing GAP; two more pages went into the same ledger to close it, and `resource_link` is what came back |
5353
| [0035](0035-a-privacy-claim-is-measured-not-asserted.md) | **A privacy claim is measured, not asserted — and this repository stays public.** The collector's own header said dispatch inputs are visible to anyone with read access. I wrote that from reasoning and never tested it; unauthenticated probing found the topic in **none** of the readable responses — run metadata, step names, timing and the artifact *listing* are `200`, while the artifact **download** is `401` and the job **logs** are `403`. A stranger learns a run's shape, not its subject. Wrong in the direction that feels safe is still wrong: a reader who finds one warning false discounts the next. Staying public because this account is on the **free** plan (checked, not inferred) where going private makes environment secrets *ignored* — a plan change, not a toggle. The claim is now executable: `bin/disclosure.mjs`, unauthenticated by construction, exit codes as the finding, and `needleChecked` so an **unchecked** thing can never print as a clean one |
54+
| [0036](0036-the-gate-can-see-single-sourcing.md) | **The gate can see single-sourcing, and says so without blocking.** Twelve checks asked whether a claim was *backed*; none asked whether it was backed more than once — so a corpus where every unknown rested on one page passed `preflight` AND `--strict`, and that corpus was this repository's own, on the day it justified ADR-0031 and ADR-0032. `corroboration` reports the SHAPE of support: `single-source`, `one-voice` (several rows, one host), or `independent` (several hosts). It **never fails on its own** — a vendor's own reference is the authority on that vendor — and joins the `POLICY_CHECKS` so `evidencePolicy=strict` is where an operator asks for the harder rule. The host distinction is the point: two pages from one vendor are one witness read twice. It immediately contradicted a commit written an hour earlier that called three `docs.github.com` pages "independent"; they are independent *pages*, not independent *sources*, and the check was right |
5455

5556
Domain vocabulary lives in [`CONTEXT.md`](../../CONTEXT.md).

‎research-kit/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -395,7 +395,7 @@ node research-kit/bin/selftest.mjs # all of it
395395
node research-kit/bin/selftest.mjs gate hook # just these files
396396
```
397397

398-
824 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
398+
829 tests, offline, no key and no network. The runner **awaits** every test, so `ok` means
399399
the assertions settled (ADR-0021), and each test is raced against a watchdog
400400
(`RESEARCH_KIT_TEST_TIMEOUT`, default 60s).
401401

‎research-kit/lib/checks.mjs‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
// A check emits findings. It never decides what a finding MEANS for the build - that is
88
// the verdict's single judgement, in lib/preflight.mjs.
99

10-
import { PATHS, resolve, exists, ageInDays } from './core.mjs';
10+
import { hostOf, PATHS, resolve, exists, ageInDays } from './core.mjs';
1111
import { captureOf, traceOf, citedIds } from './corpus.mjs';
1212
import { coverageOfUniversals } from './dimensions.mjs';
1313
import { verifyLedger } from './provenance.mjs';
@@ -477,6 +477,72 @@ function corpusShape(corpus) {
477477
return out;
478478
}
479479

480+
481+
// ---------------------------------------------------------------- 13. corroboration
482+
483+
/**
484+
* How many independent readings does a closed unknown rest on?
485+
*
486+
* WHY THIS EXISTS. Every check before it asks whether a claim is BACKED - is there a row,
487+
* is there a capture, does the chain verify, is it fresh, is it primary. None of them asks
488+
* whether it is backed more than once, so a corpus in which every unknown rested on a
489+
* single page passed preflight and `--strict` with nothing to say about it. From inside
490+
* such a corpus, a single CORRECT source and a single LUCKY one are indistinguishable.
491+
*
492+
* Found in this repository's own delivery-architecture corpus on 2026-09-21: nine rows,
493+
* all primary, all complete, all fresh - and eight unknowns each resting on exactly one of
494+
* them. Three of those eight carried architecture decisions.
495+
*
496+
* WHY IT IS A WARNING. Single-sourcing is often the right answer: a vendor's own API
497+
* reference is the authority on that API, and demanding a second opinion about it would be
498+
* ceremony. This check refuses to make that judgement - it reports the shape of the
499+
* support so a reviewer can make it. `evidencePolicy=strict` or `--strict` is where an
500+
* operator says they want the harder rule.
501+
*
502+
* WHY THE HOST MATTERS. Two pages from one vendor are not two witnesses. They are one
503+
* witness read twice, which catches a misreading and cannot catch a vendor being wrong
504+
* about itself. The finding says which of those it found, because the remedies differ:
505+
* one wants another page, the other wants another party.
506+
*/
507+
function corroboration(corpus) {
508+
const out = [];
509+
const byId = new Map(corpus.evidence.map((row) => [row.id.toUpperCase(), row]));
510+
511+
for (const unknown of corpus.unknowns) {
512+
// A KNOWN-UNKNOWN rests on nothing by definition, and unknown-closure already owns
513+
// the case of a CLOSED one citing no row at all.
514+
if (unknown.status !== 'CLOSED') continue;
515+
516+
const rows = unknown.cites
517+
.filter((id) => /^E-\d+$/i.test(id))
518+
.map((id) => byId.get(id.toUpperCase()))
519+
.filter(Boolean);
520+
if (!rows.length) continue;
521+
522+
if (rows.length === 1) {
523+
out.push(finding('warn', 'corroboration', 'single-source',
524+
`${unknown.id} rests on ${rows[0].id} alone - one reading, so a correct source and a lucky one look the same`,
525+
{ row: unknown.id, line: unknown.line }));
526+
continue;
527+
}
528+
529+
const hosts = new Set(rows.map((row) => hostOf(row.url)).filter(Boolean));
530+
if (hosts.size === 1) {
531+
out.push(finding('warn', 'corroboration', 'one-voice',
532+
`${unknown.id} cites ${rows.length} rows and all are ${[...hosts][0]} - a second reading of one source, which catches a misreading and not a source that is wrong about itself`,
533+
{ row: unknown.id, line: unknown.line }));
534+
continue;
535+
}
536+
537+
out.push(finding('pass', 'corroboration', 'independent',
538+
`${unknown.id} rests on ${rows.length} rows across ${hosts.size} hosts`,
539+
{ row: unknown.id, line: unknown.line }));
540+
}
541+
542+
if (!out.length) out.push(finding('pass', 'corroboration', 'support', 'no closed unknown to weigh'));
543+
return out;
544+
}
545+
480546
// ---------------------------------------------------------------- the registry
481547

482548
/** Order is part of the interface: a failure a person must act on precedes a warning. */
@@ -493,6 +559,7 @@ export const CHECKS = Object.freeze([
493559
{ name: 'collection-attempts', about: 'a known-unknown was reached for before it was declared unreachable', run: collectionAttempts },
494560
{ name: 'hygiene', about: 'duplicate rows, uncited captures, unparseable dates', run: hygiene },
495561
{ name: 'corpus-shape', about: 'the corpus parses and the shape is there', run: corpusShape },
562+
{ name: 'corroboration', about: 'how many independent readings a closed unknown rests on', run: corroboration },
496563
]);
497564

498565
export const CHECK_NAMES = Object.freeze(CHECKS.map((c) => c.name));

0 commit comments

Comments
 (0)