Skip to content

Commit e0edb2e

Browse files
Corroborate the last three corpora, and say plainly where the remedy does not exist
agent-interface 5 warnings -> 2, public-run-visibility 4 -> 3, root 7 -> 7. The falling numbers are the smaller half of this. U-4 of agent-interface got the best second source there is: RFC 9728 itself, the IETF standard the MCP spec defers to for the 401 -> WWW-Authenticate -> metadata chain. A different standards body, normative rather than explanatory - the one document that could have contradicted the spec instead of repeating it. U-1 and U-3 got running code: the reference TypeScript SDK ships transports for exactly the two bindings the spec calls standard, and its minimal stdio server exposes a tool with no authorization layer at all, which is what "authorization is optional" looks like implemented. The SDK was collected expecting it to close U-6 and U-7 as well. It does not. Searched over the capture rather than assumed: no occurrence of `resource_link`, none of `protocolVersion`. One row cited for four unknowns when it supports two is how a corpus inflates, and both citations would have gone green with nobody reading the tally any the wiser. Cited for two, and EVIDENCE.md states the negative. public-run-visibility got a measurement instead, and it caught an error a second reading would not have. U-1's answer, taken from the documentation, says an account is required to view run information. Probed anonymously - no Authorization header at all: actions/runs/{id} 200 actions/runs/{id}/jobs (step names) 200 actions/runs/{id}/artifacts (NAMES) 200 actions/artifacts/{id}/zip (bytes) 401 actions/runs/{id}/logs 403 actions/secrets 401 So the documented claim is true of the WEB UI and false of the REST API, and reading it as the whole exposure understated it. U-1 is corrected in place. The probe also found something nobody was looking for: the anonymous artifact listing returns NAMES, and this repository's read `research-kit-corpus-v1-gap-sea-assets`. ADR-0033 forbids the topic in a run, step or artifact name and that rule was kept - but the name carries the client_ref, and the client_refs chosen described the subject. The discipline was applied to the field it named, not to the field that renders into the public string. Recorded rather than quietly fixed: the gap is in the rule's wording. The root corpus was examined and deliberately left alone. Not budget - 841 credits remained. Its seven unknowns are vendor claims about the vendor's own product (nobody but Firecrawl can say what Firecrawl charges) or claims about THIS machine (no page knows which skill root this desktop app discovers). The available second sources are third-party comparison pages, stale often enough that citing one would raise the row count while lowering the evidence quality. A subtler trap, named so it is not walked into later: docs.firecrawl.dev and www.firecrawl.dev are different HOSTS. Citing both flips one-voice to independent on the host count while remaining one company describing itself. That is the check's known blind spot, and using it to clear a warning would be gaming the gate. What was done instead: U-8's cost claim was measured. 854 credits before, 841 after, 13 scraped pages across four collections - exactly one credit per page, on the version this machine runs. The thing itself rather than a description of it. Also closes the verdict-line gap. `--strict` and `evidencePolicy=strict` are different settings, and the summary printed the declared policy either way, so a --strict run read `FAIL ... [evidencePolicy=pluralist]` - which says pluralist failed it. It did not; the flag did. It now reads `[--strict, over evidencePolicy=pluralist]`, keeping both visible. 835 passed, 0 failed. All five corpora PASS, every chain verifies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 0732d36 commit e0edb2e

21 files changed

Lines changed: 2608 additions & 20 deletions

‎docs/ARCHITECTURE.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -177,7 +177,12 @@ not.
177177

178178
`research.mjs` (→ research-run, refused on a builder by role), `researcher-release.mjs` (→
179179
read-only R28–R32 validation and schema conformance), `handoff.mjs` (→
180-
verifyHandoff, the builder's first command), `preflight.mjs` (→ the verdict), `gate.mjs`
180+
verifyHandoff, the builder's first command), `preflight.mjs` (→ the verdict; its summary
181+
line names **the setting that decided the verdict**, printing `[--strict, over
182+
evidencePolicy=pluralist]` rather than the declared policy alone — `--strict` promotes every
183+
warning while `evidencePolicy=strict` promotes only the three `POLICY_CHECKS`, and labelling
184+
a flag-driven failure with the policy name told a reader the policy had failed them),
185+
`gate.mjs`
181186
(→ evaluate; `--staged-stdin` reads the piped path list; `--posture` answers 0 allow /
182187
1 fail-closed / 2 unreadable-and-closed, so no shell has to parse JSON), `doctor.mjs` (`--fix-arity`), `install.mjs` (deploy + `--into` per-project
183188
binding, role-aware next steps), `install-hooks.mjs` (`--fail-closed`, `--role`),

‎docs/adr/0036-the-gate-can-see-single-sourcing.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,8 @@ What an operator needs to know now: **`strict` is aspirational in this repositor
131131
standard it meets.** Anyone turning it on should expect red, and should read the findings as a
132132
worklist rather than as a regression.
133133

134+
**Fixed 2026-09-21.** The verdict now names the setting that decided it - a `--strict` run prints `[--strict, over evidencePolicy=pluralist]`, keeping the declared policy visible under the flag that overrode it. The paragraph below is what it used to do.
135+
134136
Two settings are also easy to conflate, and the verdict line encourages it: `--strict`
135137
promotes **every** warning, while `evidencePolicy=strict` promotes only the three
136138
`POLICY_CHECKS`. The summary prints the project's *declared* policy regardless, so a

‎docs/decisions/2026-09-21-agent-interface/research/BRIEF.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,3 +137,43 @@ revision and handle `-32022` whatever the cadence is.
137137
1. Review the **TODO** sections above (Contradictions, Decision) before handing off.
138138
2. Hand this file to the builder (phase 2). Re-running `node bin/brief.mjs`
139139
after edits will refuse without `--force` so your judgements are preserved.
140+
141+
## Corroboration pass, 2026-09-21
142+
143+
Three captures into the same ledger - eight entries, one chain. Five warnings became two,
144+
and the two that remain were **attempted and failed** rather than skipped.
145+
146+
**U-4 got the best kind of second source there is: the standard itself.** E-06 is RFC 9728,
147+
"OAuth 2.0 Protected Resource Metadata" - the IETF document MCP defers to for the `401` ->
148+
`WWW-Authenticate` -> metadata chain. Different standards body, different process, normative
149+
rather than explanatory. It is the one document that could have *contradicted* the spec
150+
instead of repeating it, which is what separates corroboration from echo.
151+
152+
**U-1 and U-3 got running code.** E-07, the reference TypeScript SDK, ships transports for
153+
exactly the two bindings the spec calls standard, and its minimal stdio server exposes a tool
154+
with **no authorization layer at all** - which is what "authorization is optional" looks like
155+
when somebody implements it.
156+
157+
### The part worth keeping: what the SDK did not corroborate
158+
159+
The SDK was collected expecting it to close U-6 and U-7 as well. It does not. Searched over
160+
the capture rather than assumed:
161+
162+
- **no occurrence of `resource_link`** - so U-6, how a tool returns a 71 KB corpus, still
163+
rests on `modelcontextprotocol.io` alone
164+
- **no occurrence of `protocolVersion` or version negotiation** - so U-7 does too
165+
166+
One row cited for four unknowns when it supports two is exactly how a corpus inflates, and
167+
the temptation was real: both citations would have gone green and nobody reading the tally
168+
would have known. The row is cited for U-1 and U-3, and `EVIDENCE.md` states the negative.
169+
170+
### U-7 has something better than a second page anyway
171+
172+
It was **measured**. Building this server against the specification produced a server no
173+
client could reach - `-32601: unknown method initialize` - because the shipped SDK implements
174+
the older handshake. That is the whole reason `lib/mcp.mjs` is dual-era, and it is stronger
175+
evidence about "how does a client find out and recover" than any page could be.
176+
177+
It is deliberately **not** filed as an evidence row. A measurement is not a fetched page, and
178+
`unknown-closure/no-evidence` exists to catch precisely that substitution - the same call
179+
made for U-6 of the delivery corpus, and for U-3 of the sea-assets contract.

‎docs/decisions/2026-09-21-agent-interface/research/DISCOVERY.md‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,10 @@ Status is exactly one of:
2828

2929
| ID | Unknown | Why it blocks the build | Status | Evidence |
3030
|---|---|---|---|---|
31-
| U-1 | Which transports does MCP define, and are protocol semantics the same on each? | Decides whether the collector must be written once or twice, and whether a local and a hosted deployment can share an implementation | CLOSED | E-03: exactly two standard bindings - `stdio` and `Streamable HTTP` - and "Protocol semantics are identical on every transport. A transport is a **binding**". One implementation serves both |
31+
| U-1 | Which transports does MCP define, and are protocol semantics the same on each? | Decides whether the collector must be written once or twice, and whether a local and a hosted deployment can share an implementation | CLOSED | E-03: exactly two standard bindings - `stdio` and `Streamable HTTP` - and "Protocol semantics are identical on every transport. A transport is a **binding**". One implementation serves both **Corroborated by E-07**, the reference TypeScript SDK, which ships transports for exactly those two bindings - the spec's claim, in running code on a different host. |
3232
| U-2 | Can an MCP server tell a client that long-running work has finished, or must the client wait on the open request? | A collection takes minutes. If the server cannot initiate contact, the shape is either a held-open stream or polling - and that decides the whole interface | CLOSED | E-03: "servers do not initiate JSON-RPC requests and clients do not send JSON-RPC responses" - no other message direction exists. So a server cannot announce completion out of band. E-02 (secondary) describes the in-band route: a Streamable HTTP reply may be "a request-scoped SSE stream" carrying `notifications/progress` while the request stays open |
33-
| U-3 | Is authorization required, and what does a **local** stdio server need? | Decides whether shipping an MCP server means also shipping an identity system, or whether the existing environment-variable credential is sufficient | CLOSED | E-01: authorization is "**optional**", and for stdio "you can use environment-based credentials or credentials provided by third-party libraries embedded directly in the MCP server instead". The kit's existing `tokenFromEnv` already satisfies this |
34-
| U-4 | What does a **remote** HTTP MCP server need before an agent can authenticate to it? | This is the cost of the hosted option, and it is the difference between a feature and a product | CLOSED | E-01: the full OAuth 2.1 chain - `401` with `WWW-Authenticate: Bearer resource_metadata=...`, a Protected Resource Metadata document at `/.well-known/oauth-protected-resource` (RFC 9728), authorization-server metadata discovery (RFC 8414 or OIDC Discovery), client registration either pre-arranged or via Dynamic Client Registration, then a browser authorization-code exchange. "OAuth flows ... are designed for HTTP-based transports where the MCP server is remotely-hosted" |
33+
| U-3 | Is authorization required, and what does a **local** stdio server need? | Decides whether shipping an MCP server means also shipping an identity system, or whether the existing environment-variable credential is sufficient | CLOSED | E-01: authorization is "**optional**", and for stdio "you can use environment-based credentials or credentials provided by third-party libraries embedded directly in the MCP server instead". The kit's existing `tokenFromEnv` already satisfies this **Corroborated by E-07**: the SDK's minimal stdio server exposes a tool with no authorization layer at all, which is what "optional" looks like when implemented. |
34+
| U-4 | What does a **remote** HTTP MCP server need before an agent can authenticate to it? | This is the cost of the hosted option, and it is the difference between a feature and a product | CLOSED | E-01: the full OAuth 2.1 chain - `401` with `WWW-Authenticate: Bearer resource_metadata=...`, a Protected Resource Metadata document at `/.well-known/oauth-protected-resource` (RFC 9728), authorization-server metadata discovery (RFC 8414 or OIDC Discovery), client registration either pre-arranged or via Dynamic Client Registration, then a browser authorization-code exchange. "OAuth flows ... are designed for HTTP-based transports where the MCP server is remotely-hosted" **Corroborated by E-06**, RFC 9728 itself - the IETF standard this chain is built from, and the one document that could have contradicted the spec rather than repeating it. |
3535
| U-5 | How does a client cancel work in flight, and does the mechanism differ per transport? | The collector spends money. An agent that gives up must be able to stop a run rather than abandon a request and let it bill | CLOSED | E-03: it differs by binding - "on stdio the client sends a `notifications/cancelled` notification; on Streamable HTTP it closes the request's response stream", while "the protocol-level rules are the same everywhere". E-02 names this `$/cancelRequest`, which is wrong; see the brief's contradictions section |
3636
| U-6 | How does a tool hand back an artifact the size of a corpus ZIP? | The output of this collector is a 71 KB package. A tool that cannot return it is not a design, and this was the one load-bearing GAP the gate refused the first brief over | CLOSED | E-04: a `resource_link` content item returns a URI rather than bytes - `{"type":"resource_link","uri":"file:///...","mimeType":...}` - which suits a file on the client machine exactly. Embedding is also permitted but **the spec states no maximum size anywhere on that page**, so a link is the defensible choice rather than the merely available one |
3737
| U-7 | When the protocol changes under a shipped server, how does a client find out and recover? | The protocol has already broken once. If a mismatch is silent or unrecoverable, anything built here rots quietly | CLOSED | E-05: version is declared per request with no handshake, and a server that cannot serve it **MUST** return `UnsupportedProtocolVersionError` (JSON-RPC `-32022`) carrying `data.supported`; the client **SHOULD** pick a mutually supported version and retry. `server/discover` is mandatory on servers so a client MAY ask up front. A break is therefore typed, detectable and recoverable at runtime |
@@ -49,3 +49,21 @@ Status is exactly one of:
4949
- Authorization is derived from the corpus, not asserted by a caller (ADR-0032). An MCP
5050
tool returning an artifact does not change that: the manifest still says
5151
`buildAuthorized: false` on a fresh corpus.
52+
53+
## Two unknowns left single-sourced, and what was actually tried, 2026-09-21
54+
55+
The corroboration pass closed three of five. The other two were **attempted and failed**,
56+
which is a different thing from not trying, and the corpus should say which.
57+
58+
- **U-6 (returning a corpus-sized artifact).** The reference SDK was collected expecting it
59+
to show `resource_link` in use. It contains **no occurrence of the term** - checked by
60+
search over the capture, not assumed. So the claim still rests on
61+
`modelcontextprotocol.io` alone. The SDK row is cited for U-1 and U-3 only; stretching it
62+
to cover U-6 would have bought a green check with a citation that does not support it.
63+
- **U-7 (recovering from a protocol version mismatch).** Same outcome from the same capture:
64+
no `protocolVersion`, no version negotiation. But this unknown has something better than a
65+
second page - **it was measured.** Building the server against the spec produced a server
66+
no client could reach (`-32601: unknown method initialize`), because the shipped SDK
67+
implements the older handshake. That is recorded in ADR-0034 and is why `lib/mcp.mjs` is
68+
dual-era. It is deliberately **not** filed as an evidence row: a measurement is not a
69+
fetched page, and `unknown-closure/no-evidence` exists to catch exactly that substitution.

‎docs/decisions/2026-09-21-agent-interface/research/EVIDENCE.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,6 @@ what the page actually establishes, with the quote or number that proves it.
1313
| E-03 | 2026-09-21 | P | https://modelcontextprotocol.io/specification/2026-07-28/basic/transports | **Two standard transports, one set of semantics, and servers cannot speak first.** "Protocol semantics are identical on every transport. A transport is a **binding**: it defines how messages are framed and delivered ... It does not define what the messages mean." The two standard bindings are `stdio` - "newline-delimited messages over the standard streams of a client-launched subprocess" - and `Streamable HTTP`, where "each message is an HTTP POST to a single MCP endpoint; replies arrive as a JSON object or a request-scoped SSE stream". The constraint that shapes any long-running tool: a binding "**MUST** deliver client-sent *requests* and *notifications* to the server, and server-sent *responses* and *notifications* to the client. No other message direction exists: ... servers do not initiate JSON-RPC requests and clients do not send JSON-RPC responses." Cancellation is per-binding - "on stdio the client sends a `notifications/cancelled` notification; on Streamable HTTP it closes the request's response stream" - while "the protocol-level rules are the same everywhere". Custom transports are permitted but "**MUST** preserve the JSON-RPC message format, the message patterns, and the per-request metadata model", and one over a reliable byte stream "**SHOULD** reuse the stdio framing". Note for anyone reading older material: this revision removed the connection-scoped `initialize` session and the server's ability to initiate requests, both of which earlier revisions had. Re-graded **P** from the collector's S: this is the specification. | research/raw/2026-09-21-overview-model-context-protocol-modelcontextprotocol-7296cd40.md |
1414
| E-04 | 2026-09-21 | P | https://modelcontextprotocol.io/specification/2026-07-28/server/tools | **A tool returns a LINK to a large artifact, not the bytes - and the spec sets no size limit.** Tool results carry "structured or unstructured content"; unstructured content is a `content` array that "can contain multiple content items of different types" - text, image, audio, resource links and embedded resources. The item that answers this project: `resource_link`, where "A tool **MAY** return links to Resources ... the tool will return a URI that can be subscribed to or fetched by the client", shaped `{"type":"resource_link","uri":"file:///...","name":...,"mimeType":...}`. An `embedded resource` also exists, inlining the content, and a server using one "**SHOULD** implement the `resources` capability". Two cautions a builder needs. First, **no maximum size appears anywhere on this page** - the mechanism is defined, the ceiling is not, so a 71 KB corpus ZIP is not known to be safe to embed and a link is the defensible choice. Second, a returned link "is not guaranteed to appear in the results of a `resources/list` request", so the client must use the URI it was handed rather than expecting to rediscover it. Also relevant to credential hygiene: parameters may be mirrored into HTTP headers via `x-mcp-header`, and the page warns developers "**SHOULD NOT** mark sensitive parameters (passwords, API keys, tokens, PII)" that way "as header values are visible to network intermediaries". | research/raw/2026-09-21-tools-model-context-protocol-modelcontextprotocol-920caecd.md |
1515
| E-05 | 2026-09-21 | P | https://modelcontextprotocol.io/specification/2026-07-28/basic/versioning | **Version is declared per request, there is no handshake, and the rejection is a typed error a client can act on.** "There is no negotiation handshake. Every request carries its protocol version, and the server accepts or rejects each request independently." The version travels in `_meta`, and on HTTP also in an `MCP-Protocol-Version` header. A server that does not implement the requested version "**MUST** respond with an `UnsupportedProtocolVersionError`" - JSON-RPC code `-32022` - carrying `data.supported` and `data.requested`, and the client "**SHOULD** select a mutually supported version from the `supported` list and retry". Servers "**MUST** implement `server/discover`" so a client MAY learn supported versions up front, but is not required to. The page also names the break E-03 alluded to, and dates it: **modern** revisions (`2026-07-28` and later) carry version, identity and capabilities as per-request metadata; **legacy** ones (`2025-11-25` and earlier) establish a session with an `initialize` handshake; a **dual-era** implementation supports both. This closes D-11 only partly and honestly: it settles how a version mismatch is DETECTED and recovered from, which is what a builder needs. It says nothing about who governs the specification or how often a break may arrive - see the brief. | research/raw/2026-09-21-versioning-and-compatibility-model-conte-modelcontextprotocol-c6bfb6f5.md |
16+
| E-06 | 2026-09-21 | P | https://datatracker.ietf.org/doc/html/rfc9728 | **Second source for U-4, and the only kind that counts here: the standard itself.** RFC 9728, "OAuth 2.0 Protected Resource Metadata", is the IETF document the MCP specification defers to - Section 5 "describes the use of `WWW-Authenticate` by protected resources", which is the first link in the `401` -> metadata -> authorization-server chain E-01 describes. **This is genuinely independent**: a different standards body, a different process and a different document from `modelcontextprotocol.io`, and it is normative rather than explanatory. If MCP described the chain wrongly, this row would disagree with it. It does not. | research/raw/2026-09-21-rfc-9728-oauth-2-0-protected-resource-me-ietf-0c1e8943.md |
17+
| E-07 | 2026-09-21 | P | https://github.com/modelcontextprotocol/typescript-sdk | **Second source for U-1 and U-3, from running code rather than prose - and it is honest about what it does NOT cover.** The reference TypeScript SDK ships "**server** libraries (tools/resources/prompts, Streamable HTTP, stdio, auth helpers)", naming exactly the two bindings E-03 calls standard, and its minimal example "exposes a single `greet` tool over stdio" using `StdioServerTransport` with **no authorization layer at all** - which is what "authorization is optional" (U-3) looks like when implemented. **It does not corroborate U-6 or U-7**: the README contains no occurrence of `resource_link` and none of `protocolVersion` or version negotiation. Checked by search rather than assumed, and recorded because a row cited for four unknowns when it supports two is how a corpus inflates. | research/raw/2026-09-21-github-modelcontextprotocol-typescript-s-github-f7fb7d70.md |
18+
| E-08 | 2026-09-21 | P | https://www.jsonrpc.org/specification | **Depth for U-2 rather than a new claim, and it is the layer underneath.** The JSON-RPC 2.0 specification defines a Notification as a request object without an `id`, to which a server returns no response. MCP's "servers do not initiate JSON-RPC requests and clients do not send JSON-RPC responses" (E-03) is a restriction applied on top of this, so this row explains the mechanism E-03 states as a rule. U-2 already passed corroboration and this does not change its status - collected because the design question it settles (a collection taking minutes cannot be pushed to the client) is load-bearing and rested entirely on one host. | research/raw/2026-09-21-json-rpc-2-0-specification-jsonrpc-237728dc.md |

0 commit comments

Comments
 (0)