Skip to content

Commit efe693d

Browse files
Use the kit on the kit: close ADR-0035's named gap, and find stale claims while doing it
Two gaps closed, and the second was found by asking the repository what it still admitted to rather than by remembering. 1. STALE CLAIMS: FIVE DOCUMENTS SAID SOMETHING THAT STOPPED BEING TRUE ADR-0031, ADR-0033 and the delivery corpus all still said the runner-side install path "has never executed". It has, twice - runs 35600022797 and 35608301287, both with `install the vendor CLI` and `the CLI is a major this adapter supports` succeeding. Corrected in the live documents; the DATED RECORDS are left alone. The preliminary note and live-predicate-verification were true on the day they were written and say so in their titles, and a record that is quietly revised stops being a record. ADRs take dated amendments instead, which is this repository's established pattern. Worth naming as a kit gap rather than only fixing: nothing detects a claim overtaken by events. `corroboration` now sees single-sourcing; nothing sees "this says X never happened, and X happened". 2. ADR-0035's OWN OPEN QUESTION, ANSWERED - AND IT IS THE LESS COMFORTABLE ANSWER That ADR closed with a section headed "What is NOT measured, and is therefore not claimed": what a SIGNED-IN user sees, as opposed to the anonymous caller it had probed. Collected through the collector itself, dispatched via bin/collect-remote.mjs - the agent path - as run 35620262684. "You must be logged in to a GitHub account to view workflow run information, including for public repositories." Viewing logs, searching logs, downloading logs and downloading artifacts each require only "read access to the repository". On a public repository read access is UNIVERSAL. So any person with a free GitHub account can open a collector run, read the logs - which echo the dispatched topic in the `env:` group the runner prints - and download the artifact, which is the entire corpus. The 401 and 403 ADR-0035 measured are refusals of ANONYMITY, not of strangers. The measurement was accurate about what it measured; the population it measured is not the population that matters. WHAT CHANGES, AND WHAT DOES NOT The naming rules are demoted rather than discarded - they cover the anonymous surface, listing-scrapers and a glance over a shoulder, and nothing past it. `retention-days: 7` was already set and now has a recorded reason: it bounds the window and removes nothing, and the comment says so instead of leaving seven days looking like a tidy default. Staying public STANDS. That was decided on plan grounds - a free private repository has its environment secrets and variables ignored, so going private breaks the collector rather than protecting it - and this finding does not touch that. What changes is the caveat: not "the exposure is limited to shape" but "the exposure is total to anyone with an account, and the remedy is a paid plan rather than a toggle". THE KIT UNDER ITS OWN NEW CHECK The corpus reports four corroboration warnings - three single-source, one one-voice, all docs.github.com - and they are left standing rather than argued away. This is GitHub describing its own permission model: authoritative, and not a second opinion. The check is right and the research is still worth having, which is exactly the judgement ADR-0036 said it would leave to a reviewer. Committed at docs/decisions/2026-09-21-public-run-visibility/ per ADR-0030, ledger included. From the nested root: PASS 0 blocking, 4 warning(s), 12 passing, and handoff OK - 3 ledger entries, chain verifies. Repository 829 passed, preflight PASS. Cost: 4 credits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent c340855 commit efe693d

23 files changed

Lines changed: 1378 additions & 17 deletions

‎.github/workflows/collect.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -443,6 +443,21 @@ jobs:
443443
node "$KIT/bin/artifact.mjs" validate --file "$ARTIFACT_PATH" --json
444444
node "$KIT/bin/artifact.mjs" validate --file "$ARTIFACT_PATH" --quiet
445445
446+
# `retention-days` BOUNDS the exposure. It does not remove it, and the difference is
447+
# worth stating because the naming rules above are easy to over-trust.
448+
#
449+
# On a public repository, ANY signed-in GitHub account can open this run, read these
450+
# logs - which echo the dispatched topic in the `env:` group above - and download
451+
# this artifact, which is the whole collected corpus. The documentation is explicit:
452+
# "You must be logged in to a GitHub account to view workflow run information,
453+
# including for public repositories", and viewing, searching and downloading logs and
454+
# artifacts each require only "read access to the repository", which on a public
455+
# repository is universal. The 401 and 403 an anonymous probe gets are refusals of
456+
# ANONYMITY, not of strangers.
457+
#
458+
# Seven days rather than the 90-day default, because the only dial here is how long
459+
# the window stays open. The agent path fetches within minutes, so a short window
460+
# costs nothing real. See docs/decisions/2026-09-21-public-run-visibility/.
446461
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4, resolved 2026-09-21
447462
with:
448463
name: ${{ env.ARTIFACT_NAME }}

‎docs/adr/0031-actions-first-delivery-and-pin-the-api-version.md‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -141,9 +141,18 @@ it.
141141

142142
- The next build step is a `workflow_dispatch` collector that uploads a labelled artifact,
143143
with the five properties above.
144-
- Two things remain unproven and are recorded as gaps rather than claimed: the runner-side
145-
`npm install -g firecrawl@<version>` path, which has never executed, and whether artifact
146-
download is ergonomic for the intended operator, which documentation cannot answer.
144+
- Two things remained unproven when this was written and were recorded as gaps rather than
145+
claimed: the runner-side install path, and whether artifact download is ergonomic for the
146+
intended operator.
147+
148+
**The first is closed, later the same day.** The install path has executed on a real
149+
runner and succeeded, in runs `35600022797` and `35608301287` — both reporting `install
150+
the vendor CLI` and `the CLI is a major this adapter supports` as successful. Getting
151+
there took a fix no amount of reading would have found: the npm package is
152+
`firecrawl-cli`, and the one called `firecrawl` is the SDK, which ships no binary.
153+
154+
The second is still open and always will be by this route — it is a usability question,
155+
and no vendor page answers it.
147156
- The preliminary note keeps its text. It was right about the behaviour it observed and
148157
wrong about the cause, and rewriting it would erase the most useful thing in this
149158
decision: that reading a page and running the call gave different answers, and only

‎docs/adr/0033-the-collector-refuses-rather-than-degrades.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -224,10 +224,16 @@ vulnerability: the substitution happens before the validation can run.
224224
- `live-collection.yml` loses four shell interpolations and gains four `env:` blocks.
225225
- The collector runs on `ubuntu-latest` and `windows-latest`, with `shell: bash` pinned so
226226
one script serves both.
227-
- **The runner path is still unproven.** `npm install -g firecrawl@<version>` on a runner,
228-
and `cliCompatibility()` against the CLI it installs, have never executed. This change
229-
makes that the only remaining gap and does not close it — closing it needs the
230-
environment, the secret, and one real dispatch, all of which are the owner's.
227+
- **The runner path was unproven when this was written, and is now closed.** It needed the
228+
environment, the secret and one real dispatch, all of which were the owner's to provide —
229+
and all of which arrived the same day. Runs `35600022797` and `35608301287` both executed
230+
`npm install -g firecrawl-cli@1.23.3` and `cliCompatibility()` against the CLI the runner
231+
installed, successfully.
232+
233+
The first attempt failed, which is the point of having run it at all: both workflows had
234+
been installing a package called `firecrawl` since they were written. On npm that is the
235+
JavaScript SDK, and it ships no binary. The CLI is `firecrawl-cli`. Nothing offline could
236+
have caught it — every test drives a stub.
231237

232238
## What this ADR does not claim
233239

‎docs/adr/0035-a-privacy-claim-is-measured-not-asserted.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,48 @@ silent spend, and a worse diagnostic than expected. The step's message already n
119119
right fix for the wrong reason; anybody hitting it on a Free private repository should read
120120
this paragraph first.
121121

122+
### The unmeasured half, measured — 2026-09-21, later the same day
123+
124+
The section above says what a signed-in user sees "is **unknown**, and 'unknown' is what
125+
the documentation now says rather than 'safe'". That was the right posture and it did not
126+
survive contact with the documentation.
127+
128+
Collected through the collector itself, into
129+
`docs/decisions/2026-09-21-public-run-visibility/`:
130+
131+
- "You must be logged in to a GitHub account to view workflow run information, **including
132+
for public repositories**." That is why an anonymous probe saw so little — the web
133+
surface *starts* at an account.
134+
- Viewing logs, searching logs, downloading logs and downloading artifacts each carry the
135+
same requirement: "**Read access to the repository is required to perform these steps.**"
136+
137+
**On a public repository, read access is universal.** So any person with a free GitHub
138+
account can open a collector run, read the logs — which echo the dispatched topic in the
139+
`env:` group the runner prints — and download the artifact, which is the entire collected
140+
corpus.
141+
142+
**The `401` and `403` this ADR measured are refusals of ANONYMITY, not of strangers.** The
143+
measurement was accurate about what it measured; the population it measured is not the
144+
population that matters.
145+
146+
Three consequences:
147+
148+
1. **The naming rules are demoted, not discarded.** This ADR called them "the primary
149+
control". They cover the anonymous surface — listing-scrapers, a glance over a shoulder
150+
— and nothing beyond it.
151+
2. **`retention-days: 7` on the collector now has a recorded reason.** It bounds the
152+
window; it removes nothing. The comment in `collect.yml` says so rather than leaving
153+
seven days looking like a tidy default.
154+
3. **Decision 1 stands, with a different caveat.** Staying public was decided on *plan*
155+
grounds — a free private repository has its environment secrets and variables ignored,
156+
so going private breaks the collector rather than protecting it. That is untouched. What
157+
changes is the sentence attached to it: not "the exposure is limited to shape" but "the
158+
exposure is total to anyone with an account, and the remedy is a paid plan, not a
159+
toggle".
160+
161+
The gap was named in this ADR and closed by the kit this ADR is about, which is the first
162+
time that loop has run on a question this repository asked of itself.
163+
122164
## Alternatives considered
123165

124166
**Go private anyway, to be safe.** Breaks the collector on this plan, for an exposure the

‎docs/decisions/2026-09-21-delivery-architecture/research/BRIEF.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -99,10 +99,11 @@ Two things remain unproven and are deliberately *not* unknowns in the contract,
9999
neither blocks this decision - they block the thing it recommends building. They are
100100
recorded in `research/MAP.md` under "Known gaps":
101101

102-
1. The runner-side `npm install -g firecrawl@<version>` path has never executed, and
103-
neither has `cliCompatibility()` against a CLI installed on a runner. The local metered
104-
route was verified (`../../live-predicate-verification-2026-09-21.md`); the CI route was
105-
not. Closing it needs the `live-collection` environment, its secret, and one dispatch.
102+
1. ~~The runner-side install path has never executed.~~ **Closed later the same day.**
103+
`npm install -g firecrawl-cli@1.23.3` and `cliCompatibility()` both ran and succeeded on
104+
a real runner, in runs `35600022797` and `35608301287`. Worth having named rather than
105+
assumed: the first real attempt failed, because the workflow installed `firecrawl`,
106+
which on npm is the SDK and ships no binary. The CLI is `firecrawl-cli`.
106107
2. Whether artifact download is ergonomic enough for the intended operator. A usability
107108
question; documentation cannot answer it.
108109

‎docs/decisions/2026-09-21-delivery-architecture/research/MAP.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -57,12 +57,12 @@ Neither is an unknown in the contract, because neither blocks *this* decision -
5757
block the thing the decision recommends building, and omitting them would misrepresent
5858
what has been established.
5959

60-
- **The runner-side install path has still never executed.** `npm install -g firecrawl@<v>`
61-
on `ubuntu-latest`, and `cliCompatibility()` against the CLI it installs, are asserted by
62-
`.github/workflows/live-collection.yml` and proven nowhere. The local metered route was
63-
verified on 2026-09-21 (`../../live-predicate-verification-2026-09-21.md`); the runner
64-
route was not. Closing it needs the `live-collection` environment, its secret, and one
65-
dispatch - all of which are the repository owner's to create.
60+
- ~~**The runner-side install path has still never executed.**~~ **Closed 2026-09-21.** It
61+
now has, on a real runner, twice - runs `35600022797` and `35608301287` - including
62+
`cliCompatibility()` against the CLI the runner installed. The first attempt failed, and
63+
that is exactly why the gap deserved naming rather than assuming: the workflow installed
64+
`firecrawl`, which on npm is the SDK and ships no binary at all. The CLI is
65+
`firecrawl-cli`, and no amount of reading the documentation would have found it.
6666
- **Whether artifact download is ergonomic for the intended operator.** A usability
6767
question. No quantity of vendor documentation answers it, and this corpus does not
6868
pretend to.
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# The corpus's line endings are integrity, not style.
2+
#
3+
# A capture's body hash is taken over the LF bytes the collector fetched. git's smudge
4+
# filter rewrites every text file on a default Windows checkout (core.autocrlf=true), so
5+
# without this file the whole corpus fails `body-unmodified` on arrival with nothing
6+
# actually wrong with it. This file outranks a reader's core.autocrlf. See ADR-0020.
7+
8+
research/raw/* text eol=lf
9+
*.jsonl text eol=lf
10+
research/*.md text eol=lf
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
node_modules/
2+
.DS_Store
3+
4+
# Credentials never belong in a research project (AGENTS.md Rule 6). Note that
5+
# `firecrawl env` writes the key into .env, which is why the rule says never to run it
6+
# inside a repository.
7+
.env
8+
.env.*
9+
*.pem
10+
*.key
11+
12+
# The Firecrawl CLI saves scrape output into .firecrawl/ in whatever directory it runs
13+
# from. That is a vendor cache, not evidence - the kit's own captures live in
14+
# research/raw/ with a ledger entry behind each one.
15+
.firecrawl/
16+
17+
# The kit's local review logs are not evidence.
18+
research/raw/.usage.jsonl
19+
research/raw/.failures.jsonl
20+
research/raw/.diagnostics.jsonl
21+
research/raw/.fetches.lock
22+
23+
# The chain IS evidence and must travel. The explicit negation is here so a broader
24+
# dotfile rule added later cannot silently exclude it - this repository lost its ledger
25+
# exactly that way once.
26+
!research/raw/.fetches.jsonl
27+
!research/overrides.log
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Research-first project bootstrap
2+
3+
Agent instructions for this project. Read this before planning, designing, or building
4+
anything here.
5+
6+
## How this project works: two phases, two roles
7+
8+
**Phase 1 - research (this kit).** Close every blocking unknown with fetched primary
9+
sources, then pass the gate. The output is a briefing, not code: `research/DISCOVERY.md`
10+
(the contract), `research/EVIDENCE.md` (claims with cached pages behind them), and
11+
`research/BRIEF.md` (the handoff). Phase 1 never writes product code.
12+
13+
**Phase 2 - build (someone else's job).** A builder - another agent, a different model, or
14+
a human - takes `research/BRIEF.md` plus `research/` and implements. They should not need
15+
to re-research anything: if they do, phase 1 was incomplete, and the fix is to collect the
16+
missing fact rather than to let the builder guess it.
17+
18+
The gate is the handoff point. Before it passes, phase 2 does not start.
19+
20+
## Two machines, two roles
21+
22+
The role is machine config - `role: "collector" | "builder"` in
23+
`~/.agents/research-kit.config.json`, default `collector`; declare it with
24+
`node ~/.agents/research-kit/bin/install-hooks.mjs --role builder`.
25+
26+
| | **collector** (the operator's PC) | **builder** (a sandbox, a CI box, a laptop) |
27+
|---|---|---|
28+
| holds | the Firecrawl key | no key, no Firecrawl egress |
29+
| runs | `decompose.mjs`, `research.mjs` | `handoff.mjs`, `preflight.mjs`, the build |
30+
| a missing key is | a **FAIL** | **informational** |
31+
| must | push `research/raw/` including its dotfiles | **not collect** - the collection CLIs refuse (exit 2) |
32+
33+
**If you are on a builder machine and there is no brief: you do not collect.** A page
34+
fetched by hand is not evidence in this kit, and a builder that "re-collects" a missing
35+
page forges a corpus instead of reporting a gap. Name the missing fact instead.
36+
37+
A builder's first command is:
38+
39+
```
40+
node ~/.agents/research-kit/bin/handoff.mjs
41+
```
42+
43+
The remedy depends on the cause, and the command says which: something that did not
44+
travel is the collector's to push (`git add -f research/raw/`, dotfiles included);
45+
a corpus that travelled whole and was rewritten on checkout here is fixed **here**, with
46+
`.gitattributes`, and costs no credits.
47+
48+
## Starting in the wrong place: ask which project, do not hunt for it
49+
50+
The project is the **current working directory**. The kit takes no project argument and
51+
never will. If the cwd is not the project the operator means - the kit's own repo, a
52+
parent folder, an unrelated checkout - **ask which project**, and wait. Do not search the
53+
filesystem, do not infer it from a name, do not look it up on a remote. Searching a
54+
person's disk to guess his intent reads directories nobody authorised, and the question
55+
it replaces takes one line.
56+
57+
Signals: no `AGENTS.md`, no `research/` directory, or a `research/` corpus plainly about
58+
something else. An existing corpus about another topic is the signal doing its job.
59+
60+
## Resuming after an interruption: resume, do not restart
61+
62+
The gate is a pure function of repository state, so orient from disk before touching
63+
anything. `doctor.mjs` and preflight answer *where this project is*; the last commit
64+
report and `docs/ARCHITECTURE.md` say *what was happening and why*; `git status` answers
65+
what is unfinished, and its uncommitted changes are the interrupted task - partial and
66+
unverified, work in progress, never state to trust. The corpus is already collected and
67+
spent credits are not spent again.
68+
69+
## Rule 1 - research first, then build
70+
71+
The sequence is decompose -> contract -> collect -> gate -> brief.
72+
73+
0. **Decompose** (collector machine): `node ~/.agents/research-kit/bin/decompose.mjs --topic "<topic>"`
74+
drafts `research/MAP.md` seeded with the universal checklist. The tool contains no
75+
judgment. Mark every row COVERED (citing U-## rows), DISMISSED (reason required), or
76+
GAP.
77+
1. **Write the intent** in `research/DISCOVERY.md` under `## Build intent`, and enumerate
78+
the blocking unknowns FROM the map.
79+
2. **Collect**: `node ~/.agents/research-kit/bin/research.mjs --plan research/plan.json`. Prefer the page
80+
that *owns* the fact over any write-up about it.
81+
3. **Rewrite** each auto-extracted `Finding` cell into a real claim, keeping the `Raw`
82+
cell pointing at the cached page.
83+
4. **Gate**: `node ~/.agents/research-kit/bin/preflight.mjs`. **Do not start building until it prints
84+
PASS.** Evidence must be *fetched*, not typed, and by a named transport.
85+
5. **Hand off**: write `research/BRIEF.md`.
86+
87+
There are exactly three overrides - `git commit --no-verify`, a deliberate
88+
`research/GATE_OFF`, and a repository-local `core.hooksPath` - and all three are recorded
89+
in `research/overrides.log`. If you take one, say so in your reply.
90+
91+
## Rule 2 - questions are for intent, never for facts
92+
93+
At most **3** questions, once, up front, and only about things no document can answer. If
94+
a question's answer is in public documentation, it is a research task. **Never answer
95+
"insufficient info" and stop.**
96+
97+
## Rule 3 - known unknowns are allowed, silence is not
98+
99+
Mark a genuinely unreachable fact `KNOWN-UNKNOWN` and write the day-one verification step
100+
in the `Evidence` cell. An unlabeled gap is the failure mode this project exists to
101+
prevent.
102+
103+
## Rule 4 - source quality
104+
105+
`P` primary/official carries the design. `S` secondary is context. `L` lead-only is a
106+
hint, never proof. Never invent a citation. Flag contradictions instead of averaging them.
107+
108+
## Rule 5 - cost discipline
109+
110+
Every scrape spends credits. Plan the queries in `research/plan.json` first, reuse the
111+
cache (`--refresh-days`), and check the budget with `node ~/.agents/research-kit/bin/research.mjs --status`.
112+
113+
## Rule 6 - secrets
114+
115+
The key lives in the CLI config or the environment. Never write it into this repository.
116+
117+
## The standing protocol
118+
119+
Five rules for how work here is committed and recorded. Documentation, not enforcement:
120+
no check judges prose, because a gate that judges prose is a gate that will be wrong.
121+
122+
1. **Architecture map current in the same commit.** A commit touching a declared code
123+
path (`research/kit.json`) stages `docs/ARCHITECTURE.md` with it. Enforced - and a
124+
prompt, not a proof: the gate checks the map was *staged*, not that it is *current*.
125+
2. **Commit report: five named parts** - what changed / why / what it touched / what you
126+
verified / what you got wrong and fixed. The got-wrong line is not optional; if
127+
nothing was gotten wrong it is written as "nothing to report".
128+
3. **An ADR for any choice with a rejected alternative**, dated, with the reason a future
129+
explorer would need to avoid re-suggesting it.
130+
4. **One commit per task - the revert test.** `git revert <sha>` undoes it alone.
131+
5. **A red suite stops work**, reported immediately and alone, **with the cwd recorded**.
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Start here
2+
3+
A **decision project** (ADR-0030). Read `research/BRIEF.md`; everything else is what it
4+
rests on.
5+
6+
**The question:** on a public repository, who can actually read a workflow run - its shape,
7+
its logs, and the artifact it produced?
8+
9+
## Why it exists
10+
11+
ADR-0035 measured an **anonymous** caller and found only the run's shape exposed. It closed
12+
with a section headed "What is NOT measured, and is therefore not claimed": what a
13+
**signed-in** user sees. That gap was load-bearing, because the collector's logs echo the
14+
dispatched topic and the artifact *is* the corpus.
15+
16+
This closes it, and the answer is the less comfortable one: **any signed-in GitHub account
17+
can read the logs and download the artifact.** The `401` and `403` ADR-0035 measured are
18+
refusals of anonymity, not of strangers.
19+
20+
## How it was collected
21+
22+
Dispatched through `bin/collect-remote.mjs` - the agent path, the same one an AI would
23+
use - as run
24+
[35620262684](https://github.com/StepenkoAnatoli/Research-Kit/actions/runs/35620262684),
25+
depth `quick`. Three pages, all `full`, all from `docs.github.com`.
26+
27+
The `corroboration` check reports that as `one-voice` rather than `independent`, and it is
28+
right to: this is GitHub describing its own permission model. Authoritative, and not a
29+
second opinion. The warnings are left standing rather than argued away.
30+
31+
## Verifying it
32+
33+
From **this** directory:
34+
35+
```
36+
cd docs/decisions/2026-09-21-public-run-visibility
37+
node ..\..\..\research-kit\bin\preflight.mjs
38+
node ..\..\..\research-kit\bin\handoff.mjs
39+
```
40+
41+
As of 2026-09-21: `PASS 0 blocking, 4 warning(s), 12 passing` and `handoff OK - 3 ledger
42+
entries, chain verifies`.

0 commit comments

Comments
 (0)