Skip to content

Commit 34d3779

Browse files
StepenkoAnatoliclaude
StepenkoAnatoli
and
claude
committed
Point the kit at something it knows nothing about, and watch it fail first
Step 3: run the kit on a real question in a domain neither the operator nor the reviewer works in. Every other corpus here is about GitHub, Node, MCP or this kit's own vendors, where a reviewer already knows enough to catch a bad answer. That makes them a soft test. A prior was registered IN WRITING before any collection, so the corpus could contradict it: "30 December 2025 for large and medium operators, 30 June 2026 for micro and small. Confidence: high." A year wrong on both. The prior also said, correctly, that a second postponement decided after May 2026 could not be known from memory. There was one. The answer, from the Commission: large and medium operators 30 December 2026, micro and small 30 June 2027, and micro and small ALREADY COVERED BY THE EU TIMBER REGULATION on the earlier date. That third row is what a summary drops - being small is not sufficient for the later date - and neither the prior nor a general web search surfaced it. THE FINDING THAT MATTERS MORE THAN THE DATES. The first collection returned eight captures and not one was about the EUDR: SEC exempt offerings, two CFPB regulations, federalregister.gov, California water boards, an SEC crypto release, regulations.gov, an EU climate effort-sharing page. The search matched the word "regulation" and returned US financial regulators. That corpus would have passed every structural check in this kit. Eight captures, verifying hash chain, recorded transports, no partial grades. preflight had nothing to say about it, because NO CHECK ASKS WHETHER THE EVIDENCE IS ABOUT THE TOPIC. The only thing between it and a confident wrong brief was a human reading the URLs. Isolated by re-running the identical query through the other provider: SerpApi (the default, because a key is present) 8 candidates, 0 on topic Firecrawl 17 candidates, 17 on topic Not the query - a general web search on the same string returns the Commission, the Parliament press room and the legislative train. Not the kit's code - both ran the same path. The provider failed, and auto-detect picks it. The corpus records what it is honest about: U-1 and U-3 rest on the Commission alone as accepted single-witness judgements; Wikipedia is cited for the sequence of postponements and carries a warning, because its infobox is current while its PROSE still asserts the superseded pair - the exact dates the prior held; and five of eight captures are commentary that agrees with the authority and is cited for nothing, because agreement is not confirmation. The stale-note guard caught me a third time - naming a row id inside a [single-witness: ...] reason makes citedIds count it as a citation, so the corpus reads corroborated while the note denies it. Worth noticing that the mechanism is this easy to misuse. PASS --strict, 0 warnings, 8 ledger entries, chain verifies. 861 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent e1e8e80 commit 34d3779

22 files changed

Lines changed: 2089 additions & 0 deletions

‎docs/decisions/2026-09-22-eudr-dates/.gitattributes‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# The corpus's line endings are integrity, not style.
2+
#
3+
# A capture's body hash is taken over the LF bytes the collector fetched. git's smudge
4+
# filter rewrites every text file on a default Windows checkout (core.autocrlf=true), so
5+
# without this file the whole corpus fails `body-unmodified` on arrival with nothing
6+
# actually wrong with it. This file outranks a reader's core.autocrlf. See ADR-0020.
7+
8+
research/raw/* text eol=lf
9+
*.jsonl text eol=lf
10+
research/*.md text eol=lf

‎docs/decisions/2026-09-22-eudr-dates/.gitignore‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
node_modules/
2+
.DS_Store
3+
4+
# Credentials never belong in a research project (AGENTS.md Rule 6). Note that
5+
# `firecrawl env` writes the key into .env, which is why the rule says never to run it
6+
# inside a repository.
7+
.env
8+
.env.*
9+
*.pem
10+
*.key
11+
12+
# The Firecrawl CLI saves scrape output into .firecrawl/ in whatever directory it runs
13+
# from. That is a vendor cache, not evidence - the kit's own captures live in
14+
# research/raw/ with a ledger entry behind each one.
15+
.firecrawl/
16+
17+
# The kit's local review logs are not evidence.
18+
research/raw/.usage.jsonl
19+
research/raw/.failures.jsonl
20+
research/raw/.diagnostics.jsonl
21+
research/raw/.fetches.lock
22+
23+
# The chain IS evidence and must travel. The explicit negation is here so a broader
24+
# dotfile rule added later cannot silently exclude it - this repository lost its ledger
25+
# exactly that way once.
26+
!research/raw/.fetches.jsonl
27+
!research/overrides.log

‎docs/decisions/2026-09-22-eudr-dates/AGENTS.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Research-first project bootstrap
2+
3+
Agent instructions for this project. Read this before planning, designing, or building
4+
anything here.
5+
6+
## How this project works: two phases, two roles
7+
8+
**Phase 1 - research (this kit).** Close every blocking unknown with fetched primary
9+
sources, then pass the gate. The output is a briefing, not code: `research/DISCOVERY.md`
10+
(the contract), `research/EVIDENCE.md` (claims with cached pages behind them), and
11+
`research/BRIEF.md` (the handoff). Phase 1 never writes product code.
12+
13+
**Phase 2 - build (someone else's job).** A builder - another agent, a different model, or
14+
a human - takes `research/BRIEF.md` plus `research/` and implements. They should not need
15+
to re-research anything: if they do, phase 1 was incomplete, and the fix is to collect the
16+
missing fact rather than to let the builder guess it.
17+
18+
The gate is the handoff point. Before it passes, phase 2 does not start.
19+
20+
## Two machines, two roles
21+
22+
The role is machine config - `role: "collector" | "builder"` in
23+
`~/.agents/research-kit.config.json`, default `collector`; declare it with
24+
`node ~/.agents/research-kit/bin/install-hooks.mjs --role builder`.
25+
26+
| | **collector** (the operator's PC) | **builder** (a sandbox, a CI box, a laptop) |
27+
|---|---|---|
28+
| holds | the Firecrawl key | no key, no Firecrawl egress |
29+
| runs | `decompose.mjs`, `research.mjs` | `handoff.mjs`, `preflight.mjs`, the build |
30+
| a missing key is | a **FAIL** | **informational** |
31+
| must | push `research/raw/` including its dotfiles | **not collect** - the collection CLIs refuse (exit 2) |
32+
33+
**If you are on a builder machine and there is no brief: you do not collect.** A page
34+
fetched by hand is not evidence in this kit, and a builder that "re-collects" a missing
35+
page forges a corpus instead of reporting a gap. Name the missing fact instead.
36+
37+
A builder's first command is:
38+
39+
```
40+
node ~/.agents/research-kit/bin/handoff.mjs
41+
```
42+
43+
The remedy depends on the cause, and the command says which: something that did not
44+
travel is the collector's to push (`git add -f research/raw/`, dotfiles included);
45+
a corpus that travelled whole and was rewritten on checkout here is fixed **here**, with
46+
`.gitattributes`, and costs no credits.
47+
48+
## Starting in the wrong place: ask which project, do not hunt for it
49+
50+
The project is the **current working directory**. The kit takes no project argument and
51+
never will. If the cwd is not the project the operator means - the kit's own repo, a
52+
parent folder, an unrelated checkout - **ask which project**, and wait. Do not search the
53+
filesystem, do not infer it from a name, do not look it up on a remote. Searching a
54+
person's disk to guess his intent reads directories nobody authorised, and the question
55+
it replaces takes one line.
56+
57+
Signals: no `AGENTS.md`, no `research/` directory, or a `research/` corpus plainly about
58+
something else. An existing corpus about another topic is the signal doing its job.
59+
60+
## Resuming after an interruption: resume, do not restart
61+
62+
The gate is a pure function of repository state, so orient from disk before touching
63+
anything. `doctor.mjs` and preflight answer *where this project is*; the last commit
64+
report and `docs/ARCHITECTURE.md` say *what was happening and why*; `git status` answers
65+
what is unfinished, and its uncommitted changes are the interrupted task - partial and
66+
unverified, work in progress, never state to trust. The corpus is already collected and
67+
spent credits are not spent again.
68+
69+
## Rule 1 - research first, then build
70+
71+
The sequence is decompose -> contract -> collect -> gate -> brief.
72+
73+
0. **Decompose** (collector machine): `node ~/.agents/research-kit/bin/decompose.mjs --topic "<topic>"`
74+
drafts `research/MAP.md` seeded with the universal checklist. The tool contains no
75+
judgment. Mark every row COVERED (citing U-## rows), DISMISSED (reason required), or
76+
GAP.
77+
1. **Write the intent** in `research/DISCOVERY.md` under `## Build intent`, and enumerate
78+
the blocking unknowns FROM the map.
79+
2. **Collect**: `node ~/.agents/research-kit/bin/research.mjs --plan research/plan.json`. Prefer the page
80+
that *owns* the fact over any write-up about it.
81+
3. **Rewrite** each auto-extracted `Finding` cell into a real claim, keeping the `Raw`
82+
cell pointing at the cached page.
83+
4. **Gate**: `node ~/.agents/research-kit/bin/preflight.mjs`. **Do not start building until it prints
84+
PASS.** Evidence must be *fetched*, not typed, and by a named transport.
85+
5. **Hand off**: write `research/BRIEF.md`.
86+
87+
There are exactly three overrides - `git commit --no-verify`, a deliberate
88+
`research/GATE_OFF`, and a repository-local `core.hooksPath` - and all three are recorded
89+
in `research/overrides.log`. If you take one, say so in your reply.
90+
91+
## Rule 2 - questions are for intent, never for facts
92+
93+
At most **3** questions, once, up front, and only about things no document can answer. If
94+
a question's answer is in public documentation, it is a research task. **Never answer
95+
"insufficient info" and stop.**
96+
97+
## Rule 3 - known unknowns are allowed, silence is not
98+
99+
Mark a genuinely unreachable fact `KNOWN-UNKNOWN` and write the day-one verification step
100+
in the `Evidence` cell. An unlabeled gap is the failure mode this project exists to
101+
prevent.
102+
103+
## Rule 4 - source quality
104+
105+
`P` primary/official carries the design. `S` secondary is context. `L` lead-only is a
106+
hint, never proof. Never invent a citation. Flag contradictions instead of averaging them.
107+
108+
## Rule 5 - cost discipline
109+
110+
Every scrape spends credits. Plan the queries in `research/plan.json` first, reuse the
111+
cache (`--refresh-days`), and check the budget with `node ~/.agents/research-kit/bin/research.mjs --status`.
112+
113+
## Rule 6 - secrets
114+
115+
The key lives in the CLI config or the environment. Never write it into this repository.
116+
117+
## The standing protocol
118+
119+
Five rules for how work here is committed and recorded. Documentation, not enforcement:
120+
no check judges prose, because a gate that judges prose is a gate that will be wrong.
121+
122+
1. **Architecture map current in the same commit.** A commit touching a declared code
123+
path (`research/kit.json`) stages `docs/ARCHITECTURE.md` with it. Enforced - and a
124+
prompt, not a proof: the gate checks the map was *staged*, not that it is *current*.
125+
2. **Commit report: five named parts** - what changed / why / what it touched / what you
126+
verified / what you got wrong and fixed. The got-wrong line is not optional; if
127+
nothing was gotten wrong it is written as "nothing to report".
128+
3. **An ADR for any choice with a rejected alternative**, dated, with the reason a future
129+
explorer would need to avoid re-suggesting it.
130+
4. **One commit per task - the revert test.** `git revert <sha>` undoes it alone.
131+
5. **A red suite stops work**, reported immediately and alone, **with the cwd recorded**.

‎docs/decisions/2026-09-22-eudr-dates/START_HERE.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Start here
2+
3+
A **decision project** (ADR-0030), and the first one in this repository about something
4+
other than the kit's own plumbing. Read `research/BRIEF.md`.
5+
6+
**The question:** when does the EU Deforestation Regulation actually apply?
7+
8+
**The answer:** large and medium operators from **30 December 2026**; micro and small from
9+
**30 June 2027** — *unless* already covered by the EU Timber Regulation, in which case the
10+
earlier date. Postponed twice.
11+
12+
## Why it exists
13+
14+
To test whether the kit produces research a builder could act on **in a domain it knows
15+
nothing about**. Every other corpus here is about GitHub, Node, MCP or this kit's vendors,
16+
where the reviewer already knows enough to catch a bad answer.
17+
18+
A prior was written down **before collecting**: "30 December 2025 … 30 June 2026,
19+
confidence high." That is a year wrong on both dates, and the corpus corrected it.
20+
21+
## The part worth reading even if you do not care about deforestation
22+
23+
**The first collection returned eight captures and none of them were about the EUDR** — SEC
24+
filings, CFPB regulations, California water boards. The search had matched the word
25+
"regulation."
26+
27+
That corpus would have passed every structural check: eight captures, verifying hash chain,
28+
recorded transports, no partial grades. **No check asks whether the evidence is about the
29+
topic.** `research/BRIEF.md` has the isolation showing it was the search provider, not the
30+
query and not the kit.
31+
32+
## Verifying it
33+
34+
From **this** directory:
35+
36+
```
37+
cd docs/decisions/2026-09-22-eudr-dates
38+
node ..\..\..\research-kit\bin\preflight.mjs
39+
node ..\..\..\research-kit\bin\handoff.mjs
40+
```
41+
42+
As of 2026-09-22: `PASS 0 blocking, 0 warning(s), 15 passing`, passes `--strict`, and
43+
`handoff OK - 8 ledger entries, chain verifies`.

‎docs/decisions/2026-09-22-eudr-dates/docs/ARCHITECTURE.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# ARCHITECTURE - the map of the code
2+
3+
What each module owns, the seams between them, and which decision governs each. This is
4+
a map of the **code**, not of the research: `research/MAP.md` is the topic map - a
5+
different artifact with a different job, judged by a different check.
6+
7+
The rule this map lives by: a commit touching a **declared code path** (declared in
8+
`research/kit.json`, defaulting to `src` / `lib` / `bin` / `scripts` / `app`) carries
9+
this file's update in the **same commit**. The commit gate refuses such commits without
10+
it.
11+
12+
What the rule proves is deliberately modest: the gate checks this file was **staged**
13+
with the code, not that it is current - a whitespace edit satisfies it. That makes a
14+
stale map a deliberate act rather than mere forgetting. A prompt, not a proof.
15+
16+
## The one picture
17+
18+
```
19+
_Replace this with the shape of the thing you build: the entry points, the modules
20+
they reach, and the seams between them._
21+
```
22+
23+
## Modules - what each one owns
24+
25+
| Module | Owns | Governed by |
26+
|---|---|---|
27+
| _module_ | _the one concept it is responsible for_ | _ADR or decision_ |
28+
29+
## The seams
30+
31+
_Where modules touch, what is injectable, and why testing can stay offline._
32+
33+
## Current shape, for the record
34+
35+
_Counts and invariants a reader can check against the code: how many checks, how many
36+
adapters, which defaults._

‎docs/decisions/2026-09-22-eudr-dates/research/BRIEF.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Brief - when does the EUDR actually apply?
2+
3+
- **Date:** 2026-09-22
4+
- **Gate:** PASS - 0 blocking, 0 warnings, 15 passing, and it passes `--strict`
5+
- **Corpus:** 8 captures, 8 ledger entries, chain verifies
6+
- **Collected by:** run `35723908266` via `bin/collect-remote.mjs`, search pinned to `firecrawl-cli`
7+
8+
## The answer
9+
10+
From the European Commission, which sets and publishes these dates:
11+
12+
| operator | applies from |
13+
|---|---|
14+
| large and medium operators | **30 December 2026** |
15+
| micro and small operators | **30 June 2027** |
16+
| micro and small operators **already covered by the EU Timber Regulation** | **30 December 2026** |
17+
18+
Application has been postponed **twice**: from 30 December 2024, then from 30 December 2025.
19+
20+
The third row is the part a summary loses. Being small is not sufficient for the later date
21+
— an operator already inside EUTR scope is on the earlier one. Neither the prior registered
22+
before this collection nor a general web search on the same question surfaced it. Only the
23+
Commission page states it.
24+
25+
## What this exercise was actually testing
26+
27+
Not the EUDR. **Whether the kit produces research a builder could act on, in a domain it
28+
knows nothing about** — chosen because every previous corpus here is about GitHub, Node, MCP
29+
or the kit's own vendors, where the reviewer already knows enough to catch a bad answer.
30+
31+
A prior was registered in writing **before any collection**, so the corpus could contradict
32+
it. It did.
33+
34+
> "30 December 2025 for large and medium operators, and 30 June 2026 for micro and small
35+
> enterprises. Confidence: high."
36+
37+
**That is a year wrong on both dates.** The prior also said, correctly, that a second
38+
postponement decided after May 2026 could not be known from memory. There was one. The
39+
corpus supplied exactly the thing the reviewer could not have.
40+
41+
## The finding that matters more than the dates
42+
43+
**The first collection returned nothing relevant at all**, and the gate could not tell.
44+
45+
Run on the default search path, the same topic produced eight captures: SEC exempt
46+
offerings, two CFPB regulations, federalregister.gov, California water boards, an SEC crypto
47+
press release, regulations.gov, and an EU climate effort-sharing page. **Zero about
48+
deforestation.** The search had matched the word "regulation" and returned US financial
49+
regulators.
50+
51+
That corpus would have passed every structural check in the kit. It had eight captures, a
52+
verifying hash chain, recorded transports, no partial grades. `preflight` had nothing to say
53+
about it, because **no check asks whether the evidence is about the topic.** The only thing
54+
standing between that corpus and a confident wrong brief was a human reading the URLs.
55+
56+
Isolated by re-running the identical query string through the other provider:
57+
58+
| search provider | candidates | on topic |
59+
|---|---|---|
60+
| SerpApi (the default, because a key is present) | 8 | **0** |
61+
| Firecrawl | 17 | **17** |
62+
63+
Not the query — a general web search on the same string returns the Commission, the European
64+
Parliament press room, and the legislative train. Not the kit's code — both ran the same
65+
path. The provider failed, and the auto-detect rule picks it.
66+
67+
## What the corpus is honest about
68+
69+
- **U-1 and U-3 rest on the Commission alone**, recorded as accepted single-witness
70+
judgements rather than papered over. On the dates an authority sets, a second party can
71+
only report.
72+
- **Wikipedia is cited with a warning attached.** Its infobox strikes through both superseded
73+
dates, which is what establishes two postponements. Its **body text still asserts the old
74+
pair** — the exact dates the registered prior held. One page, current in its structured
75+
data and a year stale in its prose. It is cited for the sequence and for nothing else.
76+
- **Five of eight captures are law-firm and vendor commentary.** They agree with the
77+
Commission and none of them is cited for a claim, because agreement with the authority is
78+
not independent confirmation of it.
79+
80+
## Limits
81+
82+
- **One collection, one provider, one day.** The dates have moved twice; nothing here
83+
predicts whether they move again, and the corpus would not know if they did.
84+
- **No legal reading.** This is the date obligations begin, not advice about what they
85+
require of any particular operator.
86+
- **The EUTR carve-out is quoted, not interpreted.** What counts as "already covered by the
87+
EU Timber Regulation" is a scope question this corpus did not ask.

‎docs/decisions/2026-09-22-eudr-dates/research/DISCOVERY.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Discovery Contract - when does the EUDR actually apply?
2+
3+
Started 2026-09-22.
4+
5+
## Build intent
6+
7+
A compliance question with a date answer, chosen deliberately from a domain this project
8+
knows nothing about, to test whether the kit produces research a builder could act on
9+
rather than research that merely passes its own checks.
10+
11+
"Done" is the operative application dates as of today, from the authority that sets them,
12+
with the amending instrument named.
13+
14+
**A prior was registered before any collection, so the corpus could contradict it.** It did.
15+
The prior said 30 December 2025 for large operators and 30 June 2026 for micro and small
16+
enterprises, at high confidence, with an explicit note that a second postponement decided
17+
after May 2026 could not be known. There was a second postponement.
18+
19+
## Unknowns
20+
21+
| ID | Unknown | Why it blocks the build | Status | Evidence |
22+
|---|---|---|---|---|
23+
| U-1 | What are the operative EUDR application dates today? | Any compliance plan is scheduled off this date. A year wrong in either direction is either wasted spend or a missed obligation | CLOSED | E-01: the Commission states it directly - large and medium operators **30 December 2026**, micro and small operators **30 June 2027**, and micro and small operators already covered by the EU Timber Regulation **30 December 2026** [single-witness: the European Commission is the authority that sets and publishes these dates, so a second party can only report them. One of the secondary rows in this corpus demonstrates what that reporting is worth - its own prose still carries the superseded pair while its structured data is current] |
24+
| U-2 | How many times has application been postponed, and by what instrument? | A date that has already moved twice is a date that may move again. Knowing the mechanism is how a plan stays correct rather than lucky | CLOSED | E-06 records the sequence: 30 December 2024 struck through, then 30 December 2025 struck through - **two postponements**, not one. E-08 names the outcome of the second in its title: "EU Deforestation Regulation application postponed to 30 December 2026" |
25+
| U-3 | Does the smallest-operator deferral apply to everyone small? | A carve-out that catches an operator a year early is exactly the kind of detail a summary drops | CLOSED | E-01: **no.** Micro and small operators get 30 June 2027 **unless already covered by the EU Timber Regulation**, in which case the earlier date applies. Neither the registered prior nor a general web search surfaced this; only the Commission page states it [single-witness: the EUTR carve-out is stated only by the Commission page. A general web search on the same question did not surface it at all, which is the point - a summary drops the exception and keeps the headline date] |
26+
27+
## Questions for the human (maximum 3)
28+
29+
1. None for this corpus. The dates are stated by the authority and carry an amending
30+
instrument. If anything is wanted next it is a watch on further postponement, which is a
31+
cadence question rather than a research one.
32+
33+
## Already decided
34+
35+
- Nothing. This project exists to test the kit on an unfamiliar domain, not to support a
36+
build.

0 commit comments

Comments
 (0)