Skip to content

Actions collector: refuse rather than degrade, and keep inputs out of the shell (ADR-0033) - #23

Merged
StepenkoAnatoli merged 1 commit into
mainfrom
feat/actions-collector
Sep 21, 2026
Merged

StepenkoAnatoli merged 1 commit into
mainfrom
feat/actions-collector

Conversation

@StepenkoAnatoli

Copy link
Copy Markdown
Owner

Step 5. The Actions collector, implementing ADR-0031 against ADR-0032's contract. Based on
main; both prerequisites are merged.

Scope, item by item

Asked for Done
Runs on Actions incl. windows-latest runs-on input, shell: bash pinned so one script serves both
Require the protected environment + FIRECRAWL_API_KEY environment: research-collection, secret required non-empty
Fail before spending if the secret is unavailable credential and environment checks both precede research.mjs
Never fall back to repository secrets see the honest limit below
Collect into a temporary project $RUNNER_TEMP, with git status --porcelain checked after
Run artifact.mjs create yes — and it derives authorization; the workflow cannot assert it
Upload the honestly labelled ZIP if-no-files-found: error, validated before upload
Pin X-GitHub-Api-Version: 2026-03-10 recorded in the manifest as source.apiVersion
Use the returned workflow_run_id github.run_id → source.workflowRunId
Topics out of public run and artifact names neither carries it; names decided before collection
Offline tests, no paid collection in CI 25 tests; workflow_dispatch only, asserted
Windows UI out of scope not touched

The environment is checked for existence

E-06, verbatim: a workflow referencing an environment that does not exist doesn't fail —
GitHub creates one, with no protection rules and no secrets. A typo in the name would
remove the approval gate and nothing would say so.

So the environment must prove it exists: an environment variable
(RESEARCH_KIT_COLLECTION_ENV) that an auto-created environment cannot have.

The honest limit, written into the workflow rather than implied. GitHub exposes no
scope information to a job — secrets.X resolves environment-first and falls back, and no
syntax or API available to the job distinguishes them. The marker proves the environment
was configured; it cannot prove a repository-level value didn't serve the secret.
Defeating it requires deliberately setting a repository variable of that name. The
enforcement that matters stays where GitHub actually enforces it: the protection rules.

I'd rather state that than claim a guarantee the platform doesn't offer.

A live defect this found in already-merged code

The injection test scans every workflow, and live-collection.yml had been
interpolating four dispatch inputs straight into shell bodies since it was written:

pages='${{ inputs.max_pages }}'

max_pages is free text, so '; curl … | sh; ' would have executed on the runner.
Dispatching needs write access, which bounds severity — it doesn't make it correct. Fixed
here: four env: blocks, zero interpolations left in any shell body.

A defect outside workflows entirely

Feeding the collector a hostile topic found a lib/scaffold.mjs bug:
template/research/plan.json holds "topic": "{{TOPIC}}" and substitution is textual, so
a topic with a double quote wrote a plan.json that does not parse. research.mjs then
reads an empty plan, collects nothing, and blames a file the operator never edited.

Any operator typing Why "agentic" search costs more hit this. Tokens are now JSON-escaped
for .json and verbatim for markdown. Two regression tests — both confirmed red against
the unfixed module first
, then green after restoring it.

Verified without spending a credit

Scaffold → plan → map → package → validate, run locally against a topic containing
backticks, $(…), & and quotes:

  • topic survived byte-identical into the manifest
  • appeared nowhere in the filename
  • nothing executed
  • package validated PASS / buildAuthorized: false, state COLLECTION_FAILED (correct — the paid step was skipped)

753 passed, 0 failed. Repository preflight PASS. No gate override.

Setup the owner must do, and the gap that remains

Settings → Environments → research-collection: required reviewers, the
FIRECRAWL_API_KEY secret, and RESEARCH_KIT_COLLECTION_ENV=research-collection as a
variable. Deliberately not automated — adding a credential location is yours.

Still unproven, and now the only gap: the runner-side
npm install -g firecrawl@<version> path has never executed. Closing it needs that
environment and one real dispatch.

🤖 Generated with Claude Code

…t of the shell

The first thing in this repository that spends a person's money on their behalf,
unattended, from a button. collect.yml implements ADR-0031 against ADR-0032's contract.

Separate from live-collection.yml on purpose. That one asks "does the adapter still
work"; this one asks "collect this, for a person, now". One file doing both would give
two audiences one approval list.

THE SPEND GATE IS THE ENVIRONMENT, AND THE ENVIRONMENT IS CHECKED FOR EXISTENCE

`environment: research-collection` is the only enforcement in the file that does not
depend on a step behaving - GitHub evaluates required reviewers before the first step
runs. It also has a hole the corpus names: a workflow referencing an environment that
does NOT exist does not fail, it CREATES one with no protection rules and no secrets
(E-06). A typo in the name would remove the approval gate and nothing would say so.

So the environment must prove it exists before anything is spent: an environment variable
an auto-created environment cannot have. The honest limit is written into the file rather
than implied - GitHub exposes no scope information to a job, so the marker proves the
environment was configured and CANNOT prove a repository-level value did not serve the
secret. Defeating it means deliberately setting a repository variable of that name.

A MISSING CREDENTIAL REFUSES; IT NEVER DEGRADES

No http-keyless fallback, and a test asserts the string does not appear. Falling back
would return a measurably worse corpus under the same artifact name, and the person who
asked for research would have no way to tell.

NO DISPATCH INPUT REACHES A SHELL - AND ONE ALREADY DID

`${{ inputs.x }}` inside a `run:` block is textual substitution performed BEFORE the shell
parses the script, so a topic containing a backtick or $(...) executes on the runner.
Inputs now arrive through `env:` and then through an argv ARRAY.

The test enforcing this scans EVERY workflow, and it immediately found that
live-collection.yml had been interpolating four inputs into shell bodies since it was
written - including max_pages, which is free text. Dispatching needs write access, which
bounds the severity and does not make it correct. Fixed in this commit.

A DEFECT THIS SURFACED OUTSIDE ITSELF

Feeding the collector a deliberately hostile topic found a bug in lib/scaffold.mjs that
has nothing to do with workflows. template/research/plan.json holds `"topic": "{{TOPIC}}"`
and the substitution is textual, so a topic containing a double quote wrote a plan.json
that does not parse - after which research.mjs reads an empty plan, collects nothing, and
reports a corpus problem about a file the operator never edited.

Any operator typing `Why "agentic" search costs more` hit this. Tokens are now JSON-escaped
for .json templates and left verbatim for markdown, because escaping everywhere would put
\" into prose. Two regression tests, both confirmed RED against the unfixed module and
green again after - the fix was reverted on purpose to watch them fail first.

THE WORKFLOW CANNOT ASSERT AUTHORIZATION

It packages with artifact.mjs create, which derives every authorization field from the
corpus and refuses --build-authorized outright. A freshly collected corpus is
HUMAN_REVIEW_REQUIRED with buildAuthorized false, and the ZIP says so at its top.

Names disclose nothing: neither run-name nor the artifact name carries the topic, because
E-08 measured run and artifact listings as readable by an anonymous caller on a public
repository. The workflow also says plainly what that does NOT buy - dispatch inputs are
visible on the run page, so this reduces incidental exposure rather than making a public
run private.

Verified without spending a credit: the scaffold, plan, map and packaging steps were run
locally against a topic containing backticks, $(...), an ampersand and quotes. The topic
survived byte-identical into the manifest, appeared nowhere in the filename, and nothing
executed. 25 workflow tests, offline.

Still unproven, and now the only gap: the runner-side `npm install -g firecrawl@<version>`
path has never executed. Closing it needs the environment, its secret, and one real
dispatch - all of which are the repository owner's to create.

753 passed, 0 failed. Repository preflight PASS. No gate override used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@StepenkoAnatoli
StepenkoAnatoli merged commit 96da664 into main Sep 21, 2026
3 checks passed
@StepenkoAnatoli
StepenkoAnatoli deleted the feat/actions-collector branch September 21, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant