Repository navigation
Make the live workflow's green mean something, and stop python hiding behind python - #17
Merged
Merged
Conversation
… behind python All eight findings, and all eight were mine - every one introduced by the commit that closed the previous audit. A workflow written to verify things, that then discarded the exit status of every check it ran, is the sharpest version of the lesson this repository keeps relearning: a signal that looks like verification and is not is worse than no signal. F-16 AN OLD `python` HID A USABLE `python3` (the only one in shipped code) checkPython tried `python` then `python3` and returned FAILURE at the first old interpreter. On the most common layout there is - `python` an alias for 2.7, `python3` the real one - it reported "no usable Python" with a perfectly good 3.12 one name away. Two changes: `python3` is tried first, and the search CONTINUES past an old interpreter, remembering it only so the message can name what was found rather than the less useful "no python on PATH". The test almost failed to prove this. Reordering alone satisfies the common case, so reintroducing the early return left it green. The case that actually exercises the branch - the FIRST name tried is old, a good one behind it - is asserted separately now, and goes red on its own. F-13, F-17 THE WORKFLOW SWALLOWED THE CHECKS THAT DECIDE ITS RESULT handoff and preflight both ran with `|| true`. A green run proved the repository was unchanged and nothing else - not that the corpus arrived whole, not that it passed the checks a synthetic corpus passes, not that anything was collected at all. One success predicate now, stated at the top of the file and enforced by the steps: supported CLI major, collection exited 0, a ledger exists and parses and records a successful fetch, handoff 0, preflight 0, repository unchanged. Diagnostics still print on failure - through `if ! cmd; then`, which keeps the output AND the exit status. `doctor` stays non-required, because it legitimately reports a missing key on the keyless route. That is now SAID beside the step rather than hidden in a suffix. F-14 PAID RUNS ARE SERIALISED max_pages bounds one run, not the meter: two dispatches a second apart spend concurrently while a reader reasonably believes 10 is the ceiling. `cancel-in-progress: false` is deliberate - cancelling a paid collection wastes credits already spent and can leave a scratch ledger torn. F-15 INSTALLATION FAILURE AND INCOMPATIBILITY ARE DIFFERENT FACTS `npm i -g a || npm i -g b || true` suppressed both, so "the package moved" and "this is a major we do not support" looked identical. A `cli_version` input chooses the policy explicitly: `tested` pins the version the adapter was written against and is the reproducible integration test; `latest` is surveillance and says so, where a refusal is the workflow working. F-18 THE PUBLIC SUMMARY CARRIED URL PREFIXES 60 characters of each URL - exactly enough to disclose the subject being researched and the query terms, not enough to be useful. Hostnames only. F-19 A CONFIGURED RESULT FILE FAILED SILENTLY CI reads the ABSENCE of the result file as "crashed before reporting". An unwritable path produced the same absence, so a bad permission or a typo was reported as a crash that did not happen. It says so on stderr now, while still never failing the run it reports on. F-20 "NO DEPENDENCIES" IS TRUE OF THE REPOSITORY, NOT OF COLLECTION The kit pulls nothing from npm. The metered route spawns a globally installed CLI versioned outside this repository, and the live workflow installs it at dispatch. The README draws that line now. VERIFIED node research-kit/bin/selftest.mjs -> 644 passed, 0 failed, exit 0 (642 + 2) node research-kit/bin/preflight.mjs -> PASS, 0 blocking, 0 warnings Both new guards were reproduced. The workflow guard pins the PROPERTY - no swallow on a command that decides the result - rather than the wording. The kit's own gate blocked the first attempt at this commit: runtime.mjs and selftest.mjs are declared code paths and the map was not staged with them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All eight findings. All eight were mine — every one introduced by the commit that closed the previous audit.
A workflow written to verify things, which then discarded the exit status of every check it ran, is the sharpest version of the lesson this repo keeps relearning: a signal that looks like verification and is not is worse than no signal.
F-16 — an old
pythonhid a usablepython3(the only one in shipped code)checkPythontriedpythonthenpython3and returned failure at the first old interpreter. On the most common layout there is —pythonan alias for 2.7,python3the real one — it reported "no usable Python" with a perfectly good 3.12 one name away.The test almost failed to prove this. Reordering to try
python3first satisfies the common case, so reintroducing the early return left it green. The case that actually exercises the branch — the first name tried is old, a good one behind it — is asserted separately now, and goes red on its own.F-13 / F-17 — the workflow swallowed the checks that decide its result
handoffandpreflightboth ran with|| true. A green run proved the repository was unchanged and nothing else.One success predicate now, stated at the top of the file and enforced by the steps:
Diagnostics still print on failure — via
if ! cmd; then, which keeps the output and the exit status.doctorstays non-required because it legitimately reports a missing key on the keyless route; that is now said, beside the step, rather than hidden in a suffix.F-14, F-15, F-18, F-19, F-20
max_pagesbounds one run, not the meter — two dispatches a second apart spend concurrently while a reader believes 10 is the ceiling.cancel-in-progress: falseis deliberate: cancelling a paid collection wastes credits already spent.Verified
Both new guards were reproduced. The workflow guard pins the property — no swallow on a command that decides the result — rather than the wording.
The kit's own gate blocked my first attempt at the commit:
runtime.mjsandselftest.mjsare declared code paths and the map was not staged with them.🤖 Generated with Claude Code