Add nuhuh plugin under Code Quality Testing - #327
Conversation
dudley02
left a comment
There was a problem hiding this comment.
The Stop hook executes npx --yes nuhuh@latest gate, which downloads mutable code on every run. Please pin an audited version (preferably with a lockfile) or vendor the executable so installations are reproducible.
npx --yes nuhuh@latest pulled mutable code on every run. Pin the exact version so installs are reproducible, and bump the plugin manifests to match what is published.
|
Pinned in 0285e7d. The Stop hook now runs I did not vendor the executable. nuhuh ships as a normal npm package, and checking a built bundle into this repo would make the plugin directory a lot harder to review than a one-line version string. If you would rather have it vendored, or want the hook to install through a lockfile instead of |
Upstream now pins the hook to the installing version and keeps the manifests in sync from a version lifecycle script.
|
Thanks for catching this, it was a fair hit. The same unpinned command was in the upstream repo too, so it is fixed at the source as of 0.1.10 rather than just patched here.
I verified it against the tarball published to npm, not only in tests, so a fresh install writes the pinned command. This branch now pins to 0.1.10. Still happy to vendor the executable instead if that is the standard you would prefer for this directory, just say the word. |
Adds nuhuh, a Stop hook plugin that verifies the agent's completion claims by re-running reality. When the agent declares Done, nuhuh extracts every claim from the final message (tests pass, file created, endpoint works) and verifies each one fresh, whole suite in a clean process, real exit codes, files on disk, localhost probes. A false Done is bounced back to the agent with the failing evidence, and after 3 bounces the receipt goes to the human.
It is the deterministic sibling of double-check already on this list. No LLM calls in the verification path, local only, MIT licensed. Source and a False Done Rate benchmark at https://github.com/sjh9714/nuhuh and the plugin folder mirrors the upstream .claude-plugin packaging.