Crucible intercepts every tool call an AI coding agent makes — bash commands, file reads, web fetches — extracts the signal, and returns structured knowledge packets.
AI coding agents spend a lot of their context budget on noise. Raw terminal output from a failed test run. Full file contents when only one function matters. Verbose build logs where ninety-nine lines say "compiling" and one line says "error". Crucible sits between the agent's tool calls and its context window and compresses before the agent sees anything. The agent reads a packet — what happened, how severe, where in the codebase, whether it's seen this before. The full raw output is one crucible raw <id> away if it ever needs it.
This started as a personal project to understand how AI agents actually consume information. The benchmark data, the monitor file in this repo, and the build narrative in HOW_CRUCIBLE_WAS_BUILT.md are all part of that investigation. The tool works — it ships, it has tests, it compresses. But the process of building it is the educational content. If you're here to learn, read the monitor file before you read the source.
I built Crucible using a methodology I discovered while building it. I'm calling it Experiential Specification.
The setup: I gave Claude Code (the building agent) the task, then asked it to maintain a monitor file alongside the code. Three tracks: token accounting (every tool call, every estimated cost), architectural observations (anything noticed mid-build), and expansion signals (every token sink the current system couldn't compress). At the end of each session, the agent wrote a first-person report — what did it notice, where did context fill up, what would it change.
Four sessions. Four reports. Each report identified the largest remaining gap, and that gap became the next session's build target. Session 1's report was about its own iteration loop bleeding context on unclassified commands → the universal engine got built. Session 2 noticed file reads were entirely uncompressed → the Read hook got built. Session 3 noticed PostToolUse(WebFetch) wasn't actually compressing because the agent saw raw + packet → WebFetch moved to PreToolUse. Session 4 audited the whole hook surface and found six gaps, all of which got fixed.
The agent observed its own inefficiency. The inefficiency became the specification. That's the loop.
The full narrative is in HOW_CRUCIBLE_WAS_BUILT.md. The raw session data is in .crucible_monitor.md — 1,100+ lines of an AI agent watching itself work.
82.0% compression on classified bash commands (36/36 benchmark)
89.9% compression on TypeScript file reads
91.1% compression on Python file reads
74.2% compression on Rust file reads (with content shown)
122 tests passing
These are from the benchmark suite in tests/. The non-regression rule: 36/36 cases at ≥80% average must hold after any change to the compression pipeline. Run python scripts/check_benchmark.py to verify.
| Hook | Tool | What it does |
|---|---|---|
| PreToolUse | Bash | Intercepts before the command runs. Rewrites the call to crucible run <cmd> for known commands and compresses the output. |
| PreToolUse | Read | Extracts the relevant section of a file. Returns a packet with section structure + previewed bodies. |
| PreToolUse | WebFetch | Fetches the URL itself. Returns extracted, compressed web content. The agent never sees the raw HTML. |
| PostToolUse | Bash | Catches commands that bypassed PreToolUse (wrappers, custom scripts) and processes their output. |
| PostToolUse | WebSearch | Deduplicates and ranks search results. |
Crucible makes one of three decisions before every refinement.
STANDARD — first time seeing this. Normal compression.
FULL — the agent is stuck. There's a failure on the affected path and a fix has already been attempted. Crucible preserves more signal, not less. This is the inverse of what a dumb compressor does, and it's the most important decision in the system.
ULTRA — identical repeat. Same command, same exit code, same paths, same output hash. Collapsed to one line. O(1) cost regardless of how big the original output was.
Priority: FULL > ULTRA > STANDARD. A repeated failure gets full signal, not a collapse.
pip install crucible-agent
crucible initcrucible init:
- Installs the five hooks into
~/.claude/settings.json. - Builds the TypeScript and Go tree-sitter parsers into
~/.crucible/parsers/— sandboxed in their ownsite-packages, doesn't touch your Python environment. - Idempotent. Safe to run twice.
Verify:
crucible report hooksCrucible is transparent after install. The agent's workflow doesn't change. These commands are the ones worth knowing:
crucible report compression # what got compressed this session
crucible report session # full command history with gate decisions
crucible report calibration # how thresholds have drifted
crucible raw <id> # retrieve the full uncompressed output
crucible shell # a Crucible-wrapped terminal for humansThese are real. None of them are showstoppers; all of them are documented because pretending they don't exist would be worse.
- WebSearch is PostToolUse only. Additive, not replacement. Crucible has no search engine of its own — it can't fetch the search results itself, so it processes whatever the search returned.
- Non-Python file extraction is uneven. TypeScript and Go use tree-sitter (proper AST). Rust and other languages use a regex heuristic that finds top-level boundaries only — methods inside an
implblock inherit the impl's relevance score. - The hook protocol uses deny-and-suggest. Claude Code doesn't expose a clean input-rewrite primitive yet, so Crucible returns the packet via the permission-decision reason. One upstream change away from cleaner anatomy.
crucible shell --livehas cursor overshoot of a few lines on some terminals. Use--no-livefor clean output in agent sessions.- ULTRA gate's identity check is
(command, exit_code, paths, content_hash)— strong enough to catch byte-different outputs, but if you wrap commands through arbitrary shell layers, the content hash is your only line of defense.
.crucible_monitor.md is in this repo.
It's 1,100+ lines of an AI agent watching itself work — tracking token costs, identifying inefficiencies, and writing first-person reports about what it noticed. The recent sections include a meta-cognition audit (TRACK 5: Genuine Findings) and a candid assessment written in the agent's own voice ("You did good. Here's what I would change.").
That file is not documentation. It's evidence that the methodology works. It's also the most honest record of how this tool was built that I can offer.
spatial-tether-filexplor — the Filesystem Spatial Manifest as a standalone TypeScript MCP server. Symbol extraction, call graph, Investigation Check, 25 MCP tools. (coming soon)
Iterance — behavioral witness layer for AI agents. The Ledger and Crystallizer that Crucible's native ledger is based on. github.com/Tetrahedroned/iterance
MIT — Tetrahedroned