feat: built-in bash skill (bash -c, shell parity with a native tool) - #145
Merged
Conversation
A ``bash`` builtin skill for RLM_SKILLS: run a shell command from the REPL and get combined stdout/stderr back as a string (exit code appended when nonzero, output clipped at 30k chars). Useful for mixing shell with Python in one cell (parse output programmatically) or avoiding shell-quoting via triple-quoted Python strings. Executes via ["bash", "-c", command] explicitly — not create_subprocess_shell, which resolves to /bin/sh (dash) and rejects bashisms like process substitution. Covered by tests including a bashism-parity check.
hallerite
previously approved these changes
Aug 25, 2026
…ault The skill now runs the same policy as shell tool paths: find_blocked_command refusal before execution (an agent could previously launder blocked git-history commands through `await bash(...)`), and the default timeout follows the harness exec timeout (RLM_EXEC_TIMEOUT, fallback 300) instead of a hardcoded 120. Sync subprocess.run in a thread, same output contract. Tests for both.
snimu
marked this pull request as ready for review
August 25, 2026 21:23
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 03f601b. Configure here.
The registration edit targeted the pre-rename dict shape and silently never applied — RLM_SKILLS=bash raised unknown-skill and the kernel never pre-imported it. Register it and cover the registration path with a stub-writing test (the direct-import tests missed this). Reported by cursor bugbot on #145.
Member
Author
|
Fixed in bab8cf6 — the registration edit had targeted the pre-rename dict shape and silently never applied. bash is now in _BUILTIN_SKILLS and the registration path is covered by a stub-writing test (test_enable_bash_skill_writes_stub). Good catch. |
snimu
approved these changes
Aug 26, 2026
samsja
added a commit
that referenced
this pull request
Aug 26, 2026
samsja
added a commit
that referenced
this pull request
Aug 26, 2026
snimu
pushed a commit
that referenced
this pull request
Aug 26, 2026
…mpaction handoff (#142) ## Summary The rlm harness, rebuilt on evidence from a k3-focused ablation campaign (9 models, up to 128 tasks/arm, SWE-rebench-V2): 1. **Terse coding-agent persona** replaces the verbose default system prompt (~3.2kB -> ~1.4kB) 2. **`RLM_TOOLING` preset — `dual` (default) | `tools` | `skills`** — one knob for how shell/edit are exposed 3. **Kernel-isolation prompt line** (stops project imports into the kernel venv) 4. **Actionable compaction handoff** (fenced command blocks + numbered next steps) 5. bash tool/skill share one guarded runner (`bash -c`, git-history guard, harness timeout); output-clipping removed ## The starting point: something was wrong kimi-k3 on stock rlm vs a plain bash+edit agent, same tasks: | setup | solve (easy 32) | reasoning tok/turn | solve (SWE-bench Pro 32) | |---|---|---|---| | stock rlm (ipython-only, stock prompt) | 0.53 | 327 | 0.75 | | bash+edit agent (verifiers harness) | 0.69 | 268 | 0.94 | Ablations isolated the cause: **the verbose prompt** (not the ipython interface, not `%%bash` syntax). Same tools, persona prompt: 209-232 tok/turn, solve 0.62-0.75. Frontier models are insensitive either way; kimi-k3 and opus-5 are prompt-sensitive (~2x reasoning inflation under the stock prompt). ## Final ablation: the three `RLM_TOOLING` presets (128 tasks, kimi-k3, same seeded set) | preset | n | solve | turns | reason/turn | error turns | |---|---|---|---|---|---| | `tools` | 128 | 0.711 | 36.1 | 216 | **0.8%** | | `skills` (with triple-quote guidance) | 125 | 0.704 | **33.5** | 256 | 2.5% | | `dual` | 128 | 0.695 | 35.4 | 239 | 2.1% | Converged on solve and total reasoning (~8k tok). `tools` is cleanest, `skills` fewest turns (REPL cells batch multiple actions), `dual` within noise of both. The skills preset needed two fixes, each isolated on the same 32 tasks: | skills variant | turns | error turns | |---|---|---| | buggy skill (ran /bin/sh) + no guidance | 53.5 | 4.2% | | + shell-parity fix (`bash -c`) | 35.9 | 5.0% | | + triple-quote prompt line | **33.5** | **2.5%** | The bugfix bought the turns; the prompt line bought the errors. The `skills` preset auto-renders the validated line: *"Run shell with `out = await bash(\'\'\'command here\'\'\')` — always triple-quote the command so shell quotes and multi-line scripts never need escaping..."* Cross-model check (GLM-5.3 @32): `tools` 0.61/1.3% err, `skills` 0.63 but 2.4x the reasoning tokens and 3.7% err, `dual` 0.59/0.6% err — skills-only is kimi-validated, not universal; hence **`dual` as default** (never the outlier on any model; both alternatives one env var away). ## Kernel-isolation line kimi imported project modules into the rlm kernel (~25 attempts/32 rollouts, 1 in 5 hitting ModuleNotFoundError). The line "The ipython kernel is an isolated venv without the project's packages... everything that executes project code goes through bash" drops kernel project-imports 5 -> 0, ImportErrors -> 0, at *lower* reasoning (161 vs 182 tok/turn on the probe set) — informational phrasing, no prohibition cost. ## Compaction `CHECKPOINT_COMPACTION_PROMPT` now demands runnable handoffs (exact repro/test commands + pending edits as fenced code blocks, numbered next steps): validated 0 -> ~4 command blocks per handoff, compacted-rollout solve 0.75-0.88 vs 0.53 under the stock prompt. Also checked: advance compaction notice in the system prompt is unnecessary (models stash state naturally; the at-compaction REPL note suffices; an explicit stash line doubles state-keeping but buys no solve). ## Notes - Merged with main (absorbs #145 bash skill and #146 knob removal; keeps #134 supervisor semantics — presets respected via `_selected()` in the tool registry, `RLM_SKILLS` default now preset-aware in `RuntimeConfig.from_env`). - Tests: 125 passed; the 6 `test_acp.py` failures are pre-existing on clean main in this environment. Ruff clean. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes default tooling, prompts, and compaction handoffs that directly steer agent behavior; bash/edit run on the host cwd with file writes, though git guards and timeouts are centralized. > > **Overview** > **Default agent surface** moves from IPython-only to native **`bash`**, **`edit`**, and **`ipython`** tools, with **`RLM_TOOLING`** (`dual` | `tools` | `skills`) and **`RLM_BUILTIN_TOOLS`** to override. When **`RLM_SKILLS`** is unset, skills now follow the preset via **`preset_skills()`** instead of an empty list. > > **`rlm.tools.bash`** adds shared **`run_bash`** (guarded `bash -c`, cwd, `allow_git`) used by **`BashTool`**, **`EditTool`**, and the **`bash`** skill; the IPython kernel gets **`RLM_EXEC_TIMEOUT`** / **`RLM_ALLOW_GIT`** at startup from engine policy. > > **System prompt** is shorter and tool-aware: coding-agent role lines per active tool, kernel-isolation guidance (**`PROJECT_ENV_PROMPT`**), conditional bash-skill triple-quote hints, and trimmed IPython/edit copy. **Compaction** **`CHECKPOINT_COMPACTION_PROMPT`** now requires fenced repro commands, pending **`edit`** calls, and numbered next steps. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 41e2698. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds a built-in
bashskill (enabled viaRLM_SKILLS):await bash(command=...)runs a shell command from the REPL and returns combined stdout/stderr as a string (exit code appended when nonzero, output clipped at 30k chars).Why: lets the agent mix shell with Python in one cell — capture output as a value, parse it programmatically, or sidestep shell-quoting by building commands as triple-quoted Python strings.
Implementation note: executes via
["bash", "-c", command]explicitly rather thancreate_subprocess_shell, which resolves to/bin/sh(dash on debian images) and rejects bashisms — we hitsh: Syntax error: "(" unexpectedon process substitution in ablation runs before pinning this down. A parity test (cat <(echo ...)) guards it.Tests: 4 new (output capture, bashism parity, exit-code reporting, stderr merging); suite green except 6 pre-existing
test_acp.pyfailures that also fail on clean main in this environment.Extracted from #142 so it can ship independently.
Note
Medium Risk
Introduces arbitrary shell execution from the REPL; mitigated by shared git-history blocking and timeouts, but still expands the agent’s command surface.
Overview
Adds a
bashbuilt-in skill (enable withRLM_SKILLS) so agents canawait bash(...)from IPython and get shell output as a string instead of only using separate shell tools.The new
rlm.skills.bashmodule runs one command per call viabash -c(not/bin/sh), merges stdout and stderr, appends[exit code N]on failure, honorsRLM_EXEC_TIMEOUT(default 300s), and applies the samefind_blocked_command/ git-history guard as other shell paths. Registration in_BUILTIN_SKILLSwires it into the existing session stub / pre-import flow likeedit.Tests cover echo output, bash-specific syntax (process substitution), nonzero exits, stderr, git guard behavior, timeouts, and that
enable_builtin_skills(["bash"], ...)writes the expected re-export stub.Reviewed by Cursor Bugbot for commit bab8cf6. Bugbot is set up for automated code reviews on this repo. Configure here.