feat: tool truncation + auto-compaction

README.md

@@ -6,7 +6,7 @@ The model gets a single built-in tool, `ipython`: a persistent IPython kernel fo
 
 For convenience, rlm ships built-in *skills* that can be enabled per run via `RLM_SKILLS` (comma-separated, off by default): `edit` (single-occurrence string replacement) and `search` (web search via Serper, needs `SERPER_API_KEY`). Enabled skills are pre-imported into the IPython kernel like any other skill (see [Skills](#skills)), so the agent calls `await edit(path=..., old_str=..., new_str=...)` or `await search(query=...)`.
 
-Context is reclaimed automatically: when a turn's prompt token count crosses `RLM_SUMMARIZE_AT_TOKENS`, the engine compacts the conversation into a summary and continues on a fresh branch. The IPython kernel keeps running across the compaction, so REPL state survives (see [Compaction](#compaction)).
+Context compaction is optional. When enabled, the engine compacts when 16k tokens remain below the model context window when the provider advertises it. `RLM_SUMMARIZE_AT_TOKENS` sets an explicit threshold. The IPython kernel keeps running across compaction, so REPL state survives (see [Compaction](#compaction)).
 
 Inside the IPython session, a callable `rlm` is pre-injected into the namespace. When recursion is allowed, the model can call `await rlm(...)` to spawn sub-agents. Skills supplied by the host environment (see [Skills](#skills)) are importable directly by name, e.g. `import websearch`.
 
@@ -114,9 +114,8 @@ versioned contract described above.
 | `RLM_MAX_CONCURRENT_SUBAGENTS` | `max(4, RLM_MAX_DEPTH)` | Maximum live recursive agents in a session tree. Capacity is reserved per depth to prevent nested-call deadlocks. |
 | `RLM_MAX_SUBAGENT_CALLS` | `64` | Maximum accepted recursive calls across the complete session tree. |
 | `RLM_EXEC_TIMEOUT` | `300` | Seconds per IPython execution |
-| `RLM_MAX_OUTPUT` | `-1` | Max chars returned from a tool call (`-1` disables truncation; `0` is invalid) |
-| `RLM_MAX_TOOL_OUTPUT_CHARS` | — | Preserve only a head/tail window of this many characters from raw IPython output before it enters the conversation. |
-| `RLM_SUMMARIZE_AT_TOKENS` | — | Auto-compaction threshold: when a turn's prompt tokens reach this value, the conversation is compacted into a summary. Unset disables auto-compaction. |
+| `RLM_COMPACTION` | — | Set to `1` to enable context compaction. |
+| `RLM_SUMMARIZE_AT_TOKENS` | — | Proactive compaction threshold for the current prompt, completion, and pending tool result. Setting it also enables compaction. When unset, enabled compaction uses 90% of an advertised context window. |
 | `RLM_MAX_TOKENS` | `0` | Optional completion-token budget (`0` disables) |
 | `RLM_APPEND_TO_SYSTEM_PROMPT` | — | Extra instructions appended to the generated system prompt |
 | `RLM_SYSTEM_PROMPT_PATH` | — | Path to a file whose contents fully replace the generated system prompt |
@@ -148,9 +147,11 @@ Recursive calls are created by a session-local supervisor rather than by the IPy
 
 ## Compaction
 
-There is no model-driven compaction tool. Compaction is automatic: set `RLM_SUMMARIZE_AT_TOKENS` and, once a turn's prompt token count reaches that threshold, the engine asks the model for a handoff summary and resumes the task on a fresh branch seeded with that summary. The original task prompt is dropped — the summary carries the goal forward.
+There is no model-driven compaction tool. Set `RLM_COMPACTION=1` to enable compaction. The engine reads the model context window from the provider's `/models` response and compacts when 16k tokens remain below it (small windows keep at least half). Set `RLM_SUMMARIZE_AT_TOKENS` to pin the threshold explicitly. Without a known window or explicit threshold, compaction stays off and an overflow propagates. A tool result larger than 20KB is truncated to its head and tail before it enters the conversation, with a warning naming the original size.
 
-The IPython kernel keeps running across the compaction, so all variables, imports, and in-memory data are preserved; the model is told to mention important variable names in its summary so the resumed branch knows what's available. With `RLM_SUMMARIZE_AT_TOKENS` unset, no auto-compaction occurs.
+The engine asks the model for a plain-text handoff summary and resumes the task on a fresh branch seeded with that summary; reasoning is never part of it. A provider overflow (a 400 or 413 naming a context limit) triggers the same compaction reactively from the current state. A rejected checkpoint request falls back to the last state that passed a threshold check - by definition a state with a full reserve of room - and an empty or tool-calling reply is resampled; after three failed attempts the run ends cleanly with what the conversation holds. An overflow with no history beyond the task propagates: the task alone approaches the window and there is nothing to reclaim.
+
+The IPython kernel keeps running across the compaction, so all variables, imports, and in-memory data are preserved. The model is told to mention important variable names in its summary so the resumed branch knows what is available. The same policy applies to the main agent and all recursive agents.
 
 ## Session Directory
 

src/rlm/acp.py

@@ -98,6 +98,7 @@ class _LimitsSnapshot(_ContractModel):
     max_concurrent_subagents: int = Field(gt=0)
     max_subagent_calls: int = Field(gt=0)
     max_tokens: int | None = Field(default=None, gt=0)
+    compaction: bool
     summarize_at_tokens: int | None = Field(default=None, gt=0)
     max_compactions: int | None = Field(default=None, gt=0)
     allow_git: bool

src/rlm/compaction.py

@@ -0,0 +1,161 @@
+"""Context checkpoint helpers."""
+
+from collections.abc import Mapping
+from typing import Any
+
+from openai import APIError, APIStatusError, AsyncOpenAI
+
+CHECKPOINT_PROMPT = """You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff summary another LLM can ACT on immediately to resume the task.
+
+It MUST contain, as fenced code blocks (not prose):
+- The exact shell/test command(s) to reproduce and verify — copy-pasteable, with the real path and test filter
+- Any edit still to apply, as the concrete `await edit(path=..., old_str=..., new_str=...)` call
+
+Then:
+- A NUMBERED list of remaining next steps
+- Current progress, key decisions, and constraints
+
+Be concise and concrete: prefer runnable commands over descriptions.
+
+Reply with the summary as plain text. Do not call any tools - summarize from the conversation as it stands."""
+
+REPL_NOTE = (
+    "\n\n"
+    "Note: the IPython kernel stays running across this compaction. "
+    "All variables, imports, and in-memory data are preserved. "
+    "Mention important variable names and what they contain so the "
+    "next LLM knows what's available."
+)
+
+SUMMARY_FRAMING = """Another language model started to solve this problem and produced \
+a summary of its thinking process. You also have access to the state of the tools that \
+were used by that language model. Use this to build on the work \
+that has already been done and avoid duplicating work. Here is \
+the summary produced by the other language model, use the \
+information in this summary to assist with your own analysis:"""
+
+RESERVE_TOKENS = 16_384
+"""Compact when this many tokens remain below the model context window."""
+
+COMPACTION_ATTEMPTS = 3
+"""Checkpoint attempts before compaction fails: a rejected request falls back to the
+last good snapshot; an empty or tool-calling reply is resampled."""
+
+TOOL_OUTPUT_MAX_BYTES = 20_000
+"""Middle-out truncation budget for one tool result before it enters the conversation."""
+
+_CONTEXT_FIELDS = (
+    "max_model_len",
+    "context_length",
+    "context_window",
+    "max_context_length",
+)
+_OVERFLOW_MARKERS = (
+    # OpenAI error code "context_length_exceeded"; OpenRouter relays the raw body.
+    "context_length_exceeded",
+    # OpenAI Responses/Completions: "Your input exceeds the context window of this model".
+    "exceeds the context window",
+    # OpenAI chat: "Input tokens exceed the configured limit of N tokens. Please reduce
+    # the length of the messages."; Groq words it the same way.
+    "reduce the length of the messages",
+    # vLLM: "This model's maximum context length is N tokens"; the renderers pre-flight:
+    # "Prompt length (N) exceeds maximum context length (M)"; Mistral uses the same words.
+    "maximum context length",
+    # Anthropic: "prompt is too long: N tokens > M maximum".
+    "prompt is too long",
+    # Anthropic byte-size overflow: HTTP 413 {"type": "request_too_large"}.
+    "request_too_large",
+    # HTTP proxies reject an oversized body with 413 "Request Entity Too Large".
+    "request entity too large",
+    # Google: "The input token count (N) exceeds the maximum number of tokens allowed (M)".
+    "exceeds the maximum number of tokens",
+    # xAI: "This model's maximum prompt length is N but the request contains M tokens".
+    "maximum prompt length is",
+)
+_window_cache: dict[tuple[str, str], int | None] = {}
+
+
+class CompactionFailed(Exception):
+    """Every checkpoint attempt failed - the caller ends the run cleanly instead."""
+
+
+def is_context_overflow(error: APIStatusError) -> bool:
+    details = f"{error} {error.body or ''}"
+    # An overflow is deterministic: a 400, or a 413 for a byte-size cap.
+    return error.status_code in (400, 413) and any(
+        marker in details.casefold() for marker in _OVERFLOW_MARKERS
+    )
+
+
+def default_threshold(context_window: int) -> int:
+    """Leave a fixed reserve below the window; small windows keep at least half."""
+    return max(context_window - RESERVE_TOKENS, context_window // 2)
+
+
+def _model_context_window(payload: Mapping[str, Any], model: str) -> int | None:
+    card = next(
+        (
+            item
+            for item in payload.get("data") or []
+            if isinstance(item, Mapping) and item.get("id") == model
+        ),
+        None,
+    )
+    if card is None:
+        return None
+    for field in _CONTEXT_FIELDS:
+        value = card.get(field)
+        if isinstance(value, int) and not isinstance(value, bool) and value > 0:
+            return value
+    return None
+
+
+async def discover_threshold(client: AsyncOpenAI, model: str) -> int | None:
+    key = (str(client.base_url), model)
+    if key not in _window_cache:
+        try:
+            # `models.list()` keeps provider extensions in each card's `model_extra`;
+            # a raw `cast_to` parse breaks on one Python version or another.
+            page = await client.models.list()
+            payload = {
+                "data": [
+                    {"id": card.id, **(card.model_extra or {})} for card in page.data
+                ]
+            }
+        except (APIError, AttributeError):
+            # A transient listing failure must not disable compaction for the
+            # rest of the process - leave the cache empty so the next engine retries.
+            return None
+        _window_cache[key] = _model_context_window(payload, model)
+    window = _window_cache[key]
+    return default_threshold(window) if window is not None else None
+
+
+def truncate_tool_output(text: str) -> str:
+    """Keep the head and tail of an oversized tool result and say what was cut."""
+    data = text.encode("utf-8")
+    if len(data) <= TOOL_OUTPUT_MAX_BYTES:
+        return text
+    keep = TOOL_OUTPUT_MAX_BYTES // 2
+    head = data[:keep].decode("utf-8", errors="ignore")
+    tail = data[-keep:].decode("utf-8", errors="ignore")
+    return (
+        f"Warning: truncated output (original token count: {estimated_tokens(text)})\n"
+        f"Total output lines: {text.count(chr(10)) + 1}\n\n"
+        f"{head}\n[... {len(data) - 2 * keep} bytes truncated ...]\n{tail}"
+    )
+
+
+def estimated_tokens(chars: str) -> int:
+    """Rough token count at four characters per token."""
+    return (len(chars) + 3) // 4
+
+
+def compactable(messages: list[dict]) -> bool:
+    """Whether compaction can reclaim anything - some history beyond the task exists."""
+    first_user = next(
+        (i for i, m in enumerate(messages) if m.get("role") == "user"), None
+    )
+    return any(
+        m.get("role") != "system" and i != first_user for i, m in enumerate(messages)
+    )

src/rlm/config.py

@@ -37,10 +37,8 @@ def _positive_int(value: str | int, name: str) -> int:
 
 
 def _summarize_at_tokens(value: str | int | None) -> int | None:
-    """Unset -> the 256k default; "" or "0" -> disabled; else a positive threshold."""
-    if value is None:
-        return 256_000
-    if value in ("", "0", 0):
+    """Unset, "" or "0" -> no explicit threshold; else a positive threshold."""
+    if value in (None, "", "0", 0):
         return None
     parsed = _optional_positive_int(value, "summarize_at_tokens")
     if parsed is None:
@@ -137,7 +135,8 @@ class ExecutionPolicy(_ConfigModel):
     max_depth: int = Field(default=1, ge=0)
     exec_timeout: int = Field(default=300, gt=0)
     max_tokens: int | None = Field(default=None, gt=0)
-    summarize_at_tokens: int | None = Field(default=256_000, gt=0)
+    compaction: bool = False
+    summarize_at_tokens: int | None = Field(default=None, gt=0)
     max_compactions: int | None = Field(default=None, gt=0)
     max_concurrent_subagents: int = Field(default=4, gt=0)
     max_subagent_calls: int = Field(default=64, gt=0)
@@ -171,6 +170,8 @@ def from_env(
     ) -> RuntimeConfig:
         env = os.environ if environ is None else environ
         raw_skills = env.get("RLM_SKILLS")
+        summarize_at_tokens = _summarize_at_tokens(env.get("RLM_SUMMARIZE_AT_TOKENS"))
+        compaction = env.get("RLM_COMPACTION") == "1" or summarize_at_tokens is not None
         max_depth = int(env.get("RLM_MAX_DEPTH", "1"))
         default_concurrency = max(4, max_depth)
         max_concurrent_subagents = _positive_int(
@@ -191,9 +192,8 @@ def from_env(
                 max_tokens=_optional_positive_int(
                     env.get("RLM_MAX_TOKENS"), "RLM_MAX_TOKENS"
                 ),
-                summarize_at_tokens=_summarize_at_tokens(
-                    env.get("RLM_SUMMARIZE_AT_TOKENS")
-                ),
+                compaction=compaction,
+                summarize_at_tokens=summarize_at_tokens,
                 max_compactions=_optional_positive_int(
                     env.get("RLM_MAX_COMPACTIONS"), "RLM_MAX_COMPACTIONS"
                 ),