v3.9 — Compaction, drawer, and a more reliable router for file operations - #9
Merged
Conversation
COMPACTION_ENABLED, COMPACTION_THRESHOLD, COMPACTION_KEEP_RECENT, COMPACTION_STRATEGY (rag_pointer default, llm_summary optional). MEMORY_MAX_HISTORY stays as-is, now documented as the hard-cap safety net behind compaction rather than the only retention mechanism.
New module implementing the two-strategy compaction design agreed on:
maybe_compact(history, force=False) decides WHEN to compact
(COMPACTION_THRESHOLD), a strategy function decides HOW. Two
strategies, same signature, swappable via config:
- rag_pointer (default): push the compacted block into vector
memory verbatim, replace it inline with a short pointer. No LLM
call.
- llm_summary: one LLM call per compaction, condenses the block
into prose kept inline.
Pinned messages are always excluded from what gets compacted.
9 tests covering threshold behaviour, pinned exemption, forced
compaction, and failure handling for both strategies.
…nto memory
Every history entry now carries a stable 'id' (a per-file monotonic
counter) and a 'pinned' flag. pin_message/unpin_message/get_pinned
implement the drawer ('tiroir'): pinned messages are exempt from both
compaction and the v3.8 hard-cap fallback. compact_now() forces a
compaction pass on demand (manual trigger, per the agreed design).
_apply_retention() now tries compaction.maybe_compact() first and
falls back to the v3.8 drop-oldest hard cap (pinned exempt) only if
compaction is disabled or its strategy raises -- history must never
grow unbounded either way.
Updated two existing tests whose assertions depended on the exact
pre-v3.9 dict shape (no id/pinned fields); behaviour unchanged.
GET /history -- full rolling history with ids, what the single-thread
web UI renders on load and after each turn.
GET /drawer -- currently pinned messages.
POST /drawer/pin, POST /drawer/unpin -- by message_id.
POST /compact -- force a compaction pass now, regardless of
COMPACTION_THRESHOLD.
Same auth/rate-limit dependencies as every other endpoint. 5 tests
added covering history/drawer round-trip, unknown-id 404, token
gating, and the manual compact endpoint.
Mirrors the manual /compact API endpoint for REPL usage: forces a compaction pass now, pinned messages excluded. Documented alongside the existing !clear/!trace/!remember/!recall commands.
…ompaction Chat panel now loads and re-renders from GET /history instead of staying purely client-side state, so the thread survives a page reload and every bubble carries its real message id. Each non-system message gets a 📌 pin/unpin button (POST /drawer/pin|unpin). New 'Tiroir' tab lists pinned messages via GET /drawer with a 'Retirer' action, plus a manual '🗜 Compacter maintenant' button (POST /compact).
GET /history's response model requires 'id' and 'pinned' on every
entry. Any memory.json written before v3.9 has plain {role, content}
history entries -- loading one unmigrated made every /history call
fail response validation (500), silently swallowed by the web UI's
loadHistory(), so a chat reply never appeared even though the
orchestrator answered correctly. load_memory() now backfills both
fields on read and persists the migration once.
… message The always-visible '📌 Épingler'/'📌 Épinglé' button under every bubble was too much visual noise on a full thread. Each message is now wrapped in a .msg-row (bubble + a small round pin icon), the icon staying at opacity 0 until the row is hovered. Pinned messages keep a faint permanent marker (amber, ~80% opacity) so pinned state is still readable at a glance without a label. Alignment (left/right) moved from the bubble itself to the row, since the icon needs to sit outside the bubble; the thinking indicator got the same row wrapper to keep its alignment.
The 📌 emoji renders in full color regardless of the .pin-icon opacity rules and clashes with the dark theme. Swapped for a minimal thumbtack outline (stroke=currentColor), so it correctly inherits muted/hover/ pinned coloring like any other icon in the UI.
…ault Placement: dropped the .msg-row wrapper and put the pin icon back inside the bubble itself, absolutely positioned as a small floating badge on the bubble's outer corner (opposite the tail, so it never sits over text) with a soft shadow and scale-in on hover. Pinned state gets a subtle amber glow instead of relying on color alone. Behaviour: pinning a message now pins its exchange partner too (add_exchange always assigns the assistant id right after the user id, so the partner is deducible without extra API calls) -- a question or answer read back alone tends to lose its point. Unpinning stays per-message, so either half can be removed afterward without touching the other.
Root cause of the truncated Containerfile: _remember() has hard-cut both sides of every exchange to _MAX_MEMORY_CONTENT (300 chars) since before v3.9. That was harmless when memory.json only fed the router's own next-turn prompt (a shorter prompt, not a wrong one), but the v3.9 web UI renders GET /history directly -- so a full tool result (e.g. reading Containerfile via the files tool, 1035 bytes) got silently cut to 300 chars before it ever reached the screen. Removed the truncation and the now-unused constant. Prompt-size management is v3.9's job now: MEMORY_MAX_HISTORY caps message count, compaction.py replaces old messages with a real summary once COMPACTION_THRESHOLD is crossed -- both far better suited to this than a blind per-message character cut that mangled tool output. Confirmed via SHOW_DEBUG logs: router.raw_output/router.decision showed the files tool read the full 1035-byte Containerfile correctly; the cut only happened once it round-tripped through memory.json.
…of trusting LLM_MODEL LLM_MODEL is never sent in the /completion payload for the llama_cpp provider (confirmed while debugging the truncation bug) -- unlike ollama/openrouter, where it's authoritative. It was a purely cosmetic label the person had to remember to update by hand every time they swapped GGUF files, and it silently drifted. Added get_loaded_model(url) in providers/llama_cpp.py: queries llama-server's own /props endpoint (checking a few known field names across server versions), best-effort with a short 2s timeout -- returns None on any failure so /health degrades to the configured LLM_MODEL rather than breaking. /health now uses this for the llama_cpp provider only; ollama/openrouter are untouched since LLM_MODEL is already correct for them. 9 tests added across test_providers.py (the probe itself) and test_api.py (the /health fallback behavior); 3 existing /health tests pinned to a non-llama_cpp provider so they don't depend on real network refusal timing.
…he LLM_MODEL fix - API Endpoints table: added /history, /drawer, /drawer/pin, /drawer/unpin, /compact (v3.9); corrected /health's description. - Configuration table: fixed MEMORY_MAX_HISTORY's default (was documented as 20, actually 100 since v3.8); documented LLAMA_CPP_ID_SLOT/ CACHE_PROMPT (v3.8) and all four COMPACTION_* vars (v3.9), which were entirely missing; corrected LLM_MODEL's description to explain it's informational-only for llama_cpp. - Conversation Memory section: removed the now-false '300 chars truncated' claim (the bug just fixed), added a Context compaction & drawer subsection explaining the v3.9 mechanism and its rationale versus the plain MEMORY_MAX_HISTORY hard cap.
The summary message had no 'id' when created in compaction.py. It only got one lazily on the NEXT load, via memory._migrate_history() -- which was meant for old pre-v3.9 entries, not fresh compaction summaries -- assigning whatever next_id had advanced to by then. That put the summary's id well ahead of the still-uncompacted messages right after it chronologically, which the user spotted directly in their memory.json (a summary of 59 earlier messages sitting at id 83, *after* messages 63-82 in id order despite summarizing everything before them). Fixed by assigning the summary the id of the oldest message it replaces (messages[0]['id']) at creation time, for both strategies -- that id is free for reuse since the original message no longer exists standalone. Regression test added.
…, remaining emoji - Scroll: appendMsg took an optional scroll param (default true); loadHistory() now scrolls once after the whole batch instead of once per appended message, fixing the rapid-fire scroll-flash on every full re-render (page load, after each turn, after pin/unpin). - Model name: dropped the hardcoded .slice(0, 30) that cut off real GGUF filenames (now longer since /health reports the live loaded model). #status-text gets CSS text-overflow:ellipsis with a max-width instead, plus a title tooltip showing the full name on hover. - Replaced the four remaining emoji (logo gear, token key, compact, refresh) with the same style of inline monochrome SVG already used for the pin icon, for visual consistency across the whole UI.
formatContent() rewritten from a two-regex code-fence-only formatter into a small dependency-free markdown subset: headers (h3-h5), bold, italic, links, ordered/unordered lists, inline code, and fenced code blocks -- deliberately hand-rolled instead of pulling marked.js/etc. from a CDN, since Forge is meant to keep working with no internet on the NiPoGi. Fenced "diff" blocks get special handling: +/- lines colored like real git-diff output, so an "improve this file" answer can show what changed instead of the whole file every time. Fixed a real pre-existing gap along the way: only code-fence content was ever HTML-escaped before, so raw '<'/'&' in a model's own answer outside a fence would have been injected as live markup. Everything is escaped first now, before markdown rules are applied on top. Also split .msg pre / .msg code CSS, which previously applied the same block-level padding to inline code spans as to full pre blocks. Verified with `node --check` (real syntax validation, not brace counting) and a set of representative inputs run through Node directly (multiline text, bold/italic/links, lists, headers, a code fence, a diff fence, and an XSS-attempt string confirming it comes out escaped).
Root cause behind "aide-moi a creer un fichier hello world en go" never actually creating anything: TOOL_DESCRIPTIONS already documented the write action, but _TOOL_EXAMPLES["files"] only ever showed a read example. Every other tool in this prompt (memory especially, per its own comments) already leans on the principle that a small local model follows a worked example far more reliably than a stated rule -- files was the one tool where that principle wasn't actually applied to its second action. The model always saw "read", never "write", and defaulted to answering with a code block as plain chat text instead of persisting anything -- confirmed in real usage: a later request to edit "the file you created" failed with file-not-found, because nothing had ever been written. Added a second worked example demonstrating write, mirroring the existing read one. 2 tests added confirming the example is present and its JSON is well-formed on both the outer (files) and inner (write instruction) level.
…ing file The other half of "improve this file just repeats the whole thing": even once the router correctly calls files:write (previous commit), the tool itself only ever confirmed "[ok] written N bytes" -- no diff, whether shown to the user or not. Asking the model to hand-format its own diff syntax is failure-prone for a small local model; computing one deterministically in Python (difflib.unified_diff) is not. _action_write() now reads the old content before overwriting (skipped if the existing file exceeds _MAX_READ_BYTES, same guard already used by read, so the write itself always still succeeds) and returns a ```diff-fenced unified diff instead of the bare confirmation -- which the web UI's diff renderer (already shipped) picks up and colors like real git-diff output. A brand-new path has nothing to diff against and keeps the plain confirmation; identical content reports "inchangé" rather than an empty diff block. 5 tests added: diff on overwrite, unchanged-content reporting, new-file has no diff, and the oversized-existing-file guard (write still succeeds, diff skipped).
Confirmed live: "remplace X par Y dans hello.go" (and several repeats
of it) never touched the real file -- the router had a write example
(previous commit) but no example at all for EDITING an existing
named file, so it kept answering with the ORIGINAL unmodified content
from memory/guesswork instead of the real one on disk.
Mirrors the existing memory tool's recall -> done:false -> rephrase
pattern, applied to files:
- New third files example: an edit-intent message ("remplace X par
Y dans hello.go") maps to a read with "done": false, not a write
or a chat answer.
- New step_context steering hint (elif alongside the existing
memory one): right after a [files] read result, push explicitly
toward "action":"write" with the real content + the requested
change, and away from re-reading or just answering in chat.
Caught while wiring the hint: step_context truncates every entry to
_MAX_HISTORY_ENTRY (120 chars) -- sized for compact history summaries,
not for a file the model is about to be asked to reproduce in full
with one change applied. A files:read result now gets a separate,
much higher cap (_MAX_STEP_CONTEXT_FILE_ENTRY, 4000 chars) instead,
bounded well below the files tool's own 64KB read cap to stay inside
an 8k-token local model's prompt budget.
6 new router-prompt tests (example present, hint fires only after a
read and not after a write, the 120-char cap doesn't clip a files:read
entry) plus one full orchestrator-level integration test proving the
2-step flow actually updates the file on disk and returns a real diff,
not just a plausible-looking chat answer.
…newline
Confirmed live: -print('Hello World')+print('Bienvenue') rendered as
one line instead of two. unified_diff() preserves each source line's
own ending, so a file with no trailing newline (single-line scripts,
the real case this was caught on) produces a last diff line with no
newline either -- joining it straight onto the next line with zero
separator. Display-only diff, not meant to be fed to patch, so
normalizing every line to end in \n before joining is always safe.
Regression test added with the exact real-world case (a one-line file,
no trailing newline).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's new
Context management
COMPACTION_STRATEGY):rag_pointer(pointer into vector memory, no LLM call, default) orllm_summary(condensed inline summary)POST /compact, a button in the UI, or!compactin the REPLThe drawer
GET/POST /drawer): exempt from both compaction and the hard capWeb UI
GET /history(survives a page refresh)```diffblocks, rendered like realgit diffoutputRouter reliability
filestool had a worked example for reading but none for writing — a small local model never actually followed the write action, even when explicitly askedfileseither — new read (done:false) → write pattern, mirroring the recall→answer pattern already in place for thememorytoolfiles:writenow returns a real diff (computed in Python withdifflib, never asked of the model) instead of re-posting the whole file on every editBugs found and fixed through real usage
All caught by actually using the feature, not by re-reading the code:
_MAX_MEMORY_CONTENT) was silently corrupting any content shown in the UI (e.g. aContainerfilecut off mid-word)/healthshowed a stale model name — now queriesllama-serverdirectly via/propsTests
254 tests,
rufflint/format clean.Deliberately left out