-
Notifications
You must be signed in to change notification settings - Fork 680
feat(mcp): support project reassignment in mem_update #679
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
18dd410
b7bc3a7
408929d
85ff62b
465e81a
d0ad770
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| # Maintainer Handoff | ||
|
|
||
| This document transfers the current local-fork context to another agent or | ||
| machine. It describes the state of this fork, not the upstream project's | ||
| release state. | ||
|
|
||
| ## Authoritative checkout | ||
|
|
||
| Work from this Git repository only. The Codex/Claude plugin caches and Go | ||
| module cache are generated installation copies; do not edit them as a source | ||
| of truth. | ||
|
|
||
| - Fork remote: `https://github.com/Faturrachman-dev/engram.git` | ||
| - Upstream remote: `https://github.com/Gentleman-Programming/engram.git` | ||
| - Active branch: `feat/mem-update-project-reassign` | ||
| - Latest local-and-pushed commit: `465e81a feat(provenance): record author across agent integrations` | ||
|
|
||
| The previous `engram-src` duplicate worktree was intentionally removed. Do not | ||
| recreate or use it for development. | ||
|
|
||
| ## What this branch adds | ||
|
|
||
| The branch contains the following local work on top of upstream: | ||
|
|
||
| 1. `mem_update` supports a validated `project` argument, allowing one | ||
| observation to be reassigned without raw SQLite edits. | ||
| 2. `/stats` includes `total_created` and `max_observation_id`. | ||
| 3. The HTTP server serves a dashboard at `/dashboard`, redirects `/` there, | ||
| enables browser CORS, and exposes `GET /projects/stats` for project counts. | ||
| 4. Observations support author provenance end-to-end: | ||
| - `mem_save.author` is accepted by MCP. | ||
| - When omitted, `ENGRAM_AUTHOR` is used. | ||
| - SQLite, imports, API responses, and Obsidian export retain `author`. | ||
| - The dashboard shows author information. | ||
| 5. Pi integration sets authors to `pi/<model-id>` when the active model is | ||
| available, otherwise it uses `ENGRAM_AUTHOR`. | ||
| 6. `ENGRAM_AGENT_CLI=pi` is supported for semantic conflict scanning. The Pi | ||
| runner shells out to the local `pi` CLI and uses that machine's configured | ||
| provider/model. | ||
|
|
||
| ## Architecture landmarks | ||
|
|
||
| - `cmd/engram/main.go` — CLI command wiring and environment help. | ||
| - `internal/mcp/mcp.go` — MCP schemas and tool handlers. | ||
| - `internal/store/store.go` — SQLite schema, migrations, and observation | ||
| persistence. | ||
| - `internal/llm/` — semantic conflict-scan runners; `pi.go` is the local Pi | ||
| runner. | ||
| - `internal/server/server.go` — HTTP routes, dashboard delivery, and CORS. | ||
| - `internal/server/dashboard/index.html` — embedded dashboard UI. | ||
| - `plugin/pi/index.ts` — Pi integration and memory tool requests. | ||
| - `plugin/codex/` and `plugin/claude-code/` — thin host-agent hooks. Keep | ||
| behavior and persistence policy in the Go server where possible. | ||
|
|
||
| ## Build and verification | ||
|
|
||
| Run focused verification from the repository root: | ||
|
|
||
| ```powershell | ||
| go test ./internal/llm ./internal/mcp ./internal/store ./internal/server | ||
| ``` | ||
|
|
||
| Rebuild the local executable after changing Go code, then restart MCP clients | ||
| or their sessions. The installed binary is deliberately separate from the | ||
| source checkout; never modify its generated plugin-cache files as a substitute | ||
| for rebuilding from this repository. | ||
|
|
||
| ## Working conventions | ||
|
|
||
| - Use Conventional Commits and keep the branch name in `type/description` | ||
| format. The repository ruleset rejects invalid messages. | ||
| - Do not commit generated binaries, databases, credentials, or agent caches. | ||
| - Preserve author provenance as an agent/model label, never as a credential or | ||
| captured request payload. | ||
| - Before changing a plugin hook, read `skills/plugin-thin/SKILL.md`; before | ||
| changing persistence or project resolution, read | ||
| `skills/business-rules/SKILL.md`. | ||
|
|
||
| ## Follow-up work worth checking | ||
|
|
||
| - `DOCS.md` still describes `ENGRAM_AGENT_CLI` as accepting only `claude` and | ||
| `opencode`; update it when preparing this branch for broader review. | ||
| - Add focused MCP/store tests for author migration and persistence if this | ||
| branch will be proposed upstream. | ||
| - Rebuild and smoke-test the installed executable on each machine after pulling | ||
| this branch; the executable itself is intentionally not committed. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| package llm | ||
|
|
||
| import ( | ||
| "bufio" | ||
| "bytes" | ||
| "context" | ||
| "encoding/json" | ||
| "fmt" | ||
| "strings" | ||
| ) | ||
|
|
||
| // ─── PiRunner ───────────────────────────────────────────────────────────────── | ||
|
|
||
| // PiRunner implements AgentRunner by shelling out to the `pi` CLI. | ||
| // It invokes: pi -p --mode json --no-context-files (with the prompt on stdin) | ||
| // and parses Pi's NDJSON event stream, accumulating assistant text deltas into | ||
| // the final message which is then parsed as a Verdict JSON object. | ||
| // | ||
| // Pi routes through the user's own provider configuration (e.g. a cheap | ||
| // 9router model), which makes it the low-cost background consolidation runner: | ||
| // set ENGRAM_AGENT_CLI=pi to drive `conflicts scan --semantic` with it. | ||
| type PiRunner struct { | ||
| // runCLI is the shell-out function. Defaults to defaultRunCLI. | ||
| // Tests inject a fake implementation to avoid spawning real processes. | ||
| runCLI func(ctx context.Context, name string, args []string, stdin string) ([]byte, error) | ||
| } | ||
|
|
||
| // NewPiRunner constructs a PiRunner with the real exec.CommandContext | ||
| // implementation. Tests should inject a fake via the struct field directly. | ||
| func NewPiRunner() *PiRunner { | ||
| return &PiRunner{runCLI: defaultRunCLI} | ||
| } | ||
|
|
||
| // Compare sends prompt to the Pi CLI and returns a structured Verdict. | ||
| // Invokes: pi -p --mode json --no-context-files | ||
| // | ||
| // Pi's output is NDJSON (newline-delimited JSON). Assistant text arrives as a | ||
| // stream of "message_update" events carrying "text_delta" chunks; the runner | ||
| // concatenates those chunks and parses the assembled message as a Verdict. | ||
| func (r *PiRunner) Compare(ctx context.Context, prompt string) (Verdict, error) { | ||
| args := []string{"-p", "--mode", "json", "--no-context-files"} | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- internal/llm/pi.go ---'
cat -n internal/llm/pi.go
printf '%s\n' '--- internal/llm/pi_test.go ---'
cat -n internal/llm/pi_test.goRepository: Gentleman-Programming/engram Length of output: 11922 🌐 Web query:
💡 Result: In the Pi CLI, the --no-tools and --no-context-files flags are used to disable specific agent capabilities during startup [1][2]. --no-tools (-nt): Disables all built-in, extension, and custom tools [1][3]. --no-context-files (-nc): Disables the automatic discovery and loading of AGENTS.md and CLAUDE.md context files [1][2]. For JSON output, the Pi CLI provides specific execution modes via the --mode flag [4][5]: --mode json: Operates the agent in JSON mode, outputting all events as a stream of JSON lines [4][5]. This is typically used for integration with pipelines and log processors [5]. --mode rpc: Operates in RPC mode, allowing control of the agent over line-delimited JSON through standard input and standard output [4][5]. These flags and modes can be combined to customize the agent's behavior for automation or headless environments [5]. Citations:
LLM Security (CWE-1427) Disable tools for semantic scans. Add 🤖 Prompt for AI Agents🔒 Security & Privacy | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- internal/llm/pi.go ---'
cat -n internal/llm/pi.go
printf '%s\n' '--- internal/llm/pi_test.go relevant sections ---'
sed -n '1,180p' internal/llm/pi_test.go
printf '%s\n' '--- repository references to Pi configuration/version ---'
rg -n --hidden -S 'pi|PiRunner|no-session|no-context-files' --glob '!vendor' --glob '!node_modules' --glob '!dist' --glob '!build' .Repository: Gentleman-Programming/engram Length of output: 50385 🌐 Web query:
💡 Result: In the Pi CLI, the flags --no-session and --no-context-files control how the tool manages state and project-specific instructions [1][2]. --no-session This flag enables ephemeral mode, meaning Pi will not save the current session to disk [1][3]. It is typically used for disposable tasks or automated processes where you do not need to resume the conversation later [3]. --no-context-files (or -nc) This flag disables the automatic discovery and loading of context files, specifically AGENTS.md, AGENTS.override.md, and CLAUDE.md [1][3]. By default, Pi searches for these files in your project to apply project-specific conventions, commands, and safety rules [1]. Disabling this is useful when you need to isolate the agent from unexpected project instructions or troubleshoot baseline behavior [3]. These flags can be combined with other options to customize the agent's startup state precisely [1][2]. Citations:
🤖 get_repo_knowledge executed:
Length of output: 7595 🏁 Script executed: #!/bin/bash
set -e
cat -n internal/llm/pi.go
sed -n '1,180p' internal/llm/pi_test.go
rg -n --hidden -S 'pi|PiRunner|no-session|no-context-files' --glob '!vendor' --glob '!node_modules' --glob '!dist' --glob '!build' .Repository: Gentleman-Programming/engram Length of output: 50385 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- internal/llm/pi.go ---'
cat -n internal/llm/pi.go
printf '%s\n' '--- internal/llm/pi_test.go ---'
cat -n internal/llm/pi_test.go
printf '%s\n' '--- Pi-related manifest and documentation references ---'
rg -n -S 'pi(@| version| CLI|runner)|PiRunner|no-session|no-context-files' \
--glob 'go.mod' --glob 'go.sum' --glob 'README*' --glob '*.md' --glob '*.yaml' \
--glob '*.yml' --glob '*.json' --glob '*.toml' --glob '*.sh' \
--glob '!internal/cloud/dashboard/static/**' .Repository: Gentleman-Programming/engram Length of output: 12502 🏁 Script executed: #!/bin/bash
set -e
cat -n internal/llm/pi.go
cat -n internal/llm/pi_test.go
rg -n -S 'pi(@| version| CLI|runner)|PiRunner|no-session|no-context-files' \
--glob 'go.mod' --glob 'go.sum' --glob 'README*' --glob '*.md' \
--glob '*.yaml' --glob '*.yml' --glob '*.json' --glob '*.toml' --glob '*.sh' \
--glob '!internal/cloud/dashboard/static/**' .Repository: Gentleman-Programming/engram Length of output: 12386 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- AgentRunner and semantic-scan callers ---'
rg -n -S 'type AgentRunner|\.Compare\(|Compare\(ctx|scan.*semantic|semantic.*scan' internal cmd --glob '*.go' \
--glob '!internal/cloud/dashboard/static/**'
printf '%s\n' '--- targeted semantic scan implementation ---'
fd -t f 'scan.*semantic|semantic.*scan|.*scan.*\\.go' internal cmdRepository: Gentleman-Programming/engram Length of output: 5633 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- AgentRunner and semantic-scan callers ---'
rg -n -S 'type AgentRunner|\.Compare\(|Compare\(ctx|scan.*semantic|semantic.*scan' internal cmd --glob '*.go' \
--glob '!internal/cloud/dashboard/static/**'
printf '%s\n' '--- targeted semantic scan implementation ---'
fd -t f 'scan.*semantic|semantic.*scan|.*scan.*\.go' internal cmdRepository: Gentleman-Programming/engram Length of output: 5633 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- semantic prompt construction and invocation ---'
rg -n -S 'BuildPrompt|PromptBuilder|promptBuilder|Build.*Prompt|source.*content|target.*content' \
internal/store/relations.go internal/server/server.go cmd/engram/main.go --glob '*.go'
printf '%s\n' '--- bounded source excerpts ---'
sed -n '60,125p' internal/server/server.go
sed -n '1280,1480p' internal/store/relations.goRepository: Gentleman-Programming/engram Length of output: 10517 🏁 Script executed: #!/bin/bash
set -e
sed -n '700,765p' cmd/engram/main.go
rg -n -S 'func llmBuildPrompt|llmBuildPrompt' cmd/engram internal --glob '*.go'Repository: Gentleman-Programming/engram Length of output: 2466 🏁 Script executed: #!/bin/bash
set -e
cat -n cmd/engram/llm.goRepository: Gentleman-Programming/engram Length of output: 3465 🏁 Script executed: #!/bin/bash
set -e
rg -n -S 'func BuildPrompt|type ObservationSnippet|ObservationSnippet' internal/llm --glob '*.go'Repository: Gentleman-Programming/engram Length of output: 1290 🏁 Script executed: #!/bin/bash
set -e
cat -n internal/llm/prompt.goRepository: Gentleman-Programming/engram Length of output: 3021 Sensitive Data Exposure (CWE-922) Run Pi without session persistence.
🤖 Prompt for AI Agents |
||
| raw, err := r.runCLI(ctx, "pi", args, prompt) | ||
| if err != nil { | ||
| // Propagate sentinel errors directly (e.g. ErrCLINotInstalled). | ||
| return Verdict{}, err | ||
| } | ||
|
|
||
| return parsePiNDJSON(raw) | ||
| } | ||
|
|
||
| // ─── Compile-time interface satisfaction ────────────────────────────────────── | ||
|
|
||
| var _ AgentRunner = (*PiRunner)(nil) | ||
|
|
||
| // ─── NDJSON parsing ─────────────────────────────────────────────────────────── | ||
|
|
||
| // piEvent is the generic envelope for each NDJSON line Pi emits in --mode json. | ||
| type piEvent struct { | ||
| Type string `json:"type"` | ||
| AssistantMessageEvent *piAssistantMsg `json:"assistantMessageEvent,omitempty"` | ||
| } | ||
|
|
||
| // piAssistantMsg is the payload of a "message_update" event. | ||
| type piAssistantMsg struct { | ||
| Type string `json:"type"` // text_delta | thinking_delta | ... | ||
| Delta string `json:"delta"` | ||
| Model string `json:"model,omitempty"` | ||
| } | ||
|
|
||
| // parsePiNDJSON scans Pi's NDJSON output, concatenates assistant text_delta | ||
| // chunks into the final message, and parses it as a Verdict JSON object. | ||
| // Malformed lines (Pi prints a non-JSON banner before the stream) and non-text | ||
| // events are skipped; thinking_delta chunks are ignored (reasoning stream, not | ||
| // the answer). | ||
| func parsePiNDJSON(raw []byte) (Verdict, error) { | ||
| scanner := bufio.NewScanner(bytes.NewReader(raw)) | ||
| // Pi echoes large payloads on a single line; raise the token cap well above | ||
| // bufio's 64KB default so long lines don't abort the scan. | ||
| scanner.Buffer(make([]byte, 0, 64*1024), 16*1024*1024) | ||
|
|
||
| var ( | ||
| text strings.Builder | ||
| model string | ||
| ) | ||
|
|
||
| for scanner.Scan() { | ||
| line := bytes.TrimSpace(scanner.Bytes()) | ||
| if len(line) == 0 { | ||
| continue | ||
| } | ||
|
|
||
| var ev piEvent | ||
| if err := json.Unmarshal(line, &ev); err != nil { | ||
| // Malformed line: skip and continue. | ||
| continue | ||
| } | ||
|
|
||
| if ev.Type == "message_update" && ev.AssistantMessageEvent != nil { | ||
| ame := ev.AssistantMessageEvent | ||
| if ame.Type == "text_delta" && ame.Delta != "" { | ||
| text.WriteString(ame.Delta) | ||
| } | ||
| if ame.Model != "" { | ||
| model = ame.Model | ||
| } | ||
|
Comment on lines
+98
to
+105
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🤖 get_repo_knowledge executed:
Length of output: 5339 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- internal/llm/pi.go ---'
cat -n internal/llm/pi.go | sed -n '1,180p'
printf '%s\n' '--- internal/llm/pi_test.go ---'
cat -n internal/llm/pi_test.go | sed -n '1,180p'
printf '%s\n' '--- related Pi symbols and fixtures ---'
rg -n -C 3 'message_end|assistantMessageEvent|PiRunner|runCLI|text_delta|thinking_delta' internal/llmRepository: Gentleman-Programming/engram Length of output: 38884 🌐 Web query:
💡 Result: In the pi.dev JSON event stream protocol, events follow a specific lifecycle managed by the agent [1][2]. The message lifecycle consists of three primary events: message_start, message_update, and message_end [1][3][2]. Key aspects of these events include: message_start: Indicates the beginning of a message (user, assistant, or tool result) and includes the initial message object [3][4][5]. message_update: Emitted specifically for assistant messages during streaming [3][4]. To optimize stream size, these events are delta-only, omitting cumulative message snapshots and partial state [1][6][7]. They contain an assistantMessageEvent object—typically of type text_delta—which provides the delta content and an index [1][8][5]. Because these are deltas, they should be used for live streaming UI updates rather than attempting to assemble the complete message [1][7]. message_end: Emitted when the message is complete and provides the final, authoritative version of the assistant message [1][6][7]. The overall contract for these events requires that streaming subscribers handle delta updates efficiently and rely on message_end for the final state [4][7]. Usage statistics, when reported by the provider, are included at the top level of message_update events [1][6]. The core event types and structures are defined within the pi-agent-core package, ensuring consistent behavior across JSON and RPC modes [9][5][7]. Citations:
🏁 Script executed: #!/bin/bash
set -eu
cat -n internal/llm/pi.go | sed -n '1,180p'
cat -n internal/llm/pi_test.go | sed -n '1,180p'
rg -n -C 3 'message_end|assistantMessageEvent|PiRunner|runCLI|text_delta|thinking_delta' internal/llmRepository: Gentleman-Programming/engram Length of output: 38785 🌐 Web query:
💡 Result: In the pi.dev API architecture, message_end, message, assistantMessageEvent, and text_delta are components of the JSON-based event stream protocol used for agent-assistant communication [1][2][3]. The structure of these events and their roles in the message lifecycle are as follows: message_update: This event is emitted during the streaming of assistant messages [1][4]. To keep stream sizes linear and efficient, it intentionally omits the full cumulative message snapshot [2][4]. Instead, it contains an assistantMessageEvent field [1][3]. assistantMessageEvent: This field encapsulates granular updates from the model [1][4]. It includes a type field to specify the nature of the update (e.g., text_delta, thinking_start, toolcall_delta) and a contentIndex to help clients track which part of the message is being updated [1][5][2]. text_delta: This is a specific type of assistantMessageEvent [1][5]. It is used to stream a chunk of text content from the assistant to the client [1][2][4]. The delta field within this event contains the actual string of new text [1][6][4]. message_end: This event is emitted once an assistant message is fully complete [1][5][7]. It contains the final, authoritative message object, which should be used by the client as the definitive state for that turn [1][2][3]. Summary of Workflow: 1. Clients receive a message_start event when an assistant begins responding [7][2]. 2. For text generation, the client receives a series of message_update events [2][8]. 3. Each message_update event carries an assistantMessageEvent of type text_delta containing a delta string [1][4]. The client assembles these chunks using the contentIndex [1][2]. 4. Once generation finishes, a message_end event provides the complete, authoritative message [2][4]. For tool calls, similar events exist (toolcall_start, toolcall_delta, toolcall_end) to stream arguments and complete the execution [1][5][9]. Citations:
🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository diff for the affected files ---'
git diff --unified=20 -- internal/llm/pi.go internal/llm/pi_test.go
printf '%s\n' '--- Pi JSON protocol fields ---'
curl -L --fail --silent --show-error https://pi.dev/docs/latest/json |
rg -n -C 5 'message_end|assistantMessageEvent|model|content' |
head -n 160Repository: Gentleman-Programming/engram Length of output: 40971 🏁 Script executed: #!/bin/bash
set -eu
git diff --unified=20 -- internal/llm/pi.go internal/llm/pi_test.go
curl -L --fail --silent --show-error https://pi.dev/docs/latest/json |
rg -n -C 5 'message_end|assistantMessageEvent|model|content' |
head -n 160Repository: Gentleman-Programming/engram Length of output: 40892 🏁 Script executed: #!/bin/bash
set -eu
for url in \
https://raw.githubusercontent.com/earendil-works/pi/main/packages/agent/src/types.ts \
https://raw.githubusercontent.com/earendil-works/pi-mono/main/packages/agent/src/types.ts \
https://raw.githubusercontent.com/badlogic/pi-mono/main/packages/agent/src/types.ts
do
echo "--- $url ---"
curl -L --fail --silent --show-error "$url" |
rg -n -C 4 'type AssistantMessage|interface AssistantMessage|model:|message_end|AgentMessage'
doneRepository: Gentleman-Programming/engram Length of output: 19414 Parse the terminal assistant Pi marks Update 📍 Affects 2 files
🤖 Prompt for AI AgentsSource: Path instructions |
||
| } | ||
| } | ||
|
|
||
| final := strings.TrimSpace(text.String()) | ||
| if final == "" { | ||
| return Verdict{}, fmt.Errorf("pi: no assistant text found in NDJSON stream") | ||
| } | ||
|
|
||
| // Strip optional markdown code fences before parsing the inner Verdict JSON. | ||
| if m := fenceRE.FindStringSubmatch(final); len(m) == 2 { | ||
| final = strings.TrimSpace(m[1]) | ||
| } | ||
|
|
||
| var iv innerVerdict | ||
| if err := json.Unmarshal([]byte(final), &iv); err != nil { | ||
| return Verdict{}, fmt.Errorf("%w: inner verdict from pi text: %v", ErrInvalidJSON, err) | ||
| } | ||
|
|
||
| // Validate the relation verb. | ||
| if !validRelations[iv.Relation] { | ||
| return Verdict{}, fmt.Errorf("%w: %q", ErrUnknownRelation, iv.Relation) | ||
| } | ||
|
|
||
| // Prefer a model reported by the stream, else the inner JSON field. | ||
| if model == "" { | ||
| model = iv.Model | ||
| } | ||
|
|
||
| return Verdict{ | ||
| Relation: iv.Relation, | ||
| Confidence: iv.Confidence, | ||
| Reasoning: iv.Reasoning, | ||
| Model: model, | ||
| }, nil | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Qualify the cost claim for Pi.
This example now includes
pi, but the following statement says the semantic judge costs$0with a subscription. Pi uses the configured provider and can use an API key, so that guarantee does not apply to every Pi configuration. Restrict the claim to subscription-backed runners or state that Pi cost depends on its provider configuration. (pi.dev)🤖 Prompt for AI Agents