update Agent module

.gitignore

@@ -7,6 +7,9 @@
 *.iws
 *.pyc
 *.pyo
+.pytest_cache/
+.venv/
+*.egg-info/
 .idea/
 .idea_modules/
 .settings

Agent_module/QUICKSTART.md

@@ -1,10 +1,15 @@
 # carbon_data Quickstart
 
-`Agent_module.carbon_data` is the Agent Data Infra: one `.carbondata` file =
-one self-contained knowledge base, simultaneously serving **RAG semantic
-search / long-term memory / structured queries / knowledge-graph traversal**.
+`Agent_module.carbon_data` is an Agent-native context store: one
+`.carbondata` file persists context that multiple Agents can write, retrieve,
+and hand off. It combines **semantic context retrieval / cross-agent memory /
+structured state queries / context relationship traversal**.
 
 > Companion runnable demo: [`examples/carbondata_quickstart.py`](../examples/carbondata_quickstart.py)
+>
+> The Agent module stores SQLite data. It is not binary-compatible with
+> classic Apache CarbonData columnar fact files, even though both currently
+> use the `.carbondata` suffix.
 
 ---
 
@@ -20,13 +25,16 @@ Optional dependencies are loaded on demand:
 | `hnswlib` | fast ANN for 10k+ vectors | optional (falls back to brute force when missing) |
 
 ```bash
-pip install numpy
-pip install hnswlib   # optional
+# Run from the repository root.
+python3 -m pip install -e "./Agent_module[test]"
+python3 -m pip install -e "./Agent_module[all]"  # optional HNSW support too
 ```
 
 A `.carbondata` file **is just a SQLite database**, so you can crack it open
 with any SQLite tool (`sqlite3 kb.carbondata`, DBeaver, DataGrip) for ad-hoc
-debugging.
+debugging. WAL mode can create temporary `-wal` and `-shm` sidecars while
+the database is open, so copy or back up a live store using SQLite-aware
+tools rather than copying only the primary file.
 
 ---
 
@@ -193,7 +201,8 @@ acceleration whenever:
 - `mode="vector"` (including the vector leg of `mode="hybrid"`)
 - `metric="cosine"`
 - no JSON `filters=` are present
-- `namespace=` is applied as a post-filter (with 3× over-sampling)
+- `namespace=` is applied as an adaptive post-filter that expands until
+  `top_k` is satisfied or the full index has been considered
 
 Explicit override:
 ```python
@@ -203,8 +212,8 @@ store.search(q, embedder=emb, use_hnsw="on")    # force HNSW (raises if not elig
 ```
 
 On the first search, HNSW is built from the entire `embedding` table and
-persisted to a `vector_index` blob; later add/delete calls are detected via
-a row-count comparison and trigger a lazy rebuild.
+persisted to a `vector_index` blob. API mutations explicitly invalidate the
+affected index; row-count checks also detect out-of-band adds and deletes.
 
 ### 4.3 Namespace isolation
 
@@ -226,7 +235,9 @@ with store.transaction():
 ```
 
 `ingest_text` / `ingest_table` / `remember` are already wrapped in their own
-transactions internally.
+transactions internally. Nested `transaction()` calls use savepoints: an
+inner exception rolls back the inner block, while an outer transaction may
+continue if it catches that exception.
 
 ### 4.5 Admin & ops
 
@@ -260,8 +271,7 @@ print(store.stats())
   — each file covers one milestone end-to-end.
 - **Run the suite:**
   ```bash
-  cd Agent_module
-  .venv/bin/pytest tests/carbon_data/ -q
+  python3 -m pytest Agent_module/tests/carbon_data/ -q
   ```
 
 ---
@@ -272,8 +282,7 @@ print(store.stats())
 Run the demo to see it all in action:
 
 ```bash
-cd Agent_module
-.venv/bin/python examples/carbondata_quickstart.py
+python3 Agent_module/examples/carbondata_quickstart.py
 ```
 
 For the full API surface, read the `class CarbonStore` docstring in

Agent_module/README.md

@@ -1,20 +1,31 @@
-# Agent Module — Agent Data Infra
+# Agent Module — Agent-Native Context Data
 
-`Agent_module` is a Python library that gives an LLM agent **a single
-knowledge file** to read from. That file — with the `.carbondata` extension
-— is a self-contained SQLite database that simultaneously serves the four
-data-access patterns agents actually need:
+`Agent_module` provides a data format and local store designed specifically
+for Agent access. Its primary job is to persist and share context across
+Agents — including memories, task artifacts, structured state, semantic
+fragments, and the relationships among them — so another Agent can recover
+the right context efficiently instead of replaying an entire conversation.
+
+The `.carbondata` context file is a self-contained SQLite database serving
+four complementary access patterns:
 
 | Pattern | What an agent calls it for |
 |---|---|
-| **RAG semantic search** | "Find chunks relevant to this question." |
-| **Long-term memory** | "What have I learned about this user / session before?" |
-| **Structured query** | "Give me all rows where `team = 'ml'`." |
-| **Knowledge-graph traversal** | "Which documents does this one reference, two hops out?" |
+| **Semantic context retrieval** | "Find the prior context relevant to this task." |
+| **Cross-agent memory** | "What did another Agent learn or decide earlier?" |
+| **Structured state query** | "Give me the state for this task/session/actor." |
+| **Context relationship traversal** | "Which artifacts, decisions, and tasks led here?" |
 
 One file. One Python handle (`CarbonStore`). One schema. No separate vector
 DB, KV store, graph DB, or relational DB to wire together.
 
+> **Format boundary:** the Agent module's `.carbondata` file is an
+> application-identified SQLite database. It is not the same binary format
+> as the classic Apache CarbonData columnar fact files that also use the
+> `.carbondata` suffix, and the two are not interchangeable. Treat the Agent
+> format as an experimental local-store format while its long-term naming
+> and interoperability contract is defined.
+
 ```
                    ┌─────────────────────────────┐
    agent code ───► │  CarbonStore (kb.carbondata)│
@@ -43,7 +54,7 @@ Agent_module/
 ├── QUICKSTART.md                  # full hands-on guide (read this next)
 ├── examples/
 │   └── carbondata_quickstart.py   # end-to-end runnable demo
-└── tests/carbon_data/             # 249 tests across M1–M9
+└── tests/carbon_data/             # 250+ tests across M1–M9
 ```
 
 ---
@@ -52,13 +63,15 @@ Agent_module/
 
 - Python 3.10+
 - `numpy` (required)
-- `hnswlib` (optional — enables ANN acceleration when corpus > ~10k chunks;
-  falls back to brute force when missing)
+- `hnswlib` (optional — enables ANN acceleration and is most useful for
+  larger corpora; falls back to brute force when missing)
 - `pytest` (optional — for running the test suite)
 
+From the repository root, install the module and its test dependencies:
+
 ```bash
-pip install numpy
-pip install hnswlib   # optional but recommended
+python3 -m pip install -e "./Agent_module[test]"
+python3 -m pip install -e "./Agent_module[all]"  # also install optional HNSW support
 ```
 
 ---
@@ -132,6 +145,11 @@ You now have a working `kb.carbondata` file on disk. Open it again in
 another script with `carbon_data.open("kb.carbondata")` — all your
 entities, embeddings, relations, and memories will still be there.
 
+SQLite WAL mode may create temporary `-wal` and `-shm` sidecar files while
+the store is open. A clean close normally checkpoints the database back to
+the primary file; applications must not copy a live store without its WAL
+state.
+
 ### Step 3 — go deeper
 
 [**QUICKSTART.md**](QUICKSTART.md) covers each scenario in detail:
@@ -147,7 +165,7 @@ For the full API surface, read the `class CarbonStore` docstring in
 ## Running the tests
 
 ```bash
-pytest Agent_module/tests/ -v
+python3 -m pytest Agent_module/tests/ -v
 ```
 
 Tests are split by milestone (M1 schema → M9 admin); each file is a