feat: add named agent types with API and CLI support for v1.4.0

.github/workflows/ci.yml

@@ -17,17 +17,18 @@ jobs:
         with:
           python-version: "3.12"
 
-      - name: Install flake8
-        run: pip install flake8
+      - name: Install Ruff
+        run: pip install 'ruff>=0.9'
 
-      - name: Check for syntax errors
-        run: flake8 longtrainer tests --count --select=E9,F63,F7,F82 --show-source --statistics
+      - name: Check agreed lint rules
+        run: ruff check .
 
   test:
     name: Test (Python ${{ matrix.python-version }})
     runs-on: ubuntu-latest
     needs: lint
     strategy:
+      fail-fast: false
       matrix:
         python-version: ["3.10", "3.11", "3.12"]
 
@@ -46,12 +47,50 @@ jobs:
       - name: Install package
         run: |
           python -m pip install --upgrade pip
-          pip install .[agent,dev,cli,api]
+          pip install '.[agent,research,coding,financial,dev,cli,api]'
+
+      - name: Prepare document parser assets
+        # Some Python 3.10 dependency resolutions do not install spaCy.
+        # Declare both spaCy and its compatible model; the model wheel has no dependencies.
+        run: |
+          python -m pip install 'spacy>=3.8,<3.9' 'https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl'
+          python -c "import spacy; spacy.load('en_core_web_sm')"
 
       - name: Run offline tests
+        run: pytest tests/ -v -ra
+
+  integration:
+    name: Lazy-loading integration (Python 3.12)
+    runs-on: ubuntu-latest
+    needs: lint
+    services:
+      mongo:
+        image: mongo:7
+        ports:
+          - 27017:27017
+        options: >-
+          --health-cmd "mongosh --quiet --eval 'db.adminCommand({ping: 1})'"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install system dependencies
         run: |
-          python tests/test_01_imports.py
-          python tests/test_02_loaders.py
-          python tests/test_03_tools.py
-          python tests/test_04_bot_architecture.py
-          pytest tests/test_05_cli.py tests/test_06_api.py tests/test_07_advanced_loaders.py tests/test_08_dynamic_tools.py -v
+          sudo apt-get update
+          sudo apt-get install -y libmagic-dev poppler-utils tesseract-ocr
+
+      - name: Install package and integration dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install '.[agent,dev,cli,api,integration]'
+
+      - name: Run service-backed integration tests
+        env:
+          LONGTRAINER_TEST_MONGO_URI: mongodb://localhost:27017/?serverSelectionTimeoutMS=5000
+        run: pytest tests/ -v -ra -m integration --run-integration

.gitignore

@@ -40,3 +40,6 @@ more/
 demo_longtrainer.py
 demos/LONGTRAINER_DEMO.md
 demos/*.tape
+
+# Internal roadmap
+LONGTRAINER_ROADMAP.md

CHANGELOG.md

@@ -2,6 +2,47 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.4.0] — In development
+
+Version metadata aligned to `1.4.0` on 6 October 2026. Release date, tag and publication remain pending.
+
+### ✨ New Features
+
+- Add `research`, `coding`, `financial`, and `customer_support` purposes through `create_bot(..., agent_type=...)` and the exported `AgentTypeRegistry`.
+- Resolve purpose-specific tools lazily with dedicated `research`, `coding`, and `financial` extras; fail clearly when defaults are unavailable. SQL remains deferred pending safety/configuration review.
+- Persist and restore named configurations across loads and internal document/chat-training rebuilds. Explicit tool lists replace defaults and exclude implicit global tools.
+- Ground customer support in cited knowledge-base text, rejecting web/upload augmentation and vision responses; document overrides, persistence limits and in-process Python execution in `docs/docs/agent_types.md`.
+
+### 📖 Documentation & Interfaces
+
+- Remove the internal `testing-folder/LONGTRAINER_ROADMAP.md` from the repository; public configuration guidance remains in the named-agent guide.
+
+- Add optional `agent_type` to the existing HTTP build request; invalid names return HTTP 400 with supported names.
+- Add `longtrainer build [BOT_ID] --agent-type ...` and purpose selection on `bot create`, preserving existing CLI options and explicit empty tool overrides.
+- Register the named-agent guide, document infrastructure-only YAML and interface precedence, and consolidate duplicate migration navigation entries.
+
+### 🧪 Testing & CI
+
+- Install the parser model explicitly in CI so Python 3.10 does not rely on a transitive spaCy dependency; keep all Python matrix jobs running when one fails.
+
+- Collect the full `tests/` suite on Python 3.10, 3.11, and 3.12, including rate-limiter tests and 17 relocated bug-fix regressions.
+- Make legacy script checks and lazy-loading integration setup failures fail pytest; reject tests returning non-`None` values.
+- Add 16 offline lazy-loading cases, two failure-signaling regressions, and a separate MongoDB/in-memory Qdrant CI job with an `integration` dependency extra.
+- Replace flake8 with bounded Ruff checks and record lint deferrals and verification results in `docs/phase_a_ci_integrity.md`.
+- Restore all 17 structured-output/vision tests after Phase B removes the temporary module skip.
+- Declare the community tool loader's `mypy-extensions` runtime dependency and prepare document-parser assets before offline CI tests.
+
+### 🔧 Improvements
+
+- Bound the research extra to `arxiv>=2.1,<4` because the community wrapper still calls `Search.results`, removed in arxiv 4; exercise the actual wrapper in the Python CI matrix.
+- Identify Wikipedia requests with a project user agent so live lookups satisfy the service's request policy.
+- Fall back to Yahoo Finance's news-search headlines, including source links and publication timestamps, when ticker article lookup returns no news.
+- Parse provider text blocks into strings for structured and vision responses and agent invocation and synchronous/asynchronous streaming, including Gemini responses.
+
+- Extract reusable JSON-schema validation, deterministic schema hashing, concise error feedback, and one-retry response repair into `longtrainer.structured`.
+- Preserve structured chat dictionaries and successful-only history updates; copy retry messages, remove dead message construction and the temporary Ruff lambda exception, and honor invocation configuration on both attempts.
+- Add structured bot/chat/API boundary regressions; retain unused schema registry methods without adding persistence side effects.
+
 ## [1.3.1] — 2026-05-07
 
 ### ✨ New Features