ragcodepilot

Search your codebase using plain English, right from the terminal.

ragcodepilot is a local RAG (Retrieval-Augmented Generation) code search CLI. The full RAG idea has three steps:

1. Retrieve — find the most relevant pieces of source code for a given question.
2. Augment — feed those code snippets into a prompt as context.
3. Generate — have an LLM produce an answer grounded in the retrieved code.

Retrieval is fully implemented — you can point the tool at a local repository, index it, and search with natural language queries like "how does the chunking work?". Answer generation (v0) is now available via the opt-in --answer flag, which feeds the retrieved chunks to a local Ollama model (qwen2.5-coder:7b) and prints a synthesized answer above its sources. Additional providers remain deferred; the roadmap prioritizes dense embedding reuse with recovery and actionable errors/.gitignore support; the M0 chunk-ID collision has been fixed and rechecked.

Answer mode is opt-in — without it, the tool works as a pure code search engine. Usage is through one-shot CLI commands; an interactive REPL remains deferred.

How retrieval works: The tool breaks source code into small chunks (functions, blocks of lines), converts each chunk into a numerical "fingerprint" (embedding) that captures its meaning, and stores everything in a local vector database (Qdrant). It also builds a keyword index (BM25) alongside the embeddings and combines both signals for better results. When you search, your query is matched against the stored chunks to find the closest results.

Current status

• Go CLI with Qdrant as the vector database backend.
• Local Ollama embedder using nomic-embed-text (768-dimensional vectors, auto-detected).
• Go functions/methods and named types/interfaces are chunked using go/parser AST; other languages use a 40-line sliding window.
• Chunk enrichment prepends file/language/function metadata to embedding input for improved search relevance.
• Search with dense vector lookup and optional language and repo payload filtering.
• Answer mode (--answer): feeds retrieved chunks to a local Ollama generative model (qwen2.5-coder:7b by default) and prints a synthesized answer above its sources. Opt-in; the default search path is unchanged.
• Embedding dimension auto-detection and validation (collection mismatch produces clear error with fix instructions).
• Re-indexing: unchanged hashes and index version allow a no-op. A change, addition, deletion, or version refresh currently re-chunks and re-embeds all remaining files in that repository/language scope; stale chunks are cleaned up. A same-hash representation refresh deletes old file points before writing replacement IDs, so old IDs do not survive a version migration. Index runs publish durable state, hold one writer lock per collection, and reuse validated dense vectors from the user cache when enriched inputs are unchanged. Retry/cleanup recovery remains the next M3 increment. --watch uses the same pipeline.
• Existing index/search failures include actionable remedies for missing services/models, active writers, source changes, and collection setup. The walker applies nested .gitignore rules and negation after configured exclusions; newly ignored files are removed from the index on re-index.
• Retrieval evaluation harness (ragcodepilot eval) with golden dataset, hit@k, MRR@5, recall@10, and per-stage latency percentiles.
• Collection list and delete commands.
• config.yaml is auto-loaded during indexing when present; built-in defaults are used only when it is absent.

Hybrid search is implemented: BM25 sparse vectors (k1=0.5, b=0.75) + dense vectors + Reciprocal Rank Fusion (--mode dense|sparse|hybrid, default hybrid), with additive identifier tokens and Snowball stemming on the BM25 path. Historical baseline (baseline_v6, 182 chunks, 23 golden queries): hit@5 = 0.895, hit@1 = 0.579, MRR@5 = 0.673, recall@5 = 0.789, recall@10 = 0.921. (Indexing excludes *_test.go by default — see config.yaml skip_file_patterns; this recovered hit@5 from 0.789 to 0.895 by keeping test chunks out of the top-K.) Additional retrieval components are outside the current scope. The roadmap evidence snapshot links the fresh pinned v9 self baseline (hit@5 34/35; negatives 2/4), explains calibrated negative-query scoring, and links the completed external chi evaluation and its post-fix recheck. MCP, an agent-first pivot, and multi-repo workspace commands are removed from active scope; existing search filters remain available.

Architecture

CLI
  |
  |-- index <repo-path>
  |     -> file walker
  |     -> language detector
  |     -> chunker (Go AST or sliding window)
  |     -> enrichment (prepend metadata to embedding input)
  |     -> Ollama embedder (nomic-embed-text)
  |     -> Qdrant upsert
  |
  |-- search <query>
  |     -> Ollama query embedder
  |     -> dimension validation
  |     -> Qdrant vector search (with language/repo filters)
  |     -> terminal result formatter
  |
  |-- eval --dataset <golden.yaml>
  |     -> load golden queries
  |     -> run each through search.Searcher
  |     -> compute hit@k, MRR@5, recall@10, latency percentiles
  |     -> JSON or human-readable report
  |
  |-- collections list|delete

Qdrant runs locally through Docker Compose. The Go CLI runs on the host machine and connects to Qdrant over gRPC on port 6334. Ollama runs locally and serves embeddings on port 11434.

Prerequisites

• Go 1.26.3
• Docker and Docker Compose
• Ollama with the nomic-embed-text model (required for indexing/search)
• For --answer mode only: a generative model (qwen2.5-coder:7b by default)

Getting started

Start Qdrant:

docker compose up -d qdrant

Pull the embedding model (one-time):

ollama pull nomic-embed-text

For --answer mode, also pull the generative model (one-time):

ollama pull qwen2.5-coder:7b

Tip — avoid cold-start latency: the first --answer call after Ollama starts spends 5–30s loading the generative model into memory. To keep the model resident across calls (subsequent answers take 1–5s), set:

export OLLAMA_KEEP_ALIVE=-1

This pins models in Ollama's process (costs a few GB of RAM). It is the single biggest perceived-speed improvement for --answer — see docs/knowledge/architecture_decisions.md §4.

Check the CLI from source:

go run ./cmd/ragcodepilot version

Build the project:

mkdir -p bin
go build -o bin/ragcodepilot ./cmd/ragcodepilot

Index this repository (Go files only):

go run ./cmd/ragcodepilot index --language go .

Index once, then stay running and re-index on file changes:

go run ./cmd/ragcodepilot index --language go --watch .
# Edits are detected via fsnotify, debounced, and run through the same
# pipeline as a one-shot index (full re-embed when anything changed).
# Press Ctrl-C to exit. Respects skip_dirs and skip_file_patterns
# in config.yaml.

Search indexed code:

go run ./cmd/ragcodepilot search --language go --limit 5 "embedding interface"

Search with repo filter:

go run ./cmd/ragcodepilot search --repo ragcodepilot --limit 3 "ingestion pipeline"

Combine language and repo filters:

go run ./cmd/ragcodepilot search --language go --repo ragcodepilot --limit 3 "how does chunking work?"

Generate an answer from the retrieved chunks (RAG mode):

go run ./cmd/ragcodepilot search --answer "how does change detection work?"

This prints a synthesized Answer: followed by the Sources: chunks that fed it. Use --ollama-generative-model to swap the model, or --generator fake for a hermetic (no-Ollama) canned response. Without --answer, search behaves exactly as before (retrieval only).

List collections:

go run ./cmd/ragcodepilot collections list

Delete the default collection (required when changing embedding model or enrichment logic):

go run ./cmd/ragcodepilot collections delete code_chunks

Run retrieval evaluation against the golden dataset:

go run ./cmd/ragcodepilot eval --dataset docs/eval/golden.yaml

Run eval with JSON output (for diffing baselines):

go run ./cmd/ragcodepilot eval --output json > /tmp/ragcodepilot-eval.json

Filter eval to a specific query type:

go run ./cmd/ragcodepilot eval --type navigation

Evaluate answer mode too (reference-free answer metrics, real Ollama):

go run ./cmd/ragcodepilot eval --answer

This runs the normal retrieval eval and generates an answer per query, then reports deterministic, reference-free answer metrics alongside the retrieval numbers:

• well-formed rate — answers that are non-empty.
• cited rate / all-citations-valid — positive-query answers that cite [N] chunks, and whether those citations point at chunks that were actually provided (no dangling refs).
• refusal rate (negative) — negative queries where the model correctly said "not enough information" instead of hallucinating. This is the hallucination floor. (Refusal is detected by a phrase heuristic — a diagnostic, not ground truth.)

Answer metrics are reported, never gated — they never change the exit code. Generation runs greedy (temperature 0) so re-runs are reproducible. Note this is much slower than retrieval-only eval (one LLM call per query).

Stop Qdrant:

docker compose down

CLI flags

Index

Flag	Default	Description
-collection	code_chunks	Qdrant collection name
-language	(all)	Comma-separated language filter (e.g., go,rust)
-embedder	ollama	Embedder to use: ollama, fake
-ollama-url	http://localhost:11434	Ollama server URL
-ollama-model	nomic-embed-text	Ollama embedding model
-watch	false	After the initial index, watch the repo and re-run the index pipeline on changes (blocks until Ctrl-C). Same cost as a one-shot re-index when anything changed.
-qdrant-host	localhost	Qdrant host
-qdrant-port	6334	Qdrant gRPC port

Search

Flag	Default	Description
-collection	code_chunks	Qdrant collection name
-language	(all)	Comma-separated language filter (e.g., go,rust)
-repo	(all)	Comma-separated repo name filter (e.g., ragcodepilot)
-mode	hybrid	Retrieval mode: dense, sparse, hybrid
-limit	5	Maximum number of results
-embedder	ollama	Embedder to use: ollama, fake
-ollama-url	http://localhost:11434	Ollama server URL
-ollama-model	nomic-embed-text	Ollama embedding model
-answer	false	Generate an answer from the retrieved chunks (RAG mode)
-generator	ollama	Generator for --answer: ollama, fake
-ollama-generative-model	qwen2.5-coder:7b	Ollama generative model for --answer
-answer-limit	5	Number of top chunks fed to the generator for --answer
-qdrant-host	localhost	Qdrant host
-qdrant-port	6334	Qdrant gRPC port

Eval

Flag	Default	Description
-dataset	docs/eval/golden.yaml	Path to the golden YAML dataset
-collection	code_chunks	Qdrant collection name
-output	human	Output format: human, json
-limit	10	Per-query result limit (must be ≥ 10 for recall@10)
-type	(all)	Filter queries by type (navigation, concept, behavior, negative)
-subtype	(all)	Filter queries by subtype (e.g. structural under navigation; combines with -type as AND)
-mode	hybrid	Retrieval mode: dense, sparse, hybrid
-embedder	ollama	Embedder to use: ollama, fake
-ollama-url	http://localhost:11434	Ollama server URL
-ollama-model	nomic-embed-text	Ollama embedding model
-answer	false	Also generate answers and score reference-free answer metrics
-generator	ollama	Generator for --answer: ollama, fake
-ollama-generative-model	qwen2.5-coder:7b	Ollama generative model for --answer
-answer-limit	5	Top chunks fed to the generator for --answer (retrieval metrics still use --limit)
-qdrant-host	localhost	Qdrant host
-qdrant-port	6334	Qdrant gRPC port

Configuration

config.yaml controls language extension mappings, skipped directories, and skipped file patterns for indexing.

When running index, the CLI uses this precedence:

1. Load ./config.yaml if it exists.
2. Use built-in defaults if ./config.yaml does not exist.
3. Fail fast if ./config.yaml exists but cannot be read or parsed.

Example:

languages:
  go: [".go"]
  rust: [".rs"]
  python: [".py"]

skip_dirs:
  - .git
  - vendor
  - node_modules
  - target

skip_file_patterns:
  - "*_test.go"

skip_file_patterns is a list of globs matched against each file's base name (via Go's filepath.Match) to exclude files from indexing.

• Default: when the key is omitted, it defaults to ["*_test.go"], so Go test files are excluded. This keeps test fixtures (including eval golden queries) out of retrieval and reduces top-K crowding from test chunks.
• Disable: set skip_file_patterns: [] to index everything, including test files.
• Other languages: add patterns such as "*.test.ts", "*_test.py", or "*.spec.js" as needed.

The search command does not load config.yaml. It filters against language and repo values stored in Qdrant payloads during indexing.

Development

Run all tests with race detection:

go test -race ./...

Run tests for a specific package:

go test -race -v ./internal/ingest/...
go test -race -v ./internal/qdrant/...

Build the project:

mkdir -p bin
go build -o bin/ragcodepilot ./cmd/ragcodepilot

Inject a version at build time:

mkdir -p bin
go build -ldflags "-X main.version=dev-local" -o bin/ragcodepilot ./cmd/ragcodepilot

Check formatting:

gofmt -l ./internal/

Re-indexing after changes

You must delete the collection and re-index when:

• Changing the embedding model (different vector dimensions)
• Changing the enrichment logic (different embedding input text)
• Switching between fake and ollama embedders

go run ./cmd/ragcodepilot collections delete code_chunks
go run ./cmd/ragcodepilot index --language go .

Known limitations

• Function-level chunking is Go-only (AST-based). Other languages use a sliding window.
• Re-index is not proportional to the diff. Because BM25 IDF is corpus-wide, any file change re-embeds all current files through Ollama. Change detection only skips work when the corpus is byte-identical. Fine at ~200 chunks; painful at tens of thousands. See docs/improvement/reindexing.md.
• Sparse vectors use BM25 with a softened k1=0.5 (Elasticsearch's default k1=1.2 is calibrated for long, mixed-length documents; code chunks are short and uniform, so milder TF saturation gave a much cleaner result on the May 2026 eval — hit@1 +21pp vs TF-IDF). The original plural/singular token-mismatch regression on the hasher_concept query was resolved on 2026-05-15 by additive Snowball stemming (baseline_v4). CamelCase/snake_case identifiers also keep a joined token (ChunkFile → chunkfile + chunk + file) so exact-symbol queries are not reduced to generic parts. See docs/plan/hybrid_search.md §3 for the full history and eval matrix.
• Embedding dimension is auto-detected; switching models requires collection delete + re-index.

Further docs

• MVP Roadmap
• Phase 5 v0 — --answer mode plan
• Hybrid search design + eval matrix
• Retrieval quality decisions
• System design
• Progress checklist
• Evaluation harness
• Function-level chunker
• Embedding dimension validation
• Chunk enrichment
• RAG architecture parts
• Embeddings explained
• Vector database platform comparison