The mcp service exposes the full Embeddorium lifecycle — providers, dataset preparation,
indexing, and staged search — as Model Context Protocol
tools, so an external agent can drive the same workflow the web UI drives.
It is a thin HTTP client of the FastAPI server. It holds no state, imports no
SqlStore/VectorStore, and never talks to PostgreSQL, Qdrant, or RabbitMQ directly. Every
tool is one or two calls to a documented REST endpoint, which means the MCP surface and the UI
cannot drift apart in behaviour — only in wording.
| Value | |
|---|---|
| Transport | streamable-HTTP |
| Endpoint | http://localhost:8080/mcp (host) · http://mcp:8080/mcp (inside Compose) |
| Auth | none — the same trust model as the rest of the stack |
Configuration is read once at import:
| Variable | Default | Purpose |
|---|---|---|
SERVER_URL |
http://server:8000 |
Base URL of the FastAPI server |
MCP_PORT |
8080 |
Port the MCP server listens on |
The service starts with the Compose stack. To register it with a client that reads
.mcp.json:
{
"mcpServers": {
"embeddorium": { "type": "http", "url": "http://localhost:8080/mcp" }
}
}Errors are surfaced verbatim: a non-2xx response becomes a tool error carrying the server's own
detail string, so Dataset 'x' is not prepared (preparation_status='not_prepared') reaches
the agent instead of a bare 409.
The tool set is organised as the UI is, one module per page. The ordering constraint that
matters most is the dataset/pipeline split:
a pipeline indexes an already-prepared corpus and cannot fetch or parse anything, so
launch_pipeline refuses with 409 until the dataset reaches prepared.
create_provider ──┐
├──> create_pipeline ──> launch_pipeline ──> get_pipeline_status
create_dataset ───┤ (needs a prepared dataset) │
│ │ v
└─ update_prep_config ──> prepare_dataset ──> get_dataset_status ──> search
(poll) ^
create_search_query ─── create_variable_group ──────────────────────────┘
Both prepare_dataset and launch_pipeline are asynchronous: they return as soon as the work
is queued. Poll get_dataset_status until indexable is true, and get_pipeline_status until
status is completed.
| Tool | Purpose |
|---|---|
list_provider_types |
Adapters available and the config fields each expects — read this before creating one |
discover_models |
Ask a provider's server which models it is serving, before saving the provider |
list_providers |
Configured providers, optionally filtered by model_type |
create_provider |
Create one model endpoint |
delete_provider |
Delete a provider (existing runs keep their own snapshot) |
| Tool | Purpose |
|---|---|
list_datasets |
Every dataset with its preparation state |
list_source_files |
Browse the mounted source tree for valid local paths |
create_dataset |
Declare a web seed URL or a set of local files |
get_prep_config / update_prep_config |
Read or patch validation, fetching, parsing, filtering, and crawl scope |
prepare_dataset |
Fetch and parse sources into canonical documents |
get_dataset_status |
Poll target — returns indexable |
list_dataset_targets |
Per-source progress, with skip reasons and errors |
list_dataset_documents |
The canonical documents preparation produced |
delete_dataset |
Delete a dataset and its on-disk artifacts |
update_prep_config is a patch: it merges your blocks over the current config, because the
underlying PUT replaces the whole thing and would otherwise silently reset the four blocks you
did not send. It also invalidates the prepared corpus — the dataset drops back to
not_prepared and must be prepared again.
| Tool | Purpose |
|---|---|
list_chunkers |
Chunking strategies and their settings keys |
list_pipelines |
Pipelines flattened to their key configuration |
create_pipeline |
Bind a dataset, chunker, and embedding provider. Does not launch |
launch_pipeline |
Chunk, embed, and index. Safe to retry on a failed run |
get_pipeline_status |
Poll target — chunks_embedded / chunks_pending |
list_pipeline_documents |
Per-document indexing progress |
delete_pipeline |
Delete a run |
create_pipeline probes the embedding provider for its vector dimension, because the Qdrant
collection name carries it. An unreachable provider therefore fails at creation (503) rather
than producing a run whose every embed message would die.
| Tool | Purpose |
|---|---|
list_search_queries / create_search_query / delete_search_query |
Reusable query templates |
list_variable_groups / create_variable_group / delete_variable_group |
Named positional values that fill {{0}}, {{1}}, … |
A saved query stores the template with its placeholders intact, not a rendering of it — that is what lets one question be re-run against any set of values and lets Search Comparison and Evaluation line executions up by query identity. A search of N templates × M groups runs all N × M combinations and returns each separately.
| Tool | Purpose |
|---|---|
search |
Run the staged retrieval pipeline |
list_searches |
Past executions, newest first |
get_search |
One execution with its results and per-stage snapshot |
search exposes every stage of the staged pipeline as a flat
argument: mode, the per-retriever vector_candidates / keyword_candidates,
fusion_strategy / fusion_candidates / rank_constant, reranker_provider_id, and
result_limit. Limits must narrow down the pipeline; the server rejects a config that widens
one (Final results (99) cannot exceed the candidates entering the final stage (10)).
Results come back grouped per execution, each with its rendered_text, its
variable_group_id, its stage_counts, and its search_id. Chunk text is truncated to
max_chunk_chars (default 1200, 0 disables) so a wide search cannot flood an agent's context.
The FastAPI server speaks camelCase on request and response bodies. Three payloads are deliberate exceptions and stay snake_case, because their keys are declared by plugins and stored verbatim:
| Payload | Example keys |
|---|---|
Provider config |
url, port, model_name, rerank_path |
| Chunker settings | chunk_size, chunk_overlap, window_size |
| Dataset preparation config | max_depth, verify_tls, exclude_keywords |
The tools build both shapes correctly; you pass snake_case for those three and read camelCase back on raw pass-through responses.
Ingesting a website at crawl depth 1 and a pair of local XML files, then searching both with hybrid retrieval. Arguments are shown as JSON.
1. Providers. Confirm the model exists, then create the endpoint.
Use host.docker.internal (or the ollama Compose service), never localhost — the URL is
resolved from inside the server container.
2. Datasets. A local dataset takes source-root-relative paths from list_source_files; a
web dataset takes a seed URL.
create_dataset {"name": "mcp-local-acts", "source_type": "local",
"paths": ["501022016016.xml", "501022016017.xml"]}
create_dataset {"name": "mcp-roadmap-community", "source_type": "web",
"url": "https://roadmap.sh/community"}3. Crawl scope. Depth is a preparation setting, not a dataset field. The seed page is
depth 0, so max_depth: 1 fetches the seed plus the pages it links to.
update_prep_config {"dataset_id": "<web-id>",
"config": {"schedule_discovered_links": {"max_depth": 1}}}4. Prepare, then poll.
prepare_dataset {"dataset_id": "<id>"}
get_dataset_status {"dataset_id": "<id>"}
// -> {"preparation_status": "prepared", "document_count": 12, "indexable": true}A dataset that reaches prepared with document_count: 0 means every source was skipped or
filtered — list_dataset_targets gives the reason per source.
5. Index, then poll.
create_pipeline {"name": "mcp-roadmap-mxbai", "dataset_id": "<web-id>",
"embedding_provider_id": "<embedding-id>",
"chunker": "text_recursive",
"chunker_settings": {"chunk_size": 900, "chunk_overlap": 120},
"similarity": "cosine"}
launch_pipeline {"pipeline_id": "<id>"}
get_pipeline_status {"pipeline_id": "<id>"}
// -> {"status": "completed", "chunks_embedded": 90, "chunks_pending": 0}6. Queries and variables.
create_variable_group {"name": "mcp-roadmap-topics",
"values": ["learn backend development", "roadmap"]}
create_search_query {"text": "How do I {{0}} using the {{1}} on roadmap.sh?",
"label": "mcp-roadmap-howto"}7. Search. One template against two variable groups, hybrid retrieval with RRF fusion and cross-encoder reranking.
search {"pipeline_id": "<id>",
"query_ids": ["<query-id>"],
"variable_group_ids": ["<group-a>", "<group-b>"],
"mode": "hybrid",
"vector_candidates": 30, "keyword_candidates": 30,
"fusion_strategy": "rrf", "fusion_candidates": 20,
"reranker_provider_id": "<cross-encoder-id>",
"result_limit": 4}Each execution returns its own hits and stage_counts, e.g.
{"vector": 30, "keyword": 30, "fusion": 20, "reranking": 4, "final": 4} — the shape of the
pipeline that actually ran.
- Reranking needs a rerank-capable server, and Ollama is not one. Reranking uses the
rerank_serverprovider type, which posts to an OpenAI/Jina-stylePOST {url}/{rerank_path}endpoint served by llama.cpp, vLLM, TEI, Infinity, or Cohere/Jina. Ollama serves no rerank route on any path, so it can never be that endpoint — thecross-encodercapability is deliberately not offered under theollamaprovider. A rerank failure does not fail the search: the server logs it and returns the un-reranked ordering rather than breaking an otherwise-good search. It is recorded, though — the stage snapshot carriesreranking.degraded: trueandreranking.error, and theserverlogs carryReranking failed (provider=http_rerank ...). See Running a reranker locally. - Evaluation is not exposed. Evaluation sets, relevance judgments, and evaluation runs are UI/REST only — see Evaluation.
- Search comparison and projection are not exposed. Use
get_searchto inspect executions individually. - No authentication, on this service or any other in the stack.
discover_models {"provider_type": "ollama", "config": {"url": "http://host.docker.internal", "port": 11434}} create_provider {"name": "ollama-mxbai-embed-large", "provider_type": "ollama", "model_type": "embedding", "config": {"url": "http://host.docker.internal", "port": 11434, "model_name": "mxbai-embed-large:latest"}}