forensic-fathers-library

A small static site and client-only RAG chatbot covering the founding figures of forensic science. Drop any .md files into docs/ and the chatbot will retrieve over them in the browser — no server, no API keys.

The site is published via GitHub Pages from the repo root and consists of:

• index.html — landing page listing every person.
• people/<slug>/index.html — one biography page per person (built from data/people.json).
• qr/<slug>.png — a QR code that resolves to that person's page.
• chat.html — client-only WebLLM chatbot. Retrieves context from docs.json and asks TinyLlama in-browser.
• docs/ — the corpus: any markdown files here become searchable.
• docs.json — pre-indexed chunks with embeddings (built by build_docs.py). Committed so GitHub Pages serves it directly.
• assets/css/style.css — archival-paper theme used by the static site.

Quick start

uv sync                      # install Python deps into .venv/
uv run python main.py        # build docs.json + site + QRs (full rebuild)

Open index.html for the static site or chat.html for the chatbot. Both work straight off the filesystem (or any static host) once docs.json exists.

You can also run any single step:

uv run python build_docs.py     # rebuild docs.json from docs/*.md
uv run python generate_site.py  # rebuild person pages
uv run python generate_qr.py    # rebuild QR codes

Adding content

Drop a .md file into docs/. The first # Heading becomes its title; blank lines split the body into chunks (~1500 chars each) so retrieval stays focused. Re-run python build_docs.py (or python main.py) and refresh chat.html — the sidebar list and the retriever both pick it up automatically. No code changes required.

Why it's fast now

The previous version recomputed embeddings for every document on every query, and pulled docs.json from a hardcoded raw GitHub URL on first load. The current version:

• Pre-computes embeddings at build time in build_docs.py. The chatbot reads them straight from docs.json — per-query work is a single query embedding plus N cosine similarities (N = number of chunks).
• Loads docs.json from a relative path (./docs.json), so it ships with the page instead of round-tripping to GitHub raw.
• Fetches docs.json in parallel with the WebLLM model, so the doc index is ready by the time the model finishes downloading.

Model download itself is still the dominant cost on first visit (TinyLlama is ~700 MB); subsequent visits are cached by the browser.

Project layout

.
├── data/
│   └── people.json          # Canonical biographical data (used by site builder)
├── docs/                    # Markdown corpus consumed by the chatbot
├── docs.json                # Built artifact: chunks + embeddings
├── assets/css/style.css
├── build_docs.py            # docs/ -> docs.json
├── generate_site.py         # data/people.json -> index.html + people/<slug>/
├── generate_qr.py           # data/people.json -> qr/<slug>.png
├── main.py                  # Runs all three in order
├── chat.html                # Client-side RAG chatbot (WebLLM)
├── index.html               # Generated landing page
├── people/                  # Generated person pages
└── qr/                      # Generated QR codes

Data format

data/people.json (used by generate_site.py / generate_qr.py) is a list of objects shaped like:

{
  "slug": "bernard-spilsbury",
  "name": "Bernard Spilsbury",
  "title": "Father of Forensic Science",
  "bio": "Sir Bernard Henry Spilsbury (1877-1947) was ...",
  "quote": "The truth is there for those trained to see it.",
  "links": ["https://en.wikipedia.org/wiki/Bernard_Spilsbury"]
}

docs.json (built; do not edit) is a list of:

{
  "id": "bernard-spilsbury#0",
  "source": "bernard-spilsbury.md",
  "title": "Bernard Spilsbury",
  "text": "...chunk body...",
  "embedding": [0.0123, -0.0456, ...]
}