Voice Agent Studio

An ultra-low latency, document-augmented Voice AI assistant workspace built with LiveKit Agents, the Go-based OKF (Open Knowledge Format) Engine, Groq Inference (llama-3.3-70b-versatile), Next.js, and Better Auth.

Instead of traditional vector RAG with slow semantic searches and tool calls mid-utterance, this project uses sub-millisecond OKF graph retrieval at STT time, injecting grounded context and stage/floor directives directly into the LLM context for real-time speech interaction.

---

Key Features

• OKF Engine Service Layer: Sub-millisecond concept retrieval (<1ms) via in-memory inverted indices and gRPC. It compiles uploaded documents into structured bundles on the fly.
• Groq LLM Inference & LiveKit Inference Fallback: Ultra-fast TTFT with Groq llama-3.3-70b-versatile, with seamless fallback to Gemma 4 and Gemini models.
• STT-Time Context & Directive Retrieval: Grounds responses without blocking tool-call cycles.
• Role & Persona Directives: Supports floor directives (always-on compliance/character) and situational stage directives.
• Multi-Session Tracking & Database Persistence: Powered by SQLite (chat_history.db). User workspaces and complete transcripts are preserved across sessions.
• Local LiveKit Server Support: Run LiveKit locally using your own configuration.

---

Architecture & Monorepo Structure

voice-agent/
├── apps/
│   ├── agent/               # Python Voice AI Agent
│   │   ├── src/             # LiveKit agent, Groq LLM, & STT-time retrieval
│   │   ├── tests/           # Automated pytest test suite
│   │   └── pyproject.toml   # Python dependencies (managed via uv)
│   └── web/                 # Next.js Frontend Application
│       ├── app/             # Dashboard, session studio, transcripts
│       ├── lib/okf.ts       # Server-side OKF Engine REST client
│       └── components/      # Shadcn UI, Audio Orb, and Voice controls
├── vendor/
│   └── okf-engine/          # Go-based OKF concept graph & retrieval service
│       ├── bundles/         # Knowledge & directive bundles (small, medium, large, directives)
│       └── okf              # Precompiled engine binary
├── go.work                  # Go workspace configuration
└── chat_history.db          # Unified SQLite transcript & session database

---

Quick Start Setup

1. Prerequisites & Environment

Configure .env.local inside apps/agent/.env.local:

LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
GROQ_API_KEY=your_groq_api_key
OKF_ENGINE_URL=http://localhost:8080
OKF_GRPC_ADDR=localhost:50051

And apps/web/.env.local:

LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
OKF_ENGINE_URL=http://localhost:8080
NEXT_PUBLIC_APP_URL=http://localhost:3000

---

2. Start the Local LiveKit Server

Run LiveKit Server locally:

livekit-server --config ~/.sai-local/livekit.yaml

---

3. Start the OKF Engine

Start the in-memory OKF database and gRPC server:

cd vendor/okf-engine
./okf serve --bundle bundles/small/climatechange --port 8080 --grpc-port 50051

---

4. Run the Python Voice Agent

Install dependencies and start the agent:

cd apps/agent
uv sync
uv run python src/agent.py dev

---

5. Run the Next.js Frontend

cd apps/web
bun install      # or npm install
bun run dev      # or npm run dev

Open http://localhost:3000 in your browser. Select an OKF knowledge bundle or directive bundle, upload a document, and talk in real time!