Embeddable Open-Source LLM chat built with Nim

nimble install chachachat

API reference

😍 Key Features

• Fast, compiled, embeddable into other apps
• Persistent storage of conversations and documents
• Embeddable into other programming languages via Native extensions
• Support for multiple LLM providers (Any OpenAI-compatible APIs)
• RAG (Retrieval-Augmented Generation) capabilities for enhanced responses
• Agentic function calling (OpenAI-style tools) with async tool handlers
• Easy-to-use API for building chat applications
• Bring your own UI Chatbot framework

🚀 Quick Start

import std/asyncdispatch
import chachachat

proc main(): Future[void] {.async.} =
  let llm = newOpenCodeClient(apiKey = "...", model = "deepseek-v4-flash")

  let response = await llm.chat("Hello, who are you?", proc(chunk: ResponseChunk) =
    if chunk.chunkType == chunkContent and chunk.text.len > 0:
      stdout.write(chunk.text)
  )

waitFor main()

💬 Chat API

Send a message with default options, or stream the reasoning + content chunks yourself:

import std/[asyncdispatch]
import chachachat

proc main(): Future[void] {.async.} =
  let llm = newOpenCodeClient(apiKey = "...", model = "mimo-v2.5")

  var isReasoning = false
  let response = await llm.chat("Write a haiku about Nim", proc(chunk: ResponseChunk) =
    case chunk.chunkType
    of chunkReasoning:
      if chunk.text.len > 0:
        stdout.write(chunk.text)
    of chunkContent:
      stdout.write(chunk.text)
    else: discard
  )

waitFor main()

Conversations

import std/asyncdispatch
import chachachat

proc main(): Future[void] {.async.} =
  let llm = newOpenCodeClient(apiKey = "...", model = "deepseek-v4-flash")
  let conv = newConversation(llm)

  let reply = await conv.sendMessage("Tell me a fun fact about the Netherlands")
  echo reply
  echo conv.getHistory()   # full conversation history
  conv.setTitle("Netherlands facts")

waitFor main()

⚙️ Agentic Function Calling

Define async tool handlers, register them with an Agent, and let the LLM call them in a loop until it reaches a final answer:

import std/[asyncdispatch, sequtils, strutils, strformat]
import pkg/openparser/json
import chachachat

proc getWeather(name: string, arguments: JsonNode): Future[string] {.async.} =
  let location = arguments["location"].getStr("unknown")
  return fmt"It is 22 degrees Celsius in {location} with clear skies."

proc main(): Future[void] {.async.} =
  let llm = newOpenCodeClient(apiKey = "...", model = "deepseek-v4-flash")

  let agent = newAgent(llm)
  agent.addTool(
    name = "get_weather",
    description = "Get the current weather for a given location",
    parameters = %*{
      "type": "object",
      "properties": {
        "location": {"type": "string", "description": "City name, e.g. Amsterdam"}
      },
      "required": @["location"]
    },
    handler = getWeather
  )

  let response = await agent.chat("What's the weather in Amsterdam?")
  echo response.chunks.mapIt(it.text).join("")

waitFor main()

Tool calls stream to your callback as chunkToolCall chunks, so you can show tool activity in your UI. You can also use tools directly with a Conversation via the overloaded sendMessage(input, tools, handlers).

📚 RAG

Chunk documents, embed them, and retrieve the most relevant chunks for a query:

import std/strutils
import chachachat

let doc = "A very long document about the Dutch Golden Age...".repeat(10)

let chunks = chunkDocument(doc, newChunkingOptions(
  strategy = FixedSize, chunkSize = 512, chunkOverlap = 64
))

# embed each chunk (e.g. with llm.embeddings(...)), then:
let top = findTopK(queryEmbedding, chunks, k = 3)
for match in top:
  echo match.chunk.text, " (score: ", match.score, ")"

🧩 Examples

Full runnable examples live in the examples/ directory:

• chat.nim — interactive streaming chat REPL
• functions.nim — agentic function calling with two async tools
• openrouter — usage against the OpenRouter provider

Projects using ChachaChat

Currently I'm integrating ChachaChat in the following projects

• Nimbox Desktop app for 👑 Nim, Nimble + Atlas. Build docs, Manage packages, AI chat with RAG capabilities, and more!
• Clue CLI - Nim CLI toolkit with built-in offline package documentation generator powered by ChachaChat RAG

❤ Contributions & Support

• 🐛 Found a bug? Create a new Issue
• 👋 Wanna help? Fork it!

🎩 License

MIT license. Made by Humans from OpenPeeps.
Copyright OpenPeeps & Contributors — All rights reserved.