The open-source, ultra-low latency Speech-to-Speech AI voice agent platform.
Build, deploy, and orchestrate AI phone agents in minutes — not months.
Voicely is a 100% open-source, self-hostable platform for building production-grade AI voice agents. It connects real-time telephony (Twilio), state-of-the-art LLMs (Gemini, Groq), and ultra-low latency TTS (Cartesia, Sarvam AI) into a single, seamless platform — with a beautiful dashboard, an analytics engine, and a full developer API.
We built Voicely because powerful voice AI shouldn't be locked behind $10,000/month enterprise contracts. Fork it, self-host it, extend it.
Incoming Call → Twilio Media Streams → Deepgram STT → Gemini/Groq LLM → Cartesia/Sarvam TTS → Caller
↕
Live Dashboard Sync (WebSocket)
| Feature | Description |
|---|---|
| Voice Agent Builder | Create custom AI personas with system prompts, dynamic Q&A scripts, and per-agent voice/language selection. |
| Outbound Campaign Manager | Upload a CSV of contacts and run fully automated bulk outbound AI call campaigns. |
| Ultra-Low Latency S2S | Real-time bidirectional audio streaming over WebSockets for natural, sub-second conversational AI. |
| Analytics & Evaluation Engine | Post-call semantic analysis — sentiment, intent tier, objection tracking, and AI-scored lead qualification. |
| Voice Sandbox | Test any agent live in your browser before deploying to real phone calls — no phone number required. |
| Developer API | Generate API keys and trigger calls, manage agents, and query analytics programmatically. |
| Telegram Intel Bridge | Receive live call summaries and AI evaluation results directly in your Telegram. |
| BYOK (Bring Your Own Keys) | Connect your own Twilio, Cartesia, Gemini, and Deepgram keys from the Settings UI. Zero vendor lock-in. |
| Multi-Language Support | English, Hindi, and regional Indian languages via Cartesia and Sarvam AI voice engines. |
Voicely uses a deliberately simple monolithic architecture — a single Node.js/Express backend and a React SPA frontend. This makes it trivial to understand, fork, and self-host.
The REST API is completely stateless (all session data in MongoDB), so horizontal scaling behind a load balancer works out of the box.
graph TD
subgraph "Frontend (React + Vite + Tailwind)"
FE["Dashboard & Analytics UI"]
WS_CLIENT["Voice Sandbox WebSocket Client"]
end
subgraph "Backend (Node.js / Express)"
REST["REST API Gateway\n/api/auth, /api/calls, /api/modules"]
TWILIO_WS["Twilio Media Stream Handler\n/api/streams/twiml · /api/streams/media"]
LIVE_WS["Live Call Sync Server\n/live-call (JWT WebSocket)"]
SCHEDULER["Bulk Call Scheduler (Bull Queue)"]
end
subgraph "AI Processing Layer"
STT["Deepgram Nova-2 (Real-time STT)"]
LLM["Gemini 1.5 Flash / Groq (LLM)"]
TTS["Cartesia Sonic / Sarvam AI (TTS)"]
EVAL["Post-Call Evaluation Engine"]
end
subgraph "Data Layer"
DB[("MongoDB Atlas")]
CACHE["Redis (Audio + Session Cache)"]
end
subgraph "External Services"
TWILIO["Twilio Voice API"]
TELEGRAM["Telegram Bot API"]
end
FE <-->|HTTP + JSON| REST
WS_CLIENT <-->|WebSocket| LIVE_WS
REST <-->|HTTP| TWILIO
TWILIO <-->|WebSocket Media Stream| TWILIO_WS
TWILIO_WS --> STT --> LLM --> TTS --> TWILIO_WS
TWILIO_WS --> LIVE_WS
EVAL --> TELEGRAM
REST <--> DB
TWILIO_WS <--> CACHE
SCHEDULER --> REST
- Outbound Trigger: A campaign or API call hits
/api/calls/initiate, which calls the Twilio REST API to dial the customer. - Media Stream Established: Twilio calls back to
/api/streams/twiml, which returns TwiML instructing Twilio to open a WebSocket to/api/streams/media. - Real-Time STT: Incoming audio chunks (G.711 μ-Law) stream to Deepgram Nova-2 over a persistent WebSocket connection.
- LLM Response: Deepgram final transcripts are dispatched to Gemini 1.5 Flash (or Groq) with the agent's system prompt and conversation history.
- TTS Synthesis: The LLM response is streamed to Cartesia (or Sarvam AI) and the synthesized audio is immediately piped back through Twilio — achieving barge-in-capable, fully duplex conversation.
- Live Dashboard Sync: The
/live-callWebSocket server broadcasts real-time transcripts and events to any connected dashboard session. - Post-Call Evaluation: On call completion, a final evaluation prompt is sent to Gemini to analyze the conversation — generating intent tier, sentiment, and a structured lead qualification result.
| Layer | Technology |
|---|---|
| Runtime | Node.js 18+, ES Modules |
| Framework | Express.js |
| WebSockets | ws library (Twilio Media + Live Dashboard) |
| Database ORM | Mongoose 8 |
| Authentication | JWT (RS256), Google OAuth 2.0 |
| Logging | Pino (structured JSON logging) |
| Security | Helmet, express-rate-limit |
| Validation | Zod |
| Layer | Technology |
|---|---|
| Framework | React 18, TypeScript |
| Build Tool | Vite 7 |
| Styling | Tailwind CSS v4 |
| Data Fetching | TanStack Query (React Query) v5 |
| Routing | React Router v7 |
| Auth | @react-oauth/google |
| Provider | Role |
|---|---|
| Deepgram Nova-2 | Real-time Speech-to-Text (STT) |
| Google Gemini 1.5 Flash | Primary LLM (reasoning + evaluation) |
| Groq (Llama 3) | Alternative low-latency LLM |
| Cartesia Sonic | Ultra-low latency TTS (English) |
| Sarvam AI | Regional language TTS (Hindi + Indian languages) |
| Google Cloud TTS | Fallback TTS provider |
| Twilio Voice | Telephony backbone (PSTN calls + Media Streams) |
- Node.js v18 or higher
- MongoDB instance (MongoDB Atlas M0 Free is sufficient to start)
- Twilio Account with a phone number and Media Streams enabled
- Deepgram API Key (for real-time STT)
- Gemini API Key (
gemini-1.5-flash) - Cartesia API Key (for English TTS)
git clone https://github.com/AbhigyanRaj/Voicely.git
cd VoicelyCreate backend/.env from the example:
cp backend/.env.example backend/.envThen populate your values:
# Core
PORT=5001
NODE_ENV=development
JWT_SECRET=your_super_secret_jwt_key_minimum_32_chars
# Database
MONGODB_URI=mongodb+srv://<user>:<password>@cluster.mongodb.net/voicely
# Google OAuth (for dashboard login)
GOOGLE_CLIENT_ID=your_google_oauth_client_id
# Optional: Redis (for audio caching - falls back to in-memory if not set)
REDIS_URL=redis://localhost:6379Note: Twilio, Cartesia, Deepgram, and Gemini API keys are configured per-workspace directly from the Voicely Settings UI and stored encrypted in MongoDB. You do not need to hardcode them in
.env.
Install dependencies and start the dev server:
cd backend
npm install
npm run devThe API server will start on http://localhost:5001.
Create frontend/.env:
VITE_API_URL=http://localhost:5001/api
VITE_WS_URL=ws://localhost:5001
VITE_GOOGLE_CLIENT_ID=your_google_oauth_client_idInstall dependencies and start:
cd frontend
npm install
npm run devThe dashboard will be available at http://localhost:5173.
Live phone calls require Twilio to reach your local backend. Use ngrok or Cloudflare Tunnel to expose port 5001:
ngrok http 5001In your Twilio Console, update your phone number's voice webhook to:
https://<your-ngrok-id>.ngrok.io/api/calls/twiml
For the Voice Sandbox (browser-based), also update the browser stream URL to:
https://<your-ngrok-id>.ngrok.io/api/streams/twiml
Once both servers are running, navigate to http://localhost:5173:
- Sign in with Google or create an email account.
- Go to Settings → add your Twilio, Gemini, Cartesia, and Deepgram credentials.
- Go to My Agents → create your first voice module.
- Try the Voice Sandbox to test your agent live in the browser.
- Go to Campaigns → upload a CSV and launch an outbound campaign.
Voicely/
├── backend/
│ └── src/
│ ├── controllers/ # Route handlers (auth, calls, modules, analytics)
│ ├── middleware/ # JWT auth, rate limiting, request validation
│ ├── models/ # Mongoose schemas (User, Call, Module, Lead, ...)
│ ├── routes/ # Express route definitions
│ ├── services/ # Core business logic
│ │ ├── streamingCallHandler.js # Main real-time S2S pipeline
│ │ ├── hybridTTS.js # Multi-provider TTS orchestrator
│ │ ├── cartesiaService.js # Cartesia Sonic integration
│ │ ├── sarvamService.js # Sarvam AI integration
│ │ ├── deepgramService.js # Deepgram STT integration
│ │ └── schedulerService.js # Bulk campaign queue
│ ├── utils/ # Logger, cache, DB utils, env validator
│ ├── validators/ # Zod validation schemas
│ └── websocket/
│ └── liveCallServer.js # JWT-secured live transcript WebSocket
│
├── frontend/
│ └── src/
│ ├── components/ # All React UI components
│ │ ├── AnalyticsPage.tsx # Full analytics dashboard
│ │ ├── CreateModule.tsx # Voice agent builder wizard
│ │ ├── VoiceSandbox.tsx # In-browser live test interface
│ │ └── LiveCallModal.tsx # Real-time call monitor
│ ├── contexts/ # React contexts (AuthContext)
│ ├── lib/ # API helpers, auth, TTS config, queryClient
│ └── pages/ # Page-level route components
│
├── API.md # Full REST API documentation
├── CONTRIBUTING.md # Contribution guidelines
├── render.yaml # Render.com deployment config
└── vercel.json # Vercel deployment config
Backend → Render:
# render.yaml (already included in repo)
services:
- type: web
name: voicely-backend
env: node
buildCommand: cd backend && npm install
startCommand: cd backend && npm startPush to GitHub and connect your repo in the Render Dashboard. Add your environment variables in the Render service settings.
Frontend → Vercel:
cd frontend
npx vercel --prodSet VITE_API_URL to your Render backend URL in the Vercel project settings.
For production workloads, the monolithic architecture scales horizontally behind an Application Load Balancer with sticky sessions for WebSockets.
| Component | Technology | Projected Capacity |
|---|---|---|
| API + Dashboard | AWS ECS / Fargate (auto-scaling) | 100,000+ concurrent users |
| WebSocket (Live Calls) | ALB with sticky sessions | 5,000–20,000+ concurrent streams |
| Database | MongoDB Atlas M40+ (dedicated) | Unlimited (sharded) |
| Cache | AWS ElastiCache (Redis) | Sub-millisecond audio cache |
Voicely exposes a full REST API for programmatic control. Generate an API key from the Developer tab in the dashboard.
# Trigger an outbound AI call
curl -X POST https://your-voicely-instance.com/api/calls/initiate \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"moduleId": "your_module_id",
"customerName": "Priya Sharma",
"phoneNumber": "+919876543210"
}'Contributions are what make open source great. All contributions are welcome — from bug fixes and docs to new TTS adapters or LLM integrations.
- Fork the repository.
- Create a branch:
git checkout -b feat/your-feature-name - Make your changes and write clear, descriptive commits.
- Push your branch:
git push origin feat/your-feature-name - Open a Pull Request — describe what you changed and why.
Check the Issues tab for issues labeled good first issue. These are the best place to start.
- New TTS Adapters — ElevenLabs, OpenAI TTS, Azure Cognitive Services
- New LLM Adapters — Claude, Mistral, local Ollama models
- Internationalization — More regional language support
- Mobile Dashboard — React Native companion app
- Test Coverage — Unit and integration tests for the call pipeline
Distributed under the MIT License. You are free to use, modify, distribute, and self-host Voicely — for personal or commercial projects — without any fees or restrictions.
See LICENSE for the full text.
Built by the Voicely Community
Star this repo if Voicely helps you ship faster — it helps more developers find the project.
