Parakeet Lip Sync

Parakeet Lip Sync is an open-source automatic lip-syncing tool for 2D animation. It analyzes audio recordings and generates mouth shape timings that can be exported for use in animation software.

About

Inspired by tools like Papagayo, Yolo, and Rhubarb Lip Sync, Parakeet aims to provide a modern, deep learning-powered alternative that is fully open-source.

This project is under development. Don't be surprised if it crashes 🐛, burns 🔥, eats your lunch 🍔, or fails to behave as it's supposed to 😱.

Features

• Automatic phoneme recognition - Uses deep learning to detect speech sounds
• Real-time waveform visualization - View and navigate audio with zoom/scroll
• Live mouth shape preview - See the current mouth shape during playback
• Export formats:
  • Text output (timestamp, duration, mouth shape)
  • Moho/Anime Studio timesheet (.dat)
• Keyboard shortcuts for efficient workflow
• Cross-platform - Built with DearPyGui

Installation

Using uv (recommended)

# Install uv if you haven't already
# https://docs.astral.sh/uv/getting-started/installation/

# Clone the repository
git clone https://github.com/hnhparitosh/ParakeetLipSync.git
cd parakeet_lipsync

# Run the application
uv run parakeet

Development Setup

# Sync dependencies
uv sync

# Run in development mode
uv run python -m parakeet_lipsync.main

Usage

1. Open an audio file (Ctrl+O) - Supports WAV and MP3 formats
2. Preview the audio - Use playback controls or click the waveform to seek
3. Process the audio - Click "Process Audio" to run phoneme recognition
4. Review the output - Watch the mouth shape preview during playback
5. Export - Save as text or export as Moho timesheet

Keyboard Shortcuts

Shortcut	Action
Ctrl+O	Open audio file
Ctrl+S	Save output
Space	Play/Pause
+	Zoom in
-	Zoom out
0	Fit all (reset zoom)

Roadmap

• Basic GUI with DearPyGui
• Automatic lip-sync using Allosaurus
• Audio waveform visualization
• Zoomable/scrollable waveform
• Real-time mouth shape preview with images
• Moho timesheet export (.dat)
• Keyboard shortcuts
• Support for more export formats (Toei XDTS, etc.)
• Editable timeline for manual corrections
• Custom phoneme-to-mouth-shape mappings
• Batch processing multiple files
• Custom model support (replace Allosaurus)

Tech Stack

• GUI: DearPyGui
• Audio Processing: librosa, sounddevice
• Phoneme Recognition: Allosaurus

Acknowledgments

• Papagayo-NG - Inspiration for design, UI.
• Yolo - Inspiration for UI/UX
• Allosaurus - Phoneme recognition model
• Preston-Blair for the classic mouth shape system used in animation