AI-powered classroom engagement monitoring system — built for OOSC 4.0 Hackathon (GDG x IIIT Allahabad) · Theme: "Code for Community — Where Open Innovation Meets Real-World Impact" · Track: PS2 — AI for Equitable Education Access
Under-resourced classrooms often have one teacher managing far more students than they can individually track — disengagement, silent struggling, or a student who's stopped following along can go unnoticed for weeks. ClassroomAI Monitor gives teachers real-time, objective visibility into class-wide engagement (focus, participation, drop-off moments) without adding staff or manual observation time, and keeps a session history so patterns over weeks — not just one class — become visible. That's the equitable-access angle: the same signal a well-staffed school gets from close teacher attention, made available anywhere a webcam and a laptop exist.
- 📷 Detects students via YOLOv8 object detection
- 🧠 Calculates focus score using head pose estimation with MediaPipe
- 🙋 Detects hand raises and sleeping via pose estimation
- 📱 Detects phone usage and penalizes focus score
- 🎙️ Transcribes teacher speech every 30 seconds using OpenAI Whisper
- 📊 Shows everything on a live React dashboard
- 📡 Anyone on the same WiFi can open the dashboard on their phone
- 🎬 Supports both live webcam tracking and pre-recorded video uploads directly from the dashboard
▶️ Allows starting and stopping the pipeline from the dashboard- 📚 Lets users review past sessions in the Session Analysis tab
| Layer | Technology |
|---|---|
| Person Detection | YOLOv8 (Ultralytics) |
| Face Landmarks | MediaPipe FaceLandmarker |
| Head Pose Estimation | OpenCV solvePnP |
| Pose Estimation | MediaPipe PoseLandmarker |
| Speech Recognition | OpenAI Whisper (tiny model) |
| Backend API | FastAPI + Uvicorn |
| Database | SQLite |
| Frontend | React + Vite + Recharts |
- Python 3.13+
- Node.js 18+
- Webcam
- Microphone
- Clone the repo:
git clone https://github.com/Pr1nce-Raj/ClassroomAI-Monitor.git
cd ClassroomAI-Monitor- Double-click
setup.batand wait for it to finish (5–10 minutes)
That's it. Setup only needs to be done once.
Double-click START.bat
- Starts the AI vision pipeline and live dashboard
- Automatically opens the dashboard in your browser
- You can also open
http://localhost:8000manually if needed - To share with others on the same WiFi, use the IP link printed in the terminal
Double-click DASHBOARD.bat
- Starts only the backend and dashboard to view past session data
- No camera or AI pipeline runs
- Open the Session Analysis tab
| Action | What Happens |
|---|---|
| Click "Upload Video" | Switches the pipeline from live webcam to a pre-recorded video file |
| Look at camera | Focus score goes up (green) |
| Look away | Focus score drops (red) |
| Raise hand | HAND RAISED alert appears on the dashboard |
| Hold phone | ON PHONE alert appears and focus is penalized |
| Slump head forward | SLEEPING is detected |
| Speak into mic | Transcript and Q-ratio appear after 30 seconds |
All processing happens on-device. Raw video never leaves the machine. Only numerical scores are stored in the database. Face blur mode is available via PRIVACY_MODE = True in vision/detect.py.
ClassroomAI-Monitor/
├── vision/ ← Camera, YOLO, focus score, pose detection
├── audio/ ← Whisper transcription
├── database/ ← SQLite database functions
├── api/ ← FastAPI backend
├── dashboard/ ← React frontend
├── start.py ← Main launcher
├── setup.bat ← First-time setup
├── START.bat ← Run the live system
└── DASHBOARD.bat ← View past sessions (no camera needed)
A huge shoutout to the minds behind ClassroomAI-Monitor:
| Profile | Name | Role / Contributions |
|---|---|---|
![]() Prince Raj |
Prince Raj | Team Leader — Pitch, features, debugging, repository maintainer |
![]() Md Adnan |
Md Adnan | Senior Developer — Core AI pipeline, vision system, FastAPI backend, React dashboard, primary codebase author |
![]() Vishal |
Vishal | Ideation & Dev — Presentation design, project direction & coordination |





