Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

ClassroomAI Monitor 🎓

AI-powered classroom engagement monitoring system — built for OOSC 4.0 Hackathon (GDG x IIIT Allahabad) · Theme: "Code for Community — Where Open Innovation Meets Real-World Impact" · Track: PS2 — AI for Equitable Education Access

Why This Track

Under-resourced classrooms often have one teacher managing far more students than they can individually track — disengagement, silent struggling, or a student who's stopped following along can go unnoticed for weeks. ClassroomAI Monitor gives teachers real-time, objective visibility into class-wide engagement (focus, participation, drop-off moments) without adding staff or manual observation time, and keeps a session history so patterns over weeks — not just one class — become visible. That's the equitable-access angle: the same signal a well-staffed school gets from close teacher attention, made available anywhere a webcam and a laptop exist.

What It Does

  • 📷 Detects students via YOLOv8 object detection
  • 🧠 Calculates focus score using head pose estimation with MediaPipe
  • 🙋 Detects hand raises and sleeping via pose estimation
  • 📱 Detects phone usage and penalizes focus score
  • 🎙️ Transcribes teacher speech every 30 seconds using OpenAI Whisper
  • 📊 Shows everything on a live React dashboard
  • 📡 Anyone on the same WiFi can open the dashboard on their phone
  • 🎬 Supports both live webcam tracking and pre-recorded video uploads directly from the dashboard
  • ▶️ Allows starting and stopping the pipeline from the dashboard
  • 📚 Lets users review past sessions in the Session Analysis tab

Tech Stack

Layer Technology
Person Detection YOLOv8 (Ultralytics)
Face Landmarks MediaPipe FaceLandmarker
Head Pose Estimation OpenCV solvePnP
Pose Estimation MediaPipe PoseLandmarker
Speech Recognition OpenAI Whisper (tiny model)
Backend API FastAPI + Uvicorn
Database SQLite
Frontend React + Vite + Recharts

Requirements

  • Python 3.13+
  • Node.js 18+
  • Webcam
  • Microphone

Setup (First Time Only)

  1. Clone the repo:
git clone https://github.com/Pr1nce-Raj/ClassroomAI-Monitor.git
cd ClassroomAI-Monitor
  1. Double-click setup.bat and wait for it to finish (5–10 minutes)

That's it. Setup only needs to be done once.

Running the System

Live Session (Camera Required)

Double-click START.bat

  • Starts the AI vision pipeline and live dashboard
  • Automatically opens the dashboard in your browser
  • You can also open http://localhost:8000 manually if needed
  • To share with others on the same WiFi, use the IP link printed in the terminal

Session Analysis (Offline Mode)

Double-click DASHBOARD.bat

  • Starts only the backend and dashboard to view past session data
  • No camera or AI pipeline runs
  • Open the Session Analysis tab

Demo Guide

Action What Happens
Click "Upload Video" Switches the pipeline from live webcam to a pre-recorded video file
Look at camera Focus score goes up (green)
Look away Focus score drops (red)
Raise hand HAND RAISED alert appears on the dashboard
Hold phone ON PHONE alert appears and focus is penalized
Slump head forward SLEEPING is detected
Speak into mic Transcript and Q-ratio appear after 30 seconds

Privacy

All processing happens on-device. Raw video never leaves the machine. Only numerical scores are stored in the database. Face blur mode is available via PRIVACY_MODE = True in vision/detect.py.

Project Structure

ClassroomAI-Monitor/
├── vision/          ← Camera, YOLO, focus score, pose detection
├── audio/           ← Whisper transcription
├── database/        ← SQLite database functions
├── api/             ← FastAPI backend
├── dashboard/       ← React frontend
├── start.py         ← Main launcher
├── setup.bat        ← First-time setup
├── START.bat        ← Run the live system
└── DASHBOARD.bat    ← View past sessions (no camera needed)

Demo Video

Watch on YouTube

Screenshots

Live Dashboard

Live Dashboard

Seat Heatmap

Seat Heatmap

Session Analysis

Session Analysis

👥 Contributors

A huge shoutout to the minds behind ClassroomAI-Monitor:

Profile Name Role / Contributions

Prince Raj
Prince Raj Team Leader — Pitch, features, debugging, repository maintainer

Md Adnan
Md Adnan Senior Developer — Core AI pipeline, vision system, FastAPI backend, React dashboard, primary codebase author

Vishal
Vishal Ideation & Dev — Presentation design, project direction & coordination

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages