Gesture-Controlled Virtual Mouse

A computer-vision project that allows users to control mouse movement through real-time hand gestures captured by a webcam.

The project uses Python, OpenCV, MediaPipe and PyAutoGUI to detect hand landmarks, interpret gestures and translate them into mouse actions.

This is an educational and experimental project created to demonstrate real-time hand tracking and human-computer interaction using computer vision.

Features

• Real-time hand detection
• Hand-landmark tracking
• Cursor movement using finger position
• Mouse-click gestures
• Webcam-based interaction
• On-screen landmark visualisation
• Frame-rate monitoring

Only gestures present in the current Python implementation should be considered supported.

Technology Stack

• Python
• OpenCV
• MediaPipe
• PyAutoGUI
• NumPy

Project Structure

gesture-controlled-virtual-mouse/
├── HandTracker.py
├── HandTrackingModule.py
├── assets/
│   └── .gitkeep
├── README.md
├── requirements.txt
├── .gitignore
└── LICENSE

HandTracker.py

The main application file. It opens the webcam, uses the supporting module, reads landmark positions, interprets gestures and performs supported mouse actions.

HandTrackingModule.py

The reusable supporting module responsible for initialising MediaPipe, processing frames, drawing landmarks and returning landmark positions.

If your supporting module has a different filename, update this README and ensure the import inside HandTracker.py matches it.

Installation

git clone https://github.com/namanofficial10/gesture-controlled-virtual-mouse.git
cd gesture-controlled-virtual-mouse
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

On Windows:

python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt

Run

python HandTracker.py

On some systems:

python3 HandTracker.py

Allow camera access when prompted.

How It Works

1. OpenCV captures webcam frames.
2. MediaPipe detects the hand and landmark points.
3. The supporting module returns landmark coordinates.
4. The main file evaluates finger positions and gestures.
5. Webcam coordinates are mapped to the screen.
6. PyAutoGUI performs the corresponding mouse action.

Demo

A demo is not currently included.

Later, add a GIF at:

assets/demo.gif

Then include:

![Gesture-Controlled Virtual Mouse Demo](assets/demo.gif)

Hide notifications and personal information before recording.

Privacy

• Webcam frames are processed locally.
• This project is not intended to upload or store camera footage.
• Review the source before running and confirm no recording or network logic has been added.
• Camera permissions can be revoked after use.

Limitations

• Accuracy depends on lighting and camera quality.
• Cluttered backgrounds may reduce detection quality.
• Fast movement may cause cursor instability.
• Dependency compatibility can vary by Python version.
• This project has not been validated as a production accessibility tool.

Future Improvements

• Cursor smoothing and calibration
• Configurable gestures
• Drag-and-drop support
• Scrolling and right-click controls
• Multi-hand filtering
• Low-light improvements
• Automated tests
• Sensitivity settings
• Desktop packaging
• Demo GIF

Troubleshooting

Webcam does not open

Check camera permissions, close other camera applications and verify the camera index in HandTracker.py.

Import error

Keep both Python files in the same directory and ensure the supporting module filename matches the import statement.

Cursor is unstable

Improve lighting, keep the hand fully visible and adjust smoothing or frame-margin values.

Responsible Use

Use this project only for learning, experimentation and authorised human-computer interaction research.

Author

Naman Sharma

• Portfolio: https://namanofficial10.github.io/
• GitHub: https://github.com/namanofficial10

Licence

This project is available under the MIT Licence. See LICENSE.