FaceFilter AR

FaceFilter AR is a browser-based augmented reality experiment where facial gestures control gameplay. Using MediaPipe FaceMesh, the system tracks facial landmarks in real time and detects when the player opens their mouth to catch falling objects.

The project explores how computer vision can be used as a natural input interface for playful human-computer interaction directly in the browser.

Try the game directly in your browser. Allow camera access and catch the falling items using your mouth.

Camera permission is required for the demo.

The system tracks facial landmarks and measures the distance between lip points to detect when the mouth is open. This gesture becomes the interaction mechanism used to catch falling items in the game.

• MediaPipe FaceMesh
• WebRTC Webcam
• HTML Canvas Rendering
• JavaScript Game Logic
• Web Audio API

The main challenge was connecting real-time facial tracking with responsive gameplay mechanics. The system continuously analyzes mouth movement and calculates proximity between the detected mouth position and falling objects rendered on the canvas.

Difficulty increases dynamically by adjusting spawn rate and object speed as the player's score increases.

The result is a playful AR interaction where users control gameplay entirely through facial gestures. The project demonstrates how browser-based computer vision can enable lightweight augmented reality experiences without requiring native applications.