Combat Vision

Retro arcade-inspired title screen
GIF
Landing page
In-game snapshot
GIF
Lightsaber gif

Inspiration

Vianna and Ellie have always wanted to create something using computer vision, and we have been inspired by the over-the-top battles in Mortal Kombat. But obviously, punching people in real life wouldn’t be safe or viable. Instead, the duo reimagined the concept into a fun and safe alternative: air sword fighting. Players can swing glowing lightsabers, duel each other, and still feel the thrill of combat without any physical risk. The UI is all retro arcade-inspired, bringing you back in time to experience nostalgic memories paired with modern computer vision technology.

What We Learned

We dove deep into real-time vision processing, learning how to:

Use MediaPipe Pose and Hand Landmarker models for precise body and hand tracking.
Overlay sprites and health bars on an HTML5 Canvas in sync with live motion.
Implement collision detection using simple geometry (line ↔ rectangle intersections).
Optimize performance with techniques like exponential moving averages (EMA) and frame capping.

We also sharpened our skills in front-end design and learned the importance of keeping state updates smooth to avoid flickering during gameplay.

How We Built It

Front-end: React + JavaScript for app structure and state management.
Computer Vision: MediaPipe Tasks Vision to detect poses and hand landmarks.
Rendering: HTML5 Canvas for real-time drawing of hitboxes, health bars, and weapons.
Game Logic: Implemented collision math to detect saber strikes and deal damage.
Creative Assets: All sprites were drawn by hand using Aseprite, making the game feel unique and personal. All art in the game is 100% original creations.

Challenges We Ran Into

Hitbox Calibration: Getting bounding boxes to feel fair and consistent was tricky.
Flickering Weapons: Early versions had lightsabers flickering when detection dropped frames. We solved this by smoothing positions and keeping sprites persistent.
Performance: Running real-time computer vision in the browser required careful optimizations to maintain responsive gameplay and it can only be ran locally as of right now because calling to a public URL causes slow runtimes and problems with responsiveness.