Broomhandle Game

Don't you wish you could fly on a broom and collect floating golden rings? What if you could control the broom's flight by magically waving your fingers in the air?

Comment

Inspiration

When thinking about this year's theme "ignite the magic within," I knew I wanted to play with Google's MediaPipe library. Being able to get accurate and relative finger positions from a single camera in 3D space in relatively close to real time via machine learning is just mind blowing! I have always wanted to create a 3D broom flight game, and marrying hand gestures with broom flight felt like the most magical thing I could think of.

What it does

It's a simple collecting game! You're able to play in a variety of ways, but the most magical way is by leveraging your device's built in or wired live camera. I find that it works best on desktop, but it is not impossible to play on a stronger mobile device too. Can you collect all the rings?

Hand Controls

The hand tracking mode is the most interesting, in my opinion. Movement is based on several factors. To move forward, extend your index finger upward and show it to the camera. Notice, how the angle of your finger determines how Byte turns on his broom! Want to rise higher? Extend your middle finger alongside your index finger. With the two extended fingers shown to the camera, Byte will slowly rise. Want to lower byte to towards the ground? simply add your ring finger, so that you are now showing three fingers to the camera. The three fingers tells the system that Byte should descend towards the ground. Want to pause the game? extend your palm towards the camera with all fingers extended, holding your hand thusly tells the game you would like to pause.

Keyboard and Mouse Controls

By choosing the Keyboard and Mouse play mode, you can leverage the WASD keys as well as the mouse to maneuver our broom riding pooch.

Gamepad Controls Mode

By selecting the Gamepad mode, you can leverage a standard XBOX style controller. The angle of the left stick determines the forward thrust, you can also use the left stick for turning Byte. The L1 and R1 buttons trigger ascention, while the L2 and R2 buttons descend the character towards the ground.

How I built it

I built a basic scaffolding using Svelte and SvelteKit, hosted it on Railway, and leveraged Google's MediaPipe library as well as the Threlte ecosystem, which itself leverages Three.js and the Rapier physics engine. Once I had a working baseline, I brought Claude and other AI models in to modify parts, investigate bugs, and develop features into the game you have before you.

Since I had more time than I originally thought, I decided to add background music:

Abstraction Music Loop Bundle on Itch.io

NOTE: This asset bundle is licensed as Public Domain (CC-0 - https://creativecommons.org/publicdomain/zero/1.0/)

I also made a custom sound effect for when you pick up a ring using the FL Studio Mobile App for Android.

Challenges I ran into

Getting all the various parts to work together was a challenge initially. MediaPipe is not a simple library to integrate, but once I was able to get reliable data from it, things sorta fell into place. When I was integrating game-pad controls, the AI models had different assumptions as to how the controller was mapped, so I had to manually investigate the button layout by hand, which took a while.

I also wanted to incorporate the UGA Hacks mascot, Byte as the main character, but I had never authored a 3D model and put it in a game before. The existing GLTF ecosystem provided by Threlte is buggy at best, so I had to fiddle with the compiled component in order to get it properly imported. The 3D model was made in Nomad Sculpt for Android, drawn on a Samsung Tablet with Wacom pen input.

Accomplishments that I'm proud of

  • This is my very first ever mostly cohesive game! It's very simple, but I think the scope was appropriate for this weekend's events, and a single person team.
  • This was my second ever Hackathon, and the first time I've made something like this solo.
  • I love the 3D model of Byte! SO CUTE! I will def leverage Nomad Sculpt more for this in the future.
  • One of my favorite things about the web is that its easy to share what you make with others, I am proud that I could share this with my friends as I was making it.

What I learned

  • Don't try and do too much!
  • Get lots of sleep!
  • Be early for meals, not late.
  • For MediaPipe in particular, set the number of hands to 2, since it will ignore any additional hands that it sees, and the first one it notices might not be the one you want it to have seen.

What's next for the Broomhandle Game?

  • More levels!
  • Better graphics and shaders!
  • More player characters, brooms, and game modes, as well as ways to play the game.
  • I also want to figure out more fun gestures too, and maybe incorporating more than one hand next time too.
  • Magic spell casting?

Built With

Share this project:

Updates