Jazzhands

• 

  Jazz Hands Screensaver (Hands up to play!)

• 

  Jazz Hands Main Menu (3 Levels to choose from)

• 

  Jazz Hands Thriller Level, match the hand symbols to score points

• 

  Jazz Hands Physical Arcade Machine

• 

  Jazz Hands Tutorial

Team 29 (Toby Clark & Amber Swarbrick)

Inspiration

Both of being fans of Guitar Hero and Just Dance, we wanted to experiment with what Computer Vision could look like in a game within a retro capacity. Sort of like the arcade machine from the future!

What it does

Jazzhands is an arcade machine with no physical controls. Instead, the user uses their hands to control the menus and play 3 bespoke, rhythm based levels. Similar to Guitar Hero, the user must match the gesture on the screen with their own hands, earning points. It was designed for ease of use, with user tutorials at the start, and intuitive gameplay throughout. (Although, you'd probably have to play it to see this...)

How we built it (boring techy stuff!)

JazzHands was constructed in GameMakerStudio2, communicating over TCP to a Python3, OpenCV & Mediapipe Computer Vision Server. All assets were made/modified in Aseprite, with audio being edited with Audacity.

How we built it (epic cardboard thing!)

1. Obtain Cardboard
2. Spill glue on the cardboard
3. Construct monolithic pillars of card in tribute to a deity.
4. Stick and repeat.
5. Make it pretty (Optional).
6. Insert the brain (Macbook Pro M1 2022)

Challenges we ran into

We wanted to create an end-to-end retro experience, which drove us to build a physical arcade machine. If there was a record for longest time spent walking in a Hackathon, I think we would've won. Taking a 1hour 30 trip to buy physical supplies on-top of developing the game itself was certainly a struggle.

One technical challenge we managed to overcome was the lack of support for webcam/CV within GameMaker. In true Hackathon style, we bodged a client-server connection and ran the CV externally to the game, communicating through sockets.

Accomplishments that we're proud of

In this hackathon we took the extra time to polish the game, in pursuit of providing optimal user experience. We felt this seamlessly integrated the gesture control, to the point of actions becoming second-nature when playing the game. (Thanks Aseprite!)

What we learned

We both went into this with little experience of computer vision in video games, but managed to get a prototype working quite quickly. Learning tools like OpenCV, Mediapipe, and (now removed from the repo), Tensorflow. We're both fortunate that such great tools are readily available to developers.

What's next for Jazzhands

We both had an absolute whirlwind designing, developing and testing Jazzhands. We were quite skeptic at the start but could not be happier with how it turned out. This project has surprised us, educated us, and made us laugh; even after listening to 8bit Creep for the 122nd time with 0 sleep. There is absolutely no doubt we will continue working on this, to achieve the following targets:

• Integrate 2 Player Multiplayer
• Fix the Jank.
• At some point in the far future, Steam?
• Leaderboard! (Global?)

Built With

• aseprite
• gamemakerstudio2
• mediapipe
• opencv
• python