StrumSpace

Our Beautiful Landing page, which also has it's very own 3D Guitar model embedded in it.
Our dashboard page with all the information and features of the app summarized
Our AI Model detecting frets using Computer vision and anotating the frets for player
Live playing of chords and correction with our AI Models!
Here is the song catalog for us to choose a song from some sample songs

Inspiration

We were inspired to create StrumSpace by our shared struggles learning guitar. As a team of tech enthusiasts and musicians, we realized that many beginners face challenges with finger placement, chord transitions, and timing. Existing tools lacked real-time, interactive feedback, so we set out to build a smarter solution. By combining computer vision, AR, and audio analysis, we aimed to make practice sessions more engaging and effective for self-learners and hobbyists alike.

How we built it

We built StrumSpace by combining computer vision, AR, and audio analysis into a real-time interactive tool. Using YOLOv8, we trained a custom object detection model to recognize fretboard zones from a webcam feed. We overlaid chord shapes as visual dot guides aligned with the detected frets, helping users place their fingers correctly. To verify performance, we integrated real-time audio detection that listens for the correct chord before allowing progression. The entire system was developed in Python with OpenCV for live video processing, and the model was trained and fine-tuned on Google Colab using a labeled dataset. Through collaboration and iterative testing, we built a cohesive experience that bridges music learning and immersive technology.

Challenges we ran into

While building StrumSpace, we faced several challenges. Training a YOLOv8 model to detect fretboard zones was difficult due to subtle visual differences and varying lighting conditions. Achieving accurate alignment between the detected fret zones and the AR chord overlays required careful calibration and geometric reasoning. On the audio side, implementing real-time chord detection was tricky, especially filtering background noise and ensuring reliable pitch recognition. We also encountered integration issues when combining computer vision, audio processing, and UI in a single pipeline without performance drops. Debugging these components as a team took time and coordination, but overcoming these challenges ultimately strengthened our system and collaboration.## Accomplishments that we're proud of We’re proud of several key accomplishments in building StrumSpace. First, we successfully trained and deployed a custom YOLOv8 model that accurately detects individual fret zones in real time—a challenging task given the subtle visual differences across frets. We also implemented an interactive chord overlay system that dynamically guides users with visual cues, aligned perfectly to their guitar’s fretboard. Additionally, integrating real-time audio verification added an extra layer of intelligence, ensuring users receive immediate feedback based on what they play. Most importantly, we’re proud of how we collaborated as a team across disciplines to turn a complex idea into a working, intuitive AR learning tool.

What we learned

Throughout the development of StrumSpace, we learned how to integrate diverse technologies into a single, cohesive system. We deepened our understanding of computer vision by training and fine-tuning a custom YOLOv8 model for a niche use case. We gained experience in designing real-time AR overlays that align dynamically with detected physical objects. On the audio side, we explored real-time sound analysis and how to match audio input with expected outcomes. We also learned how to optimize system performance to handle video, detection, and audio processing simultaneously. Most importantly, we learned the value of collaboration, iterative problem-solving, and adapting quickly when technical challenges arose.

What's next for StrumSpace

Next for StrumSpace, we plan to enhance its capabilities and user experience. We aim to expand chord support to include barre chords and custom user-defined shapes. We also want to introduce strumming pattern recognition to provide rhythmic feedback, not just chord accuracy. Another key goal is to develop a progress tracker that visualizes user improvement over time. On the technical side, we plan to optimize the system for mobile and AR headsets, making it more accessible and immersive. Eventually, we envision StrumSpace becoming a full-fledged virtual tutor with lesson modules, feedback loops, and real-time multiplayer practice modes for collaborative learning.