Scale Sense

• 

  Landing page

• 

  Scale page

• 

  Score page

Our Story

As a team of students involved in musical extracurriculars such as UCF Marching Knights, we understand the frustration of learning and practicing endless scales. Mastering your scales is important to improving your skills as a musician, and playing scales is a common requirement for ensemble auditions. As students passionate about music, we built Scale Sense to fill the gap between traditional music practice and technology.

Scale Sense listens to, analyzes, and assesses musical pitches in real-time, using a machine learning JavaScript library ml5.js. Scale Sense will pick up the frequencies of sounds, convert these frequencies to megahertz via CREPE, a machine learning model for estimating pitches, and then convert the megahertz to the appropriate note. Then, it gives a real-time assessment of tuning and accuracy while the student practices, supporting over 40 scales including Major, Minor, Harmonic, and Melodic. Finally, Scale Sense also supports transposition of the scales for all concert instruments. For example, if you play an instrument such as French horn, it is pitched in F. This means a scale in French horn may look different than a scale for another instrument. Scale Sense's ability to transpose scales creates support for all concert instrument players to practice.

Scale Sense is built on a tech-stack comprised of ml5.js, p5.js, and Convolutional Representation for Pitch Estimation (CREPE). ml5.js allows us to take a trained model such as CREPE, and use it's intelligence to make conversions of sound. p5.sound library allows us to filter these sounds. This way, we were able to build a series of algorithms that gather pitches from a sound translator. With these pitches, we conditionally assess if the user is playing the right note compared to the scale they want to practice.

While our team has plenty of experience with music and scales, we were very new to pitch conversion and web audio at the start of the project. The scope of the project was ambitious for us, but we were determined to make an accessible and intuitive product to help other students. During the development of the project, our inexperience with these technologies was a challenge, and we spent many hours familiarizing ourselves with it before we could begin. Additionally, the main assessing algorithm required us to implement complex logical processes to make judgements on note accuracy and give a real-time assessment. There was a lot of trial and error on which logic would output the expected result, but it was well worth it in the end.

Completing an end-to-end user interface that implements the logical algorithms we created, along with including seamless designs and intuitive interactions was a huge accomplishment. Testing with our actual French horn instruments was an incredibly proud moment, as we saw firsthand how applicable this tool is to the everyday life of students like us who are passionate about music. Ultimately, the ability to combine technology and music to help create an accessible and free learning tool for music lovers was an great honor.

What's next for Scale Sense? We know there is a lot you can do with music, and a lot of ways to refine the way you practice music. We aim to continue to refine our application to further help students create productive practicing experiences.

Built With

• bootstrap
• crepe
• css
• html
• javascript
• machine-learning
• ml5.js
• node.js
• p5.js
• p5.sound