TeachAR

Inspiration

According to the American Academy of Orthopaedic Surgeons & Nationwide Children, around 20 percent of children experience some kind of forearm fractures, which typically requires 3 to 6 months to heal. These temporary (and permanent) disabilities can negatively impact children’s education in a period that is most critical for child development and learning. TeachAR seeks to address this problem by making learning more accessible. Moreover, TeachAR utilizes life gamification to reinforce learnings, making TeachAR a perfect tool for learning regardless of the user’s abilities. Furthermore, we hope to utilize the connected lenses functionality to create a shared experience that is unique to the AR world.

What it does

TeachAR utilizes the speech to text functionality to allow users to verbally answer questions. For instance, one type of question TeachAR may ask users includes identifying spelling errors. The users can then state which word they believe is spelled incorrectly and the app will respond accordingly. This discards the need of physically writing down the answers, allowing those who physically cannot write the ability to effectively learn. Additionally, with the Snap Spectacles we can tangibly reward users for answering questions correctly through points and cool animations. Thus, gamifying learning by enhancing the feelings of success each time we complete a question. This type of gamification has been shown to be advantageous for motivating behavior improvements.

How we built it

To successfully implement TeachAR, we utilized the VoiceML model through Lens Studio in congruence with Javascript and various art assets. First we planned the user experience, we wanted users to be greeted by a start screen before entering the game and an end screen for when the users finished the game. After we finalized our user flow, we extracted keywords from the user's voice input using the VoiceML model to trigger certain responses (based on whether the user was correct or not). Finally, we put everything together by implementing the transitions between multiple scenes.

Challenges we ran into

Since this is the first time we used Lens Studio there was a big learning curve for our team. We struggled with finding which assets would work well for the spectacles. This includes the vfx assets and understanding how the VoiceML worked behind the scenes.

Accomplishments that we're proud of

However, we were able to overcome these challenges by being resilient, flexible and being quick on the uptake of learning. We are most proud of how we were able to create a working application starting from zero knowledge on Lens Studio. Moreover, the application can be later scaled to diverse domains.

What we learned

Through this experience, we were able to become familiar with various aspects of Lens Studio. This includes the VoiceML model, the vfx assets and Javascript.

What's next for TeachAR

We hope to utilize the new skills we learned to further advance TeachAR by transforming it into a shared connected experience. We would also include different types of questions, multiple academic disciplines, and multi language support. Doing so will make TeachAR even more accessible and educationally helpful. Moreover, by making TeachAR a multi user experience in AR we can make learning an exciting experience that people of all ages can look forward to.

Built With

• javascript
• lensstudio
• vfx
• voiceml