Vibe Tracker

Making personal health manageable.

Comment

Inspiration

I have pain in my right shoulder, it has been clearly because of my poor posture sitting in front of my laptop. I wish there would have been an assistant who would have reminded me every time I slouch for more than 15-20 seconds to correct my posture. We realized this could be a good utility application for most of the office workers. MediaPipe offers pose detection and landmarking of the body.

And, A study found that the average office employee spends 1,700 hours a year in front of the computer. i.e. 6 hours a workday. There is good potential to track the emotional wellbeing by measuring the expressions and accordingly

What it does

With the help of live video input,

  • Feature 1: Measures the slouch intensity and notifies if its beyond the threshold. (using Mediapipe)
  • Feature 2: Measures the expressions elicited by browser-window/applications. (using Hume)

How we built it

1st Iteration: We used vanilla HTML/CSS/JS to run the application, soon realising the shortcomings, i.e. only restricted to the browser. Only Posture correction feature was added using Mediapipe. 2nd Iteration: We used Electron to build a native app, which was able integrate both the features that is expression measurements and slouch detection. We used Hume API for expression measurements.

Challenges we ran into

  • We teamed up 24 hours before the deadline. : shortage of time
  • Deciding which platform to go with, considering the permissions and other aspects.

Accomplishments that we're proud of

  • Ethan a full-stack developer tagging with beginner- Akshay.
  • Using Gemini API to calculate the slouch intensity. (This was surly not the intended use case)
  • Breadth of various technology stacks used on the project.

What we learned

  • Learning a novel approach to detect the slouch using Gemini-Mediapipe API.
  • Learned about contributing and working on open-source projects.
  • Team work
  • Working towards deadline.

What's next for Vibe Tracker

  • Gait analysis for detection of early onset of orthopedic conditions.
  • Better UI, be able deploy for end user.
  • Monitor emotional health and produce better insights using Hume's measuring expressions.
Share this project:

Updates