Handora

Handora turns rehab into play. Using wearable sensors and AI, it tracks finger motion and guides patients through music-based games - making recovery engaging, measurable, and empowering for all ages.

Comment

💡 Inspiration

Handora began with a story that touched our team. One of our members shared about his uncle, who lives with ALS and gradually lost muscle control. Once a passionate pianist, he found rehabilitation exercises repetitive and disheartening. We asked ourselves, what if therapy could feel like playing music again?
Research shows that music and gaming stimulate neuroplasticity, helping the brain rewire after stroke, spinal cord injury, or ALS. Our goal: make rehabilitation engaging, gamified, and emotionally meaningful.

🎮 What it does

Handora is a wearable rehabilitation system that helps patients restore hand movement through interactive games.

  • A smart glove tracks finger and wrist motion using flex sensors and an IMU (BNO055).
  • Data is processed to compute rehab metrics like range of motion, smoothness, and consistency.
  • Our AI assistant, “Dora,” offers real-time feedback, encouragement, and voice-based navigation, making it accessible even for users with limited mobility.

Each movement becomes part of a music-based game, turning repetitive rehab into a rewarding, creative experience.

⚙️ How we built it

  • Hardware: Custom-designed glove using flex sensors + IMU for accurate angle tracking.
  • Backend: FastAPI with Supabase for data storage and metrics computation.
  • Frontend: Next.js + Pixi.js for real-time game rendering and visualization.
  • Arduino/Hardware: (https://github.com/Jhua03/natHack) for flex glove
  • Communication: BLE (Bluetooth Low Energy) for low-latency streaming between glove and app.
  • AI: OpenAI + Whisper integration for Dora’s voice assistant.

We implemented calibration flows, adaptive thresholds, and real-time graphs using WebSockets.

🧩 Challenges we ran into

  • Ensuring stable, low-noise sensor readings under varying conditions
  • Handling BLE disconnections and maintaining synchronization between data streams
  • Designing an ergonomic, comfortable glove suitable for long-term wear
  • Achieving smooth gameplay with low latency between motion and visual feedback

🏆 Accomplishments that we're proud of

  • Developed a fully functional prototype integrating hardware, software, and AI
  • Achieved real-time motion tracking with <50 ms latency
  • Created a calibration and analytics pipeline tailored to individual patient abilities
  • Made therapy fun, personalized, and data-driven, encouraging continued rehab at home

📚 What we learned

We learned the power of interdisciplinary collaboration—how hardware, software, and design converge to create meaningful health tech. We gained hands-on experience with Bluetooth communication, signal processing, and real-time UX design for accessibility.

🚀 What's next for Handora

We plan to:

  • Expand the game library with adaptive difficulty and personalized music therapy
  • Conduct pilot testing with rehabilitation clinics and ALS communities
  • Integrate cloud-based progress tracking and clinician dashboards
  • Develop multi-sensor calibration for fine motor control analysis

Our vision is to make at-home rehabilitation engaging, measurable, and empowering—for everyone.

Built With

Share this project:

Updates