SignGloves

• 

  SignGloves

• 

  ESP32 bracket CAD

• 

  wrist strap CAD 1

• 

  wrist strap CAD 2

• 

  full device CAD

• 

  soldered sensors

• 

  translation page

• 

  voice matching

• 

  login page

Inspiration

American Sign Language is a primary language for many Deaf individuals, yet communication barriers still exist in everyday interactions. These barriers often limit independence, accessibility, and inclusion. We were inspired to use science and technology to reduce this gap by building a wearable, real-time translation system for Deaf users to communicate without relying on interpreters or impersonal tools. Our goal was not only translation, but to improve equity for the Deaf community by harnessing technology to support accessibility and inclusion.

What it does

SignGloves is a wearable device that translates sign language into spoken words in real time. By detecting finger touch combinations, the system converts signed letters and words into full sentences and speaks them aloud using ElevenLabs AI speech synthesis.

How we built it

On the hardware side, we designed a custom wrist-mounted bracket in OnShape to house the ESP32 microcontroller, along with an adjustable pin-and-tuck strap system to fit different wrist sizes. The entire assembly was 3D-printed in TPU95 on a BambuLab P1P for comfort and flexibility, secured with M3 heat inserts in the bracket to mount the ESP32 without damaging the printed part. For the sensors, we designed and manufactured our own touch sensors. We mapped our hands tracking our range of motion across our palm and backs of our fingers to find the optimal positions for the sensors. Then we manufactured these sensors using a waterjet cutter out of aluminum. Each pad was then wired and soldered to connect to the ESP32’s touch-sensing pins, allowing finger contact combinations to be detected.

On the software side, we build a web application using Next.js for the frontend and Supabase for authentication and session handling. Each touch sensor combination was mapped to ASL letters and common words, then encoded as binary bitmasks within the ESP32’s C++ firmware. The ESP32 streams gesture data through Node.js WebSocket bridge to the browser in real time, where letters are assembled into words and sentences.

Challenges we ran into

One of our main challenges was designing and implementing our own touch sensors from scratch. We had never previously worked with capacitive touch sensing on the ESP32 so determining sensitivity thresholds and ensuring consistent detecting required a lot of experimentation and iteration.

ElevenLabs Integration

One of the main goals of our project was to use ElevenLabs and make it a focal part of our system. After signing up for the provided credits, we were impressed by the range of capabilities available beyond basic text-to-speech. We used the list-similar-voices endpoint in a socially meaningful way by allowing Deaf users to find a voice that sounds like them, rather than relying on a generic default voice. This enabled us to create a system that doesn't just speak for you, it speaks like you. The text-to-speech endpoint was used to convert signed gestures into spoken output in real time. Recognizing that ElevenLabs is more than just a text-to-speech platform, we intentionally explored its more niche features, such as voice customization and voice design to enable users to further customize their voice to make it truly their own. We are especially proud that this integration goes beyond functionality to prioritize user identity and representation, making the technology feel personal thanks to ElevenLabs.

What we learned

Through SignGloves, we learned how to design and manufacture wearable assistive hardware, build reliable sensor systems, and create real-time connections between embedded devices and web applications.

What's next for SignGloves

SignGloves has the potential to significantly improve everyday communication accessibility by reducing barriers between Deaf and hearing communities. Our next step is to integrate motion sensors to capture full hand and wrist movement, allowing the system to recognize signs that rely on motion rather than solely finger positioning. Expanding beyond fingerspelling to more expressive ASL gestures would enable more natural communication. We also plan to explore scalable manufacturing methods and potential commercialization paths in education, assistive technology, and accessibility-focused markets, with the goal of making SignGloves affordable and widely accessible.

Built With

• elevenlabs
• esp32
• node.js
• supabase
• waterjet