Inspiration
When an autistic child tips from calm to meltdown it can feel like a light-switch—one moment everything’s fine, the next there’s shouting, self-hitting, or bolting across a parking lot. Researchers have shown the body usually warns us first: heart-rate shoots up, variation in the time between heartbeats increases, vocal pitch tightens, and movement grows jerky up to a minute in advance. Yet the wearables on the market today either track location or log skin sweat after the storm, giving caregivers no real time to act. We wanted to flip that script and hand families an early-warning system they could use today.
What it does
- Listens to the body, not just behaviour. The watch streams heart-rate and the variation in heartbeat timing once a second, plus raw wrist-motion data.
- Grabs tiny voice snapshots. Every three seconds it pipes a short, silent microphone clip to the phone using Apple’s Continuity Microphone feature—no pop-ups, no always-on recording.
- Builds a 26-number snapshot. Six numbers from motion data, six numbers from heart-rate data , and fourteen numbers from voice (thirteen average mel-frequency cepstral coefficients, which capture how we hear the tone of someone’s voice, plus the average pitch of the voice)
- Adds it all up A lightweight machine learning model trained on public and private recordings decides “low risk” or “high risk” every three seconds.
- Tells the grown-ups. If the risk crosses a certain threshold, the caregiver’s phone buzzes and shows a one-tap de-escalation tip (for example, “dim the lights, offer noise-reducing headphones”).
How we built it
- Data first. We used publicly available speech recordings showing emotional tone, publicly available wearable stress data, and two hours of our own watch recordings with local families.
- Simple features, small model. Instead of storing huge spectrograms, we average each speech coefficient and pitch into single numbers, giving a tidy fourteen-element voice profile that anyone can reproduce in Python.
- Friendly software stack. SwiftUI on the watch and phone, core Machine Learning for motion and heart-rate features, and TensorFlow Lite for the fourteen-number voice network (model file under 40 kilobytes, so it runs comfortably on an Apple Watch Series 6)
Challenges we ran into
- No silent background recording on the watch
- Apple’s watch operating system blocks always-on microphone streams; we discovered that the Continuity Microphone can quietly send audio to the phone without user taps.
- Apple’s watch operating system blocks always-on microphone streams; we discovered that the Continuity Microphone can quietly send audio to the phone without user taps.
- Data scarcity
- Very few studies record heart rate, wrist motion and voice all at once; we merged three datasets and still collected more recordings ourselves.
- Very few studies record heart rate, wrist motion and voice all at once; we merged three datasets and still collected more recordings ourselves.
- Keeping it tiny
- Converting the voice model to eight-bit integers caused drops in accuracy until we fine-tuned each channel with a small representative sample.
Accomplishments that we’re proud of
- A live demonstration that achieves a 71 percent balance between precision and recall on our pilot data, all running on an off-the-shelf Apple Watch.
- Parents who tested it said the gentle watch tap felt “less alarming” than a loud phone notification.
- 100 percent open-source code so researchers can build on our work immediately.
What we learned
- Combining body signals outperforms any single sensor: voice alone topped out at about 53 percent accuracy, but adding heart-rate variation and wrist motion boosted performance by nearly 20 percent.
- Privacy-driven limits can inspire better design: short voice clips respect autonomy and reduce false alarms.
- Caregivers value simple, timely alerts far more than complex dashboards—haptic feedback first, charts later.
What’s next for AutiCalm
- Field pilot with ten local families this fall to gather over one hundred hours of labeled data.
- Personalized baselines so the system learns each child’s normal heart-rate patterns and voice range.
- Android Wear support using comparable sensors to make the solution more affordable.
- Open data release under a Creative Commons license to accelerate research in autism support technology.
AutiCalm gives caregivers those precious seconds of warning so moments of calm can turn into supportive hugs instead of emergency calls.
Log in or sign up for Devpost to join the conversation.