Vigil

Vigil architecture
Vigil deployed on Apple Watch
Local models running on sample exercise data (active, but non-threat situation)
Email automatically sent by Vigil

Inspiration

In a kidnapping or abduction scenario, the victim often can't reach for their phone, open an app, or press an SOS button. What if your body could call for help on its own? The Apple Watch is already strapped to over 100 million wrists, tracking heart rate, acceleration, rotation and various other metrics 24/7. We built Vigil to turn that passive data into an active lifeline by detecting panic situations autonomously, recording evidence, and immediately contacting emergency response with location and intelligent incident summaries, all without requiring a single tap.

What it does

Vigil runs silently on your Apple Watch, continuously monitoring your heart rate, accelerometer motion, and GPS data. It automatically:

Detects a panic event using two independent on-device ML models (both must agree to minimize false alarms).
Record 10 seconds of audio from the Watch microphone plus sensor and GPS data as evidence.
Uploads the package to a cloud model for heavyweight threat confirmation using audio and GPS data analysis.
Dispatches an emergency phone call and email in real time containing your GPS coordinates, an audio recording of the event, and an intelligent incident summary if the threat is confirmed.

No buttons. No phone. Just your Watch.

How we built it

Sensors

Vigil collects data from CoreMotion (accelerometer + gyroscope at 10Hz), HealthKit (heart rate and heart rate variation), and CoreLocation (GPS) every second.

On-device detection

We built a two-stage biometric analysis model that runs entirely on the Apple Watch using Swift:

SVM: Anomaly detection, trained using unsupervised learning on a dataset of only normal activity that we collected; detects abnormal biometrics.
Probabilistic model: Mathematical model of threat probability based on heart rate and movement data. Panic triggers only when both models exceed their thresholds, dramatically reducing false positives.

If both models exceed the decision threshold, the watch begins recording 10s of audio through its microphone and sending the sensor stream to the cloud.

Cloud inference

Our sensor data and the audio recording is converted to base64 and uploaded via a HTTP request to a Modal serverless endpoint. OpenAI’s gpt-4o-audio-preview analyzes the audio and location context to confirm whether the threat is real. We then use GeoPy to run GPS movement analysis, detecting addresses, distance, and speed.

Emergency dispatch call

On confirmed threat, Vigil uses Twilio for an automated voice call to emergency services containing an AI voiceover with live GPS coordinates and recorded audio of the event. It also uses SMTP for an emergency email containing a continual database of streamed GPS coordinates and an intelligent incident summary.

Challenges we ran into

Apple Watch edge integration Apple products are notoriously difficult to work with, and things are only harder on the Watch. We had to build in Xcode and Swift; with only two Apple Watches, we had to test a lot of code on the watchOS simulator, which often didn’t translate perfectly to physical deployment.
Local inference We had to design the local infrastructure (initial decision threshold models and data sent to cloud stack) to be sufficiently lightweight and efficient to run seamlessly and continuously on Apple Watch hardware. We constantly run lightweight detection locally on the Apple Watch as a gating mechanism for the powerful cloud models, reducing energy and inference costs.
Data collection We obviously didn’t have any positive samples of sensor readings from kidnappings, so we had to innovate with model design. We collected over 1,000 negative readings (normal activity), and used it for unsupervised training. We experimented with different architectures like an LSTM-autoencoder, but ultimately settled on using an SVM for anomaly detection, which worked well and provided a lightweight method easily deployable on the Apple Watch.

Accomplishments that we're proud of

Single-device stack: Vigil runs entirely through the Apple Watch, without needing to connect to your iPhone or Mac, ensuring it stays active regardless of where you go. We maximize the functionality of the Apple Watch to get tons of data from a lightweight edge device.
Full emergency response pipeline: Vigil doesn’t just detect kidnapping: In under 1min, it records audio, processes GPS information, and automates a call to emergency services with the relevant info for authorities to take immediate action.
Robust classification We tackle a challenge that lacks training datasets of positive examples and that occurs in a highly variable environment. We built a pipeline of three different threat classification models: the two local biometric models as gatekeepers and the powerful cloud model for detailed contextual analysis. This multi-layer system maximizes recall, which is crucial in this threat detection scenario, while also prioritizing precision so as to minimize both cloud model calls and false kidnapping alerts.

What we learned

We learned that systems are a lot more functional and robust in simulation than on edge hardware. Sometimes, models or features would work perfectly on the computer, but just break when deployed to the Watch. Building on an Apple Watch forced us to build systems that are robust to variable data and lightweight.

Furthermore, through this project, we deeply understood how data really moves throughout the internet. We had to figure out how to convert an audio file into a large base64 string, alongside biometric data, push to our modal endpoint, and properly decode it back on the python server. It helped us understand how to bridge the gap between edge devices and the cloud.

What's next for Vigil

Broader threat detection: Extend the models to detect a broader range of threat events, like collisions and physical assault patterns
iOS companion app: A paired iPhone app for event history, contact management, and connectivity backup
Emergency contact network: Let users configure trusted contacts who receive real-time GPS tracking and notifications of threat events