SpeakSafe

Landing Page
Login Page
Homepage
Voice Verification Page
Verification Successful Page
Pop-up after voice is successfully verified

Inspiration

The Spark of Inspiration

As a team fascinated by both technology and psychological thrillers we have always been captivated by the line between what is real and what is fabricated. The rise of AI deepfake technology felt like a plot twist unfolding in the real world and the statistics are staggering. With reported losses from imposter scams many now powered by AI voice cloning exceeding $2.6 billion last year alone it is clear this is a massive threat. We read alarming articles about criminals using these clones to impersonate executives or family members in distress. It struck us that the one thing we considered uniquely human our voice was becoming a dangerous vulnerability

This led us to think about the financial sector where a single phone call can move millions. Traditional security like a password or a mother's maiden name feels archaic in an age of constant data breaches. We realized the next frontier of digital crime would not just be about stolen information but about stolen identities. That is where the idea for SpeakSafe was born: a security system that does not just ask what you know but confirms who you are

What it does

SpeakSafe is a web-based security tool designed to detect deepfake audio in real time and protect users from fraudulent access to sensitive information such as online banking credentials. When a user attempts to log in or verify their identity SpeakSafe analyzes the audio or video feed to check for manipulation. If a deepfake is detected access is blocked and the user receives an alert. This adds an additional layer of security to prevent impersonation attacks and protect sensitive financial data.

How we built it

We built SpeakSafe as a full-stack application:

Frontend: React with Tailwind CSS for a clean and responsive web interface
Backend: Flask serving API endpoints and handling user requests
ML Model: Python using both pretrained deepfake detection models and custom-trained models for higher accuracy
Database: Redis for session management and quick access to user verification data
Other Tools: TensorFlow and PyTorch for the ML models Cloudflare Turnstile for bot protection

The system integrates the ML model into the backend so that audio is analyzed in real time allowing the application to immediately block suspicious access attempts

Challenges we ran into

Real-time detection: Ensuring the ML model could analyze audio fast enough to prevent access without noticeable delay
False positives: Balancing sensitivity so legitimate users are not blocked while still catching deepfakes
Integration: Connecting the frontend backend and ML model seamlessly so that the workflow from login to detection is smooth
Deployment: Handling serverless constraints while deploying a Flask backend and ML model to cloud infrastructure

Accomplishments that we're proud of

Built a fully functional real-time deepfake detection system
Successfully integrated audio verification into the login flow for added security
Designed a clean and responsive web interface that alerts users immediately when suspicious activity is detected
Developed a workflow that prevents fraudulent access to sensitive banking information using AI-based analysis

What we learned

The importance of balancing ML accuracy and performance in real-time applications
How to structure a full-stack security application with frontend backend and ML integration
Best practices for secure user authentication in the age of AI impersonation
The practical challenges of deploying ML-powered applications in serverless or cloud environments

What's next for SpeakSafe

Expand support to video deepfake detection in addition to audio
Improve ML model accuracy to reduce false positives
Integrate with more financial platforms for broader adoption
Explore biometric verification features like voiceprint authentication for added security