Ocular

Ocular
Ocular Dashboard
Ocular Contacts
Ocular Manual Contacts

Inspiration

Our inspiration comes from the Ring doorbell, a market leader that alerts homeowners and lets them interact with visitors. We saw an opportunity to go further by creating an AI that handles requests on its own, recognises familiar faces, and either takes messages or notifies homeowners directly. We built this conversational AI to guard your door around the clock. With integrated TTS technology, it can do just about anything, with or without your involvement, ensuring a smarter and safer home security experience while making guests feel welcome.

What it does

Ocular leverages real-time facial recognition to instantly determine whether a visitor is a family member or friend. When it detects someone familiar, its tone becomes warm and welcoming; if it detects an unfamiliar face, it adopts a more formal tone to inquire about their presence. Beyond greetings, Ocular logs every entry and exit, providing a continuous and reliable home security. What makes Ocular special is its deeply conversational and contextually aware abilities as it remembers the homeowner's preference: whether that's for package deliveries or catching up with friends. Ocular always ensure personalised interactions every time. By integrating advanced TTS and computer vision with AI, Ocular is paving the way for the next era in smart home security.

How we built it

Ocular is built using a combination of cutting-edge technologies. We start with OpenCV and OpenVINO to detect any visible face. Once detected, Ocular captures images and uses an OpenAI LLM to convert the visitor’s speech to text, which is then processed through Neuphonic's API for text-to-speech responses. The LLM extracts key details from the conversation like the visitor’s name and purpose—which, along with the captured images and user confirmations, are stored in Google Firebase. This ensures Ocular remembers every interaction, enhancing both personalisation and security with zero downtime.

On the user interface side, we built a dashboard using React and Vite to display all contacts, notes, and alerts. A Python Flask backend facilitates data exchange between the dashboard and Firebase, while React Charts provides smooth data visualisation for real-time analytics.

Challenges we ran into

The biggest challenge we ran into was that OpenVINO excels at extracting faces but struggles with accurate recognition. Even with high similarity scores, variations like turned faces, side profiles, or changes in yaw and pitch often triggered false positives in Firebase. To minimise these errors, we increased the similarity threshold and relied on contextual information to match images to the correct person, ultimately opting for an image-first, context-based approach.

A secondary challenge was related to camera quality. Blurry images or sudden movements sometimes led to new face detections, resulting in inaccuracies. We addressed this by implementing a delay to ensure the face was captured correctly before triggering recognition.

Accomplishments that we're proud of

We're proud of Ocular because it's our first project to merge real-time facial recognition with rich, contextual speech information. It was a steep learning curve, but we managed to deliver a doorbell interaction that feels natural and accurate. Ocular is truly innovative in the sense that it combines facial recognition with contextual details to create a security system that doesn't just react when you're present but works independently. It welcomes guests, remembers every interaction, and remains effective even when you're not at your phone. This fusion of technology and genuine connection marks a real step forward in home security.

What we learned

We learned that innovation is built on a foundation of passion and perseverance. Working on Ocular, we discovered that merging real-time facial recognition with contextual speech data is as challenging as it is rewarding. Every obstacle from fine-tuning detection accuracy to managing unpredictable camera inputs taught us lessons in resilience and creativity. More than just a smart doorbell, Ocular has become our stepping stone toward creating technology that understands and protects our homes, even when we're not there. This journey has reinforced our belief that building something truly sustainable means laying down the foundations today for a safer, more connected future.

What's next for Ocular

We're excited to continue evolving this technology into a truly integral part of home security and smart living. In the near future, we're exploring deeper integrations with additional smart home devices and sensors, so Ocular can offer even more proactive, personalised experiences. Imagine a system that not only greets you but also learns your daily rhythms, adjusts to your environment, and even anticipates your needs. We're also focusing on refining its accuracy and responsiveness, ensuring it becomes more intuitive over time. Our goal is to build a platform that not only secures your home but also enriches your everyday life, setting the foundation for a future where technology and human connection go hand in hand.