Protego

This is the top of the main screen on the site where you can view information of your previous scans etc.
This shows the scanning history functionality of the app where you can track what you've scanned before.
This is some of the configurable settings that you can adjust to your individual wants/needs.
This shows how you can connect your gmail account.

Inspiration

One of our team members was hacked after clicking a link on Instagram sent by a friend, whose account had already been compromised. That simple mistake made us realize how easily anyone can fall victim to online scams, even when the threat comes from a seemingly trusted source. Online fraud has become more personal and pervasive than ever. Scammers now use convincing fake profiles, phishing links, and impersonation tactics that make it nearly impossible to tell what’s real and what’s not. The scale of the problem is staggering: over $1 trillion was lost globally to scams last year, affecting more than 600 million people. Since 2022, phishing attacks have surged by over 1,200%. This growing crisis is both expensive and erodes trust in the digital spaces we rely on every day. We built Protego to change that. Our goal is simple: to help people feel safe online again by detecting suspicious links, fake accounts, and potential scams before harm is done. Protego is our approach to creating a more secure internet for everyone.

What it does

Protego functions as an intelligent, "invisible assistant" designed to provide continuous, real-time protection against online scams and fraud. Its core capability is to actively monitor user screen activity, analyzing content across emails, websites, and social media profiles for potential threats.

When Protego detects suspicious elements (such as phishing links, deceptive content, or patterns indicative of a scam), it automatically flags these potential threats. It alerts users before they can become victims, offering a vital moment to reassess and disengage from harmful interactions. This proactive approach is essential in combating the sophisticated and rapidly evolving nature of online fraud. By acting as an ever-watchful guardian on your screen, Protego aims to neutralize threats at their source, contributing to a more secure and trustworthy digital environment.

How we built it

We built Protego as an Electron desktop app for cross-platform compatibility while keeping all processing local for privacy. The system combines multiple AI and security APIs into a unified real-time threat detection pipeline.

For URL analysis, we integrated URLScan.io's VM sandbox API for behavioral analysis of suspicious links, detecting malware and phishing attempts. We built a caching layer for instant results on previously scanned URLs and implemented a queue system to handle rate limits. BrightData provides WHOIS lookups for domain age and registration data. These signals combine with URLScan.io verdicts to produce 0-100 risk scores through our scoring algorithm. We found URLs to scan from 3 locations: 1) if the user copied it to their clipboard, 2) if the user navigated to it, and 3) using OCR via Tesseract.js to extract all URLs visible on a screen.

The AI screen monitoring feature uses Reka AI's vision model to analyze screenshots on an opt-in basis. Users can choose manual scanning via Cmd+Shift+S or automatic 10-second intervals. Screenshots are sent to Reka's API with specialized prompts to detect scam patterns, returning risk scores with detailed explanations.

For Gmail integration, we used Google OAuth for read-only email access. Beyond scanning email text and checking sender domain age, we implemented LinkedIn identity verification through BrightData's LinkedIn API. When someone emails as "John Smith from Company X", we query LinkedIn to find their real company and email domain, flagging clear impersonation attempts.

Using Electron’s IPC communication, we combined all of these various features into one application with an easy to use dashboard and easily configurable settings.

Challenges we ran into

We found integrating OCR into electron to be pretty tricky, especially because tesseract.js isn't the most accurate thing. We also found it hard at times to integrate APIs like Bright Data's API and Reka's API just given their large response delay at times. Beyond that, we also had to pivot our application a few times after realizing that things we wanted to make wouldn't work (we envisioned having Deepgram voice transcription for phone call scam functionality but then we realized that this was a desktop app, not a phone app).

Accomplishments that we're proud of

Completing our first-ever hackathon! :)
Integrating OCR to detect scams hidden in images
Being able to monitor on-screen activity and flag unsafe links in real time
Building a fully functional prototype in such a short time frame!

What we learned

Prior to CalHacks, all team members were new to participating in a hackathon. With limited experience in both hackathons and computer science, we dove passionately into solving a meaningful problem—scamming. Throughout the process, we learned how to work effectively as a team by delegating responsibilities and tasks within the project. We explored new technologies and implemented partner software and APIs such as Reka and Bright Data, allowing us to bridge different tech fields. We also developed problem-solving skills, learning not only how to debug syntax and logic issues but also how to overcome external limitations, like Instagram blocking scraping or Electron behaving unexpectedly with OCR. In addition, we focused on designing an application that balanced effectiveness with usability, ensuring that it remained feature-rich yet accessible and intuitive. Integrating various solutions across different parts of the project was another major challenge, as it required constant synchronization and incorporation of updates from all sides. Coming from diverse backgrounds in engineering, computer science, and business, we also learned to align our strengths to create the most cohesive and high-impact product possible. Most importantly, we learned to think from the perspective of a consumer. Since the primary goal of our application was to maximize real-world impact, we made deliberate design choices to ensure it was user-friendly, accessible, and practical for anyone to use.

What's next for Protego

We see Protego growing far beyond just link and profile detection. Our next step is integrating voice protection as part of a larger mobile app, allowing Protego to identify potential scam calls or AI-generated voices that mimic real people. With deepfake technology becoming more common, we want Protego to be able to catch those fake personas before they cause harm. We’d also like to be able to connect Outlook integration as well (Azure’s approval process seems considerably more difficult with more required approval checks). Beyond that, we’d look to achieve improvements on the accuracy and speed of URL scanning as well as real-time screen scanning.