ScholarLens

Instead of manually checking dozens of websites, ScholarSpy continuously crawls and aggregates data from trusted sources such as government portals, universities, NGOs, and private organizations.

Comment 1

Inspiration

Finding scholarships is extremely difficult for students, especially in developing countries. Scholarship information is scattered across hundreds of government portals, university pages, NGOs, and private websites. Many students miss opportunities simply because they don’t know where to look, what they’re eligible for, or when deadlines are approaching.

What it does

ScholarSpy is an intelligent scholarship discovery platform that: Scrapes scholarship data from multiple trusted sources . Stores and indexes scholarships in a searchable database Allows students to filter scholarships by: Category Education level Income criteria Deadline Eligibility Provides real-time updates and deadline alerts .Offers a simple, clean dashboard for students

How we built it

Backend Python + FastAPI for API development Selenium for automated web scraping ElasticSearch for fast full-text search and filtering Docker for running ElasticSearch locally PostgreSQL / SQLite for structured data storage

Frontend React + Vite Clean UI for scholarship listing & filtering REST API integration with backend

Challenges we ran into

Scraping dynamic websites that block bots or load content asynchronously Normalizing inconsistent data formats from different sources Setting up ElasticSearch locally using Docker Handling duplicate scholarships from multiple sources Ensuring fast search performance for large datasets

Accomplishments that we're proud of

Built a working end-to-end system within hackathon time Successfully integrated ElasticSearch for real-time filtering Designed a scalable scraping architecture Created a useful product with real social impact

What we learned

Practical experience with web scraping at scale How search engines like ElasticSearch work internally Docker fundamentals and containerized services Designing APIs for real users, not just demos Importance of data cleaning and validation

What's next for ScholarSpy

Email & WhatsApp deadline notifications User profiles with saved scholarships Admin dashboard for approving scraped data Multi-country and international scholarships Cloud deployment for production scale

Share this project:

Updates