AgentNow

Inspiration

We picked the ServiceNow challenge because the proposal felt like the perfect way to combine web development with retrieval-augmented generation (RAG). It also gave us a great opportunity to get more comfortable working with Gemini API while solving a real-world problem.

What it does

AgentNow is a Gemini-powered assistant that listens to customer conversations and, in real time, recommends the most relevant ServiceNow accelerators. It highlights popular accelerators while also surfacing those that may be underutilized, ensuring balanced recommendations. With a microphone interface, AgentNow provides immediate, data-driven feedback during meetings so solution consultants and sales teams can respond faster and more effectively.

How we built it

We integrated Google’s Gemini API for natural language understanding and combined it with a RAG pipeline connected to an accelerator knowledge base. The frontend dashboard was built with Next.js and Tailwind (with shadcn/ui components), visualizing metrics like accelerator popularity and coverage. Real-time audio transcription was handled through Whisper-based models, feeding directly into the recommendation engine.

Challenges we ran into

Being only a 2-person team instead of 4 meant we had to cover a lot of ground, from frontend design to backend ML, which stretched our time and energy.
We originally wanted AgentNow to listen directly to the computer’s system audio, but quickly realized this was out of scope for a hackathon and would take multiple days to implement properly.
The design was tricky with deciding how to layout all of the pages, the front end was finally started on the second day after some deliberation.

Accomplishments that we're proud of

Building an end-to-end working prototype in a short time with a team of two.
Creating a system that not only recommends accelerators but also visualizes gaps and underused content.
Working with ElevenLabs for the first time to create a better experience for voice chat

What we learned

How to combine real-time speech recognition with retrieval-augmented generation.
The importance of designing interfaces that support, rather than distract from, live conversations.
We learned how to divide responsibilities effectively as a 2-person team, balancing ML, backend, and frontend work.