Bison Buddy

Project Architecture

Inspiration

We drew inspiration from our college application days, during which waiting weeks for a reply from admissions officers about a question was frustrating. Therefore, we aimed for Bison Buddy to be a one-stop solution for all questions related to Howard University. Recent development of enterprise AI chatbots also inspired us to pursue our goal.

What it does

Bison Buddy acts as a personal guide to a prospective student by answering all their questions related to Howard University, whether it is about campus life, academics, opportunities, or even the weather in DC. It also supports voice interaction where users can interact with the chatbot through audio.

How we built it

We use Azure to instantiate an OpenAI model. We use Azure blob storage to store Howard University-specific + User-Specific files and implement RAG. So, when we want to train the OpenAI model on Howard University-specific data, we would store the data on one instance of the blob storage, and run indexing on it. This indexing is used by the Cognitive Search API whose results are the basis for our OpenAI's response. For Howard University-specific data, some of the candidates for indices were Admissions, Application, Research, etc. As mentioned, users can also upload their data that they would like the admission bot to know to give a more personalized and accurate response. We store it on the same blob storage, but the preprocessing is done on the webserver itself. We have used Azure's Document Intelligence API to get text out of the file format, then we run an NLP on the text to create many indexes out of that information. If OpenAI doesn't find the information the user asked for, we can do a basic GPT query as well. If the information was stored, and GPT just failed to index and return it properly (which happened to us because we couldn't use Semantic Search and had to use Keyword Search ), the user can report it, then the application can make use of user's intelligence to reindex the data (For us, we have just rephrased the most similar index and made it another index.)

Challenges we ran into

While building upon the azure-search-openai-demo, we ran into a major challenge. We didn't have access to Azure-auth, but we decided to tackle the problem a bit differently. We have a fully functional backend (with a minimal frontend that we will show in the demo), but we have not wired it with the frontend yet. So, all the functionalities that the user would get only after logging in don't have a great interface yet. So, users uploading documents, reindexing the data, and feedback have minimal frontend.

Accomplishments that we're proud of

Voice Feature, Feedback loop (Reindexing), UI, NLP (Document Intelligence)

What we learned

A lot about Azure Infrastructure, functions, and cloud hosting. We also saw how different cloud component interact with one another, especially how Search AI interacted with the indexing on our Blob Storage.

What's next for Bison Buddy

Email summarization from all domains of Howard University (@bison.howard.edu / @howard.edu) on the chatbot itself. We have also implemented this summarization function in a branch.

We will also add a deadline Reminder Notification for users.