Minutes.ai

Use AI to write complete notes for your meetings. Given Realtime audio data from meetings, Minutes.ai transcribes, separates into idea sized chunks of sentences, summarizes, and writes bullet points.

Comment

Inspiration

Meetings. So many meetings. While brainstorming ideas for the hackathon, we realized that we were spending a considerable amount of time documenting what we were doing- precious time that could have been spent actually hacking along. This is where the idea of Minutes.ai was born.

What it does

For To Hacks 2022, we've engineered an app which allows users to forego the typical administration required when holding meetings. Minutes.ai takes in audio held in meetings, transcribes it, intelligently separates it into bite size chunks of ideas, and then summarizes it into point form, all in Realtime. Through Artificial Intelligence, this allows the person assigned the task of writing notes to have a template in which they can work from, so all the members can focus on what really matters- the content of the meeting.

How we built it

  1. We first utilized Assembly API's powerful audio to text transcribing features, using Node.js, web sockets, Axios, and ExpressJS to connect to their services and receive Realtime audio data.
  2. We then separated the transcribed audio of these into bite sized chunks of "ideas", and sent them towards OpenAI to be parsed through by the DaVinci Engine.
  3. Each bite size chunk is converted through the DaVinci engine to note form, allowing meeting minutes notes to be effectively summarized, and bullet point form ideas to be generated.
  4. Finally, these summarized and bullet point form ideas are sent back to our app, where it is then displayed in our React custom-coded text editor for the meeting holders to look back on and edit.

The front-end text editor was first designed in Figma and implemented using JSX, CSS, and React, using minimalist design conventions. We've also designed the text editor to give you the ability to create new instances of notes as well!

Challenges we ran into

While working on this project, we ran into a series of technical issues, which we managed to overcome.

  • Initial issues with GitHub merge conflicts, code being rewritten over each other- we overcame this through strong communication, the use of Github branches, and the installation of Github Kracken- a GUI representation of version control. We also established a git pushing and pulling naming convention (feat = feature, chore = back-end work, etc), in order to keep things neater.
  • Communication between the front end, and the back end- we resolved this by reorganizing our file structure to include two separate folders with their distinct purposes, and communicating with them via fetch requests.
  • API communication issues- originally, we had many issues with getting Assembly AI and Open AI to send data to and from each other. After staying up until 4am to resolve these issues, we finally got it to work!

Accomplishments that we're proud of

We're proud that we were able to persevere through all the bugs and glitches to produce a product that we would actually use in real life! We wish that the work we did now will prevent much more work for anyone looking to hold meetings in the future, and that this product will be useful in both academic settings, and professional settings.

What we learned

  • API GET requests
  • Working with JS promises via Axios
  • Power naps help a lot
  • Testing and communication is super important- otherwise misunderstandings may occur
  • Work to people's strengths. If one person knows Express, and the other knows front-end, let them fly!

What's next for Minutes.ai

  • Developed exporting options for Google Docs, Word, etc.
  • Allow Markdown options for our built-in text-editor
  • Incorporate a login system, so permanent data can be stored
  • Host our app on a website- we considered utilizing Google Firebase in order to do so, but realized it couldn't instantiate a back-end.

Built With

Share this project:

Updates