DocJoy: AI notetaking aide for medical professionals

Click "Start Record" to automatically transcribe and generate medical reports during patient consultations.
Easily edit and update reports with simple prompts.
Reports generated by DocJoy adhere to NHS standards for consistency and clarity.

💡 Inspiration

In the fast-paced environment of medical practices, doctors are constantly juggling multiple tasks during patient visits. Not only do they need to listen and hold a conversation with the patient, but they also must pose the right questions, form a diagnosis, and document the visit all at once. This multitasking can often shift the focus away from the patient, as doctors spend a significant amount of energy taking notes. This lengthy documentation process extends patient wait times and the overall duration of visits, as each interaction requires substantial time to record and organize necessary information for both diagnosis and treatment. These reports are crucial; they serve as a vital part of a patient's medical history and are essential for continuity of care.

👀 Why this is Important

According to the NHS think 70% of a clinician's time is spent on admin. NHS Long Term Workforce Plan The researchers believe 44% of all admin work can be automated, in other words around 31% can be automated. The report sites the Topol Review study showing that even one minute of time in the NHS could equate to "400,000 hours of emergency department consultation time and 5.7 million hours of GP consultation time."

According to a case study in Chiswick doctors were extending appointments by 10-15 minutes for general administrative tasks. Case study in Chiswick If we reduce from 10 minutes to 5 minutes that could equate to around 28.5 million hours of GP consultation time and 2 million hours of emergency department consultation time.

Every visit relies on the accuracy and availability of these records. Unfortunately, the current system places an undue burden on medical professionals, leading to longer wait times and potentially rushed visits that can affect the quality of care provided. If doctors spend less time on time-sucking admin they can more effectively educate the patient and consult them on any concerns using colloquial rather than medical terminology.

⚙️ What it does

DocJoy is an AI-driven note taking aide designed to revolutionize the way medical professionals interact with their patients. DocJoy records the conversation first medical consultations and uses advanced speech recognition and natural language processing technologies to transcribe the recording. It not only captures the dialogue from the recording but also intelligently organizes the information into coherent, structured medical notes.

Seamless Integration: Integrate smoothly into existing medical systems, operating unobtrusively in the background during patient visits.
Intelligent Transcription: Transcribe spoken words from doctor patient visits. It understands medical terminology and contextual nuances, which allows it to generate accurate and detailed notes.
Smart Organization: Categorize information into predefined sections such as symptoms, diagnosis, tests ordered, and treatment plans. This structuring mirrors traditional medical reports, in the NHS, making them easy for many healthcare professionals to understand.
Instant Report Generation: Create a complete, professionally formatted report. This report can be reviewed and edited by the physician before being added to the patient’s medical record or sent directly to the patient.

🛠️ How we built it

We used ReactJS for the front end and NodeJS for the back end to make everything work smoothly and look good. The Gemini AI sorts the conversation into categories like symptoms and diagnoses and turns it into a clear, structured report.

💪🏻 Challenges we ran into

Handling Long Conversations: We needed to ensure our Gemini AI could manage long data streams effectively, so doctors get continuous and reliable reports. Also, transcribing speech accurately in real-time posed a significant challenges. To address this, we implemented a system where conversations are recorded separately and then uploaded for processing by Gemini AI. This approach ensures the accuracy and reliability of the transcription, enhancing the overall usability of our reports.

👑 Accomplishments that we're proud of

We're proud to have enhanced doctor efficiency in documenting medical visits, significantly reducing the time spent on paperwork. Additionally, our system presents health records in a clearer, more understandable format for patients, facilitating better comprehension of their medical information. This dual improvement benefits both doctors and patients, streamlining processes and improving communication.

🧠 What we learned

We learned how to effectively integrate Gemini AI in a real-world healthcare setting, balancing innovation with practicality. Understanding the intricacies of speech recognition and natural language processing was crucial.

🚀 What's next for DocJoy

We are set to launch DocJoy in real medical settings for extensive testing in UK. This will help us collect insightful feedback from healthcare professionals and patients, enabling us to fine-tune our system for optimal performance.
We aim to introduce DocJoy to global markets by adapting it to comply with local regulations and customizing the report formatting to meet country-specific standards. This step ensures that DocJoy can be effectively integrated into diverse healthcare systems.
We plan to scale DocJoy into a full-fledged electronic medical record system. This expansion will support the digitization of healthcare systems, streamlining processes, and improving the accessibility and management of patient records across the healthcare industry.

Youtube: https://youtu.be/M6JfWzW-yPo