Our AI writing assistant, WriteUp, can assist you in easily writing any text. Click here to experience its capabilities.

GastroGPT Outperforms General Models in GI Clinical Tasks

View Original View Raw

Summary

GastroGPT, a novel specialty-specific, clinically-oriented artificial intelligence model, was found to outperform leading general-purpose large language models (LLMs) in a proof-of-concept study. In 10 simulated patient cases, GastroGPT scored higher in all tasks, including assessments, diagnostic test recommendations, management, multidisciplinary care and referrals, follow-up plans, and patient counseling/education. By democratizing access to expert-level GI care, GastroGPT has the potential to provide quality GI care to underserved patient populations, especially in low- and middle-income countries. The model has not yet been compared to real people, but the lead investigator believes it is not inferior and might be superior to non-expert physicians for expertise-requiring questions.

Q&As

What is GastroGPT and what are its advantages?
GastroGPT is a novel specialty-specific, clinically-oriented artificial intelligence model that demonstrates superiority in overall utility and in key clinical tasks of gastroenterology when compared with leading general-purpose large language models (LLMs). Its advantages include the potential to provide quality GI care to underserved patient populations, especially in low- and middle-income countries, and the potential to save time by automating clinical tasks that usually require experts.

How was the study conducted to compare GastroGPT with general artificial intelligence models?
The study was conducted by comparing GastroGPT with three general-purpose LLMs (OpenAIs GPT-4, Google's Bard, and Anthropic's Claude). An expert panel was used to assess the responses of GastroGPT in comparison to the three general-purpose LLMs. For the evaluation, the experts helped to generate 10 simulated patient cases that were closely representative of reality.

What tasks did the AI models perform in the study?
The AI models performed seven clinical tasks for each case: assessment, additional history gathering, diagnostic test recommendation, management, multidisciplinary care and referral, follow-up plan, and patient counseling/education.

What was the primary outcome of the study?
The primary outcome of the study was overall performance across tasks.

How did GastroGPT compare to real people in the study?
In the study, GastroGPT performed better than all three models in most domains. Simsek said that in his experience, it is definitely not inferior to real people, and if the comparison is a non-expert physician for an expertise-requiring question, then he believes it is superior.

AI Comments

👍 The study shows that GastroGPT significantly outperforms general AI models in gastroenterology tasks, providing an exciting first step in democratizing access to expert-level GI care globally.

👎 The AI model is not yet able to outperform real experts in its tasks, and should always be supervised by healthcare providers until it demonstrates valid results.

AI Discussion

Me: It's about a new artificial intelligence model called GastroGPT that specializes in gastroenterology tasks such as patient assessments, diagnostic recommendations, patient counselling, and treatment plans. It outperformed general AI models in a head-to-head comparison study.

Friend: Wow, that's really cool. What are the implications of this?

Me: Well, it could be a huge help in providing quality GI care to underserved patient populations, especially in low- and middle-income countries. It could also save time by automating clinical tasks that usually require experts, and bring expertise to settings where specialist doctors are hard to access. It could also be used for screening patient cases and flagging high-risk situations, providing second opinions on complex cases, catching potential errors or inconsistencies, automating components of care plans and referrals, being available 24/7 for patient questions and triage, and supporting research and education.

Action items

Research other AI models that are being developed for specialty-specific clinical tasks.
Explore the potential applications of AI models in providing quality GI care to underserved patient populations.
Consider the ethical implications of using AI models in patient care and develop guidelines for their use.

Technical terms

GastroGPT: GastroGPT is a novel specialty-specific, clinically-oriented artificial intelligence model.
Large Language Models (LLMs): Large Language Models are general-purpose AI models that can chat about any topic.
OpenAIs GPT-4: OpenAIs GPT-4 is a general-purpose AI model developed by OpenAI.
Google's Bard: Google's Bard is a general-purpose AI model developed by Google.
Anthropic's Claude: Anthropic's Claude is a general-purpose AI model developed by Anthropic.
ChatGPT4: ChatGPT4 is a general-purpose AI model that can chat about any topic.
Inflammatory Bowel Disease (IBD): Inflammatory Bowel Disease is a group of chronic disorders that cause inflammation or ulceration in the small and large intestines.
United European Gastroenterology (UEG) Week 2023: United European Gastroenterology Week is an annual event that brings together leading experts in gastroenterology from across Europe.