Assessing AI Chatbots' Efficacy in Ophthalmic Triage and Referrals: A Comparative Study

PIYUSH JAIN,*
Ankita Mishra,
DEEPIKA PRIYADARSHINI,
Radhakanta Bhoi

Abstract

The burgeoning field of artificial intelligence (AI) presents innovative tools for augmenting healthcare practices, particularly in the challenging domain of ophthalmology. This study evaluates the efficacy of AI chatbots, specifically OpenAI ChatGPT (GPT-3.5), Google Gemini, and WebMD, compared to human ophthalmology trainees in triage and referrals for common ophthalmic conditions. A single-center study was conducted at the MKCG Medical College, Berhampur, Odisha, involving six ophthalmology trainees. The performance of the AI chatbots was measured in terms of diagnostic accuracy and triage categorization. Results indicated that physician respondents listed the appropriate diagnosis among the top three suggestions in 95% of cases, with Google Gemini achieving 90%, ChatGPT 85%, and WebMD 20%. High concordance was observed between physician and AI recommendations for investigations and referrals. This study demonstrates the promising potential of AI chatbots in supporting triage and referral decisions for ophthalmic conditions. While human expertise remains paramount, AI tools can serve as valuable adjuncts, enhancing diagnostic accuracy, efficiency, and patient care. Future research should focus on refining AI algorithms, integrating clinical data, and exploring real-world implementation strategies The burgeoning field of artificial intelligence (AI) presents innovative tools for augmenting healthcare practices, particularly in the challenging domain of ophthalmology. This study evaluates the efficacy of AI chatbots, specifically OpenAI ChatGPT (GPT 3.5), Google Bard, and WebMD, compared to human ophthalmology train