DefinePK

Abstract

Objective: To evaluate the diagnostic accuracy of artificial intelligence (AI) tools compared with human clinicians in identifying oral lesions.
Material and Methods: A comparative cross-sectional diagnostic accuracy (quantitative) study research has been conducted at the Department of Oral and Maxillofacial Surgery, Fatima Jinnah Dental College, Jinnah Hospital, Karachi, following approval from the ethical board, from November 2023 to January 2024.involved an examination consisting of 20 MCQs. These questions had options of responses titled A to D. The exam, aligned as multiple-choice questionnaire by an expert of that content field, was administered via Google Forms.
Results: Among the 35 participants, 14.3% were LLMs and 85.7% were house officers, with the mean ages of house officer’s 29.07±4.05 years. The majority of participants were male. In terms of diagnostic accuracy, LLMs and house officers performed similarly, with correct answer rates of 80.0% and 83.3%, respectively, and no statistically significant difference (p=0.855). Additionally, artificial intelligence tools demonstrated varying efficiencies, with OpenAI achieving 91.4% correct answers, PopAI at 77.1%, and Gemini at 80.0%.
Conclusion: The findings notably demonstrate that LLMs exhibit a level of performance that is on par with that of senior dental house officers, thus underlining their promise as a valuable tool in augmenting diagnostic capabilities within clinical settings.
Keywords: Artificial intelligence, Diagnosis, Human intelligence, Large language models, Oral lesions.