Assessing Artificial Intelligence as a Diagnostic Support Tool for Surgical Admissions in the Emergency Department.

Authors

Jamie Rice, Eoin Ó'Briain, Conor J Kilkenny, Richard E Hogan, Tom V McIntyre, Dara Kavanagh, Paul C Neary, James M O'Riordan, Shaheel M Sahebally

Published in

Journal of surgical education. Volume 82. Issue 10. Pages 103676. Sep 01, 2025. Epub Sep 01, 2025.

Abstract

Artificial intelligence (AI) is increasingly being used in healthcare for data analysis and decision support. This study assesses the potential of AI (specifically ChatGPT-4o) as a diagnostic support tool for surgical admissions in the Emergency Department and compared its accuracy with that of on-call surgical trainees.
This was a single-institution retrospective study conducted in December 2023. Primary outcomes comprised the agreement and accuracy of diagnoses between AI and trainees. The secondary outcome measure was the similarity of treatment plans. Agreement was defined as the percentage of cases in which both AI and the surgical trainee concurred on the diagnosis or specific aspects of the management plan. Accuracy was defined in both groups as the proportion of provisional diagnoses that corresponded with the definitive diagnosis, as confirmed by subsequent imaging and/or clinical documentation.
One hundred patients were included in the study. The mean age of presenting cases was 54 (±20), years. Abdominal pain was the most commonly reported symptom in provisional diagnoses (68%). The accuracy of provisional diagnoses compared to CT-confirmed diagnoses was 76% for the AI system and 74% for the surgical trainees (p = 0.744). Substantial agreement between the AI and trainee's initial diagnosis was observed (κ = 0.73). There were no statistically significant differences between AI & surgical trainee in provisional diagnosis (p = 0.754), decisions to bring patients to the operating theatre (p = 0.540) and antibiotic administration (p = 0.122). The overall agreement rates were 85%, 58% & 74% respectively.
Commercially-available AI models demonstrate similar diagnostic ability to junior surgical trainees and may serve as a useful decision-support system. These models could be incorporated into electronic record systems as an adjunct to enhance decision-making.

PMID:
40896918
Bibliographic data and abstract were imported from PubMed on 03 Sep 2025.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.