Large language model-assisted CAD-RADS classification: structured report extraction, accuracy, and clinical decision-making risk of ChatGPT-4o/5 and DeepSeek-R1.

Authors

Haorong Rong, Xiyi Huang, Xinjie Chen, Lizhu Ouyang, Yuelu Li, Xin Lin, Jialing Pan, Yunjing Li, Fan Xie, Fusheng Ouyang, Qiugen Hu, Baoliang Guo

Published in

Frontiers in cardiovascular medicine. Volume 13. Pages 1909068. Epub Oct 09, 2026.

Abstract

To compare ChatGPT-4o, ChatGPT-5, and DeepSeek-R1 in assigning main Coronary Artery Disease Reporting and Data System (CAD-RADS) categories from findings-only coronary computed tomography angiography (CCTA) reports and to characterize management-relevant classification errors.
This retrospective diagnostic-accuracy study included 2,100 category-balanced findings-only CCTA reports (300 per CAD-RADS category). Two experienced CCTA radiologists graded the English-translated reports; 12 disagreements were resolved by a third. ChatGPT-4o was evaluated in standard mode, ChatGPT-5 in Thinking mode, and DeepSeek-R1 in Deep Thinking mode; model generation and inference mode therefore varied together. Exact-match accuracy with 95% Wilson confidence intervals (CIs), paired comparisons, and unweighted Gwet AC1 with 95% CIs were calculated. Management-relevant transitions and digital subtraction angiography (DSA) findings among high-risk downgrades were analyzed.
DeepSeek-R1 had an exact-match accuracy of 82.57% (95% CI, 80.89-84.13) and AC1 of 0.797 (95% CI, 0.778-0.816). ChatGPT-5 had an accuracy of 80.19% (95% CI, 78.43-81.84) and AC1 of 0.769 (95% CI, 0.749-0.789), whereas ChatGPT-4o had an accuracy of 53.62% (95% CI, 51.48-55.74) and AC1 of 0.460 (95% CI, 0.435-0.485). Cochran's Q test was significant (Q = 657.87; df = 2; P < 0.001); after Holm correction, DeepSeek-R1 also outperformed ChatGPT-5 in the paired comparison (P = 0.026), and both outperformed ChatGPT-4o (both P < 0.001). ChatGPT-4o downgraded 127/900 (14.11%) reference high-risk reports to CAD-RADS 3 or lower. Of these 127 cases, 65 underwent DSA within three months; DSA supported the original high-risk assessment in 56/65 (86.15%). The corresponding high-risk downgrade proportions were 37/900 (4.11%) for ChatGPT-5 and 23/900 (2.56%) for DeepSeek-R1.
The DeepSeek-R1 Deep Thinking and ChatGPT-5 Thinking configurations achieved higher accuracy than ChatGPT-4o in standard mode. Because model generation and inference mode varied together, the results do not isolate a model-family effect. LLMs merit further evaluation for secondary structuring when no documented CAD-RADS category is available, but should not overwrite a radiologist-assigned category or independently determine management.

PMID:
42920484
Bibliographic data and abstract were imported from PubMed on 24 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.