Agreement of Multimodal Large Language Models and Novice Readers With Expert-Based Radiographic Scoring of Presumptive Septic Carpal Arthritis in Calves.

Authors

Ayşe Gölgeli Bedir, Esra Modoğlu, Tuğçe Kartal, Büşra Kibar, Mümin Gökhan Şenocak, Ömer Tarık Orhun, Ferda Turgut, Uğur Ersöz, Yakup Kocaman, Yasemin Akçora, Sıtkıcan Okur, Latif Emrah Yanmaz

Published in

Veterinary radiology & ultrasound : the official journal of the American College of Veterinary Radiology and the International Veterinary Radiology Association. Volume 67. Issue 6. Pages e70252.

Abstract

Septic arthritis is an important cause of morbidity in calves, and radiographic interpretation may vary between observers, especially when structured scoring systems are used by less experienced readers. This single-center retrospective reader study evaluated whether general-purpose multimodal large language models (LLMs) could support rubric-based radiographic scoring of presumptive septic carpal arthritis in calves. Fifty calves aged 0-3 months with clinical findings consistent with presumptive septic carpal arthritis were included, and one carpal radiograph per case was assessed. Two novice veterinary surgeons and three multimodal LLMs, ChatGPT-5, Gemini-2.5 Pro, and Claude Sonnet-4, independently scored each case using a Constant-based ordinal radiographic scoring framework. Expert consensus served as the operational reference standard. Agreement with the reference was assessed using exact agreement, agreement within one score category, and quadratic weighted kappa. Novice 1 showed the highest concordance, with a mean exact agreement of 55.6%, mean ±1 agreement of 89.8%, and substantial agreement (mean κ_w = 0.68; 95% CI, 0.56-0.80). ChatGPT-5 was the best-performing model, achieving moderate agreement (mean exact agreement, 53.0%; mean ±1 agreement, 82.6%; mean κ_w = 0.58; 95% CI, 0.41-0.72). Claude Sonnet-4 and Gemini-2.5 Pro showed slight agreement overall. The best-performing novice reader remained more reliable than the evaluated models, although ChatGPT-5 performed comparably to, or better than, one novice evaluator in selected parameters. Selected LLMs may have limited adjunctive value for structured review or training, but not as replacements for human interpretation.

PMID:
42802979
Bibliographic data and abstract were imported from PubMed on 28 Sep 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.