Assessment of CDASI scoring by a multimodal large language model: a comparative study with expert assessors.

Authors

Marco Fornaro, Vincenzo Venerito, Swapnasha Panigrahi, Sara Sabbagh, Florenzo Iannone, Latika Gupta

Published in

Rheumatology international. Volume 46. Issue 7. Jul 08, 2026. Epub Jul 08, 2026.

Abstract

Large language models (LLMs) with multimodal capabilities may support automated assessment of cutaneous disease activity in dermatomyositis (DM). We evaluated the performance of Claude v3.5 Sonnet in scoring the Cutaneous Dermatomyositis Disease Area and Severity Index (CDASI) compared with expert rheumatologists. Thirty published DM cases with available clinical images were retrospectively analyzed. Two expert rheumatologists independently scored CDASI activity and damage domains. Claude v3.5 Sonnet assessed the same images using structured prompting based on CDASI definitions. Agreement was evaluated using intraclass correlation coefficients (ICCs) with a two-way random-effects model for absolute agreement. The LLM demonstrated better agreement for activity than for damage assessment. Global CDASI activity showed good agreement with expert raters (ICC 0.71, 95% CI 0.45-0.86), whereas damage reliability was lower (ICC 0.41, 95% CI 0.25-0.67). Agreement was moderate for activity-related domains including erythema (ICC 0.61), scaling (ICC 0.57), and erosions (ICC 0.57), while lower concordance was observed for chronic damage-related features such as poikiloderma (ICC 0.47, 95% CI 0.10-0.73). Hand assessments demonstrated the strongest performance, particularly for periungual changes (ICC 1.0, 95% CI 1.0-1.0) and global hand scores (ICC 0.95, 95% CI 0.91-0.97). The model also markedly reduced evaluation time compared with clinicians (42 s vs. 8.4 min per case), corresponding to an approximately 92% reduction in scoring time. Multimodal LLMs demonstrated promising agreement with expert raters for CDASI activity assessment and markedly improved efficiency, although lower reliability in damage domains reinforces the continued need for physician oversight.

PMID:
42420687
Bibliographic data and abstract were imported from PubMed on 09 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.