Comparative performance of nine large language models on professional Parkinson's disease tasks: a multidimensional benchmark study.

Authors

Ludi Xie, Rumeng Wei, Li Xiong, Rui Lu, Kangwen Peng, Wenlin Ma

Published in

Frontiers in neurology. Volume 17. Pages 1970459. Epub Oct 30, 2026.

Abstract

Parkinson's disease (PD) requires both disease-specific knowledge and contextual clinical reasoning. However, most evaluations of medical large language models (LLMs) emphasize closed-ended questions and factual accuracy, providing limited insight into broader dimensions of performance on professional PD tasks.
Nine contemporary LLMs were evaluated using a 200-item English-language professional PD benchmark spanning five clinical domains, four difficulty levels, and multiple question formats. Responses were assessed for expert-rated factual accuracy, clinical reasoning, Global Quality Scale (GQS), and Flesch-Kincaid Grade Level (FKGL).
The nine models generated 1,800 responses. Between-model separation was minimal for expert-rated factual accuracy (Kendall' s W = 0.031) but greater for FKGL (W = 0.235), Clinical Reasoning (W = 0.256), and GQS (W = 0.301). Between-model separation in expert-rated factual accuracy was small for closed-ended questions (W = 0.020, p = 0.001) but greater for open-ended questions (W = 0.121, p < 0.001). Expert-rated factual accuracy remained weakly discriminative across PD domains and difficulty levels, while GQS showed the greatest domain-specific separation in Safety & Ethics (W = 0.431). Flagship variants consistently showed higher Clinical Reasoning and GQS, but not uniformly higher factual accuracy.
Contemporary LLMs showed limited between-model separation in expert-rated factual accuracy within this benchmark and scoring framework, whereas greater differentiation emerged in open-ended tasks, clinical reasoning, overall response quality, and readability. Multidimensional evaluation may therefore provide a more informative characterization of model performance and help inform task-specific model selection and oversight in medical applications.

PMID:
43031820
Bibliographic data and abstract were imported from PubMed on 14 Nov 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.