Clinical safety, content coverage, and patient-centered language of AI responses to periodontal complaint-based queries: a comparative study of four large language models.

Authors

Çağrı Esen, Mehmet Gül, Fatih Karayürek

Published in

Odontology. Jul 07, 2026. Epub Jul 07, 2026.

Abstract

This study aimed to evaluate the clinical safety, informational completeness, and patient-centered language of responses generated by four large language model (LLM)-based systems to standardized periodontal complaint-based queries. Seven standardized symptom-based periodontal queries were developed based on commonly reported patient complaints and submitted in Turkish to Copilot, Gemini, Claude, and ChatGPT. Responses were evaluated using a structured rule-based framework consisting of a Content Coverage Score (CCS), Risk of Harm Score (RHS), and Patient Language Score (PLS). Assessments were performed by two blinded human reviewers and one blinded AI-based evaluator. Human consensus, AI consensus, and combined evaluator scores were calculated. Agreement between evaluators was assessed using intraclass correlation coefficients (ICC), while human-AI differences and inter-model comparisons were analyzed using non-parametric statistical tests. Excellent agreement was observed between human reviewers, repeated AI evaluations, and human-AI consensus scores (ICC range: 0.911-0.925; p < 0.001). No significant difference was found between overall human and AI consensus scores (p = 0.985). Across the four AI systems, no statistically significant differences were observed in CCS or PLS scores in the human, AI, or combined evaluator analyses (all p > 0.05). PLS scores were generally high across models, indicating good linguistic accessibility for patients. No clearly harmful guidance was identified by the human reviewers. Overall, LLM-based systems generated clinically safe, reasonably comprehensive, and generally patient-accessible responses to common periodontal complaint-based queries. Although these systems may serve as supplementary sources of periodontal health information, they cannot replace individualized clinical evaluation and professional dental consultation.

PMID:
42412385
Bibliographic data and abstract were imported from PubMed on 07 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.