Patient questions and artificial intelligence answers: Evaluating chatbots in aspirin-exacerbated respiratory disease.

Authors

Carter J Segal, Noah G Braunstein, Jillian C Bensko, John V Bosso, Erin K O'Brien, Kathleen M Buchheit, Jordan Glicksman, Elina Jerschow, Christine Reger, Andrew A White, Adam N Williams, Alan D Workman, Tanya M Laidlaw

Published in

Allergy and asthma proceedings. Volume 47. Issue 5. Pages 343-347. Sep 01, 2026.

Abstract

Background: Aspirin-exacerbated respiratory disease (AERD), also known as nonsteroidal anti-inflammatory drug-exacerbated respiratory disease, is a chronic condition that is both clinically complex and often difficult for patients to understand. As patients increasingly turn to online tools for medical information, it is important to evaluate the quality of the responses they may receive. Objective: This study assessed the medical accuracy and readability of responses generated by ChatGPT 5.1, Gemini 2.5 Flash, and Claude Sonnet 4.5 to 12 common questions that patients often ask their providers about AERD. Methods: The 12 questions were developed by clinicians and patients; the resulting chatbot responses were de-identified and reviewed by an expert panel of 11 AERD specialists who rated each response for medical accuracy on a 10-point scale. Readability was measured by using readability and grade-level analysis programs. Results: Gemini's responses had the highest overall ratings for medical accuracy, followed by Claude and ChatGPT. The models generated responses written at a postgraduate reading level, which is substantially higher than recommended targets for patient-education materials. Questions with more complex responses showed greater variability in expert ratings, potentially reflecting differences in clinical interpretation. Conclusion: Analysis of these findings suggests that, whereas chatbots may improve access to medical information, their current outputs require clinician review and substantial simplification before they can be reliably used for patient education.

PMID:
42698173
Bibliographic data and abstract were imported from PubMed on 05 Sep 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.