Authors
Man Sun, Dan Zang, Huan Zhou, Jun Chen
Published in
Frontiers in cellular and infection microbiology. Volume 16. Pages 1955356. Epub Oct 09, 2026.
Abstract
Immune checkpoint inhibitor-related liver injury (ICI-LI) has heterogeneous phenotypes, a complex differential diagnosis, and evolving management. Gut microbiome research provides biological plausibility for immune regulation during ICI therapy, but evidence specific to ICI-LI remains limited and is not yet clinically actionable. We evaluated whether publicly accessible large language model (LLM) chatbots can organize this mixed evidence base and communicate its uncertainty.
We developed a disease-specific benchmark of 25 questions covering immunological mechanisms, microbiome and immune regulation, diagnosis, biomarkers, and management of ICI-LI. The benchmark addressed ICI-LI broadly and included a focused microbiome component. Four publicly available LLM platforms were evaluated using DISCERN, EQIP, the Global Quality Score, the JAMA Benchmark Criteria, and six readability indices. These instruments assessed information quality, structure, transparency, and patient-facing readability rather than factual accuracy or clinical decision-making.
Performance differed across DISCERN, EQIP, and JAMA scores, whereas the Global Quality Score showed limited discrimination. Question-level variation was most apparent for items requiring evidence appraisal or translation. All platforms produced structured responses, but none met the prespecified patient-facing readability benchmark. Readability was interpreted separately from clinician-facing information synthesis.
Disease-specific multidimensional evaluation is needed before LLM-generated information can be considered reliable for microbiome-informed clinical communication or decision support. Evaluation should emphasize evidence calibration and distinguish biological plausibility, association, and clinically actionable evidence.
PMID:
42920487
Bibliographic data and abstract were imported from PubMed on 24 Oct 2026.
Read full publication at:
Please sign in
to see all details.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 16
- Comments 0