IVCM-Insight: automated interactive interpretation of in vivo confocal microscopy.

Authors

Zijun Zhang, Shangyang Jiang, Zhiqun Wang, Yang Zhang, Kexin Chen, Mingda Wei, Qingquan Shi, Yuan Wei, Jinding Pang, Xinxin Lu, Qingfeng Liang

Published in

NPJ digital medicine. Jun 23, 2026. Epub Jun 23, 2026.

Abstract

In vivo confocal microscopy (IVCM) is a critical ophthalmic examination that provides in vivo cytological and neurological information essential for diagnosing corneal and certain systemic diseases, but its clinical utility is limited by time-consuming interpretation and the need for subspecialty expertise. We developed IVCM-Insight, an artificial intelligence (AI) system integrating image-text contrastive learning with large language models (LLMs) for automated report generation and interactive question answering (QA). Based on 30,368 IVCM images and 4155 paired clinical reports, the model was trained with contrastive alignment, image-conditioned language modeling, and multi-image consistency loss to produce structured diagnostic reports while a domain-adapted LLM supported patient-centered QA. Automated evaluation showed strong agreement with the reference reports: Bilingual Evaluation Understudy (BLEU)-1 to BLEU-4 scores were 0.69, 0.58, 0.47, and 0.41, Recall-Oriented Understudy for Gisting Evaluation (ROUGE-L) was 0.67, Consensus-based Image Description Evaluation (CIDEr) was 1.85, and Metric for Evaluation of Translation with Explicit Ordering (METEOR) was 0.66. In addition, the multi-label classification achieved an accuracy of 0.96 and an F1 score of 0.80. Manual assessment by corneal specialists rated report accuracy (4.17), completeness (4.19), coherence (4.70), and diagnostic support (4.06), with excellent inter-rater reliability; QA outputs achieved high accuracy (4.33), relevance (4.54), and non-harmfulness (4.81). Representative cases, including cytomegalovirus, fungal, and Acanthamoeba keratitis, demonstrated accurate detection of key findings and clinically safe explanations. To our knowledge, IVCM-Insight is the first dedicated AI system for comprehensive IVCM interpretation, with potential to enhance diagnostic efficiency, strengthen physician-patient communication, and broaden access to advanced corneal imaging across care settings.

PMID:
42337008
Bibliographic data and abstract were imported from PubMed on 24 Jun 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.