Automated risk scoring for venous thromboembolism using large language models with expert knowledge-augmented prompting: a multicenter validation study.

Authors

Jing Ma, Dingyi Wang, Yaqian Zhang, Zhaofei Chen, Kaiyuan Zhen, Mengyao Li, Jingrui Li, Yunfan Shi, Xiaomeng Zhang, Rui Liang, Feiya Xu, Siqi Ma, Jintao Chen, Yanshuang Lyu, Yuanhua Yang, Zhe Cheng, Yutao Guo, Lei Xia, Wei Wang, Bing Liu, Cunbo Jia, Shuai Zhang, Qian Gao, Yunxia Zhang, Zhu Zhang, Wanmu Xie, Guohui Fan, Shengfeng Wang, Zhenguo Zhai

Published in

NPJ digital medicine. Jul 09, 2026. Epub Jul 09, 2026.

Abstract

Venous thromboembolism (VTE) is a common but preventable complication in hospitalized patients, yet standardized risk scoring using unstructured electronic health records (EHRs) remains challenging. We retrospectively analyzed anonymized EHRs from a nationwide multicenter cohort across 30 hospitals. Using stratified random sampling, 50 cases from 10 hospitals were used for development and 200 cases from the remaining 20 hospitals for testing. Expert knowledge-augmented prompts for the Padua and Caprini scores were developed through a Delphi process involving 19 specialists and compared with basic and complex prompts. Performance was evaluated using six open-source large language models (LLMs), with prevalence-adjusted bias-adjusted kappa (PABAK) and F1 score as primary metrics. Expert knowledge-augmented prompts showed the highest performance in both datasets. In the test dataset, mean item-level PABAK and F1 scores were 0.97 and 0.96 for Padua, and 0.97 and 0.93 for Caprini; most items showed PABAK and F1 scores above 0.90. Risk stratification was better for Padua (PABAK 0.92; F1 score 0.96) than for Caprini (PABAK 0.73; F1 score 0.64), with processing times of 6.07 s and 12.82 s per case, respectively. These findings indicated that LLMs with expert knowledge-augmented prompting could efficiently support automated Padua and Caprini scoring and risk stratification in thrombosis prevention workflows.

PMID:
42426240
Bibliographic data and abstract were imported from PubMed on 10 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.