Identifying anaphylaxis using weakly-supervised prediction models and natural language processing.

Authors

Brian D Williamson, David J Cronkite, Onchee Yu, Arvind Ramaprasan, Sharon Fuller, Jennifer Covey, Erika Kiniry, Daniel Park, Robert Winter, Jill Whitaker, Michael F McLemore, Saranrat Wittayanukorn, Danijela Stojanovic, Yueqin Zhao, Sarah Dutcher, David S Carrell, Lisa A Jackson, Jennifer C Nelson, Joshua C Smith

Published in

medRxiv : the preprint server for health sciences. Jun 17, 2026. Epub Jun 17, 2026.

Abstract

Scalable computable phenotyping algorithms are critical for conducting high-throughput disease-outcome research in large, distributed-data electronic health record (EHR) and claims data settings. We developed and evaluated a claims- and EHR-based computable phenotyping algorithm for anaphylaxis, a rare acute condition that is challenging to accurately identify using claims data alone.
Potential anaphylaxis events came from two healthcare systems (Kaiser Permanente Washington [KPWA] and Vanderbilt University Medical Center [VUMC]). We engineered features from clinical text using automated natural language processing (NLP) methods. We then developed a phenotyping algorithm using four NLP- and diagnosis code-based silver labels (proxies for the gold-standard labels). Gold-standard abstracted outcomes were used to evaluate algorithm performance.
The largest area under the receiver operating characteristic curve (AUC) was 0.931 for an NLP-based silver-label model at KPWA. Depending on the model and healthcare system site, positive predictive value (PPV) and sensitivity at the threshold of predicted probability that maximized F1 score ranged from 0.52 to 0.77 (PPV) and 0.78 to 1 (sensitivity).
NLP-based silver-label models had large AUC at KPWA but not at VUMC. This may be because clinical text at KPWA is only available for outpatient encounters and secure messaging. High sensitivity for identifying anaphylaxis can be obtained using our best-performing models.
The best-performing models had better PPV and sensitivity tradeoffs than prior bespoke anaphylaxis models with costly, manually curated features. The simplicity of the approach compared to traditional phenotyping methods allows it to be deployed easily at multiple health care systems.

PMID:
42369512
Bibliographic data and abstract were imported from PubMed on 29 Jun 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.