Transporting a pediatric surgical site infection prediction model to the electronic health record: external validation, harmonization, and recalibration.

Authors

Carrie T Chan, Mark J Pletcher, Karthik Balakrishnan, Yulin Hswen, Aaron Scheffler

Published in

Journal of the American Medical Informatics Association : JAMIA. Sep 25, 2026. Epub Sep 25, 2026.

Abstract

To externally validate a pediatric surgical site infection (SSI) prediction model, assess institutional and temporal transportability, and evaluate recalibration strategies to support adaptation of registry-derived predictions for clinical decision support (CDS).
An elastic-net logistic regression model developed using a national pediatric surgical registry (2012-2022) was validated in an institutional electronic health record (EHR)-derived cohort (10 450 procedures) and in a newly released, temporally distinct cohort from the same registry (301 999 procedures). We assessed discrimination, calibration, and overall performance, used a closed testing procedure to guide recalibration, and evaluated clinical utility using decision curve analysis.
Performance was preserved in the registry cohort, with c-statistic 0.75, Brier score 0.03, calibration slope 1.00, and observed/expected (O/E) ratio 1.01. Direct application to the institutional EHR cohort produced inflated predicted risk and poor performance, with c-statistic 0.60, Brier score 0.07, calibration slope 0.15, and O/E ratio 0.25. After omitting open-ended diagnosis-code indicators, performance improved, with c-statistic 0.70, Brier score 0.03, calibration slope 0.81, and O/E ratio 1.35. Logistic recalibration restored calibration slope and O/E ratio to 1.00. Decision curve analysis showed positive net benefit across prespecified thresholds.
These findings highlight a key challenge for clinical translation: models that perform well under internal or registry-based validation may not be directly transportable to real-world EHR environments. Predictor misalignment-differences in how model predictors were captured and measured across registry and EHR data-emerged as a key barrier to direct institutional deployment, underscoring the importance of implementation-aware external validation before clinical use.
Addressing diagnosis-code predictor misalignment and applying parsimonious recalibration supported institutional adaptation of a pediatric SSI prediction model, providing a pragmatic pathway for translating registry-derived models into real-world clinical practice. This work sets the foundation for prospective evaluation of EHR-embedded CDS to enable individualized SSI risk estimation at the point of care.

PMID:
42789965
Bibliographic data and abstract were imported from PubMed on 26 Sep 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.