Authors
David Fernández-Narro, Pablo Ferri, Juan Miguel García-Gómez, Carlos Sáez
Published in
Studies in health technology and informatics. Volume 332. Pages 47-51. Oct 02, 2025.
Abstract
Out-of-distribution data , data coming from a different distribution with respect to the training data, entails a critical challenge for the robustness and safety of AI-based clinical decision support systems (CDSSs). This work aims to investigate whether real-time, sample-level quantification of epistemic uncertainty, the model's uncertainty due to limited knowledge of the true data-generating process, can act as a lightweight safety layer for health AI and CDSSs, targeting model updates and spotlighting human review. To this end, we trained and evaluated a continual learning-based neural network classifier on quarterly batches in a real-world Mexican COVID-19 dataset. For each training window, we estimated the distribution of the prediction epistemic uncertainties using Monte Carlo Dropout. We set a data-driven uncertainty threshold to determine potential out-of-distribution samples at 95% of that distribution. Results across all training-test time pairs show that samples below this threshold exhibit consistently higher macro-F1 and render performance virtually invariant to temporal drift, while the flagged samples captured most prediction errors. Since our method requires no model retraining, sample-level epistemic uncertainty screening offers a practical and efficient first line of defense for deploying health-AI systems in dynamic environments.
PMID:
41041744
Bibliographic data and abstract were imported from PubMed on 03 Oct 2025.
Read full publication at:
Please sign in
to see all details.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 55
- Comments 0