Federated artificial intelligence monitoring service (FAMOS): an in silico feasibility study.

Authors

Aysha Luis, Andrew Scarsbrook, Mariusz Grzeda, Jesus Perdomo Lampignano, Matt Clark, Bob Wheller, Jack Baldwin, James Cairns, Simran Dhesi, Mark Hall, David J Lowe, Haris Shuaib

Published in

BJR artificial intelligence. Volume 3. Issue 1. Pages ubag016. Epub Sep 21, 2026.

Abstract

To evaluate feasibility of a federated AI monitoring service (FAMOS) for post-deployment surveillance of third-party AI applications used in chest X-ray (CXR) interpretation.
FAMOS was deployed at 2 NHS Trusts using a federated architecture enabling local data processing while maintaining data governance compliance. De-identified CXRs from patients aged >18 years were retrospectively identified, along with relevant patient attributes (age, sex, inpatient status, image orientation, season, artefact). Chest X-rays were processed by 2 AI applications to simulate real-world deployment. FAMOS analyzed input data, AI inference values, and longitudinal human-AI agreement as proxy indicators of data, prediction, and behavioral drift. Input monitoring used image feature embeddings analyzed with principal component analysis and Hotelling's T² statistics, while risk-adjusted cumulative sum charts were applied to sequentially monitor AI inference values. human-AI agreement was evaluated at 10 time points over 3 months.
Input monitoring identified a small proportion of outlier examinations, predominantly associated with modifiable image quality issues. Artificial intelligence inference values remained stable across most findings for both vendors, with limited drift events detected. Human-AI agreement patterns differed between sites, remaining stable at 1 site while increasing over time at another, suggesting evolving automation bias.
Real-time, federated monitoring of deployed radiology AI systems is technically and operationally feasible within clinical environments. Multi-domain platform-based monitoring provides scalable, independent oversight and may function as an early-warning system supporting identification of emerging risks following deployment.
This study introduces a federated, multi-domain monitoring framework integrating sociotechnical indicators for continuous post-deployment surveillance of clinical radiology AI tools.

PMID:
42818595
Bibliographic data and abstract were imported from PubMed on 01 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.