A Hybrid Human-AI Workflow for Ultrasound Report Quality Control: A Multicenter Performance and Efficiency Evaluation.

Authors

Xuan Liang, Hui He, Chunzheng Liu, Rui Sang, Jialin Zhu, Xiaoqing Wang, Wenjing Hou, Deyi Li, Junguang Wang, Yuanyuan Li, Jiayu Su, Xiaoyi Wang, Xi Wei

Published in

Academic radiology. Oct 17, 2026. Epub Oct 17, 2026.

Abstract

This study evaluated a hybrid human-AI workflow for ultrasound report quality assurance using multicenter retrospective data.
786 anonymized ultrasound reports covering 22 examination categories were collected from four hospitals. Reports containing naturally occurring or implanted errors were matched across arms. GPT-4o and DeepSeek-v3 performed initial screening before review by six human reviewers (HRs). HR with blinded expert over-reading (HR-ex) served as control. Accuracy, false-positive (FP) and false-negative (FN) rates, and review time were compared. Supplementary validation compared HR and DeepSeek-assisted review using 1680 consecutive reports without implanted errors.
Hybrid workflow (HYB) accuracy was comparable to HR (4o-HYB 80.2%, v3-HYB 82.5%, HR 82.3%) but lower than HR-ex (87.3%). Both HYBs showed higher FN rates than HR-ex (4o-HYB 34.2% and v3-HYB 27.3% vs HR-ex 16.9%, both p < 0.001). Severity assessment did not indicate preferential omission of clinically significant errors. Human verification reduced LLM-generated FNs and FPs. Median review time decreased substantially from 51.0 s (HR-ex) to 11.0 s (4o-HYB) and 12.0 s (v3-HYB). In supplementary validation (error prevalence, 9.6%), v3.2-HYB and HR showed FN rates of 22.6% and 24.3%, FP rates of 7.7% and 10.0%, and median review times of 15.4 and 24.3 s, respectively.
The LLM-assisted HYB achieved accuracy comparable to single-physician review while substantially reducing audit time. These findings require prospective real-world validation before routine clinical implementation.

PMID:
42892597
Bibliographic data and abstract were imported from PubMed on 18 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.