In search of truth: evaluating concordance of AI-based anatomy segmentation models.

Authors

Lena Giebeler, Deepa Krishnaswamy, David Clunie, Jakob Wasserthal, Lalith Kumar Shiyam Sundar, Andres Diaz-Pinto, Klaus H Maier-Hein, Murong Xu, Bjoern Menze, Steve Pieper, Ron Kikinis, Andrey Fedorov

Published in

Journal of medical imaging (Bellingham, Wash.). Volume 13. Issue 6. Pages 062204. Epub Apr 03, 2026.

Abstract

Artificial intelligence based methods for anatomy segmentation can help automate characterization of large imaging datasets. The growing number of similar functionality models raises the challenge of evaluating them on datasets that do not contain ground truth annotations. We introduce a practical framework to assist in this task.
We harmonize the segmentation results into a standard, interoperable representation, which enables consistent, terminology-based labeling of the structures. We extend 3D Slicer to streamline loading and comparison of these harmonized segmentations and demonstrate how standard representation simplifies review of the results using interactive summary plots and browser-based visualization using the OHIF Viewer. To demonstrate the utility of the approach, we apply it to evaluating segmentation of 31 anatomical structures (lungs, vertebrae, ribs, and heart) by 6 open-source models-TotalSegmentator 1.5 and 2.6, Auto3DSeg, MOOSE, MultiTalent, and CADS-for a sample of computed tomography scans from the publicly available National Lung Screening Trial dataset.
We demonstrate the utility of the framework in enabling automating loading, structure-wise inspection, and comparison across models. Preliminary results ascertain the practical utility of the approach in allowing quick detection and review of problematic results. The comparison shows excellent agreement segmenting some (e.g., lung) but not all structures (e.g., some models produce invalid vertebrae or rib segmentations).
The open-source resources developed include segmentation harmonization scripts, interactive summary plots, and visualization tools. These resources assist in segmentation model evaluation in the absence of ground truth, ultimately enabling informed model selection.

PMID:
41948672
Bibliographic data and abstract were imported from PubMed on 08 Apr 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.