Automatic recognition of anatomical structures and surgical phases in robot-assisted minimally invasive esophagectomy (RAMIE) using deep learning: a retrospective cohort study.

Authors

Romy C van Jaarsveld, Yiping Li, Ronald L P D de Jong, Franco Badaloni, Gino M Kuiper, Tim J M Jaspers, Marcel Breeuwer, Fons van der Sommen, Richard van Hillegersberg, Yasmina Al Khalil, Jelle P Ruurda

Published in

Surgical endoscopy. Jul 09, 2026. Epub Jul 09, 2026.

Abstract

Curative treatment of resectable esophageal cancer comprises neoadjuvant chemoradiotherapy and esophagectomy. Robot-assisted minimally invasive esophagectomy (RAMIE) is the preferred technique; however, learning RAMIE is challenging due to the complex chest anatomy, patient positioning and zoomed-in camera view. Computer-aided anatomy recognition holds promise for improving surgical navigation. This study aims to develop real-time anatomy recognition and surgical phase recognition algorithms for the thoracic part of RAMIE using deep learning and to understand the challenges of current state-of-the-art algorithms.
A retrospective single-center cohort study was conducted on prospectively collected RAMIE videos at University Medical Center Utrecht, The Netherlands. Two datasets were created: an anatomy segmentation dataset with 1504 frames from 53 videos, annotated for eight anatomical classes and four surgical instruments; and a surgical phase dataset, with 38 videos labeled with thirteen distinct phases. Several deep learning models were trained and tested using both datasets.
The SegNeXt model achieved the most accurate segmentations with an overall overlap score of 0.72 for all classes, of which the surgical instruments were best detectable. The best surgical phase recognition model achieved an overall accuracy of 82.8% and revealed loss of accuracy during phase transitions. Class imbalance affected both datasets which led to less frequent appearing classes performing lowest.
This study identified suitable models for computer-aided, real-time anatomy and surgical tool segmentation and surgical phase recognition for the thoracic part of RAMIE. Acceptable performance was achieved while both datasets are highly complex. Performance is expected to improve by further expanding and diversifying the datasets.

PMID:
42423778
Bibliographic data and abstract were imported from PubMed on 09 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.