SUDA: Simultaneous unsupervised knowledge distillation and adaptation of foundation models for efficient pathological image analysis.

Authors

Lanfeng Zhong, Kun Qian, Weiren Zhao, Tian Shen, Shan Xu, Jianming Li, Guotai Wang

Published in

Medical image analysis. Volume 113. Pages 104177. Jun 26, 2026. Epub Jun 26, 2026.

Abstract

Pathology foundation models have greatly advanced pathological image analysis due to their generalizable feature representation capabilities after learning from a large dataset. However, they still face two critical limitations: first, their large model size and high computational cost hinder clinical deployment; second, domain shifts between pretraining and downstream datasets degrade performance on target tasks. While existing Knowledge Distillation (KD) methods have shown effectiveness in model compression, they typically rely on fully labeled data and overlook domain shifts between pretraining and downstream tasks. To overcome these limitations, we propose Simultaneous Unsupervised knowledge Distillation and Adaptation (SUDA) that integrates KD with self-supervised learning, enabling effective adaptation and compression of foundation models into lightweight ones with high performance on target domains. Specifically, SUDA introduces Dual Instance Discrimination Distillation (DI2D) that contains teacher-guided instance and relation-wise distillation and student self-guided instance discrimination distillation. We further propose Masked Consistency Modeling (MCM) to improve the student model's feature consistency under masking, enabling better adaptation to downstream pathology datasets without human annotations. Extensive experiments on two patch-level and three WSI-level pathology image datasets demonstrate that our SUDA outperformed existing knowledge distillation and self-supervised learning methods under different student and teacher structures. What is more, the student after SUDA matches or even surpasses the performance of the teacher model with only 0.018× parameters, significantly reducing the computational cost.

PMID:
42364292
Bibliographic data and abstract were imported from PubMed on 28 Jun 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.