Foundation Models and Multimodal Learning for Medical Data with Applications in Oncology.

Authors

Asim Waqas, Aakash Tripathi, Siddharth Sivaram, Farzana Ali, Shiba Kuanar, Yashbir Singh, Hanieh Ajami, Orhan Unal, Yuankai Huo, Abdussalam Elhanashi, Qiangqiang Gu, Khaled Younis, Cody Heath Savage, Iryna Hartsock, Nikolas Koutsoubis, Samir Alejandro Nacer, John Michael Templeton, Matthew B Schabath, Ghulam Rasool

Published in

Journal of imaging informatics in medicine. Oct 13, 2026. Epub Oct 13, 2026.

Abstract

The rapid expansion of medical data, including radiological imaging, digital pathology, molecular profiling, electronic health records, and longitudinal clinical narratives, has created new opportunities for AI in healthcare, while also exposing the limitations of traditional unimodal modeling approaches. Multimodal learning addresses these challenges by integrating heterogeneous data sources, and recent advances in foundation models, large-scale pretrained models designed for broad task transfer, have further accelerated this shift. Together, these developments have given rise to multimodal foundation models that aim to learn generalizable, aligned representations across diverse clinical modalities. In this review, we provide an overview of multimodal foundation models in medical AI, with a particular focus on oncology. We first outline key medical data modalities and the machine learning objectives that motivate multimodal integration, including predictive, representational, and alignment-based goals. We then examine core architectural design patterns and training paradigms underlying multimodal foundation models, such as modality-specific encoders, fusion strategies, attention mechanisms, contrastive pretraining, and scalable self-supervised learning. Building on this foundation, we survey representative multimodal models across radiology, pathology, clinical text, and molecular domains. We further review applications of multimodal foundation models across the cancer care continuum, including early detection, diagnosis and subtyping, prognosis, therapy response prediction, and longitudinal survival modeling. Finally, we discuss critical challenges that limit clinical translation, including cross-institutional heterogeneity, temporal misalignment across modalities, weak supervision, interpretability, regulatory constraints, and workflow integration, and outline future directions for advancing safe and effective deployment in real-world oncology practice.

PMID:
42870148
Bibliographic data and abstract were imported from PubMed on 14 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.