Authors
Xili Deng, Yi Zhao, Nanqing Liu, Yaping Zhang, Chengfu Yang, Zhanxuan Hu
Published in
Neural networks : the official journal of the International Neural Network Society. Volume 206. Issue Pt C. Pages 109901. Nov 14, 2026. Epub Nov 14, 2026.
Abstract
Existing vision-language models (VLMs) exhibit strong zero-shot generalization, yet remain fundamentally limited in Unsupervised Multi-label Image Classification (UMIC), primarily due to their inadequate multi-object perception and the intrinsic modality gap between vision and language. In particular, global representations tend to entangle multiple concepts, hindering accurate recognition of co-existing objects. To address these challenges, we revisit UMIC from a decomposition-then-classification perspective. Instead of predicting multiple concepts holistically, we first decompose an unlabeled multi-label image into multiple single-concept visual instances, thereby explicitly improving multi-object perception. We then perform classification based on aggregated predictions. Building upon this paradigm, we propose ClassifierAlign, a fully training-free framework that jointly leverages visual foundation models (VFMs) and VLMs. Specifically, we derive a lightweight visual classifier directly in the visual feature space, and semantically align it with CLIP's textual classifier, effectively bridging the vision-language modality gap without updating parameters of the pretrained foundation models. This design further enables seamless integration of powerful self-supervised VFMs (e.g., DINO), yielding robust and transferable representations without requiring spatial alignment. Extensive experiments demonstrate that ClassifierAlign achieves competitive performance with fine-tuning-based methods, while significantly reducing computational overhead and eliminating the need for costly annotated data. The source code for this work can be found at https://github.com/dengxili/ClassifierAlign.
PMID:
43033644
Bibliographic data and abstract were imported from PubMed on 15 Nov 2026.
Read full publication at:
Please sign in
to see all details.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 6
- Comments 0