Authors
Kuofeng Gao, Jiawang Bai, Shaobo Min, Shu-Tao Xia, Wei Liu, Zhifeng Li
Published in
IEEE transactions on pattern analysis and machine intelligence. Volume PP. Aug 26, 2026. Epub Aug 26, 2026.
Abstract
Recent studies have identified vulnerabilities in contrastive vision-language pre-training (CLIP) models to backdoor attacks. Attackers manipulate models to exhibit targeted behaviors upon encountering specific triggers while maintaining normal performance on clean inputs. Existing backdoor attacks on CLIP primarily focus on the pre-training stage or require extensive data for fine-tuning, which limits their practicality in few-shot transfer learning scenarios. In this work, we introduce BadCLIP, a novel backdoor attack that leverages prompt learning to effectively inject malicious behaviors into CLIP models even in data-scarce scenarios. Our BadCLIP operates under two different scenarios. (1) BadCLIP-W assumes white-box access to models and consists of a learnable trigger and a trigger-aware context generator. To optimize performance, we introduce a trigger warm-up strategy during training. (2) BadCLIP-B functions under black-box constraints to models. To enhance attack performance, we first generate a trigger pattern by a black-box optimization process due to the absence of gradient information. Then, we adopt the trigger pattern to poison the training datasets to inject backdoors via prompt learning. Comprehensive experiments demonstrate that BadCLIP-W and BadCLIP-B achieve high attack success rates while maintaining comparable accuracy on clean images.
PMID:
42647717
Bibliographic data and abstract were imported from PubMed on 27 Aug 2026.
Read full publication at:
Please sign in
to see all details.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 23
- Comments 0