Date
June 15, 2024
Author
Hai Huang, CISPA Helmholtz Center for Information Security; Zhengyu Zhao, Xi’an Jiaotong University; Michael Backes, CISPA Helmholtz Center for Information Security; Yun Shen, NetApp; Yang Zhang, CISPA Helmholtz Center for Information Security
2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024)
Large language models (LLMs) have demonstrated superior performance compared to previous methods on various tasks, and often serve as the foundation models for many researches and services. However, the untrustworthy third-party LLMs may covertly introduce vulnerabilities for downstream tasks. In this paper, we explore the vulnerability of LLMs through the lens of backdoor attacks. Different from existing backdoor attacks against LLMs, ours scatters multiple trigger keys in different prompt components. Such a Composite Backdoor Attack (CBA) is shown to be stealthier than implanting the same multiple trigger keys in only a single component. CBA ensures that the backdoor is activated only when all trigger keys appear. Our experiments demonstrate that CBA is effective in both natural language processing (NLP) and multimodal tasks. For instance, with 3% poisoning samples against the LLaMA-7B model on the Emotion dataset, our attack achieves a 100% Attack Success Rate (ASR) with a False Triggered Rate (FTR) below 2.06% and negligible model accuracy degradation. Our work highlights the necessity of increased security research on the trustworthiness of foundation LLMs.
Resources
The paper can be found at: https://aclanthology.org/2024.findings-naacl.94.pdf