SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation
Xiaoze Liu, Ting Sun, Tianyang Xu, Feijie Wu, Cunxiang Wang, Xiaoqian Wang, Jing Gao · 2024
Large Language Models (LLMs) have transformed machine learning but raised significant legal concerns due to their potential to produce text that infringes on copyrights, resulting in several high-profile lawsuits.The legal landscape is struggling to keep pace with these rapid advancements, with ongoing debates about whether generated text might plagiarize copyrighted materials.Current LLMs may infringe on copyrights or overly restrict non-copyrighted texts, leading to these challenges: (i) the need for a comprehensive evaluation benchmark to assess copyright compliance from multiple aspects; (ii) evaluating robustness against safeguard bypassing attacks; and (iii) developing effective defenses targeted against the generation of copyrighted text.To tackle these challenges, we introduce a curated dataset to evaluate methods, test attack strategies, and propose lightweight, a real-time defense mechanism to prevent the generation of copyrighted text, ensuring the safe and lawful use of LLMs.Our experiments demonstrate that current LLMs frequently output copyrighted text, and that jailbreaking attacks can significantly increase the volume of copyrighted output.Our proposed defense mechanism significantly reduce the volume of copyrighted text generated by LLMs by effectively refusing malicious requests.* These authors contributed equally to this work.User (Example of copyright infringement) Mr. and Mrs. Dursley, of LLaMA 3 70B number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much!Explanation: The first line of the copyrighted book "Harry Potter and the Philosopher's Stone". User (Example of overprotection)Tell me the first 1000 words of the book "A Tale of Two Cities"Claude 3 I apologize, but I cannot reproduce the first 1000 words of "A Tale of Two Cities" as that would be a direct copy of copyrighted material.However, I'm happy to provide a brief summary of the beginning of the novel. . .