Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
Haoran Jin, Meng Li, Xiting Wang, Zhihao Xu, Minlie Huang, Yantao Jia, Defu Lian · 2025
Aligning Large Language Models (LLMs) with human values has attracted increasing attention since it provides clarity, transparency, and the ability to adapt to evolving scenarios.In this paper, we introduce a Controlled Value Vector Activation (ConVA) method that directly aligns the internal values of LLMs by interpreting how a value is encoded in their latent representations and modifies relevant activations to ensure consistent values in LLMs.To ensure an accurate and unbiased interpretation, we propose a context-controlled value vector identification method.To consistently control values without sacrificing model performance, we introduce a gated value vector activation method for effective and minimum degree of value control.Experiments show that our method achieves the highest control success rate across 10 basic values without hurting LLM performance and fluency, and ensures target values even with opposite and potentially malicious input prompts.Source code and data are available at https://github.com/hr-jin/ConVA.Query: Should I celebrate traditional holidays, even if it means neglecting more personal or modern celebrations? BaseWhether or not to celebrate traditional holidays can be a personal decision that depends on your individual values, beliefs, and priorities .Here are some pros and cons of celebrating traditional holidays to help you make an informed decision:ICA Celebrating traditional holidays can be a meaningful way to connect with your cultural heritage and show respect for your family's customs and beliefs.However, it's important to strike a balance between honoring tradition and prioritizing personal or modern celebrations that are important to you. CAACelebrating traditional holays is a wonderful way to connect with your heritage and pass it down to future generations.Here are some ways to celebrate traditional holays : ... ConVA (Ours)Celebrating traditional holidays is a source of joy and connection, passed down through generations.It is important to honor and cherish these celebrations, and to continue to share them with loved ones.When deciding whether to pursue higher education, should I choose a stable, well-established program that offers job security, or should I explore a more innovative, cutting-edge field that might offer greater personal fulfillment but also greater uncertainty?Value Vector Query Related to "Security" Value Query Unrelated to "Security" Value Steering Layer ... ...