MAEM: A Multi-Aspect Extraction Model for Enhanced Embedding in RAG
Ningyuan Yi, Chen Liu, Yue Wang, Jianjun Yu · 2025
Retrieval Augmented Generation can effectively reduce hallucinations in LLMs during question-answering, with embedding models directly influencing its performance. While current embedding models improve encoding through large-scale training, they often overlook the potential or explicit multi-aspect information within the text. To address this, we propose the Multi-Aspect Extraction Model (MAEM), an eigen decomposition-based approach that extracts and integrates text aspects into a unified vector for enhanced retrieval, and introduce a regularization loss function to assist in training. We utilized LLMs to create the Policy-Corpus dataset and validated the model on both Policy-Corpus and FiQA. Incorporating MAEM and regularization into GTEbaseimproved NDCG@10 by 3.8 points on FiQA and 4.56 points on Policy-Corpus. Respectively, achieving results comparable to larger models using a smaller parameter model.