A Hybrid Framework for Enhanced Prompt Generation: Integrating Mean Templates, Fine-Tuned Models, and Clustering Mechanisms
Zeyu Wang, Beichen Liu, 典子 江谷, Lisha Zhou, Xinshi Li · 2025
The effective generation of prompts is crucial for optimizing the performance of language models. This article introduces a novel hybrid method aimed at improving prompt prediction accuracy by combining a mean prompt template, fine-tuned Mistral models, clustering strategies, and gating mechanisms. Our approach strives to achieve a balance between broad adaptability and specific task accuracy, leveraging a multi-layer component structure. The proposed model integrates components such as MistralForCausalLM and MistralForSequenceClassification to enhance prompt selection and alignment. Experimental results demonstrate significant improvements across multiple performance metrics, highlighting the effectiveness of our model compared to existing methods, while also ensuring data consistency through a comprehensive preprocessing strategy involving various large language models to simulate structural diversity in prompt generation.