DST-GFN: A Dual-Stage Transformer Network with Gated Fusion for Pairwise User Preference Prediction in Dialogue Systems
Kowei Shih, Zhenghao Deng, Xiang Chen, Yuanzhe Zhang, Li Zhang · 2025
User preference prediction is important for customizing responses in large language models (LLMs) for dialogue systems. This paper presents DST-GFN (Dual-Stage TransformerGated Fusion Network), a model made to predict preferences from two responses generated by LLMs. DST-GFN uses a DualStage Transformer Encoder, a Gated Fusion Block (GFB), and a Hierarchical Contextual Fusion (HCF) layer to find connections and differences between responses. The model processes prompt-response pairs and the relationship between the two responses with two encoders. A gating method then combines the outputs. The final prediction is made using a Softmax layer. It applies a weighted cross-entropy loss and $L 2$ regularization to reduce class imbalance and overfitting. Tests show that DST-GFN performs better than models like BERT, LSTM, and GRU. Its key parts, the Dual-Stage Encoder and Gated Fusion Block, are also shown to be effective. The model can also work in multilingual and multimodal situations.