Dual-oriented Disentangled Network with Counterfactual Intervention for Multimodal Intent Detection
Zhanpeng Chen, Zhihong Zhu, Xianwei Zhuang, Zhiqi Huang, Yuexian Zou · 2024
Multimodal intent detection leverages diverse modalities for a comprehensive understanding of user intentions in real-world scenarios, playing a critical role in modern taskoriented dialogue systems.While existing methods have made progress in modal alignment and fusion, they overlook two vital limitations: (I) Close entanglement of multimodal semantics with modal structures; (II) Insufficient learning of the causal effects of semantic and modality-specific information on final predictions in end-to-end training.To address these limitations, we introduce the Dualoriented Disentangled Network with Counterfactual Intervention (DuoDN).DuoDN consists of a Dual-oriented Disentangled Encoder that decouples semantics-and modality-oriented representations, and a Counterfactual Intervention Module that uses causal inference to understand causal effects by injecting confounders.Experiments on three benchmark datasets demonstrate DuoDN's superiority over existing methods, with extensive analysis validating its advantages.