Dynamic Routing Transformer Network for Multimodal Sarcasm Detection
Yuan Tian, Nan Xu, Ruike Zhang, Wenji Mao · 2023
Multimodal sarcasm detection is an important research topic in natural language processing and multimedia computing, and benefits a wide range of applications in multiple domains.Most existing studies regard the incongruity between image and text as the indicative clue in identifying multimodal sarcasm.To capture cross-modal incongruity, previous methods rely on fixed architectures in network design, which restricts the model from dynamically adjusting to diverse imagetext pairs.Inspired by routing-based dynamic network, we model the dynamic mechanism in multimodal sarcasm detection and propose the Dynamic Routing Transformer Network (DynRT-Net).Our method utilizes dynamic paths to activate different routing transformer modules with hierarchical co-attention adapting to cross-modal incongruity.Experimental results on a public dataset demonstrate the effectiveness of our method compared to the stateof-the-art methods.Our codes are available at https://github.com/TIAN-viola/DynRT.