A Multimodal Entity Linking Approach Incorporating Topic Concepts
Pengyuan Li, Yongli Wang · 2021 International Conference on Computer Information Science and Artificial Intelligence (CISAI) · 2021
The characteristics of social media posts are ambiguous, noisy, and not rich in context and semantics, which poses a significant challenge to the task of entity linking. To this end, we propose a multimodal entity linking method TCMEL that 1) integrates topic concepts by Clustering entity concepts to clarify text topics, 2) extracts context features from both text and image using Bi-LSTM and CNN respectively, and 3) aggregates contextual information extracted from subject concepts, text, images and lexical symbols and calculates the similarity between knowledge base entities and text mentions. We demonstrate the effectiveness of the proposed TCMEL method by conducting comparative experiments on the MELBench dataset containing multimodal data from multiple domains.