Application of English semantic understanding in multimodal machine learning
Luo Yun · 2024
This article explores the application of English semantic understanding in multi-modal machine learning, especially focusing on the two core tasks of sentiment analysis and information retrieval. By conducting experiments on the public multimodal datasets CMU-MOSI and Flickr30k, this study evaluates the performance difference between multimodal methods that integrate text, image, and audio information and single-modal methods. Experimental results show that the multi-modal method significantly improves the accuracy in the sentiment analysis task, reaching 90%. In the information retrieval task, the multi-modal method also shows higher precision, recall and F1 score. . These findings demonstrate the effectiveness of multimodal learning in improving the model's ability to understand complex English semantics. In addition, this article also discusses the future research directions of multimodal technology in the field of English semantic understanding, including exploring more advanced modal fusion technology, developing robust multimodal models, and improving the transparency and interpretability of the model.