Image Information Prompt: Tips for Learning Large Language Models
Yin Zhang⋆ · Advances in transdisciplinary engineering · 2024
Sentiment text classification is a natural language processing technique for recognizing and extracting subjective information in text, such as emotions, attitudes, and opinions. Traditional models such as BERT and XLNet have achieved brilliant results in text classification problems. With the rise of large language model technology, using generative large models to improve text classification accuracy has become a new research direction. However, compared with traditional classification models, large language models have a slight disadvantage in language classification tasks. In this paper, we propose a prompt-based approach to enhance the accuracy of large language models for text classification using image prompt information on multimodal datasets. First, we illustrate the principle of consistency between image and textual information. Second, we propose a multimodal framework for sentiment analysis of images and text, which realizes the prediction of sentiment tendency for both image and textual data by injecting the image prompt information into the text and into the Large Language Model. Finally, we designed experiments and evaluated them using real multimodal datasets to verify the effectiveness and accuracy of the framework.