Multimodal Approach to Recommend Movie Genres Based on Multi Datasets
Preksha Khant, Bharat Tidke · Indian Journal of Science and Technology · 2023
Objectives: The main purpose of this study is to implement the multimodal as well as the multilabel approach of accurate movie genre classification using the Actor, Director, Writer, Poster Images, and Synopsis data. The various kind of data collected from IMDB as well as existing datasets are used to train the models. Methods: The 5 deep learning models are created on Poster Images, Text Data, Actors Data, Directors Data, and Writers Data. These models are then combined using the weighted average ensemble model of deep learning. The final output gives the prediction scores of the top 3 genres for a given movie. The main deep learning model used for poster image model training is CNN. The LSTM model is used for text/synopsis model training. The multilabel along with the multimodal approach yielded good results in terms of accurate predictions than the existing models. Findings: The total F1 Score is 0.65. Along with good results, it has some limitations which are further discussed in the paper. Novelty: The novelty of the work lies in 3 major aspects, firstly the consideration of the Actor, Director, and Writer data for the genre classification, as usually, an actor does a particular kind of movie, a writer writes a particular type of movie and director directs a particular type or genre movie. Although there are many exceptions but usually this is the case. The second novelty lies in combining the models using the weighted average method wherein every model is assigned a weight for how much it will contribute to the final genre classification. And lastly, the novelty lies in how the weights are defined i.e., we used the correlation method to determine the weights of every individual model for genre classification. Keywords: Movie Genre Classification; CNN; Multimodal Movie Genre; Multilabel Movie Genre; LSTM; Deep Learning