Construction of User Multimedia Fusion Cognition Theory in Digital Culture Driven by Multimodal Data
Hui Lin · 2024
This research addresses the simply stacking status of multimedia convergence resources in digital cultural services. It focuses on the users' cognitive state associated with the use of multimedia combinations. The aim of the study is to construct a new theory of user multimedia fusion cognition in digital culture. The main research question is how to explain users' cognitive state during the utilization of multimedia fusion resources in DC. To answer that, a series of sub research questions have arisen. A mixed methodology of user experiment and algorithm optimization is designed. Following the research path, preliminarily, sub question 1 and sub question 2 have been solved. In the preliminary study, we designed an EEG-eye movement collaborative experiment to examine whether digital cultural multimedia fusion resources bring cognitive differences to users and attempted to use multimodal subjective and objective data collaborative modeling to predict user cognitive load. The findings showed that there were differences of users' cognition in digital cultural multimedia convergence and the multimodal subjective-objective synergistic prediction effect was the most optimal.