Multimodal Recognition of Emotions with Application to Mobile Learning
María Lucía Barrón Estrada, Ramón Zataraín Cabada, Claudia Guadalupe Aispuro-Gallegos · 2018
A great variety of emotional recognition systems have been implemented, but few have been applied in the real world due to the high cost of the necessary technology and due to the low percentage of recognition effectiveness, when one does not work with spontaneous emotions. This paper presents the initial implementation of a system of multimodal recognition of emotions using mobile devices and the creation of an affective database through a mobile application. The recognizer works into a mobile educational application to identify user's emotions as they interact with the device. The emotions that the system recognizes are engagement and boredom. The affective database was created with spontaneous emotions of students who interacted with an educational mobile application called Duolingo and a mobile information gathering application called EmoData. The developed system has a regular percentage of effectiveness for audio recognition and position recognition. The corpus used for training is still small. We believe that as the number of registrations in the affective database increases, the precision will improve.