A Siam-LSTM Model for Multichannel EEG on VR Motion Sickness Recognition
Chengcheng cheng Hua, Zhanfeng Zhou, Jianlong Tao, Ying Yan, Jia Liu, Rongrong Fu, Lining Chai · IEEE Sensors Journal · 2024
The combination of prior knowledge of electroencephalography (EEG) and deep learning always receives better results in EEG pattern recognition. Thus, we propose end-to-end models combining functional brain network (FBN) and Siamese long short-term memory (Siam-LSTM) model and apply them to a virtual reality motion sickness (VRMS) recognition task. Siam-LSTM is a key module in the proposed models to process EEG signal electrode by electrode via the shared layers. The models apply 1D-convolutional neural network (CNN) and Siam-LSTM to extract temporal features, and then, Pearson correlation coefficients of the features among the electrodes are computed as the FBNs. The 2D-CNN and dense networks are used to extract the FBN feature and do regression analysis. In final, a VRMS-related EEG dataset is used to verify the models by tenfold cross-validation, while the VRMS level is assessed by the subjective simulator sickness questionnaire (SSQ). The experimental results show that the two major proposed models named BC-CNN-LSTM and BC-CNN-LSTM2 obtain the best performance in regression, i.e., mean square errors (mses) between the predicted SSQ scores and the real SSQ scores are$26.89 \; \pm \; 8.89$and$24.75 \; \pm \; 4.70$, respectively. The proposed models provide a new idea for long short-term memory (LSTM) to process multielectrode EEG signals and outperform the state-of-the-art methods in VRMS recognition.