Decoding Silent Speech in Japanese from Single Trial EEGS: Preliminary Results
Yamaguchi H Yamazaki T, Yamamoto K Ueno S · Journal of Computer Science & Systems Biology · 2015
Materials and Methods Subjects, tasks and electrical recordingsTen healthy student volunteers (two females and eight males; mean age: 23.7 ± 1.42 years) participated in Experiment I, whose procedures were approved by the Ethics Committee for Human Subject Research, Faculty of Computer Science and Systems Engineering, Kyushu Institute of Technology.Informed consents were obtained from all the students in writing for the procedures prior to the experiment.All the subjects were right-handed according to the Edinburgh inventory [17].The subjects were requested to speak "rock", "paper" or "scissors" (/ gu:/, /pa:/ or /tʃɔki/ in English pronunciation of Japanese, respectively) into a microphone (MS-STM87SV, ELECOM CO., LTD., Japan) in the learning phase or to silently speak it in the decoding phase, according to visual cues.After the subjects gazed for 3 s a point presented at the center of a monitor 62 cm away from the subjects, a line drawing of a hand indicating "rock", "paper" or "scissors" was presented for the next 3 s.Only the fixation point was presented for the next 3 s.Then, when the point disappeared, the subjects overtly or covertly spoke "rock", "paper" or "scissors" corresponding to the line drawing presented just before (Figure 1).The line drawings were randomly presented ten times for each janken.Nineteen active electrodes (AP-C100-0155, DIGITEX LAB.CO., LTD., Japan) were affixed to the scalp according to the International 10-20 system.Additive six channels were included for electromyograms (EMGs) and electrooculograms (EOGs), so that AbstractWe propose a new scheme for speaker-dependent silent speech recognition systems (SSRSs) using both single-trial electroencephalograms (EEGs) scalp-recorded and speech signals measured during overtly and covertly speaking "janken" and "season" in Japanese.This scheme consists of two phases.The learning phase specifies a Kalman filter using spectrograms of the speech signals and independent components (ICs), whose equivalent current dipole source localization (ECDL) solutions were located mainly at the Broca's area, of the EEGs during the actual speech.In case of the "season" task, the speech signals were transformed into vowel and consonant sequences, and these relationships were learned by hidden Markov model (HMM) with Gaussian mixture densities.The decoding phase predicts spectrograms for the silent "janken" and "season" using the Kalman filter with the EEGs during the silent speech.For the silent "season", the predicted spectrograms were inputted to the HMM, and which "season" was silently spoken was determined by the maximal log-likelihood among each HMM.Our preliminary results as training steps are as follows: the silent "jankens" were correctly discriminated; the silent "season"-HMMs worked well, suggesting that this scheme might be applied to the discrimination between all the pairs of the hiraganas.