An Investigation on the Speech Recovery from EEG Signals Using Transformer
Tomoaki Mizuno, Takuya Kishida, Natsue Yoshimura, Toru Nakashika · 2024
In recent years, brain-machine interfaces (BMI) have been researched to enable people who cannot physically speak or who are placed in situations where they cannot speak to engage in speech communication. However, synthesizing complete speech from ElectroEncephaloGraphy (EEG) signals remains a challenging task. In this paper, reconstructing speech from EEG signals, a Transformer-based model has been developed on the basis of data from listening to the speeches of two people, one male and one female. The objective of this study is to investigate the potential to reconstruct speech from EEG signals, including the characteristics of the corresponding speaker’s speech and the potential to reconstruct speeches containing the corresponding linguistic content, by training a single model on both male and female speeches. Our findings reveal that our model can generate two distinctly different speakers’ speeches from the EEG signals of the two speakers’ speeches. Additionally, the EEG signals appear to contain information about speaker characteristics that can be reconstructed from the EEG signals.