Improving Speech Recognition for Japanese Deaf and Hard-of-Hearing People by Replacing Encoder Layers
Kaito Takahashi, Yukoh Wakabayashi, Kengo Ohta, Akio Kobayashi, Norihide Kitaoka · 2024
Communication between hearing individuals and those with hearing impairments generally involves sign language, written communication, and speech. It has been reported that more than half of Japanese people with hearing impairments communicate using speech. Therefore, speech recognition systems available for individuals with hearing impairments are demanded. However, speech recognition systems trained on speech from hearing individuals do not achieve high recognition accuracy for speech from individuals with hearing impairments. In this study, we propose a method to replace the encoder layer of the speech recognition model based on SSL to achieve high-accuracy speech recognition for speech from individuals with hearing impairments. By this method, we improved the recognition performance for significantly speech from individuals with hearing impairments.