CNN-based character recognition for a contextless text input system in immersive VR
Szymon Kuś, Robert Szmurło · 2021
The paper investigates the applicability of deep neural networks and transfer learning procedure to developing a handwriting recognition system for typing contextless content in a Virtual Reality system. Inputting special characters in such systems is still a nontrivial task, even nowadays. Most algorithms for editing text focus on contextual text recognition from speech or gestures. Unfortunately, these approaches fail for text containing not only letters additionally in the context of words, phrases, and trivial punctuation characters. In the paper, the authors explore the possibility of typing more advanced structures, which for example, can be observed in source code editing. The aim of the paper is to verify if modern machine learning algorithms can facilitate text typing in a context-free environment. The authors have chosen a transfer learning procedure utilizing already trained convolutional layers from ALEXNET [1], followed by deep network classification layers. The methods and procedures implemented for the Virtual Reality system used to acquire data necessary to train the recognition model are also presented.