Benchmarking Transformer-Based Transcription on Embedded GPUs for Space Applications
Marika E. Schubert, Alan D. George · 2021 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · 2021
Speech transcription is a necessary tool for backend applications commonly found in voice assistants. Transcription is typically performed using cloud-based servers or custom hardware, but those resources are not always amenable to space environments due to size, weight, power, and cost constraints. Therefore, it is important to determine the performance of and optimal conditions for running transcription on hardware that is feasible for deployment in a space application. This research investigates and evaluates the performance of the wav2vec2 speech transcription engine, the current state-of-the-art model for this domain with and without optimizations. The target hardware, the NVIDIA Xavier NX Jetson embedded GPU, was chosen for its modern GPU architecture and small form factor. In addition to examining the input scaling behavior, we evaluate the hyperparameters of the clustered attention optimization, and average power and energy for inference relative to the operating power mode of the device. The clustered attention model outperformed the improved-clustered model for large input sizes, but the wav2vec2 model without clustering performed better for small input sizes. The clustered model energy per inference (13.90 J) was less than energy per inference of the improved-cluster model (15.03 J) and the vanilla softmax model (15.85 J). All models meet real-time speech processing requirements necessary to perform onboard inference entirely on a space system.