Digital music interfaces for motor rehabilitation: a motion capture and machine learning approach
Raquel Lucena Peris · Zenodo (CERN European Organization for Nuclear Research) · 2023
In this work, a digital music interface software has been developed for helping motor rehabilitation in stroke patients by capturing the movement using a webcam. To improve patients’ quality of life using the digital interface, the Processing pro-gramming language has been used to design the interactive interface. Some Python scripts have been used to load the deep learning models that detects the body key-points. Finally, the Wekinator software has been used to learn the positions and predict poses from new incoming data poses. As a result of the previous steps, the interface offers different tasks to improve motion on patients. Two of the tasks have been designed to be used simultaneously with a music therapist or a musical song. On the one hand, while the patient’s position is mapped to a musical sound, the music therapist accompanies the patient with an instrument. On the other hand, a song is selected with the interface and reproduced. The patient keeps the tempo with the movements and accompanies the song with some music notes or drums. The other two tasks are designed so that the patient can use them without much help; it is a game that combines cognitive and motor rehabilitation. The interface has been tested on a few stroke patients and on some healthy people to test its potential. After the sessions, some questionnaires were done to: evaluate the results and compare the digital instrument with a traditional instrument, asses the understability and utility of the interface and check the progress of the patients. The results show that the digital instrument interface has potential as a tool for stroke rehabilitation, but a longer and larger study needs to be done to have robust results.