PhonoNet: Multi-stage deep learning for raga preservation in hindustani classical music
Sauhaarda Chowdhuri · The Journal of the Acoustical Society of America · 2019
Hindustani classical music is an ancient improvisational form of music based on ragas, melodic frameworks which have no written representation and are passed down through a fading oral tradition. The proposed system, PhonoNet, aims to preserve Hindustani classical music by providing computational prediction of ragas, so singers can receive live feedback when learning. PhonoNet also creates a visual format for documenting and preserving raga information. First, the system computes the short-term Fourier Transform of the input audio data to form a chromagram representation of the notes being sung. These data are then augmented using a transpositional data augmentation algorithm and split into chunks for use as training inputs for a deep convolutional neural network. The convolutional network's filters are analyzed using a saliency visualization algorithm and modified with a recurrent layer to allow processing of full-length songs. The convolutional system achieves 78.9% validation accuracy for raga prediction on 150 second audio chunks. The joint raga prediction system achieves a new state-of-the-art 98.9% accuracy for raga prediction on full-length songs. Future work can extend the proposed hierarchical system to other tasks with long temporal sequences and extend the data augmentation and visualization algorithms to different applications of audio processing.