Synthetic AI Data Pipeline for Domain-Specific Speech-to-Text Solutions
Anderson Luiz Karl, Guilherme Sales Fernandes, Leonardo Augusto Pires, Yvens Rebouças Serpa, Carlos Caminha · 2024
In this article, we propose a pipeline to fine-tune domain-specific Speech-to-Text (STT) models using synthetic data generated by Artificial Intelligence (AI). Our methodology eliminates the need for manually labelled audio data, which is expensive and difficult to obtain, by generating domain-specific data with a Large Language Model (LLM) combined with multiple Text-to-Speech (TTS) solutions. We applied our pipeline to the radiology domain and compared the results with different approaches based on the availability of domain-specific data, varying from the total absence of domain-specific data to the use of only domain-specific high-quality data (ground truth). Our performance improved the accuracy of the baseline by 40.19% and 10.63% for the WhisperX Tiny and Small models, respectively, which, although performed worse than the results from using the ground truth, shows that it is possible to achieve good results with minimal cost and effort. Finally, the result analysis shows a good insight into the amount of action necessary to achieve good results based on the availability of real data.