Cultures of the AI paralinguistic in voice cloning tools
Ada Ada Ada, Stina Hasse Jørgensen, Jonas Fritsch · Designing Interactive Systems Conference · 2024
With AI-based voice cloning tools becoming more accessible to designers, we deem it imperative to understand their paralinguistic capabilities, limitations and cultures. Paralinguistics as a field of study is concerned with how you say something rather than what you say, and new AI-based statistical voice synthesis tools differ significantly from previous methods. As such, they require asking novel questions and provoking new thoughts. This paper contributes by analyzing and evaluating various voice cloning platforms by looking at how they describe their own ability to produce three different paralinguistic elements: laughter, stuttering and pacing. We focus on text-to-speech and hybrid approaches to voice cloning, and follow up our analyses by attempting to produce these three paralinguistic elements using the voice cloning platform ElevenLabs’ voice synthesis tools. Conclusively, we draw on our results to pose questions for further investigation into what kinds of AI paralinguistic cultures can and should be designed.