Recombinant speech synthesis: Natural text-to-speech synthesis with prosodic control.
Esther Klabbers, Taniya Mishra, Jan P. H. van Santen · The Journal of the Acoustical Society of America · 2009
Unit selection text-to-speech synthesis methods rely on large corpora to cover all phoneme sequences in as many prosodic contexts as possible. This coverage is rarely complete except in limited domains. This becomes particularly salient when using prosodic markup to generate specific prosodic patterns (e.g., emphatic stress). An architecture is proposed combining the naturalness of unit-selection synthesis with the requirement of prosodic control. The speech corpus consists of multiple sub-corpora, each optimized to cover a “linguistic subspace”; subspaces include phoneme sequences, left-headed feet, sentence structures, and paralinguistic categories. The system relies on the superpositional model of intonation to decompose natural pitch contours into component contours, e.g., phrase curves (corresponding to phrases) and accent curves (corresponding to left-headed feet); on analogous methods for timing; and on hybridization methods to implement paralinguistic features. During synthesis, phoneme sequences, curves, and parameters are generated from the sub-corpora, optionally modified as per prosodic control tags, and “re-combined.” The explicit representation in terms of component curves allows for complete prosodic control, while the naturalness of the prosodic patterns is guaranteed by extracting these curves from natural speech and smoothly modifying them, thereby preserving important natural detail. [Research supported by NSF grant 0205731, “Prosody generation in child-oriented speech.”]