EXPECTATIONS FOR ASSESSMENT TECHNIQUES APPLIED TO SPEECH SYNTHESIS
Katherine S. Morton · 2024
Introduction — the Importance of Assessment Reliable methods of assessing the quality and acceptability of speech output from synthesis systems would be useful to two groups: designers of the hardware and software and users of voice output products. Potential users of synthesizers are currently able to choose among a fairly small range of commercially available products, but this number will grow as the use for voice output systems becomes more obvious. At some point it will be necessary to agree on a method for judging which will be the most suitable device for the task. Researchers designing these new products need a method of assessing which are the best among existing designs with a view to determining why they are good, and ideally will be able to measure their own designs against a standard. Although it is not difficult to see the need for assessment, it has not been a simple matter to agree on techniques of evaluation, and a number of questions can be asked: for example, it is not clear whether techniques should be objective or subjective — or a combination of both. Additionally, how much weight should be placed on where the synthesizer will actually be used? For example, the conditions over a telephone line will be different from an airplane cockpit, and yet different in a classroom. Is it essential to measure performance on specific tasks, or is a rating as to general use sufficient? Is it possible to assess potential performance, or should tests be limited to actual performance? Does the cost of the device, or ease of use, i.e. ‘value for money’ enter into assessment procedures, or are these considerations irrelevant?