Techniques for Comparing Objective and Subjective Speech Quality Tests

Stephen D. Voran

Abstract: Objective (or instrumental) tests of speech quality have been proposed as ways to reduce the need for expensive and time-consuming subjective (or auditory) tests. Both types of tests attempt to quantify the range of opinions that listeners express in response to a group of speech transmission or storage devices, but objective test results often show measurable deviation from subjective test results. This deviation may be judged to be acceptable if the objective test offers significant savings of time and expense. This cost-performance judgment cannot be made without a meaningful statistical measure of the deviation that is likely to be associated with the objective test. This paper offers a number of techniques that compare both the central tendencies and the uncertainties of two tests. The resulting statistics have direct, intuitive interpretations.

(bochum.pdf)

Disclaimer: Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.

For questions or information on this or any other NTIA scientific publication, contact the ITS Publications Office at ITSinfo@ntia.gov or 303-497-3572.

Back to Search Results