Proceedings of the IEEE First International Workshop on Quality of Multimedia Experience (QoMEX 2009), pp. 133-138, San Diego, July 29-31, 2009.
Gradient Ascent Paired-Comparison Subjective Quality Testing
doi: 10.1109/QOMEX.2009.5246964Cite This Publication
Stephen D. Voran and Andrew A. Catellier, “Gradient Ascent Paired-Comparison Subjective Quality Testing,” in Proceedings of the IEEE First International Workshop on Quality of Multimedia Experience (QoMEX 2009) pp. 133-138, San Diego, July 29-31, 2009..
Stephen D. Voran 
and Andrew A. Catellier
Abstract: (This paper won the QoMEX 2009 Best Paper Award.) Subjective testing is the most direct means of assessing audio, video, and multimedia quality as experienced by users and maximizing the information gathered while minimizing the number of trials is an important goal. We propose gradient ascent subjective testing (GAST) as an efficient way to locate optimizing sets of coding or transmission parameter values. GAST combines gradient ascent optimization techniques with paired-comparison subjective test trials to efficiently locate parameter values that maximize perceived quality. We used GAST to search a two-dimensional parameter space for the known region of maximal audio quality as proof-of-concept. That point was accurately located and we estimate that conventional testing would have required at least 27 times as many trials to generate the same results.
Keywords: protocols; switches; audio recording; error correction codes; humans; Quality assessment; Redundancy; Robustness; software quality; software testing
For technical information concerning this report, contact:
Stephen D. Voran
Institute for Telecommunication Sciences
(303) 497-3839
svoran@ntia.gov
Disclaimer: Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.
For questions or information on this or any other NTIA scientific publication, contact the ITS Publications Office at ITSinfo@ntia.gov or 303-497-3572.