Institute for Telecommunication Sciences / Research / Quality of Experience / Video Quality Research / Standards / Subjective Tests / J.av-dist proposal

Proposal for New Audiovisual Quality Recommendation, 2011

ITU-T Recommendation P.910, ITU-T Recommendation P.911, and ITU-R Recommendation BT.500 have been successfully used for many years to perform video quality and audiovisual quality subjective assessments. These Recommendations were initially designed around the paradigm of a fixed video service that transmits video over a reliable link to an immobile cathode ray tube (CRT) television located in a quiet and non-distracting environment, such as a living room or office. These Recommendations have been updated and expanded as technology shifted and have proven to be valuable and useful for the displays and questions addressed in their original scopes.

However, the initial premise of these Recommendations does not include the new paradigms of internet video and distribution quality television. One new paradigm of video watching is an on-demand video service transmitted over an unreliable link to a variety of mobile and immobile devices located in a distracting environment, using LCDs and other flat-screen displays. This new paradigm impacts key characteristics of the subjective test, such as the viewing environment, the listening environment, and the questions to be answered.

Discussion

The devices and usage scenarios of interest will be called "internet video and distribution quality television". The focus is on the quality perceived by the end-user, eliminating the specialized needs of broadcasters and contribution quality, which are well addressed by existing ITU Recommendations. Internet video is increasing at an alarming rate of 39% in the next 3 years with over 700 million paid subscribers. Users of internet video and distribution quality television are moving from one device to another and from one environment to another throughout the day, perhaps even observing the same video using multiple devices. For example, someone might start watching a sporting event on their computer using IPTV, move to an over-the-air broadcast in their living room when the IPTV connection displays a rebuffering event, and then switch to a mobile internet device (MID) or even a smart phone when leaving the house. Thus, subjective quality assessments into internet video and distribution quality television ask unique questions that are not considered in the existing Recommendations. For brevity, this discussion will focus on the test environment.

Consider the pristine viewing environment defined by ITU-R Rec. BT.500, with its exact lighting conditions and non-distracting walls.  The intention is to remove the impact of the viewing environment and listening environment from the experiment. For some subjective audiovisual quality experiments, this is not appropriate. First, consider an experiment that investigates the quality of service observed by videoconferencing users in an office with florescent lights and the steady hum of a computer. Second, consider an experiment that analyzes a communications device for emergency personnel. A highly distracting background may be a critical element of the experiment design (e.g., to simulate video watched inside a moving fire truck with sirens blaring). The impact of environment is an integral part of these experiments.

For the many experiments where a pristine environment is not appropriate and a realistic user environment is required, looser environmental constraints may be appropriate. In the current Recommendations the light levels, backlighting, color temperature, sound isolation, and such are specified conservatively, rather than empirically determining thresholds of perceptibility. Since there does not appear to be evidence that these thresholds are appropriate for testing the new viewing paradigms, more realistic environmental constraints should be considered. This will require a new testing Recommendation that can assess, for example, internet-delivered television together with traditionally delivered cable television.