Institute for Telecommunication Sciences / Research / Quality of Experience / Video Quality Research / Video Home
Video Quality Research
The video quality project helps public and private sectors optimize networked video transmissions and provide the best video quality possible. To deliver high-quality video to users, whether commercially or in public safety and surveillance uses, video streaming applications need to dynamically balance available streaming bandwidth with video resolution, providing the best quality video possible without requiring excess time for video buffering.
Develop Video Quality Metrics
ITS is renowned for developing objective metrics that predict human perception of video quality. Until 2011, ITS developed metrics that compare a pristine original video with an impaired version of that video. We distribute free software for out-of-service applications.
Although it has long been possible to statistically compare the quality of a video to another reference video, it remains challenging to measure the quality of a single video with no reference. Hence, the term no-reference (NR) metric is used to describe the type of model we are currently seeking to develop.
Source code and a framework for developing NR metrics is available on GitHub. Click here for an introduction to this NR metric framework and objective metrics that measure video quality.
Our article "Why NR metrics for image and video quality lack accuracy and reproducibility" provides a comprehensive overview of NR metrics. We examine 26 independent evaluations of NR metrics (previously published) and analyze 32 NR metrics on six IQA datasets and six VQA datasets (new results). Where NR metric developers claim Pearson correlation values between 0.66 and 0.99, our measurements range from 0.0 to 0.63. None of the NR metrics we analyzed are accurate enough to be deployed by industry. Performance evaluations that indicate otherwise are based on insufficient data and highly inaccurate. We will examine development strategies, tools, datasets, root cause analysis, and our baseline metric for collaboration, Sawatch.
Analyze New Video Technologies
ITS performs and publishes independent analyses of new video technologies, to help US industry understand the quality impact of new video codecs, transmission networks, and video display technologies. These analyses usually involve subjective tests at our specialized laboratory facilities.
ITS has a long history of helping first responders understand how their video quality needs differ from typical consumers. Click here for more information.
Improve Research Methods
Rapid advances in video technology require new analysis methods. ITS investigates improved methods for designing and performing subjective tests. We also develop new statistical analysis methods to analyze subjective test data and objective metric data.
Share Data & Information
ITS is committed to openly collaborating with national and international colleagues to solve complex video quality assessment problems. The most impactful open issues cannot be resolved by any single organization in isolation. Our publications demonstrate a long history of openly collaborating with diverse organizations and publishing our findings, to inform US industry.
ITS sponsors the Video Quality Experts Group (VQEG) through leadership, website support, and email reflectors. VQEG facilitates communication among academia, industry, and the International Telecommunications Union (ITU). VQEG provides a venue to share cutting-edge research, understand the video quality problems of diverse video systems, develop new video quality assessment methods, and validate video metrics prior to standardization.
To facilitate and foster collaborative research and development in the area of consumer video processing and quality measurement, ITS hosts the Consumer Digital Video Library (CDVL). Where licensing terms permit, ITS subjective video quality datasets and studio quality video sequences are openly shared on CDVL.
Support International Standards
ITS supports international standardization efforts of the International Telecommunications Union (ITU) on topics related to video quality. As VQEG's Independent Lab Group (ILG) Co-Chair, ITS led and coordinated the independent validation of objective video quality metrics prior to standardization. Click here for more information on video quality standards.
Our current focus is the proposed merger of ITU-T Rec. P.910, P.911, and P.913 into an updated P.910. Study Group 12 work item P.910rev includes improved methods for subjective testing developed by subject matter experts over the past decade.