Institute for Telecommunication Sciences / Research / Quality of Experience / Video Quality Research / Guides and Tutorials / Description of VQM Tools

Description of Video Quality Metric (VQM) Software

The VQM software tools provide both standardized and non-standardized methods for measuring the video quality of digital video systems. Each VQM software tool is described below.

Each VQM tool estimates how people perceive video quality. These VQM algorithms compare the processed video (output) with the original video (input). These models are suitable when the original video is good quality or better. Quality problems from the camera and original production are not considered. 

VQM software is available royalty-free to all interested parties from the GitHub VQM repository and may be freely used for both non-commercial and commercial purposes in accordance with the license terms set out in the repository.

Batch Video Quality Metric (BVQM) Software 

BVQM is a Windows program for performing out-of-service / lab bench testing. BVQM can processing and analyses of multiple video scenes and multiple video systems at once. BVQM reads video sequences from files, and reports results to the screen. BVQM includes a variety of calibration options, quality models, and graphical presentation of results. For an overview of the BVQM tool, click here.

  • BVQM_pc32_v20 has BVQM version 2.0, compiled for 32-bit Windows operating systems.
  • BVQM_pc64_v20 has BVQM version 2.0, compiled for 64-bit Windows operating systems.
  • BVQM_src_v20 has MATLAB source code for BVQM 2.0.
  • Older versions of BVQM are also available for download. Some of these older versions are compiled for Linux (lx, no longer supported).

Command Line Video Quality Metric (CVQM) Software

CVQM is a Windows command line program for performing out-of-service / lab bench testing. The video quality calibration and metric algorithms within CVQM are basically identically to those offered by BVQM. The two differences are that (1) CVQM is called from a command line (e.g., Windows Accessory "Command Prompt"), and (2) CVQM cannot compare the results from multiple video clips to improve calibration accuracy.  CVQM runs on one pair of video files at a time, and writes results to files. For an overview of the CVQM tool, click here.

  • CVQM_pc32_v30 has CVQM version 3.0 compiled for the 32-bit Windows operating system.
  • CVQM_pc64_v30 has CVQM version 3.0 compiled for the 64-bit Windows operating system.
  • CVQM_src_v30 has MATLAB source code for CVQM version 3.0.
  • Older versions of CVQM are also available for download. Some of these older versions are compiled for Linux (lx, no longer supported). If you want to view or modify the MATLAB source code, CVQM will be easier to understand than BVQM.

PSNR

This command-line MATALAB software calculates peak signal to noise ratio (PSNR) according to ITU-T Rec. J.340. Also computes PSNR with variable frame delays removed (PSNR-VFD). These metrics are also available in the BVQM and CVQM software.

VFD

This command-line MATLAB software calculates variable frame delay (VFD) between an original video sequence and a processed video sequence. This algorithm and code is used by BVQM, CVQM, and PSNR software packages.

FDF

This command-line MATLAB software calculates fraction dropped frames (FDF) from VFD statistics. This algorithm and code is used by BVQM, CVQM, and PSNR software packages.

Separate Algorithms from BVQM/CVQM

The following tools are integrated into the BVQM and CVQM software. These stand alone versions allow increased flexibility.

FDF_pc is a Windows command line program for performing No Reference (NR) and Reduced Reference (RR) measurements of the Fraction of Dropped Frames (FDF) in a processed video stream. The algorithm is fully described here.

  • FDF_pc32_v12 has FDF version 1.2, compiled for 32-bit Windows operating systems.
  • FDF_pc64_v12 has FDF version 1.2, compiled for 64-bit Windows operating systems.
  • FDF_src_v12 has MATLAB source code for FDF version 1.2.
  • Older versions of FDF are also available for download.

PSNR_pc is a Windows command line program for performing Full Reference (FR) measurements of Peak Signal to Noise Ratio (PSNR) of a processed video stream. The algorithm is fully described here.

  • PSNR_pc32_v12 has PSNR version 1.2, compiled for 32-bit Windows operating systems.
  • PSNR_pc64_v12 has PSNR version 1.2, compiled for 64-bit Windows operating systems.
  • PSNR_src_v12 has MATLAB source code for PSNR version 1.2.
  • Older versions of PSNR are also available for download.

VFD_pc is a Windows command line program for performing Full Reference (FR) measurements of Variable Frame Delays (VFD) in a processed video stream. The algorithm is fully described here.

  • VFD_pc32_v12 has VFD version 1.2, compiled for 32-bit Windows operating systems.
  • VFD_pc64_v12 has VFD version 1.2, , compiled for 64-bit Windows operating systems.
  • VFD_src_v12 has MATLAB source code for VFD version 1.2, .
  • Older versions of VFD are also available for download. 

Obsolete Tool: IVQM

IVQM is available upon request.

The In-Service Video Qualtiy Metric (IVQM) was a "proof of concept" software implementation. This software demonstrates that the reduced reference (RR) algorithms provided within BVQM and CVQM can actually run in-service. IVQM calibrates and estimates the quality of video being transmitted live. This software captures the source video and processed video at separate locations, calculates video quality, then repeats. It operates on two computers, one upstream and one downstream and utilizes an IP-based reduced-reference connection. For an overview of the IVQM tool, click here.

The 1st problem with IVQM is that the video capture code is compatible with only a few video capture devices. That is, the video capture code is obsolete.  Video capture is supported using VfW, WDM, or other MATLAB compliant video capture devices.

The 2nd problem is that IVQM is incompatible with the modern technique of integrating the camera and encoder into PC software. IVQM needs a camera that outputs video onto a video cable, such as the S-Video or Composite that are used between DVD players and some televisions. That video signal must then be split so that it goes to a capture card (for IVQM) and also to the videoconferencing system. This is not possible for most modern videoconferencing systems, which use either integrated cameras or USB cameras.

Both of these problems are solved by supplementary software provided by Acreo. That tool allows IVQM to capture IPTV video streams.

  • IVQM_pc_v25 as IVQM version 2.5, compiled for 32-bit Windows operating systems.
  • IVQM_pc_v24_iptv has the previous release, IVQM version 2.4, compiled for 32-bit Windows operating systems plus compatible with an extension from Acreo that allows capture from streamed video (IPTV-interface).
  • IVQM_src_v25 has MATLAB source code for IVQM version 2.5
  • Older versions of IVQM are also available for download.
  • IVQM is no longer supported.

Obsolete Tool: VQM_pc & VQM_lx

VQM_pc and VQM_lx are available upon request.

VQM_pc (compiled for Windows) and VQM_lx (compiled for Linux) have been replaced by BVQM. This software was developped under a cooperative research and devleopment agreement (CRADA) by ITS and Intel. This was the first version of the ITS video quality models that was suitable for use by people outside of our laboratory. The source code cannot be redistributed.

The problem is that this code was written in 2004. The code has not been recompiled for modern operating systems. We do not know whether or not the code can be compiled for Windows XP, Vista and 7. We will try to recompile this code for Windows as time permits. If you have a computer with an older operating system, this code will run faster than BVQM/CVQM, because it is implemented in C++ instead of MATLAB.

VQM_pc/VQM_lx is a program for performing out-of-service / lab bench testing. The tool provides a simple control file interface to the video calibration and quality measurement models. Video sequences must be stored in files; user commands are entered in a text file; and results are emailed to the user or stored in output files. The VQM_lx/VQM_pc tool provides a graphical user-interface to video calibration algorithms, quality measurement models, video playback and conversion tools. The tool can also create a test sequence of multiple video scenes, which may then be sent through a video transmission system for quality analysis. A specialized color bar can be included, to improve calibration. Limited video capture capabilities (VfW) are included in the PC version.

  • VQM_pc/VQM_lx are no longer supported.