Institute for Telecommunication Sciences / Research / Quality of Experience / Video Quality Research / Software / Web-Enabled Subjective Test Software
Web-Enabled Subjective Test (WEST)
The Web-Enabled Subjective Test (WEST) software package offers a solution to the problem of gathering subjective testing data from multiple locations and multiple portable or computing devices. The software is freely available for download.
Modern web browsers and most smartphones and tablets support HTML 5 and its related technologies, including the <video> tag. This fact, along with programmable web server logic, enables the collection of audio, video, or multimedia subjective test data in unique new ways.
Originally written to conduct an over-the-top device comparison study, the software has been generalized for more commonly occurring test scenarios. WEST uses server-client infrastructure which allows one to conduct experiments measuring the delivered quality to any number of different mobile devices or computers whether on a local network or distributed throughout the world. Stimuli encoded using H.264 can be decoded using most modern web browsers; most smartphones and tablets contain hardware H.264 codecs. WEST differs from Tally, another subjective testing software package, in that stimuli are actually played back on the device connecting to the server.
Features
The server software is written in PHP and interacts with the Apache web server and a PostgreSQL database. The client software is written in HTML, CSS and JavaScript and is compatible with most modern web browsers. The entire package is available on GitHub for seamless collaboration.
The WEST software enables data collection in three different modes—forced choice, free response and transcript. The forced choice mode should be selected for the common Absolute Category Rating (ACR) type of testing. The free response mode allows for open-ended responses to questions. The transcript mode is useful for intelligibility testing.
The software allows the experiment designer to specify an arbitrary number of viewing sessions each with an arbitrary number of stimuli per session. Stimuli may be played back in a specified order or with a unique random order for each subject. WEST also allows the designer to ask an arbitrary number of questions per stimuli. When the experiment has been completed, results are available in a comma separated table.
The WEST software manual describes how to install a basic Web server stack and configure it to run the WEST software to run on an Ubuntu Linux machine.
Download
The manual for the WEST software is freely available for download as an NTIA Handbook.
The WEST software package is available by clicking the link below the next paragraph. The 7 MB package includes the server and client-side code necessary to configure and run a subjective test on a properly configured Web server along with video files that are useful for verifying the functionality of a properly configured test.
The download contains software developed by NTIA. NTIA does not make any warranty of any kind, express, implied or statutory, including, without limitation, the implied warranty of merchantability, fitness for a particular purpose, non-infringement and data accuracy. NTIA does not warrant or make any representations regarding the use of the software or the results thereof, including but not limited to the correctness, accuracy, reliability or usefulness of the software or the results. You can use, copy, modify, and redistribute the NTIA-developed software upon your acceptance of these terms and conditions and upon your express agreement to provide appropriate acknowledgments of NTIA's ownership of and development of the software by keeping this exact text present in any copied or derivative works.
The software is also available as a repository on GitHub. Help us make the software better!