July 2021 | Journal Article
Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR)
Cite This Publication
Pablo Pérez, Lucjan Janowski, Narciso García, and Margaret H. Pinson, “Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR) ,” IEEE Transactions on Multimedia 24 (July 2021): 3442–3454. https://doi.org/10.1109/TMM.2021.3098450
Pablo Pérez, Lucjan Janowski, Narciso García, and Margaret H. Pinson 
Abstract: Recent studies have shown that it is possible to characterize subject bias and variance in subjective assessment tests. Apparent differences among subjects can, for the most part, be explained by random factors. Building on that theory, we propose a subjective test design where three to four team members each rate the stimuli multiple times. The results are comparable to a high performing objective metric. This provides a quick and simple way to analyze new technologies and perform pre-tests for subjective assessment.
Keywords: subjective quality evaluation; video quality; experiment design; video quality assessment
For technical information concerning this report, contact:
Margaret H. Pinson
Institute for Telecommunication Sciences
(720) 601-7314
mpinson@ntia.gov
For funding information concerning this report, click this link.
Funding Information
Performing Agency
Disclaimer:
Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.
For questions or information on this or any other NTIA scientific publication, contact the ITS Publications Office at ITSinfo@ntia.gov or 303-497-3572.