Institute for Telecommunication Sciences / Research / Quality of Experience / Video Quality Research / First Responder Use Case / First Responders
Video Quality for First Responders
First responders rely on audio and video communication technology for mission critical tasks. ITS has conducted various investigations into differences between first responder and entertainment requirements for video systems. Several of these produced simulate first responder videos that is available on the Consumer Digital Video Library. Actual videos typically cannot be distributed due to privacy or litigation concerns.
A note about the term Public Safety Communications Research (PSCR). This term originally referred to a collaboration between NTIA/ITS and NIST to conduct research into new technologies for first responders. Around 2016, NIST renamed one of their divisions PSCR. For clarity, we have removed the acronym PSCR from ITS webpages that refer to earlier research conducted by ITS, but it may still appear in associated publications.
Datasets with First Responder Videos
The following datasets explore the first responder use case: ITS4S, ITS4S2, ITS4S3, ITS4S4, ITSnoise, Public Safety #1, Public Safety #2, and VCRDCI. See the datasets web page for more information.
Circa 2005, Mission-Critical Communications: Video Quality for First Responders
This white paper describes our motivation for conducting research into improving the way video technologies serve the first responder community.
Circa 2013, Guide to Defining Video Quality Requirements for First Responders
Project Lead Carolyn Ford, circa 2007 to 2013
In 2008, the U.S. Department of Homeland Security (DHS) sponsored ITS to launch investigations into specialized video system requirements for first responders.
DHS and ITS also sponsored a series of open meetings of the Video Quality in Public Safety (VQiPS), which invited public safety practitioners, federal partners, manufacturers, and representatives of standards making bodies to improve the way video technologies serve the public safety community. VQiPS efforts culminated in this Guide to Defining Video Quality Requirements and its associated Recommendations Tool, which were designed to help practitioners with little or no technical expertise in video describe requirements for their video quality needs and select key video system components.
See this white paper for the tools and reports produced by VQiPS.
ITS also produced two datasets (Public Safety Datasets #1 and #2). Insights from these datasets and discussions within VQiPS inspired ITS to create task based subjective test methods. ITS led efforts to standardize those methods, which now appear in ITU-T Rec. P.912, "Subjective video quality assessment methods for recognition tasks."
First Responder Problems with Modern Video Systems
Project Lead Margaret Pinson, 2016 to 2022
Using funds from a spectrum sale, NIST sponsored ITS to investigate problems that first responder have with modern video systems. ITS efforts included talking with first responders; raising awareness of first responder needs; commissioning footage that simulates first responder use cases (e.g., law enforcement, fireground, and surveillance); creating datasets that showcase first responder use cases; analyzing the performance of NR metrics made available by other researchers; developing the NRMetricFramework; and improving NR metric Sawatch.
This research led to updates in ITU-T Rec. P.910 that simplify the task of subjectively assessing the quality requirements of specific use cases. Basically, the experiment applies conventional subjective test techniques with three changes: 1) the videos depict the use case, 2) the instructions ask subjects to envision the use case, and 3) the subjects must have use case expertise. Other organizations have found this experiment design to be valuable when studying image quality for health applications.
Publications
- Margaret H. Pinson, “Why No Reference Metrics for Image and Video Quality Lack Accuracy and Reproducibility,” Journal Article, March 2023
- Robert Grosso and Margaret H. Pinson, “VMAF Compression Ratings that Disregard Camera Impairments (VCRDCI) Dataset,” Technical Memorandum NTIA TM-22-564, September 2022
- Joel Dumke and Margaret H. Pinson, “ITSnoise: An Image Quality Dataset With Sensor Noise,” Technical Memorandum NTIA TM-22-563, September 2022
- Margaret H. Pinson, “Analysis of No-Reference Metrics for Image and Video Quality of Consumer Applications,” Technical Memorandum NTIA TM-20-547, January 2020
- Margaret H. Pinson and Samuel Elting, “ITS4S4: A Video Quality Study of Camera Pans,” Technical Memorandum NTIA TM-20-545, December 2019
- Margaret H. Pinson, “ITS4S3: A Video Quality Dataset With Unrepeated Videos, Camera Impairments, and Public Safety Scenarios,” Technical Memorandum NTIA TM-19-538, April 2019
- Margaret H. Pinson, “ITS4S2: An Image Quality Dataset With Unrepeated Images From Consumer Cameras,” Technical Memorandum NTIA TM-19-537, April 2019
- Margaret H. Pinson, “Caught in the Act: Law-Enforcement Feedback on Video Surveillance Recordings,” Conference Paper, January 2019
- Margaret H. Pinson, “Gaps in Public-Safety Cameras,” Journal Article, August 2017
- Margaret H. Pinson, “Technology Gaps in First Responder Cameras,” Technical Memorandum NTIA TM-17-524, May 2017
- Joel Dumke, “Visual acuity and task-based video quality in public safety applications,” Conference Paper, February 2013
- Margaret H. Pinson, Stephen Wolf, and Robert B. Stafford, “Video Performance Requirements for Tactical Video Applications,” Conference Paper, May 2007
- Kenneth R. Tilley, “Major telecommunications institute seeks volunteers to help with testing the performance of first-responder video,” Journal Article, July 2006