ITS continues to contribute significant research in support of clear and effective communications for fire, police, medical, and other public safety officials.
ITS is supporting NIST in developing methods of measurement for Mission Critical Voice (MCV) Key Performance Indicators (KPIs). These MCV KPIs are
- Speech Intelligibility
- Audio Quality
- Mouth-to-Ear Latency
- Access Time
- Probability of Access
- Probability of Retention
Each of these can influence the usability of an MCV system and, in turn, the ability to efficiently and correctly perform first-responder functions. In extreme situations the KPIs may actually be connected to levels of success in saving lives or property. The KPIs are especially important as agencies consider migrating from Land Mobile Radio (LMR) to LTE-based MCV systems.
The ITS-developed speech intelligibility measurement tool ABC-MRT16 provides a foundation for several of the listed KPIs. In an ideal situation this tool uses 1200 different recorded words to determine the average intelligibility of an MCV system. But using 1200 words is impractical in some KPI measurement cases (access time, probability of access, and probability of retention). So ITS uses software simulations of analog and digital LMR and LTE-based MCV systems to identify sets of test words that meet the various criteria associated with measuring different KPIs. We use multiple simulation trials to capture important causes of variation, like noise and timing relationships. The millions of measurements that result flow into our statistical analysis procedure.
The Bigger Picture:
Our stakeholders have emphasized the following goals for MCV systems:
- High speech intelligibly (listener can easily understand what is being said)
- Listener can recognize the voice of a known talker
- Listener can detect stress in a talker’s voice
- High audio quality (background sounds are clear but at level that does not obscure the speech)
We have studied each of these areas and, following stakeholder input, we have focused on degradations caused by background noises and radio channel impairments. In our studies we often design, conduct, analyze, and document objective and subjective tests to discover when the conditions in the above goals are not met, the severity of those failures, and potential mitigation strategies. Most often we measure speech intelligibility, report conditions that reduce speech intelligibly, and investigate technology alternatives that can lessen those reductions.
- Stephen Voran and Andrew Catellier, Intelligibility Robustness of Five Speech Codec Modes in Frame-Erasure and Background-Noise Environments," NTIA Technical Report TR-18-529, December 2017.
- Andrew Catellier and Stephen Voran, “Intelligibility of Selected Speech Codecs in Frame-Erasure Conditions,” NTIA Technical Report TR-17-522, November 2016.
- Stephen Voran and Andrew Catellier, “Speech Codec Intelligibility Testing in Support of Mission-Critical Voice Applications for LTE,” NTIA Technical Report TR-15-520, September 2015.
- David J. Atkinson, Stephen Voran, and Andrew Catellier,“Intelligibility of the Adaptive Multi-Rate Speech Coder in Emergency-Response Environments,” NTIA Technical Report TR-13-493, December 2010.
- David J. Atkinson and Andrew Catellier, “Intelligibility of Analog FM and Updated P25 Radio Systems in the Presence of Fireground Noise: Test Plan and Results,” NTIA Technical Report TR-13-495, May 2013.
- David J. Atkinson and Andrew Catellier, “Intelligibility of selected radio systems in the presence of fireground noise: Test plan and results,” NTIA Technical Report TR-08-453, June 2008.
- Andrew Catellier and Stephen Voran, “Relationships Between Intelligibility, Speaker Identification, and the Detection of Dramatized Urgency,” NTIA Technical Report TR-09-459, November 2008
- Andrew Catellier and Stephen D. Voran, “Speaker Identification in Low-Rate Coded Speech,” Proceedings of the 7th International MESAQIN (Measurement of Audio and Video Quality in Networks) Conference, Prague, Czech Republic, May 2008.
- Stephen Voran, “Listener Detection of Talker Stress in Low-rate Coded Speech,” Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), pp. 4813-4816, Las Vegas, March 31-April 4, 2008.
Some first-generation digital radio systems fell short of these goals under certain conditions, most often conditions involving moderate to loud levels of background noise. But these noise environments (e.g. sirens, engines, rescue equipment, pumps and nozzles) are inherent in public safety operations and clear communications in their presence is a must.
The late Mr. DJ Atkinson heard reports of these communication breakdowns, took them seriously, and investigated carefully. He established a formal process of study at ITS based on the standardized speech intelligibly test called the Modified Rhyme Test (MRT). DJ’s tireless efforts, passionate engagement of the work, and commitment to the best interests of the stakeholders are widely recognized and contribute to his remarkable and enduring legacy. DJ’s work established ITS as a key source for rigorous, impartial, speech intelligibility test results, and the contributions continue to this day.
Speech intelligibility examples can be heard here: Speech Intelligibility Demo
MRT resources are available here: Modified Rhyme Test Audio Library