Institute for Telecommunication Sciences / About ITS / 2020 / ITS Releases Open Metadata Extensions for Sharing and Reusing RF Measurement Data

ITS Releases Open Metadata Extensions for Sharing and Reusing RF Measurement Data

April 2, 2020

The Open, Public, Electronic, and Necessary (OPEN) Government Data Act, signed into law on January 14, 2019, requires federal agencies to publish their information online as open data, using standardized, machine-readable data formats. In addition, metadata must be included to ensure data can be retrieved, downloaded, indexed, and searched. ITS has been making collections of spectrum measurement data publicly available for decades. These data can be, and frequently are, used to validate the propagation models that underlie technical engineering studies for optimizing spectrum sharing, improving frequency management and reuse, and planning and coordination of telecommunications systems.

ITS has traditionally used several proprietary or internally-designed metadata formats to store and retrieve these recorded signal datasets. Those formats served ITS well, but did not necessarily facilitate sharing the datasets publicly. To enable collaboration and data sharing, the scientific and engineering community needs a standard way to describe recorded data. To address the need for a widely accepted standard, a project was kicked off at the DARPA Brussels Software Defined Radio (SDR) Hackfest in early 2017 to specify a standard Signal Metadata Format (SigMF) as a way to describe sets of recorded digital signal samples.

SigMF, now supported by the GNU Radio Foundation, is generic enough to be used for any recording of a digitized time-series signal. To use it for sharing and reusing the radiofrequency (RF) measurement data that ITS collects, some specialized extensions were needed. ITS developed, and released through GitHub on March 5, 2020, the sigmf-ns-ntia namespace extension that describes ITS’s open data format for recorded signal datasets.

This repository contains a number of markdown files that document new SigMF metadata extensions that define a common set of machine-readable metadata to facilitate greater reuse and better management of ITS generated data. In addition, this repository contains Java software to make it easy to read and write the metadata specified in the markdown files and a readme within the MATLAB® directory to provide instructions on how to use the Java software within MATLAB.

Development of the sigmf-ns-ntia extensions was motivated by the ITS Spectrum Monitoring Project, designed to collect the data essential to optimizing the use of spectrum resources. Through long-term, continuous measurement of the RF environment from multiple sensors, the project seeks to provide spectrum managers with real-time information about the use of radio frequencies.

Two years ago, the ITS Spectrum Monitoring team released a first reference implementation of a sensor-control Application Programming Interface (API) proposed as part of the IEEE 802.22.3 Spectrum Characterization and Occupancy Sensing (SCOS) standard. The scos-sensor software is a secure web-based API that provides a common interface to discover capabilities, schedule tasks, and retrieve data from  individual sensors regardless of the underlying hardware. The metadata extensions defined in sigmf-ns-ntia ensure the data from scos-sensor is accompanied with sufficient metadata to describe the data provenance and support additional processing, storage and retrieval. With the release of sigmf-ns-ntia, the ITS Spectrum Monitoring Project has now provided the key to widespread re-use of the data collected by similar systems.