Institute for Telecommunication Sciences / About ITS / 2024 / IORS 2024

Inaugural International Open RAN Symposium (IORS) Opens Global Conversation

The inaugural International Open RAN Symposium (IORS) convened September 17–19, 2024, in Golden, Colorado, United States of America. NTIA welcomed over 250 participants at the inaugural Symposium from over 20 countries.

Montage of images from IORS 2024
From plenary sessions, to breakout room, to informal and impromptu technical conversations, IORS 2024 offered many opportunities for discussion and exchange of ideas.

A primary goal of IORS was to accelerate the global adoption and deployment of interoperable Open Radio Access Networks (RAN). Interesting and insightful discussions were enabled by the diverse mixture of mobile network operators (MNOs), government, industry, academia, and community organizations like the Telecom Infra Project (TIP), Open Testing and Integration Centers (OTIC), and OpenAirInterface Foundation. Highlights included:

  • Opening keynotes from Don Graves (Deputy Secretary of Commerce) and Sarah Morris (Principal Deputy Assistant Secretary) of the U.S. Department of Commerce
  • Opening keynote from Steven Lang (Ambassador, U.S. Coordinator for International Communications and Informational Policy) of the U.S. Department of State
  • ATIS announcement of the ATIS Open RAN Minimum Viable Profile (MVP).
  • Success stories from worldwide lab pairs for repeatable, consistent Open RAN testing
  • Lightning talk introductions from 47 organizations
  • Interactive workshop discussions on 25 technical topics
  • Tour of DISH Wireless’s new Open RAN Center for Integration and Deployment (ORCID) in Cheyenne, Wyoming
  • Technical successes on consistent, repeatable worldwide testing

Day 1 focused on topics of interest to all attendees. On Days 2 and 3, IORS diverged into separate policy and technical tracks with similar numbers of attendees. NTIA’s Office of International Affairs (OIA) organized and led the policy track; NTIA’s Institute for Telecommunication Sciences (ITS) organized and led the technical track, aligning with the goals of each of our organizations.

NTIA’s OIA sponsored IORS 2024, paving the way for a series of an annual symposia and interim virtual meetings. The technical focus in 2024 was to create a community of practice for consistent and repeatable Open RAN testing. Insights from the IORS Technical Track are summarized below. A longer Executive Summary briefly addresses insights from the IORS Policy Track as well.

Reliability and stability: MNOs raised the concern that the testing community focuses too much on lab capabilities and test plans and not enough on the overall long-term performance of the Device Under Test (DUT). Major MNOs need Open RAN solutions to achieve wireline-equivalent performance before they will consider adding Open RAN equipment to their networks. Even though Open RAN is in the development stage, the Open RAN community must start considering reliability and stability now to lay the groundwork for long term performance tests on real networks.

Building trust: There are currently about 20 OTIC labs, but a gap remains between the capabilities of OTIC labs and the private labs of major MNOs. Before MNOs widely deploy Open RAN equipment, they need more confidence in OTICs’ ability to generate consistent, repeatable test results in time, location, test environment, and DUT configuration. In addition to repeating the same test in multiple labs, another idea is to share the effort and run tests across a federation of interconnecting OTICs to take advantage of geography, equipment, environment, and test capabilities at each lab. Attendees stressed the importance of communication between MNOs and OTICs at events like IORS.

Workforce development and training for new labs: Many OTIC labs receive government funding and without continued funding some OTIC labs may fail. From a business standpoint, it is impractical to expect that existing labs will train new labs, since doing so would mean training the competition.

A more practical strategy seems to be workforce development, in the form of a new academic program that provides cross-functional and hands-on training for Open RAN configuration and testing and confers a master’s degree. The curriculum would need to integrate six skill sets: 

  • Artificial intelligence and machine learning (AI/ML)
  • Information technology (IT), including networking and cybersecurity
  • Quality assurance
  • Software engineering
  • Systems engineering
  • Wireless RF and embedded hardware engineering

Brownfield deployment: Most OTIC labs are focusing on standalone 5G (greenfield), which is not of interest to most MNOs (EchoStar/DISH in U.S. and Rakuten Mobile in Japan are exceptions). OTICs need to focus on non-standalone networks that must operate older equipment like LTE/4G (brownfield) for situations that dominate most of the MNOs (in the U.S. the tier1 brownfield operators are AT&T, Verizon, T-Mobile).

Performance parity: MNOs need performance metrics and testing in real life environments, so that they can compare Open RAN products to traditional vendor products. Some critical requirements are missing, like RF environment specifications, fully fleshed out test plans, methods to share data after removing sensitive content, and whether to focus on basic interface testing or to also perform advanced testing. Test cases must be evolved to become repeatable. Perhaps 80% of this can be general testing performed by any OTIC lab, with the remaining tests vendor specific.

Performance benchmark tests need to be defined to represent operator deployments (e.g., number of UEs, radio channel impairments, transport network impairments, mobility profiles, mixed end user services, and application traffic). KPIs for performance test cases need to be defined and include accessibility, retainability, availability, mobility, energy consumption, and efficiency. Input parameters need to be clearly specified (e.g. TDD slot formats, carrier bandwidth).

Authority: Which organization is the ultimate authority for consistent, repeatable testing—the O RAN ALLIANCE, TIP, or the Third Generation Partnership Project (3GPP)? Currently the O-RAN ALLIANCE establishes Open RAN specifications, drives feature development, and drives testing with its Test and Integration Focus Group (TIFG). The 3GPP does not monetize its work via testing. Testing is becoming a business case for OTIC labs.

The best solution would be a collaboration among all three organizations, to build industry trust and bridge the gaps between OTIC, TIP, and 3GPP. Testing labs have a large presence worldwide. To encourage collaboration and knowledge sharing between worldwide test labs, seed funding may be required.

Non-repeatability across labs: Sources of non-repeatability across labs include imprecise test specifications (3GPP and the O-RAN ALLIANCE have workgroups striving to resolve this), different testbed configurations (peer review may help resolve this), negligence (spot checks help resolve this), gross negligence (could be avoided by accreditation), and intentional system gaming (certification authorities could prevent this). 

Accreditation, badging, and certification: Test plan specifications and success criteria for badging must be tightened, and test cases must evolve to become consistent and repeatable. It is critical to define a common test suite, common pass/fail criteria (lower/upper bound), machine-readable outputs, and standard test reports. Labs need to automate testing to the greatest extent possible, ensure zero-trust in security testing, and the test results need to have some form of peer review. Open-source solutions can be used as a reference implementation and spur continuous integration (CI), continuous development (CD), and continuous testing (CT).

Is there value in badges and certificates: do they enable plug and play? We should focus on system level certification, perhaps using blueprints based on regional MVPs. Lab accreditation will not improve the quality of lab testing beyond what is described in a test plan. If a test plan is ambiguous or incomplete, the accreditation simply ensures that the lab follows the test plan, ambiguities and all. 

Open test data for R&D: Availability of data from real system deployments (e.g., traffic profiles) is hindered by device vendor NDAs, the need to protect user privacy, regulations, business interests, licensing issues, and the need to sanitize data before sharing. The immediate sources of data include open-source projects, academic datasets, public domain mobility traces (e.g., taxi and metro) from large cities, application-layer traffic statistics, and measurements from research testbeds.  

System Integration (SI): Open RAN promises lower capital expenditures (CapEx) and lower operational expenditures (OpEx), but achieving those savings requires overcoming significant SI and testing challenges. There will probably be a hybrid SI model of in-house, third-party, and vendor-based based on operator needs. OTICs may serve as SI as an independent third-party entity for specific operators and vendors.

Next Steps for Technical Track

For technical track next steps, NTIA/ITS will lead the effort to:

  • Hold virtual meetings every two months
  • Create some version of an IORS wiki
  • Share IORS outcomes and future work with TIP, O-RAN ALLIANCE, ACCORD, ATIS, etc.
  • Create an inventory of current lab collaborations
  • Organize and lead the second IORS in 2025 (location TBD)