August 1995 | NTIA Technical Report TR-95-319-2
Performance Evaluation of Data Communication Services: NTIA Implementation of American National Standard x3.141, Volume 2. Experiment Design
Martin J. Miles
Abstract: The six volumes of this report are: Volume 1. Overview; Volume 2. Experiment Design; Volume 3. Data Extraction; Volume 4. Data Reduction; Volume 5. Data Analysis; Volume 6. Data Display This volume shows how to design an experiment to evaluate the performance of a data communication service as specified by ANS X3.l4l. It reviews some statistical concepts required for experiment design and analysis, including dependence between consecutive trials. It discusses objectives from both the user's and the vendor's point of view. It describes the criteria for selecting performance parameters and their appropriate analysis. It discusses the conditions that define the population of each function of a data communication service. It shows how to assign performance values (some of which cause excessive delays to be considered failures). It shows how to conduct a preliminary test to characterize the service. This test provides sample sizes required to achieve specified precisions, and it estimates the test duration for each. It discusses possible designs for the experiment and how to determine the sample size if a preliminary characterization test is either not conducted or if the objective of the experiment is to determine the acceptability of a performance parameter value.
Keywords: performance parameters; American National Standards; precision; sample size; analysis of variance; confidence level; data communication services; dependent trials; factors; performance measurements
To request a reprint of this report, contact:
Ed Drocella, Chief,
Spectrum Engineering and Analysis Division
Office of Spectrum Management
National Telecommunications and Information Administration
Disclaimer: Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.