Post

Statistical Guidance for Reporting Diagnostic Test Results with Qualitative Outcomes

This guidance addresses the reporting of results from studies evaluating diagnostic devices with qualitative outcomes (positive/negative) in PMAs and 510(k)s. It focuses on appropriate statistical practices for reporting results and identifies common inappropriate practices, with special attention to discrepant resolution issues.

  1. Consult with FDA early to discuss study design and statistical analysis plans
  2. Select appropriate benchmark:
    • Use reference standard if available
    • If reference standard impractical, use on subset of subjects
    • If no reference standard, consider constructing one
    • If no reference standard possible, use agreement measures
  3. Design study population carefully:
    • Include full spectrum of disease states
    • Include confounding conditions
    • Include diverse demographics
    • Use final device version and instructions
    • Include multiple qualified users
  4. Document and report:
    • Complete study context and methods
    • Raw data in 2x2 tables
    • Appropriate performance measures with confidence intervals
    • Results by site and subgroups
    • Full accounting of all subjects
  5. Avoid inappropriate practices:
    • Do not use discrepant resolution
    • Do not eliminate equivocal results
    • Do not use altered outcomes
    • Do not compare to algorithms using the test
    • Do not mix intended use and other populations

Key Considerations

Clinical testing

  • Study population must include subjects across entire range of disease states
  • Must include subjects with relevant confounding medical conditions
  • Must include subjects across different demographic groups
  • Must use final version of device according to final instructions
  • Must include multiple users with relevant training and expertise
  • Must cover range of expected use conditions

Labelling

  • Must clearly describe designated reference standard if constructed
  • Must define condition of interest and intended use population
  • Must describe conditions of use (operator experience, facility, controls, specimen criteria)
  • Must report performance measures with confidence intervals
  • Must report results separately for intended use population

Other considerations

  • CLSI EP12-A: User protocol for evaluation of qualitative test performance
  • CLSI GP10-A: Assessment of clinical accuracy using ROC plots

Original guidance

  • Statistical Guidance for Reporting Diagnostic Test Results with Qualitative Outcomes
  • HTML / PDF
  • Issue date: 2007-03-12
  • Last changed date: 2020-03-19
  • Status: FINAL
  • Official FDA topics: Medical Devices, Biostatistics
  • ReguVirta summary file ID: c4fbbcad300c6d5161c028857324d965
This post is licensed under CC BY 4.0 by the author.