Statistical Guidance for Reporting Diagnostic Test Results with Qualitative Outcomes
This guidance addresses the reporting of results from studies evaluating diagnostic devices with qualitative outcomes (positive/negative) in PMAs and 510(k)s. It focuses on appropriate statistical practices for reporting results and identifies common inappropriate practices, with special attention to discrepant resolution issues.
Recommended Actions
- Consult with FDA early to discuss study design and statistical analysis plans
- Select appropriate benchmark:
- Use reference standard if available
- If reference standard impractical, use on subset of subjects
- If no reference standard, consider constructing one
- If no reference standard possible, use agreement measures
- Design study population carefully:
- Include full spectrum of disease states
- Include confounding conditions
- Include diverse demographics
- Use final device version and instructions
- Include multiple qualified users
- Document and report:
- Complete study context and methods
- Raw data in 2x2 tables
- Appropriate performance measures with confidence intervals
- Results by site and subgroups
- Full accounting of all subjects
- Avoid inappropriate practices:
- Do not use discrepant resolution
- Do not eliminate equivocal results
- Do not use altered outcomes
- Do not compare to algorithms using the test
- Do not mix intended use and other populations
Key Considerations
Clinical testing
- Study population must include subjects across entire range of disease states
- Must include subjects with relevant confounding medical conditions
- Must include subjects across different demographic groups
- Must use final version of device according to final instructions
- Must include multiple users with relevant training and expertise
- Must cover range of expected use conditions
Labelling
- Must clearly describe designated reference standard if constructed
- Must define condition of interest and intended use population
- Must describe conditions of use (operator experience, facility, controls, specimen criteria)
- Must report performance measures with confidence intervals
- Must report results separately for intended use population
Other considerations
- Must use appropriate benchmark (reference standard when available)
- Must avoid discrepant resolution practices
- Must avoid elimination of equivocal results
- Must avoid using test outcomes altered by discrepant resolution
- Must avoid comparing to algorithms that use the new test outcome
- Must report complete accounting of all subjects and test results
- Must report results by clinical site and demographic subgroups
Relevant Guidances
- CLIA Waiver Applications for In Vitro Diagnostic Devices - Demonstrating Simplicity and Insignificant Risk of Erroneous Results
- Dual 510k and CLIA Waiver by Application for In Vitro Diagnostic Tests
- Content and Decision-Making Process for 510k Submissions: Determining Substantial Equivalence
- Design Considerations for Medical Device Pivotal Clinical Studies
Related references and norms
- CLSI EP12-A: User protocol for evaluation of qualitative test performance
- CLSI GP10-A: Assessment of clinical accuracy using ROC plots
Original guidance
This post is licensed under CC BY 4.0 by the author.