Clinical Evaluation of Software as a Medical Device (SaMD)
This guidance focuses on the clinical evaluation activities needed for Software as a Medical Device (SaMD), providing a harmonized approach for demonstrating the clinical association, analytical validation, and clinical validation of SaMD. It aims to establish a common understanding of clinical evaluation principles for demonstrating safety, effectiveness and performance of SaMD.
What You Need to Know? 👇
What are the three key components of SaMD clinical evaluation?
Clinical evaluation consists of valid clinical association (establishing scientific validity between SaMD output and clinical condition), analytical validation (ensuring accurate technical performance), and clinical validation (demonstrating clinically meaningful outcomes in target populations).
When is independent review more important for SaMD clinical evaluation?
Independent review is more important for higher-risk SaMD that treat/diagnose serious and critical healthcare conditions, or drive clinical management in critical situations. Lower-risk SaMD may rely on internal design reviews or self-declaration.
How can manufacturers leverage real-world performance data for SaMD improvement?
Manufacturers can continuously collect post-market data including safety information, performance studies, and user feedback to understand real-world SaMD performance, potentially leading to updated definition statements and improved functionality through evidence-based modifications.
What evidence sources can demonstrate valid clinical association for SaMD?
Valid clinical association can be established through literature searches, professional society guidelines, clinical studies in peer-reviewed journals, consensus statements, or by generating new evidence through secondary data analysis or clinical trials.
What performance metrics are commonly used for SaMD clinical validation?
Key metrics include sensitivity, specificity, positive/negative predictive values, likelihood ratios, odds ratios, number needed to treat/harm, confidence intervals, and clinical usability assessments to demonstrate meaningful clinical outcomes.
How does SaMD risk categorization influence clinical evaluation requirements?
The SaMD categorization framework (based on healthcare situation criticality and information significance) determines the depth of clinical evaluation needed, with higher-risk categories requiring more rigorous evidence generation and independent review processes.
What You Need to Do 👇
Recommended Actions
- Establish clear SaMD definition statement and risk categorization
- Develop clinical evaluation plan covering:
- Valid clinical association
- Analytical validation
- Clinical validation
- Generate appropriate clinical evidence based on risk category
- Implement independent review process for higher risk SaMD
- Establish continuous monitoring system for real world performance
- Develop post-market surveillance plan
- Maintain documentation of clinical evaluation process
- Implement change management process for SaMD modifications
- Ensure appropriate labeling and user instructions
- Establish process for ongoing assessment of clinical evidence
Key Considerations
Clinical testing
- Clinical evaluation should be an iterative and continuous process as part of the quality management system
- Clinical validation measures the ability of SaMD to yield clinically meaningful outputs
- Level of clinical evaluation should be commensurate with the risk posed by the SaMD
- Clinical validation can be demonstrated through existing data, extrapolated data, or new clinical data
Non-clinical testing
- Analytical validation measures the ability to accurately and reliably generate intended technical output from input data
- Verification and validation activities should be conducted as part of quality management system
- Evidence can be generated through use of curated databases or previously collected patient data
Human Factors
- Clinical usability and user interface considerations must be evaluated
- Safety considerations for device users, use environments and user interfaces must be assessed
Software
- Software verification and validation must be conducted
- Continuous monitoring of software performance should be implemented
- Software changes must be managed through appropriate processes
Labelling
- Clear definition statement about intended use must be provided
- Limitations of the SaMD relevant to clinical performance must be identified
- Instructions for use must be appropriate for intended users
Safety
- Safety data collection and monitoring required
- Real world performance data should be collected to monitor safety
- Risk assessment should be considered when conducting clinical evaluation
Other considerations
- Independent review of clinical evidence recommended for higher risk SaMD
- Real world performance data should be leveraged for continuous learning
- Post-market surveillance should be conducted
- Changes to SaMD definition statement may require regulatory review
Relevant Guidances 🔗
- Content of Premarket Submissions for Device Software Functions
- Off-The-Shelf Software in Medical Devices: Documentation Requirements for Premarket Submissions
- Software Validation for Medical Device Production, Quality Systems, and Device Components
- Cybersecurity in Medical Devices: Design, Implementation, and Premarket Submissions
- Clinical Decision Support Software Functions: Device vs Non-Device Classification
- Policy for Device Software Functions and Mobile Medical Applications
- Changes to Medical Device Definition for Software Functions Under the 21st Century Cures Act
Related references and norms 📂
- ISO 14155:2011: Clinical investigation of medical devices for human subjects - Good clinical practice
- ISO 14971:2007: Application of risk management to medical devices
- ISO 62304/A1:2015: Medical device software – Software life-cycle processes
- IEC 62366-1:2015: Medical Devices – Part 1: Application of usability engineering to medical devices
- ISO 82304-1:2016: Health software – Part 1: General requirements for product safety