Post

Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions

This guidance provides a general risk-informed framework for assessing the credibility of computational modeling and simulation (CM&S) used in medical device regulatory submissions. It applies specifically to first principles-based models (e.g., physics-based or mechanistic models) but not to standalone statistical or data-driven models like machine learning or AI models. The guidance aims to help improve consistency and transparency in reviewing CM&S, increase confidence in using CM&S in regulatory submissions, and facilitate improved interpretation of CM&S credibility evidence.

  1. Define question of interest and context of use for the computational model
  2. Perform model risk assessment considering decision consequence and model influence
  3. Identify and categorize planned credibility evidence across verification, validation and other categories
  4. Define credibility factors and set goals commensurate with model risk
  5. Generate credibility evidence through planned studies and/or analysis of existing data
  6. Perform adequacy assessment to determine if evidence supports model use
  7. Document limitations and rationale for model credibility
  8. Prepare comprehensive credibility assessment report following recommended format
  9. Consider submitting Q-submission to get FDA feedback on credibility assessment plan
  10. Include both model details report and credibility assessment report in regulatory submission

Key Considerations

Clinical testing

  • In vivo validation results can be used as credibility evidence, including clinical or animal data
  • For patient-specific models, subject-level comparison between simulation and clinical data should be performed
  • Population-based validation comparing model predictions to clinical datasets at a population level can be used
  • Clinical trial results can provide validation evidence if appropriate statistical techniques and regulatory requirements are followed

Non-clinical testing

  • Bench test validation under well-controlled laboratory conditions is recommended when possible
  • Both prospectively planned validation and validation against retrospective datasets can be used
  • Previously generated validation results may be applicable if relevance to current context of use can be demonstrated
  • Calculation verification and uncertainty quantification should support validation results

Software

  • Code verification must demonstrate correct implementation of numerical algorithms
  • Software quality assurance and numerical code verification should be performed
  • For models in device software, both software and model verification/validation are important but differ in scope

Labelling

  • Context of use should be clearly described
  • Model limitations should be explicitly stated
  • Any assumptions should be documented and justified

Safety

  • Model risk assessment should consider:
    • Decision consequence (potential severity and probability of harm)
    • Model influence relative to other evidence
  • Risk mitigation procedures should be documented
  • Safety thresholds and model predictions relative to these should be evaluated

Other considerations

  • ASME V&V 40: Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices
  • ISO 14971: Medical devices — Application of risk management to medical devices
  • ISO/TR 24971: Medical devices — Guidance on the application of ISO 14971

Original guidance

  • Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions
  • HTML / PDF
  • Issue date: 2023-11-17
  • Last changed date: 2023-11-16
  • Status: FINAL
  • Official FDA topics: Medical Devices, Digital Health, Premarket
  • ReguVirta summary file ID: 587b485351179688548d0034b5de05b6
This post is licensed under CC BY 4.0 by the author.