Navigating the impact of AI in Medical device certification: A comprehensive insight
In a recent discussion during the RAPS meeting in Brussels, Natascha Cuper, representing Kiwa Medical of the Notified Body Kiwa Dare, engaged with Rachel Mead of BSI to delve into the pivotal role of artificial intelligence (AI) in the medical device industry. As AI technology takes flight, there's a growing urgency to investigate its influence on medical devices. In this article we will explore the complex world of AI-based medical devices, emphasizing the need for rigorous clinical evaluations and robust regulatory compliance.
The essence of Clinical Evaluation Reports (CERs)
In the medical devices sector, the clinical evaluation of these innovations is crucial. Central to this process is the Clinical Evaluation Report (CER), which serves as a comprehensive documentation of the medical device's clinical performance. Notably, the advent of AI adds layers of complexity, demanding a deeper level of specificity beyond a mere acknowledgment of machine learning as the underlying technology.
Defining AI's clinical relevance
To be able to navigate the regulatory landscape successfully, it is necessary to specify the AI-based medical device's intended clinical use and its anticipated benefits. This specification needs to be framed within the context of the existing state-of-the-art and what sets the device apart from conventional medical practices. It is strongly recommended for manufacturers to focus on three pivotal components:
- Establishing a valid clinical association or scientific validity
- Detailing technical and analytical performance
- Validating clinical performance
Adhering to GDPR and overcoming common CER shortcomings
Natascha Cuper underscores the imperative of General Data Protection Regulation (GDPR) compliance and sheds a light on common shortcomings observed in CERs submitted for AI-based medical devices: “What we want to see, of course, is the selection criteria and the sample size of your training and validation data sets, and how representative they are for the intended use. Have any potential biases in the datasets been considered, and what are the actual demographics – which should include all intended medical indications or applications? We also want to see any pre-processing of the data, for example, labeling, the ground truth, and how you decided to pick processes.”
It is essential to address these concerns proactively to ensure seamless regulatory approval.
What Notified Bodies and EU regulators seek
One of the key takeaways from the discussion was the expectations of Notified Bodies and EU regulators concerning the Certification of Suitability (CEP) reports and CERs. Natascha Cuper summarizes their desires as follows:
- Selection criteria and sample size: Providing detailed insights into the selection criteria and the size of training and validation datasets. Transparency regarding the representativeness of these datasets for the intended use is important.
- Addressing bias: Manufacturers must demonstrate a thorough consideration of potential biases in the datasets, including a comprehensive demographic representation across all intended medical indications or applications.
- Data Pre-processing: Detailed information about data pre-processing, such as labeling, ground truth, and the rationale behind the chosen processes.
Post-market considerations for AI models
Beyond pre-market regulatory hurdles, post-market considerations are of importance. Maintaining the performance of AI models and preventing any degradation is paramount. Natascha Cuper said: “I want to underline the importance of collecting real-world prospective clinical data, particularly if the clinical evaluation initially relied on retrospective datasets. In addition to that I advise manufacturers to establish a robust plan for model maintenance. Adhering to their own Quality Management System (QMS) procedures for change management is essential. Moreover, notifying the relevant Notified Body about any changes that could impact the device's application is equally crucial.”
Questions? Contact us!
The integration of AI in the medical device industry brings both opportunities and challenges. Navigating this dynamic landscape requires a clear approach, from meticulous clinical evaluations to ensuring post-market performance. Regulatory compliance and transparency are essential, ensuring that AI-based medical devices contribute positively to the healthcare ecosystem while meeting the highest standards of safety and efficacy. We can imagine you have questions about this. If so, please do not hesitate to contact us at medical@kiwa.com.
Read more about the discussion at the RAPS meeting: Experts outline current knowledge on AI and medical devices | RAPS