MHRA Publishes AI Airlock Phase 2 Report on Regulatory Challenges for AI Medical Devices
The Medicines and Healthcare products Regulatory Agency (MHRA) has published the Phase 2 report of its AI Airlock programme, an initiative designed to explore regulatory challenges associated with artificial intelligence (AI) in medical devices and in vitro diagnostic medical devices (IVDs).
The report brings together findings from case studies, simulation workshops and stakeholder discussions involving regulators, manufacturers, healthcare professionals and technical experts. It identifies areas where additional regulatory clarification may be needed to support the safe deployment of AI technologies within healthcare.
Performance Evaluation for AI-Powered IVDs
One of the key areas explored during Phase 2 was the evaluation of performance for AI-powered IVDs.
According to the MHRA, existing expectations relating to sensitivity, specificity, analytical performance, clinical validity and scientific validity remain applicable to AI-based IVDs. However, the report notes that additional metrics may sometimes be required to provide a more comprehensive understanding of device performance.
The programme also explored situations where AI systems may identify findings beyond the range of traditional human assessment, raising questions regarding appropriate comparators and validation approaches.
Managing Variability in Real-World Environments
The report highlights that AI-powered devices can be particularly sensitive to variations in data quality and deployment conditions.
For digital pathology applications, factors such as staining intensity, scanner type, image resolution and image processing methods may influence model performance. Participants also highlighted the importance of reference datasets that adequately reflect real-world laboratory variability, demographic diversity and challenging clinical cases.
The MHRA notes that governance, maintenance and independence of such datasets remain important considerations for future regulatory discussions.
Lifecycle Management of AI Systems
Another major focus of the programme was the management of changes to AI systems after they have been placed on the market.
The report discusses the relationship between Post-Market Surveillance (PMS) activities and Predetermined Change Control Plans (PCCPs), highlighting how both mechanisms may support the lifecycle management of AI devices.
Workshop participants indicated that clearer expectations regarding significant changes, performance thresholds and change management processes could help reduce uncertainty for manufacturers and Approved Bodies.
Explainability and Human Oversight
The programme also examined how explainability can support the safe and effective use of AI technologies.
According to the report, different stakeholders require different forms of explanation. Regulators may need information regarding how a model was developed and controlled, while healthcare professionals may require case-specific explanations to support clinical decision-making.
The findings further emphasise the importance of maintaining meaningful human oversight throughout the lifecycle of AI medical devices.
Recommendations for Manufacturers
The report includes several recommendations directed at manufacturers.
These include:
Considering real-world deployment conditions during validation activities;
Implementing appropriate human oversight mechanisms;
Documenting limitations within training and validation datasets;
Justifying the selection of performance metrics;
Integrating PMS activities and change management planning from the early stages of development.
The report also encourages manufacturers to consider explainability and change management requirements throughout the product lifecycle rather than treating them as standalone activities.
Why This Matters for Medical Device and IVD Manufacturers
For manufacturers developing AI-enabled medical devices and IVDs, the report provides insight into regulatory topics that continue to receive attention from the MHRA.
Several themes discussed throughout the programme are likely to be relevant for organisations developing or maintaining AI-based technologies, including:
AI performance evaluation;
Dataset quality and representativeness;
Explainability;
Human oversight;
Post-market surveillance;
Predetermined Change Control Plans (PCCPs);
Lifecycle management of AI systems.
The report highlights the increasing focus on ensuring that AI-enabled devices remain safe, effective and appropriately controlled throughout their lifecycle.
