New EU AI Regulation and GAMP® 5
This article describes how ISPE GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition) and related GAMP Good Practice Guides can be effectively applied to help meet the requirements of the proposed European Union (EU) artificial intelligence (AI) regulation for qualifying GxP-regulated systems employing AI and machine learning (ML).
On 21 April 2021, the EU Commission presented the long-awaited draft on the regulation of AI. The document is based on a number of reports from the EU Commission and aims to ensure citizens’ trust in AI systems. The regulation is the first targeted legal regulation of AI. As such, it will have great significance in Europe and the rest of the world in relation to the development and use of AI. The AI regulation applies alongside the General Data Protection Regulation (GDPR), as systems must comply with both, e.g., when using personal data for training algorithms or when using AI systems for automatic decisions with legal effect for the data subjects1.
The GAMP guidance may potentially prove useful for other areas and industries in supporting the quality assurance activities and methods described in the draft regulation, at the discretion of the organizations involved. GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition) covers AI/ML components and their life cycles2, and the GAMP® RDI Good Practice Guide: Data Integrity by Design covers the data life cycle aspects of such systems, which can help ensure data integrity, a key requirement for these types of applications3.
Description of the AI Regulation1
The AI regulation contains four types of regulations:
- Prohibition of the use of certain AI systems (Article 5)
- Special requirements for the use of AI systems that are considered to present a high risk (Articles 6–51)
- Transparency requirements for AI systems interacting with humans (Article 52)
- A framework for voluntary “codes of conduct” for AI systems that are not high-risk systems (Article 69)
Prohibited AI systems are ones that harm people physically or psychologically with subliminal techniques or by exploiting vulnerabilities, that implement “social score cards” by monitoring citizens, and that use special forms of facial recognition/personal recognition.
High-Risk Systems
The focus of the AI regulation is to regulate high-risk systems, which are defined as those within eight areas:
- Biometric identification and categorization of natural persons
- Management and operation of critical infrastructure
- Education and vocational training
- Employment, worker management, and access to self-employment
- Access to and enjoyment of essential private services and public services and benefits
- Law enforcement
- Migration, asylum, and border control management
- Administration of justice and democratic processes
For management and operation of critical infrastructure, this includes AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating, and electricity. For employment and worker management, this includes AI systems intended to be used for the recruitment or selection of natural persons, notably for advertising vacancies, screening or filtering applications, and evaluating candidates in the course of interviews or tests. For access to private and public services, this includes AI systems intended to be used to dispatch or to establish priority in the dispatching of emergency first response services, including by firefighters and those administering medical aid.
For these systems, for example, a risk management system and a quality assurance system must be established, just as requirements for human involvement, transparency, robustness, cybersecurity, and correctness must be established. This is where GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition)2 and the GAMP® RDI Good Practice Guide: Data Integrity by Design3 can be useful. The GAMP® 5 framework and other GAMP guides already contain strong and mature guidance on the establishment of quality assurance systems and risk management systems, and on ensuring the integrity of data, which is essential for robustness and correctness.
Appendices
Appendix D11 in GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition) focuses on AI/ML2. It provides a basic understanding of AI, the use of static and dynamic ML subsystems in industry, and guidance on how to ensure compliant integration and fitness for use in a regulated environment. It also presents an overview of a risk-based, regulatory-compliant AI/ML life cycle framework that aligns with GAMP® 5 principles and phases (concept, project, and operation).
It describes the importance of data integrity to the overall quality of AI/ML, in addition to presenting an understanding of inherent risks, and acknowledges the iterative nature of developing AI/ML as a subsystem within the overarching IT application and/or business solution, all in conjunction and support of good software quality engineering practices.
Appendix S1 in GAMP® RDI Good Practice Guide: Data Integrity by Design3 examines the area of ML and the importance and implications of data integrity on the outcomes of what “machines” are able to process and/or learn from the data made available to them. Both Appendix D11 and Appendix S1 describe a life cycle approach, from concept to project (i.e., data modeling and evaluation) and operation, including deployment and continuous monitoring.
AI Technical Documentation and GAMP® 5
In GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition)2, Article 111 describes the technical documentation needed. It outlines that it shall contain at least the following information, as applicable to the relevant AI system, as shown in the following tables.
| A General Description of the AI System, Including: | GAMP® 5 Sections and Appendices that Support These Requirements |
|---|---|
| Its intended purpose, the person(s) developing the system, date, and system version | D6–System Descriptions |
| How the AI system interacts or can be used to interact with hardware or software not part of the AI system, where applicable |
|
| Versions of relevant software or fi rmware and any requirement related to version update | D6–System Descriptions |
| Description of all forms in which the AI system is placed on the market or put into service |
|
| Description of hardware on which AI system is intended to run |
|
| Marking and internal layout of products when the AI system is a component of products, photographs, or illustrations showing external features | Not supported by GAMP® 5 |
| Use and installation instructions |
|
Conclusion
AI and ML are transforming the way in which industry is doing business and processing data. The pharmaceutical industry is increasingly relying on such innovative technologies to automate many functions previously performed by humans. As computer systems become more integrated and datasets become more extensive, computer science is advancing our ability to learn from that data and draw conclusions.
| A Detailed Description of the Elements of the AI system and the Process for Its Development, Including: | GAMP® 5 Sections and Appendices that Support These Requirements |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
| Detailed Information about Monitoring, Functioning, and Controlling the AI System, Including: | GAMP® 5 Sections and Appendices that Support These Requirements |
|---|---|
|
|
| Detailed description of the risk management system in accordance with Article 9 |
|
| Description of any change made to the system through its life cycle |
|
|
Not covered or supported in GAMP® 5 |
| Copy of the EU declaration of conformity | Not covered or supported in GAMP® 5 |
| Detailed description of the system in place to evaluate the AI system’s performance in the postmarketing phase in accordance with Article 61, including the postmarketing monitoring plan referred to in Article 613 |
|
Underlying algorithms are sophisticated enough to begin making robust decisions in the form of AI. The listed requirements in the draft regulation for developing and operating high-risk AI systems are all based on good engineering practice. Many activities in GAMP® 5 and supporting guidance, like GAMP® RDI Good Practice Guide: Data Integrity by Design, are also based on good engineering practice and, as such, can serve as the basis for how to fulfill the listed requirements.
Even though high-risk AI systems are not evaluated as GxP systems, it will be beneficial to use the GAMP-based quality activities from the company’s quality management system. GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition) and related GAMP Good Practice Guides can be effectively applied to help meet the requirements of the proposed EU AI regulation for GxP-regulated systems employing AI/ML that fall under the scope of that regulation. GAMP guidance may also prove useful for any organization wishing to meet the quality assurance requirements of the draft regulation for other AI/ML systems.
