New GAMP® Guide Addresses Challenges Posed by AI-Enabled Computerized Systems

Since 2019, members of the GAMP^® Community of Practice (CoP) Software Automation (SA) and Artificial Intelligence (AI) Special Interest Group (SIG) have collaborated and consistently produced guidance to support practical implementation of AI-enabled computerized systems. By the summer of 2023, only a few months after the release of GPT-3 (the third-generation Generative Pre-trained Transformer), GAMP SA and AI SIG Leaders were inundated with questions, indicating a rise in concern, and the growing set of challenges our industry colleagues were facing. We agreed that it was time to create a GAMP® AI Guide to enable companies to efficiently and effectively implement AI-enabled computerized systems.

Life science professionals around the world shared their concerns and expressed their fears as they struggled to understand technology and sought guidance. Only a limited number of pioneers within industry and ISPE were collaborating on AI topics, many from the GAMP CoP. The members of the GAMP SA and AI SIG representing this group of pioneers were flooded with questions asking how, if at all, AI could be validated. From these discussions, several recurring themes and key challenges emerged:

• Knowledge management: the need to build AI literacy skills
• Evolving regulatory expectations: regulatory uncertainty from a lack of clear guidance
• Suppler relationships: lack of common language and clarity on responsibilities
• Pace of innovation: rapid technological advances outpacing organizational readiness
• Risk management: uncertainty in determining an appropriate control strategy by lack of technology understanding
• Data: often fragmented, collected, and structured for its intended purpose that may not be fit for the purpose of AI development
• Cybersecurity: new cybersecurity threats aiming to exploit the dependency on data and complexity of models
• Legal considerations: connecting the wider legislative landscape, including cross-sector AI regulation and established statues, e.g. concerned with data privacy

The result of these challenges: Too many AI initiatives failed or stopped at pilot stages. The failure rate of AI projects is estimated at 80%, which is considerably higher than general IT projects 10 years ago.1 These projects originating from a need or an opportunity aiming for tangible benefits, such as: Higher efficiency in helping to manage increasing workloads, improved support for patients through AI-enabled devices, better process control, and yield increase in pharmaceutical manufacturing, among others. Each project failure in these areas is not only a loss of investment resources but in many cases also has consequences for patients. The same being true for AI-enabled systems in operation that lack robustness, reliability, and control. It was time for a change.

The recently published ISPE GAMP® Guide: Artificial Intelligence, addresses challenges posed by AI-enabled computerized systems in the pharmaceutical and medical device industries. It offers a cost-effective, best-practice framework to help ensure these systems are high-quality, effective, fit for intended use, and comply with relevant regulations. It is meant to facilitate a common understanding of AI’s possibilities and limitations within GxP-regulated environments. It serves as a reference point for stakeholders— including regulated companies, suppliers, service providers, and regulators—on best practices in developing, implementing, and using AI-enabled systems. With a primary focus on quality and compliance, the Guide supports innovation in settings where patient safety, product quality, and data integrity are paramount. It covers both managerial and organizational topics, as well as technical aspects.

Users of the Guide can harness various business benefits:

• Facilitating risk-based approaches in conjunction with enabling innovation to allow for focusing efforts and allowing for efficiency while achieving compliance
• Supporting organizations in choosing system designs that fit the organization’s overall maturity and level of experience
• Having a common language to support collaboration across diverse stakeholders as a key factor in improving successful implementation of AI-enabled computerized systems
• Establishing an effective monitoring and maintenance strategy, while demonstrating ongoing control
• Facilitating continual improvement during operation, particularly by using newly generated data

With a shared framework provided by the ISPE GAMP® Guide: Artificial Intelligence, industry can streamline processes in the design, implementation, and use of AI-enabled computerized systems. This enables the maintenance of patient safety, improved efficiency through risk-based approaches, application of critical thinking, and opportunities for continuous improvement.

This Guide is useful to a broad set of stakeholders:

• GxP regulated companies (e.g., in the areas of pharmaceuticals, biologics, advanced therapy medicinal products, or medical devices)
• Suppliers (e.g., equipment, hardware and software)
• Service providers (e.g., contract research organizations, contract development and manufacturing organizations, contract testing laboratories or specialty labs)
• Consultancies supporting other organizations in the design, implementation, and use of AI-enabled computerized systems

Additional stakeholders may include academia and healthcare providers involved in regulated activities or committed to safeguarding patient safety, product quality, and data integrity. The Guide is also relevant to those engaged in early research outside traditional GxP-regulated areas.

Regulators and healthcare authorities may find the Guide a valuable reference for understanding industry perspectives on best practices for AI. By offering a holistic view of achieving high-quality AI-enabled computerized systems in GxP regulated environments, the Guide serves as an industry response to recent discussion and reflection papers published by regulatory and health authorities. Selected regulatory initiatives include:

• US Food and Drug Administration (FDA) Center for Evaluation and Research Artificial Intelligence in Drug Manufacturing2
• US FDA Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products – Discussion Paper and Request for Feedback3
• European Medicines Agency (EMA) Reflection paper on the use of Artificial Intelligence (AI) in the medicinal product lifecycle4^, 5
• US FDA Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products (Draft Guidance)6
• US FDA Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations (Draft Guidance)7

Alignment With Established Guidance and Standards

The new Guide bridges general concepts of ISPE GAMP^® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition)8 with the characteristics of AI, expanding on considerations provided in Appendix D11 to achieve a holistic framework. Guidance such as ISPE GAMP^® Guide: Records and Data Integrity9 and the ISPE GAMP® Records and Data Integrity Good Practice Guide: Data Integrity by Design10 have been incorporated in the new ISPE GAMP® Guide: Artificial Intelligence. The Guide considers current industry regulation and guidance as well as available standards and frameworks, such as ISO/IEC 42001:2023.11

Concepts Developed in This Guide

The Guide aims to define the overarching framework for developing and using AI-enabled computerized systems, introducing extensions to existing guidance, new concepts, and practical details:

• Risk management: Quality Risk Management (QRM) is central to ensuring patient safety, product quality, and data integrity. The Guide outlines AI-specific risk considerations and a framework for identifying and controlling risks.
• Scalable life cycle activities: Promoting a risk-based approach and use of critical thinking, the Guide encourages scalable life cycle activities.
• Regulated company and supplier collaboration: Guidance is provided to enhance supplier assessment and monitoring in the context of AI-enabled systems.
• Roles and responsibilities: The Guide addresses evolving roles and responsibilities, augmenting existing ones and introducing new roles. It highlights quality assurance unit (QAU) activities that support decision-making and ensure compliance of AI-enabled computerized systems.
• Data and model governance and management: The Guide introduces concepts like fit-for-purpose data, considerations on data quality, and an extended view on data governance to build robust data foundations. Model governance strategies further ensure decision traceability throughout development and use.
• Ideation, design, and proof-of-concept: To support AI project setup, the Guide outlines key concept-phase elements, including proof-of-concept components that form the foundation for transitioning to the project phase.
• Model development: The Guide provides model development guidance, covering key steps, risks, decision-making, and documentation throughout the life cycle.
• Testing of models and AI-enabled computerized systems: The Guide includes testing approaches for models and AI-enabled systems, using fit-for-purpose data, performance indicators, human assessments of usability and suitability, and considerations for robustness and fairness.
• Ongoing monitoring: The Guide considers the use of live data for ongoing monitoring to ensure control and support continual improvement.
• Change management: The Guide supports comprehensive decision-making in change management, addressing both business and quality drivers.
• CAPA management: The Guide outlines typical incident scenarios, remediation approaches, and incident and Corrective and Preventive Action (CAPA) considerations specific to AI-enabled systems.
• Knowledge management: Guidance is provided on integrating specialized expertise—such as data scientists and ML engineers—into established roles like data and system owners. The Guide also emphasizes AI literacy and strategies for building organizational competencies.
• Trustworthy AI: The Guide promotes responsible AI use through transparency, human oversight, and bias mitigation, facilitating responsible and ethical use of AI.
• Explainable AI: Guidance is provided to support human-AI-team collaboration in GxP regulated processes.
• Dynamic systems: Expanding on GAMP^® 5 (Second Edition) Appendix D11, the Guide addresses dynamic systems across concept, project, and operational phases.
• Cybersecurity: Cybersecurity guidance includes considerations for adversarial attacks on data and models.
• AI as and in medical device: The Guide integrates medical device practices—such as design reviews and device-specific risk management—into broader strategies for safe and effective AI-enabled systems.

Conclusion and Summary

Producing the ISPE GAMP® Guide: Artificial Intelligence has been an enlightening journey, integrating existing concepts and building a comprehensive understanding of subject matter by close interdisciplinary collaboration. The Guide highlights both the complexity of AI-enabled computerized systems and the value of teamwork across disciplines.

The AI Guide team looks forward to the impact this resource will have in supporting industry implementation of AI-enabled systems. We see the Guide as the beginning of a longer journey. As technology advances, challenges across GxP areas will grow—data volumes will grow, processes will become more complex, and personalized medicine will move closer to reality. We believe that AI can offer means to address these challenges and unlock new horizons in life sciences, if we understand AI as an integral part of a broader digital strategy—not as a one-size-fits-all solution.

We invite all stakeholders to join us in this journey, collaborating, co-creating, and co-inventing the future of life sciences. The ISPE AI CoP and GAMP SA and AI SIG welcome new members.

Purchase Guide

Figure 1: Overview of new and extended concepts developed in the Guide