The Dual Role of AI in GxP IT Validation: Validation Enabler and Validation Subject

To place this discussion in context, in the earlier ISPE blog posts of the Validation 4.0 Series, the author traced the evolution of Validation 4.0 from concept to regulatory confirmation. The first blog post, “Concluding Compliance Challenges with Validation 4.0”1 proposed a seven-pillar framework for next generation of computerized system validation (CSV)/computer software assurance (CSA) anticipating where regulatory thinking was headed before guidance arrived. The second blog post, “Concluding Validation 4.0 with Computer Software Assurance (CSA) and Annex 11 Framework”2 assessed those pillars against the final US Food and Drug Administration (US FDA) CSA guidance3 and European Medicine Agency’s (EMA) revised draft Annex 114 and concluded that five of the seven pillars identified had been substantively absorbed into the new regulatory frameworks.

Two of the pillars were not prominent:

1. Process and data flows as foundation for validation activities
2. Continuous control and cognitive compliance

Figure 1: Comparing the seven Validation 4.0 pillars identified in the first blog post against the confirmed pillars from the second blog post shows that process and data flows and continuous control and cognitive compliance were not carried forward.

AI brings the two less-prominent Validation 4.0 pillars back into practical relevance because AI use is often embedded within process execution rather than introduced as a standalone system, and because its behavior may require ongoing monitoring rather than one-time validation. Process and data flows make AI visible; continuous control makes it governable over time. This aligns with the human-centric principle often associated with Industry 5.0: technology should support and empower workers, while accountability remains with qualified personnel.

However, AI can empower only when its outputs can be trusted and in GxP environments trust is demonstrated through validation. Validating AI, however, differs from validating traditional systems as the AI system’s behavior is never frozen—it is shaped by models, data, instructions, and how users engage with it and it can shift in operation. This then gives AI-era validation a natural definition—establishing risk-based assurance that a system performs consistently within its defined intended use range. The word range acknowledges the reality of non-deterministic outputs requiring performance boundaries. On lines of Industry 5.0, AI-era validation can be the AI-enabled evolution of Validation 4.0—where the five confirmed pillars are strengthened, and the two less-prominent pillars (process/data flows and continuous control and cognitive compliance) become operationally essential.

Continuous Control and Cognitive Compliance

The cognitive compliance part in this pillar encouraged the use of AI systems and tools to facilitate system validation. While discussing this, the author had surfaced a peculiar dilemma in the second blog post2—while neither US FDA nor EMA guidance said anything about use of AI tools to support validation activities, industry’s regulatory attention had moved toward AI validation with industry guidance like ISPE GAMP® Guide: Artificial Intelligence7 and Draft guidelines: New annex 22 – Artificial intelligence5.

Over the last few months, AI usage for supporting system validation has evolved to many use cases across the system validation lifecycle with industry in different stages of deployment and adoption of these use cases. Notably, the same AI tools can be also used extensively in validating AI systems which brings a unique aspect of circular nature of AI in validation.

This dual role of AI introduced at the outset is both a challenge and an opportunity. It is a challenge because it introduces questions: Can AI tools enabling validation be considered a GAMP® 1 category tool like other validation supporting tools or must it have a higher classification and related controls? And can an AI tool credibly test or monitor an AI system built on the same underlying model? It is an opportunity because AI can help address some of the very weaknesses that make AI governance difficult.

To understand the opportunities of this circular nature, it is useful to examine some of the weakest links in AI governance and validation—some general to AI systems, some most acute in citizen-developed agents:

1. Risk Classification Integrity: Initial risk classification is typically based on the process the system supports and an intended use questionnaire. In a standard system, the initial declarations are reviewed and progressively verified and corrected if required part of the software development lifecycle. In case of citizen-developed agents, no such lifecycles follow—the questionnaire is often both the first and last artifact, a self-declaration by the tool’s own builder. AI can help close this gap by understanding and reading the data inputs. For example, a developer may report a tool as not containing any sensitive privacy information, but AI can read the data types to flag potential privacy sensitive data.
2. Intended Use Variance: A standard system is developed with specific user requirements, and the functionalities are limited based on that. For an AI system, the range of output can be potentially far greater than the original intended use driven by creativity of the end users. Restricting usage to specific use cases may not be always possible. This creates a testing problem—if the range of realistic use cases cannot be fully enumerated, test scenario coverage becomes difficult to define and attempting comprehensive coverage quickly escalates test volume beyond what manual approaches can sustain. An AI enabled validator can help to identify a broader range of likely use cases, personas, and misuse scenarios and generate test prompts at the scale this variance demands.
3. Evolving Specifications and Data: AI systems and more specifically citizen developed agents have potentially frequent changes. AI systems may be more sensitive to change than traditional systems with any small change in instructions, retrieval source, input data, or configuration setting having significant impact on output. This creates pressure on change control and regression testing. Standard automated testing can help but it may not be sufficient for change impact analysis and delta/additional testing. AI-assisted automated regression testing can make it scalable. AI-assisted testing is well covered by a recent GxP and AI tools: Compliance, Validation and Trust in Pharma article published by EY.6 Some AI agent platforms are already introducing built-in testing modules. This direction is encouraging, provided testing methods are documented, acceptance criteria are justified, and results are reviewed.
4. Drift: Even without deliberate changes, AI performance can degrade over time. In some cases, performance degradation may be gradual and become visible only after repeated use. This brings us to the continuous control part in this pillar. Evolving specification and Data as well as drift makes continuous monitoring more important than in traditional computerized systems. An AI validator can support based on defined performance characteristics with flags raised based on identified triggers.
5. Data and Integrations: Another practical risk in current AI models is data source variability in formats and varying levels of integrations maturity with different source systems. Many models and tools have limitations in reading certain file formats, and complex document structures.

The biggest problem is not only that the model may fail to read the complete dataset. The bigger risk is that it may produce an output as if it had read and analyzed the full dataset. This represents both a data completeness and accuracy failure, directly affecting ALCOA+ principles and a hallucination risk, where the system extrapolates beyond what was actually processed.

AI enabled validation tools should therefore include safeguards to verify input completeness. These may include accuracy and confidence indicators, file readability checks, row and column count verification, source-to-output reconciliation, and restrictions preventing the tool from making claims beyond the data processed. Where AI is used to support validation review, the tool should clearly state what it reviewed, what it could not review, and what assumptions it made.

Process and Data Flow as Foundation of Validation Activities

This pillar positions process and data flows as the starting point for validation activities rather than user requirements. They support the direct derivation of user requirements, understanding of business processes, risk definition, and all related downstream activities.

This has become more important with use of AI tools supporting validation. AI enablers may not always be detected by traditional governance models because they do not always appear as new standalone systems. They may arrive as changes to how existing processes are executed or as embedded capabilities within platforms already deployed. A governance model based on system boundary may miss them or detect it late when the system gets reported as an inventory item. A governance model at process and data flow level detects it by design—the moment an AI tool begins automating a step of a process or consuming regulated data from another source, the process flow and data flow changes which should trigger appropriate actions. This also helps with risk assessment as risk inheritance kicks in directly from the process step and no new classification framework may be necessary.

The same logic also extends to actual GxP-critical business activities where certain steps are being facilitated by AI.

Deriving user requirements directly from process steps could become much easier and faster with the aid of AI. Mapping AI as part of process and data flows could also give confidence to auditors which exact process steps are automated, where humans in the loop and any other control steps could come into the picture to mitigate residual risks associated with AI use.

This would require two good practices: 1) All automated AI or AI-enabled activities are consistently and clearly mapped in process and data flows with a view of where human-in-the-loop is involved; 2) a robust version and change control for process and data flow changes to be implemented.

Confirmed Five Pillars of Validation 4.0 in AI Era

Beyond the two open pillars we discussed, how AI enhances the confirmed pillars of Validation 4.0 to support standard system validation and how they support AI validation is also outlined below.

Conclusion

Validation 4.0 represented a movement toward risk-based and assurance-oriented validation. As the table above shows, AI strengthens each of the five confirmed pillars while simultaneously raising new questions within most of them.

The AI-enabled evolution of Validation 4.0 can represent the next step: the integration of intelligent tools into the validation lifecycle while simultaneously establishing robust governance for those tools. AI as a validation enabler can reduce manual burden, improve completeness, accelerate reviews, and strengthen risk-based decision-making. AI as a validation subject requires careful attention to intended use, data integrity, change management, monitoring, and human oversight. The two dimensions cannot be separated. The same AI capability that helps validate systems must itself be validated or governed according to its GxP impact and both best anchored in robust process and data flows. This is also where the human-centric principle of Industry 5.0 becomes a practical test. The decision to embed AI in any validation or business process should rest on whether it genuinely supports and empowers the person reviewing its output. If the human rework required to correct and verify an AI generated draft approaches the effort of authoring it from scratch, it is better to stay manual.

As the paradigm moves from point in time validation to continuous monitoring and from human-authored validation drafts to AI-generated ones, human in the loop skills become even more important. Two capabilities become central: a) critical review of AI output: The expert author and quality reviewer role evolves to include an AI-output assessor trained to detect hallucination, bias and incomplete data; b) business analysis: continuous monitoring configuration needs definition of performance characteristics and changing thresholds. This is important as irrespective of how mature AI models become, regulators would trust proper execution and review of activities impacting human safety only by qualified human personnel.