Critical Deficiencies Undermine QRM Effectiveness

This alarming statistic was highlighted by Alice Redmond, PhD, Chief Strategy Officer at Commissioning Agents, Inc. (CAI), during the “ICH Q9(R1) - Implementation: A Framework for Demonstrating PQS Effectiveness” session at the 2025 ISPE Europe Annual Conference in London. Her presentation underscored a critical reality that despite years of comprehensive training in pharmaceutical quality systems (PQSs), ICH Q10 implementation, and additional industry initiatives, serious quality defects continue to arise. This is a persistent challenge that the ICH Q9(R1) revision specifically aims to address through enhanced quality risk management (QRM) practices.

An Improved QRM Approach

The original ICH Q9 guidance served the industry well for over a decade, but the pharmaceutical landscape has since undergone dramatic transformation. Today’s complex global supply chains, accelerated development timelines, advanced manufacturing technologies, and heightened regulatory expectations have created challenges that the initial framework simply wasn’t designed to address.

Recognizing this gap, the ICH Q9(R1) revision represents a comprehensive response to these modern realities, delivering a more robust, clarified approach to QRM that the industry needs.

Through extensive industry consultation and analysis, four critical deficiencies were identified that have been undermining QRM effectiveness across the pharmaceutical sector:

1. High levels of subjectivity in risk assessments and QRM outputs: Redmond emphasized how current risk assessments are heavily influenced by the methodology employed, the individuals conducting the assessment, and even the topic under investigation. How a risk is defined also plays a crucial role in both risk management and QRM.
2. Managing supply and product availability risks: The revision places significant emphasis on developing risk-based tools that can be applied across the entire supply chain. This focus directly addresses the mounting supply challenges evidenced by the drug shortage statistics, a problem that continues to worsen without systematic intervention.
3. Lack of understanding regarding “formality” requirements: Redmond highlighted the struggle to determine appropriate levels of formality in QRM processes by the industry as a whole, leading to either over-engineering simple risks or under-analyzing complex scenarios.
4. Enhanced clarity on risk-based decision-making: The updated guidance provides clearer direction on risk review activities and introduces refined definitions for risk identification processes, addressing long-standing ambiguities that have hindered effective implementation.

Industry’s Response

To gauge how organizations are responding to these challenges, the conference session included live polling, which revealed telling insights into the industry’s current state regarding the Q9(R1) adoption. When asked about implementation approaches for ICH Q9(R1), two strategies emerged as clear frontrunners: “No formal process, updating organically” and “Driven by corporate initiative,” with “GAP assessment - remediation and procedure updates” following closely behind.

Perhaps more revealing was the response to what constitutes the biggest implementation challenge: A decisive 33 votes identified “Mindset shift to proactive risk management” as the primary obstacle. This result prompted significant discussion among the expert panel. The session was moderated by Redmond and Caroline Cuel, Global Quality & Compliance Management Review & Governance at Roche. Panel members were:

• Kevin O’Donnell, PhD, Market Compliance Manager and Senior GMP Inspector, Health Products Regulatory Authority and ICH Q9(R1) Rapporteur
• Graham Carroll, Lead Senior GMDP Inspector, Medicines and Healthcare Products Regulatory Agency (MHRA)
• Arie Menachem, VP, Quality Audits & Inspections, Novo Nordisk

Cuel offered perspective on this challenge, noting that, “Sometimes we still really need to understand the benefit of proactive risk management so that teams work in a continual improvement environment that allows for a robust PQS.” Redmond observed that effective operations rely on both a solid understanding of risks and the integration of risk management into standard procedures and practices.

A successful ICH Q9(R1) implementation requires more than procedural updates; it demands cultural transformation that embeds risk thinking into every operational aspect.

This observation highlights a fundamental truth. A successful ICH Q9(R1) implementation requires more than procedural updates; it demands cultural transformation that embeds risk thinking into every operational aspect. The path forward is clear: organizations must move beyond superficial compliance activities to embrace the proactive, integrated approach that ICH Q9(R1) envisions. This requires companies to actively implement the necessary steps to ensure teams are knowledgeable and possess a strong understanding of how to integrate risk management into every stage of their processes.

A Global Company Case Study Done Right

The theoretical foundation of ICH Q9(R1) comes alive by looking at real-world application examples, as in Redmond’s presentation on a global company’s implementation approach.

Strategic Foundation: Comprehensive Gap Assessment

The company began with a rigorous global gap assessment, examining policies, standard operating procedures, and existing risk assessments alongside internal audit findings. This thorough baseline evaluation, conducted in March 2023, provided a critical foundation for QRM.

Training Innovation: Ditching the Instructors

Rather than relying on conventional instructor-led sessions, the company deployed hybrid training combining recorded content with interactive question-based components. The key differentiator was the active integration of participant feedback. Trainees identified improvement areas, highlighted complexity challenges, and shared what worked and what didn’t. These became insights that directly informed the evolution of the training. “For risk management to really work in a holistic way, being built into the day-to-day systems, we need to make it easier for that to happen, and this is a perfect example of that. Getting that critical feedback to build understanding,” said Redmond.

Tiered Training Excellence

The company’s three-tier training approach proved particularly effective:

• Level 1: Broad familiarization with targeted focus groups (technicians, operators, lab analysts)
• Level 2: Role-specific training refined through iterative feedback
• Level 3: Functional training on risk assessment best practices using real scenarios of previous risk assessments performed by the company

Knowledge Management Champions

Perhaps most notably, the company used the provided ICH Q9(R1) training materials to educate site and local QRM champions who carried out global risk mapping across the organization. This knowledge led to the development of a comprehensive knowledge management (KM) QRM database, representing a valuable opportunity for broader implementation across organizations.

Demonstrate PQS Effectiveness

Kevin O’Donnell presented insight into how ICH Q9(R1) serves as not only guidance but also as a framework for demonstrating PQS effectiveness that could qualify companies for regulatory relief opportunities if used correctly. (See the appendix table featured in ICH Q10). O’Donnell emphasized how Q9(R1) provides a framework to help teams demonstrate that their PQS system is indeed effective, robust, and centered on risk. He provided a compact and concise breakdown of what PQS robustness looks like in practice in a system, describing its three fundamental characteristics:

1. Patient protection: Ensuring products meet quality, safety, and efficacy standards
2. Comprehensive design and implementation: Incorporating appropriate risk management principles with monitored effectiveness
3. Clear organizational integration: Defining, communicating, and implementing PQS roles throughout the organization

The Early Warning Imperative

A critical Q9(R1) concept highlighted by O’Donnell was developing early warning systems for internal activities as well as suppliers to detect emerging product availability risks. This proactive approach directly addresses the supply chain vulnerabilities contributing to the drug shortage crisis discussed previously.

Tackling Formality Through Three Pillars

Defining formality in QRM is heavily contingent on three interconnected concepts:

1. Uncertainty: Acknowledging that changes may not proceed as planned
2. Importance: Recognizing critical process impacts
3. Complexity: Understanding how process sophistication influences risk assessment requirements

According to O’Donnell, KM is the uncertainty reducer, particularly focusing on hazards that could emerge from change implementation. He emphasized that KM is what permits an organization to reduce levels of uncertainty.

The Change Control Challenge

Regulatory observations reveal a troubling pattern: investigators frequently identify defective quality processes stemming from inadequate change controls. One company’s 2012 study found that approximately 28% of serious deviations at one of its sites were directly related to poorly assessed and implemented change controls, which had occurred earlier.

O’Donnell challenged attendees to ask themselves, “How much risk reduction will this change control provide to the overall process? To the product? To the patients?” He added, “Wouldn’t it be great if you could show regulatory bodies that 90% of your proactive change controls from the year prior lead to a measurable risk reduction this year?” This perspective shift, from viewing change controls as compliance burdens to measuring them as risk reduction investments, represents the mindset transformation that ICH Q9(R1) seeks to catalyze.

The Knowledge–Risk Integration GAP

Despite widespread recognition of their interdependence, a 2021 survey conducted by Technical University Dublin revealed a stark implementation gap: although 96% of respondents agreed that QRM and KM are highly interdependent, only 4% reported intentionally integrating these disciplines within their organizations. This disconnect represents perhaps the greatest opportunity for Q9(R1)-driven improvement. O’Donnell reinforced the idea of both risk and knowledge being infinitely connected— one will inevitably impact the other. This cycle is continuous and perpetual, as knowledge is always evolving and should be applied to inform risk.

Subjectivity in the QRM Space

O’Donnell strongly recommended the PIC/S paper titled “How to Evaluate and Demonstrate the Effectiveness of the Pharmaceutical Quality System with regard to Risk-based Change Management.” This resource provides practical checklist tools, particularly valuable for ICH Q12 implementation activities, offering concrete guidance for organizations seeking to operationalize Q9(R1) principles.

O’Donnell also addressed one of Q9(R1)’s most challenging aspects: the hidden influence of cognitive biases and subjectivity in risk assessments. When tackling subjectivity, tools such as modifying risk assessment worksheets to have built-in columns to force teams to document the GMP controls that support probability ratings, detection ratings, and severity ratings can be used. This structural approach transforms subjective impressions into evidence-based evaluations grounded in objective data and knowledge —the cornerstone of effective risk-based decision-making.

Three Essential Actions for Global Implementation

To translate theory into practice, O’Donnell concluded with three concrete actions that companies worldwide can implement immediately to ensure quality standards are maintained and product availability remains uncompromised:

1. Establish practices and procedures to operationalize the integration of QRM and KM: Moving beyond recognition of their interdependence to systematic implementation
2. Define the type of knowledge created by QRM and how this should be managed: Ensuring that risk assessment outputs become valuable organizational assets rather than archived documents
3. Develop formal mechanisms where KM serves as input in all risk assessments: Creating structured processes that leverage existing knowledge to inform new risk evaluations

Panel Discussion: Diving Deeper Into Implementation Challenges

The session’s final segment included a discussion of the most contentious aspects of Q9(R1) from both the regulator and industry perspectives. The segment included live audience polling that would reveal surprising insights about how the industry interprets and applies Q9(R1) guidance. The discussion opened with a provocative examination of subjectivity in formality.

Confronting the Subjectivity Challenge

The discussion began with a fundamental question that gets to the heart of Q9(R1) implementation: What, if any, work is your company doing to implement the ICH Q9(R1) guidance on subjectivity in QRM? This was followed by another question: How much of an issue do you think subjectivity in QRM is?

Interestingly, one answer dominated the poll results: “It can be a significant problem in risk assessments; it can lead to nonscientific risk ratings, ineffective risk control strategies, a false sense of security, and poor risk-based decision-making.”

This overwhelming consensus revealed a critical industry insight: although organizations clearly recognize that subjectivity is inextricably linked to QRM outcomes and understand its fundamental importance, many lack awareness of practical approaches to reduce its impact or systematically tackle its influence on decision-making processes.

The polling exposed a troubling gap between awareness and action, whereby teams understand the risks but haven’t systematically addressed them. Redmond brought to light the various sources of subjectivity within teams, pointing out that individual backgrounds—such as prior project experience, education, and site-specific knowledge—can all influence how risks are assessed. Menachem commented that this disconnect needed to be addressed.

This observation underscores a fundamental truth that subjectivity isn’t a character flaw; it’s an inevitable consequence of human diversity and our experiences and perspectives. The solution lies not in eliminating human judgment but in structuring processes to minimize its uncontrolled influence through careful attention to risk problem statements, tool selection, and team training.

O’Donnell noted that the study of subjectivity traces back to experimental psychology work from the 1970s that identified human heuristics and biases. From a regulatory perspective, Carroll noted that risk management has historically presented inspection challenges, particularly regarding scoring methodologies and their justification. He proposed that case studies can be a valuable learning resource for companies and should be provided in the training materials. However, he also highlighted the difficulties in making such case studies available to companies as a regulator due to the extensive anonymization which each case study must undergo. Industry organizations such as ISPE can facilitate peer learning and experience sharing in the QRM space, filling a critical gap that regulators cannot easily address.

FMEA Subjectivity: A Detailed Examination

The discussion then focused on specific manifestations of subjectivity in failure mode and effects analysis (FMEA), with Redmond posing the question: “Where might one encounter subjectivity when performing a FMEA, and how might this subjectivity adversely affect the risk assessment?” Carroll identified several critical areas where bias commonly infiltrates FMEA processes, including in identifying failure modes, severity, and detection ratings. He highlighted that some companies determine risk levels based on the number of detection mechanisms employed; for instance, categorizing a risk as medium when two mechanisms are in place, and as low when three are present. However, there is often little understanding of how effective each mechanism really is.

The Threshold TRAP

Carroll also addressed the dangers of rigid cut-off limits, noting that most risks falling below arbitrary thresholds remain unaddressed—and this only accounts for risks that were successfully identified in the first place, leaving significant awareness gaps for future quality issues. Echoing this concern,

O’Donnell commented that weighing more on “severity” than “probability of occurrences” can be useful, posing a thought-provoking scenario that challenges conventional FMEA practice by asking, “What if there were no RPN thresholds? How different really is a score of 58 compared to a 62 in the case of a RPN threshold of 60. In many cases a 58 requires no action, whereas a 62 requires risk reduction.”

Menachem advocated for tool diversification beyond FMEA, specifically recommending risk grids that use descriptive words rather than ordinal numbers. His key insight: investing more effort in the definitions of the ratings beforehand can potentially reduce the likelihood of a biased or subjective result. He also suggested that tools such as Ishikawa diagrams and the 5 whys can be used to focus teams on where the risk actually lies.

The Formality Finale

The final poll revealed encouraging progress in formality adoption, with over half of attendees voting that their companies are actively implementing ICH Q9(R1) guidance. However, for the 15 respondents uncertain about their company’s formality efforts, Menachem advised that every facility should prioritize developing internal risk facilitation experts as a foundational step toward effective formality implementation.

A case study on a company that experienced significant stability failures after changing primary packaging of a product was discussed by O’Donnell where he shared a question that was posed post-investigation: “Does one even need a risk assessment when the problem has already been identified? What probability would you assign to a stability failure when you already know that you have a stability failure?”

This observation highlighted the essence of effective risk management and reinforced O’Donnell’s earlier point regarding the potential overuse of risk assessments, serving as a valuable reminder that formality should enhance problem-solving approaches. When issues have already manifested, our energy may be better invested in robust root cause analysis and strengthening GMP controls rather than conducting redundant risk assessments. The session ultimately reinforced that formality in risk management comes down to applying the right tool at the right time with the right level of rigor.

Redmond shared her perspective on the so-called “golden hour,” describing how when an issue arises, particularly when complexity is involved, one needs to record as much as one can, as these records ultimately build the evidence and knowledge foundations that are crucial in supporting an investigation.

Conclusion

The collective wisdom from this panel of industry and regulatory experts crystallizes into three fundamental imperatives for companies embarking on ICH Q9(R1) implementation. These include: 1) successful implementation demands moving beyond procedural updates to achieve a critical mindset shift to proactive risk management, embedding risk-based thinking into daily operations, 2) the transformative power of systematically integrating KM with QRM and the inextricable link between risk and knowledge, and 3) the expert consensus on addressing subjectivity.

This last imperative was highlighted by Carroll’s call for demonstrations of effective strategies for subjectivity minimization during investigations and Menachem’s emphasis on redefining rating definitions and diversifying risk assessment tools. This reinforces that both subjectivity control and the appropriate application of formality are critical enablers of effective QRM. For additional information, please visit www.ich.org/page/quality-guidelines, select “Q9 QRM,” and then the Q9 (R1) Training Materials Zip file at the end of the section.