When Is a Category Not a Category?

Categorization has traditionally been (mis)used by many organizations to create a tick-box approach to validation, attempting to categorize entire systems as “standard” or “configured” and/or defining the life cycle approach depending only on the category rather than on overall risk to patient safety.

This article discusses the use of component categorization within risk management and is based on work done during the update of ISPE GAMP^® Good Practice Guide: GxP Process Control Systems1 but will be equally applicable to using categorization within other system types.

GAMP^® and Software Categories

The concept of software categories within GAMP^® goes back several decades and has been refined with each subsequent issue of the guide, with “operating systems, firmware, standard software packages, configurable software packages and custom (bespoke) software” in GAMP^® 4 becoming “infrastruc-ture software, tools and IT services, standard system components, configured components and custom applications and components” in ISPE GAMP^® 5: A Risk-Based Approach to Compliant GxP Computerized Systems (Second Edition).2

ISPE GAMP^® 5 Second Edition also brings an increased focus on applying critical thinking when using categorization. This article seeks to explore further what is intended. ISPE GAMP^® 5 Second Edition1 clarifies that it recommends whole-system categorization only in the context of defining the approach to supplier assessment (Section 12.2 Using the GAMP Categories): ‘When coupled with critical thinking, risk assessment and supplier assessment, categorization can be part of an effective quality risk-management approach. Categorization is not intended to provide a checklist approach to validation. Life cycle activities need to be added, removed and scaled based on the nature of the components and the identified risks.

There are two main ways to use the categories:

• Whole-system assessment: on a whole-system level, the category of the main component may be used to help define the approach to supplier assessment. Combining categorization with an assessment of system GxP impact can help to decide the extent of supplier assessment required.
• Functional risk assessment: in a functional risk assessment, categorization can help increase objectivity in the assessment process. The increased risk derives from a combination of greater complexity and less user experience...’.2

Everywhere else in the text of ISPE GAMP^® 5 Second Edition, wording has been updated to clarify that categorization applies to the (often multiple) components or products that make up a system.

A quick and early whole-system categorization—or one based on the most significant component (such a standard or configurable core product)—might be useful in some cases in making an early and informal (not detailed or prescriptive) evaluation of the likely required deliverables at the highest level. This approach is often useful for suppliers of product that need to position their products on the continuum as standard or configurable products, as opposed to custom.

Risk Scenarios and Residual Defects

Our aim when using the concept of categorization has always been to reduce the risk to patient safety, product quality, and data integrity by implementing appropriately scaled life cycle activities. To decide what scaling is appropriate, it is necessary to identify the risk scenarios associated with our business process and for each scenario understand the following questions:

1. What is the severity of the impact on patient safety, product quality, and data integrity?
2. What is the probability of the scenario occurring?
3. What is the detectability of the scenario?
4. What mitigations can be put in place to reduce the risk to patient safety, product quality, and data integrity?

Where a risk scenario involves a software component either as a potential trigger or as a potential mitigation, it becomes important to assess the inherent likelihood of there being residual defects in that component once it is in operation (see Figure 1).

Each of the software components highlighted in Figure 1 form part of the risk analysis and each therefore requires an understanding of the likelihood of residual defects that might lead to a failure occurring within it. To be useful, that assessment needs to be done at a component level not at an overall system level because the component is of interest in the context of the risk assessment.

The inherent likelihood of residual defects is affected by:

• The complexity of creating the item (e.g., a defect is far less likely to be introduced by setting a parameter than by writing 1,000 lines of custom code)
• Novelty (everything was once custom code but if it has already been tested as part of releasing the component and used many times in similar situations, then it is much less likely to have residual defects)

The inherent likelihood of residual defects is a continuum within which all components will lie, ranging from low likelihood where a component is simple and already proven in use to high likelihood where a component is very complex and very novel (see Figure 2).

Categorization is a tool that can help us understand the inherent likelihood of residual defects by grouping components with similar attributes. ISPE GAMP^® 5 Second Edition groups components based on the question, “What complexity of activity has to be done to create the component for this specific application?” The software categories as defined in ISPE GAMP^® 5 Second Edition Appendix M4 fit onto the same continuum that defines the likelihood of there being residual defects in a component once it is in operation, placed relative to each other in a way that represents where they typically reside on this continuum (see Figure 3).

Table 1: Examples of critical thinking around software categorization.

Critical Thinking Around Categorization		Possible Actions As A Result
Software component A involves particularly complex configuration; it has a higher-than-typical risk of residual defects.		Increased verification activities, e.g., a formal configuration review, additional module testing
Software component B is provided by a less competent supplier; it has a higher- than-typical risk of residual defects.		Increased supplier surveillance arrangements, particularly during verification phases led by supplier
Software component C is newly released to the market rather than proven in use; it might also have a higher-than-typical risk of residual defects.		Increased verification activities Increased hypercare arrangements early in operational phase
Software component D is an identical copy of a configuration that has already been tested and proven in use; the reuse has a lower-than-typical likelihood of residual defects.		Reduced verifi cation activities so that only the parameterization of each instance requires testing, not the repeated configuration

The Need for Critical Thinking

Understanding that the scale shows where each category of component typically resides makes it clear that there can be components that do not reside within the “typical” range for their categorization (e.g., a very complex configured component might have a higher likelihood of residual defects than a very simple custom component). Critical thinking is therefore needed when considering the components involved in a risk scenario.

In all the examples in Table 1, the software components require only configuration as part of the application life cycle. Critical thinking identifies which present a higher or lower risk of residual defects and therefore how life cycle activities might appropriately be scaled:

These examples also make it clear that sometimes there may be alternative options for a system component, some of which will involve much more customization and have a higher likelihood of residual defects than others. Critical thinking in the planning phase of a project can allow the most appropriate solution to be selected.

As stated in ISPE GAMP^® 5 Second Edition Section 12.1.1, the following are what needs to be emphasized when using categorization as a tool:

• Computerized systems are generally made up of a combination of components from different categories; the categories should be viewed as a continuum.
• The software category is just one factor in a risk-based approach; the life cycle activities should be scaled based on the overall GxP impact, complexity, and novelty of the system (derived from the criticality of the business process supported by the system).
• Software categories still bring benefit in deciding the rigor of supplier assessment and when judging the probability of a failure or defect occurring in a system.2

Conclusion

Categorization can be a useful starting point for assessing the likelihood of failure of an individual software component, but critical thinking needs to be applied to determine the actual likelihood.

Categorization at a whole system level adds little value and should be avoided except either as a first step, before any detail of individual software components is available, to help decide appropriate scaling for supplier assessment, or as a quick and early whole system viewpoint based on the most significant component to give an early and informal (not detailed or prescriptive) evaluation of the likely life cycle deliverables.