By: Philippe Cini, PhD; Gretchen Allison; Gerald Leister; Eda Ross Montgomery, PhD; Julia O’Neill; Paul Stojanovski; Michael Thomas; and Arne Zilian, PhD
As an organization begins to develop a process-capability program, it must determine how often process capability should be calculated, and who should be involved in the review and discussion of those indices. Capability index calculation requires a minimum number of data points; statisticians often recommend 20 or more if the results are to be meaningful. Hence, process-capability indices that measure lot-to-lot variability cannot be calculated too early in the development process or the start of commercial manufacturing.
The minimum frequency is annual, given the regulatory requirement of assembling annual product reviews (APRs), which typically report process-capability indices, assuming that the number of available data points is sufficient. Process capability calculations also support continued process verification (CPV) programs, so the frequency of the calculation may also be set by a company’s CPV program.
At the other end of the spectrum, capability indices could theoretically be recalculated with every new batch, although this is unlikely to yield significant additional information unless a dramatic change occurred with the last batch. Other tools, such as process control charts, may be more useful for detecting minor within- or between-batch changes. Some survey responses below indicated that the more robust an organization’s product and processes are, the less frequently they conduct capability calculations.
Current frequency is annual. Implementing quarterly program based on risk. If Ppk is > 1.3 will evaluate annually.
- Survey Participant
Some respondents calculate process capability for all batches manufactured as part of a campaign, which allows them to compare product and process performance across campaigns. One respondent indicated that "[I]f a process change has been made, it may be wise to increase the frequency with which process capability is calculated to ensure that the change is not resulting in unintended situations. Once that concern has been alleviated, the frequency of calculation of process capability can be reduced." Another said that "[F]or products for which very few batches are manufactured, they will calculate process capability less frequently due simply to the scarcity of the data."
Because process-capability indices are simple performance measures that allow organizations to monitor how the robustness of a product or a process evolves over time (from one campaign, year, or month to the next), they are excellent tools that can inform both technical and nontechnical managers in their resource-allocation decisions for continuous-improvement projects.
Clearly, there is no single way to set the frequency with which capability indices should be calculated. At the same time, it is apparent that criteria or rules for that purpose should be defined and used consistently across an organization. For this area, respondents rated themselves 3.7 on average, indicating that process capabilities are periodically calculated, summarized, and shared with site leadership (although the frequency of calculation may vary from site to site), and the results are used to drive continuous improvement efforts.
Respondents also indicated that in 2–3 years they expect to reach an average maturity level of 4.5. At that level, the frequency with which process capability is calculated has been standardized across the organization manufacturing network. Process capabilities, in conjunction with a suite of other statistical analysis tools, are used continuously by all levels of the organization to track and communicate process performance and drive continuous improvement.
A consistent basis for specification is essential for comparing process capabilities across sites, manufacturers, etc. The concept of process capability rests on the comparison of actual manufacturing results to a meaningful specification range. In their purest form, drug-product specifications should represent the needs of the patients receiving them.
To set specifications for pharmaceutical products based on patient needs, however, clinical experience would have to cover relatively broad ranges for each product attribute. This kind of clinical data is rarely available, unfortunately. In its absence, manufacturers rely on an assortment of other approaches, including assessments of achievable variation based on process development experience and analytical method performance, USP compendia specifications where available, specifications established for similar products manufactured with the same technology platform, bioequivalence studies, and previous manufacturing history.
The latter is the most challenging situation for process capability. Since most process-capability indices are based on a ratio of the process-variation range to the specification range, when specifications are based on process variation (commonly referred to as "process-capability specifications"), the calculation becomes circular. Process capability will inevitably fall close to 1.0, since both the numerator and denominator are based on process variation. To avoid this, the regulatory agency (often driven by risk consideration) may request tighter limits based on the available test data during the approval process. These would automatically result in process capabilities lower than 1.0.
Some products where specs are tight, therefore lower capability but capable.
- Survey Participant
Analytical method specs are typically based on USP. Product specifications such as control release dissolution are based on bioequivalence study and may require some discussion/negotiation with the FDA.
- Survey Participant
Survey participants rated their current state of specification bases at an average of 3.5. This indicates that, where possible, specifications are based on knowledge of the product attribute ranges necessary to achieve safety and efficacy. Participants cited examples of the current mix of bases for specification setting (below). This shows that when specifications are based on previous manufacturing history, it will inevitably fall close to 1.0.
When asked where they would like their program to be in 2–3 years, the average response was 4.2. Some participants provided examples of mature specification-setting approaches already in place, including: "Specs linked to quality target product profile."
The quest for more clinically relevant specifications is the subject of a newly formed ISPE team called the Clinically Relevant Specification Work Group. When consistently set, specifications that truly represent patient needs open the possibility for meaningful process-capability comparisons as envisioned in the FDA’s quality metrics draft guidance. Without universal specifications for a given product, comparisons of process performance across sites or manufacturers remains a challenge.
On average participants rated themselves 3.6 for their current state; this reflects consistent use of a process-capability approach, calculations, and metrics across an organization. At this level, methodologies to outline data set size, confidence limits, number of batches, effect of normality, and indication for using qualitative vs. quantitative results are documented in guidelines or SOPs across at least part of the organization (local, regional, and/or global).
Organizations chose to use short-term (Cpk) or long-term capability indices (Ppk) and control charts to monitor their processes. Of the 15 survey participants, eight used Cpk, four used Ppk and, three used both Cpk and Ppk, depending on the situation. The key is to be consistent in the calculation and use of such metrics.
When respondents were asked how much further they would like to take their programs in 2–3 years, the average rating was 4.8. At that stage, capability results are used to set long-term product and process robustness improvement strategies, which are integrated into the organization’s culture and management review processes. According to one respondent, "We are working on site harmonization around using consistent metrics,implementation of remediation plans, and identification of future process-capability opportunities."