IT Services: Applying Good IT Practice & Automation

The ISPE GAMP® Good Practice Guide: IT Infrastructure Control and Compliance2 defines typical IT infrastructure components and processes that form the IT quality management system (Figure 1).

Traditionally, IT infrastructure qualification practices have been employed to ensure that IT infrastructure is appropriately specified, designed, configured, and deployed. Now, advances in IT practices, service models, service management tools, and automation provide an opportunity to establish and maintain the qualified state of IT infrastructure in a robust and efficient manner that minimizes the risk of IT infrastructure failure affecting regulated business applications and data.

Embracing Infrastructure Automation

As reported in a panel discussion, “FDA and Industry Collaboration on Computer Software Assurance (CSA),” 3 at the Institute of Validation Technology’s 20^th annual Computer and IT Systems Validation conference, 23 April 2019, the FDA and industry team’s recommendations are to:

• Embrace automation in the management of IT infrastructure
• Use electronic means rather than paper documentation
• Leverage continuous data and information for monitoring and assurance

This approach improves quality and process control while lowering quality, security, and integrity risks.

The team reported case studies on replacing manual, paper-based, and error-prone test evidence and specification maintenance with an automated, error-free approach based on standard tools. In these case studies, the time savings were ten-fold (i.e., the automated approach takes only a tenth of the time of the manual method).

Risk Profile of It Infrastructure

The primary risks resulting from a failure of an IT infrastructure environment relate to:

• Data protection and data integrity
• Business application availability and performance

IT infrastructure architectures incorporate widely used industry standard components (GAMP® Category 1 hardware and software) that typically include error detection and self-correction features, leading to a low failure rate and a high probability of threat and error detection.

Threats to the IT Infrastructure environment largely come from cyberattacks, unauthorized access, system and component failure, or inadequate re-source provisioning (storage capacity, processing capacity). These risks are continuous, and it is therefore imperative that the currency of IT infrastructure controls is maintained (e.g., through security patching) and monitoring is in place to provide early detection of any threat. IT infrastructure design incorporates a high degree of resilience that mitigates both single-point and complete failure.

Further, IT infrastructure supports business applications that hold, process, and transmit regulated records. The completeness and accuracy of these regulated records are largely governed by the business processes supported by these business applications. IT infrastructure is a secure platform that hosts these applications and data and does not directly impact regulated records. This ensures that the risk to patient safety, product quality, and regulated data integrity resulting from an IT Infrastructure failure is low.

IT infrastructure also supports nonregulated business applications and data. Therefore, the IT infrastructure cannot effectively be partitioned into GxP and non-GxP. As such, common IT practices and controls are used to manage IT infrastructure supporting both GxP and non-GxP operations.

Industry-standard IT management practices, electronic service management tools, modern IT service models, automation, and continuous monitoring are essential for ensuring the performance, security, and integrity of the IT infrastructure environment.

IT Quality Management System

An IT quality management system based on well-established, cross-industry IT governance standards such as IT Infrastructure Library (ITIL)4 is essential to the effective management of the IT infrastructure environment. The IT quality management system establishes robust processes, technologies, and subject matter expertise to effectively manage the IT infrastructure environment in accordance with risk. Further, such IT quality management systems use metric-driven continuous improvements to enhance IT controls maturity. Examples of IT practices include the following.

• General management practices:
  • Strategy management
  • Portfolio management
  • Architecture management
  • Workforce and talent management
  • Continual improvement
  • Measurement and reporting
  • Risk management
  • Information security management
  • Knowledge management
  • Organizational change management
  • Project management
  • Relationship management
  • Supplier management
• Service management practices:
  • Business analysis
  • Service catalog management
  • Service design
  • Service level management
  • Availability management
  • Capacity and performance management
  • Service continuity management
  • Monitoring and event management
  • Service desk
  • Incident management
  • Service request management
  • Problem management
  • Release management
  • Change management
  • Service validation and testing
  • Service configuration management
  • IT asset management
• Technical management practices:
  • Deployment management
  • Infrastructure and platform management
  • Software development and management

These IT practices are consistent with the expectations of the ISPE GAMP® Good Practice Guide: IT Infrastructure Control and Compliance.2

Service Management Tools

Electronic service management tools that incorporate configuration management databases (CMDBs) and electronic workflows supporting change management, configuration management, and incident and problem management are integral to the IT quality management system.

The CMDB supports effective management of the configuration status of IT infrastructure components and business applications. Electronic workflows ensure adherence to processes and collaboration of IT subject matter experts across global organizations.

IT Qualification vs. IT Quality Management

Traditional qualification activities can be integrated into the IT quality management system and service management tools, avoiding the need for one-off protocols and paper records management. For example, if we consider a backup service, work instructions can be created within the service management tool to define how a new server or storage device is added to the backup solution. Backup scheduling is configured within the backup tool to ensure backups are scheduled at the right frequency. Alerts are configured to automatically notify of failures. Evidence of backup logs and backup restoration tests can be captured in a secure repository. This can all be achieved without the need to create and execute standalone protocols. In essence, the backup deployment, configuration, and monitoring become part of the operational processes of the IT quality management system.

Product Quality Management vs. IT Quality Management

There are fundamental differences between the quality management system directly supporting the regulated product life cycle, and the IT quality management system supporting information technology and infrastructure.

Quality management systems supporting the product life cycle include risk management processes such as deviation management, corrective and preventive action (CAPA), and change management. The processes engage business and quality assurance subject matter experts to effectively evaluate and manage risks impacting patient safety and product quality.

Similar processes are included in the IT quality management system, but they are focused on IT risks relating to availability, performance, and information security. These processes include incident and problem management and change management that engage IT subject matter experts in the evaluation and management of IT risks.

Interfaces may be established between the IT quality management system and the product life-cycle quality management system (Figure 2) to hand off potential risks relating to patient safety and product quality. For example, an IT incident may potentially impact GxP data (e.g., data loss or corruption). Such incidents should be communicated to business and quality assurance functions so that the business and regulatory impacts can be assessed and mitigated. The CMDB will support IT in identifying IT incidents that could potentially impact GxP.

IT Infrastructure Life Cycle and Automation

The IT infrastructure life cycle comprises:

• Resource provisioning
• Configuration management
• Monitoring
• Compliance
• Optimization

Resource provisioning utilizes Infrastructure as Code (IaC) and virtual machine templates to provision new servers and services that are configured in accordance with IT standards. Infrastructure code and templates are subject to version control using code management tools. Changes to code and templates are fully auditable in the event of an inadvertent or unauthorized change.

Configuration management ensures that infrastructure code and templates automatically provision a standard configuration. Thus, code and templates can be verified once and used many times when provisioning like resources.

Monitoring uses tools that monitor IT availability, performance, incidents, and security vulnerabilities. Automated alerts are directly sent to support teams to enable a timely response. Self-correcting technologies allow for adjustments in configuration to address performance and other issues. Security log monitoring identifies and reports potential unauthorized access attempts.

Compliance is monitored to minimize the risk of deviation from standard configurations. Environments are automatically audited against configuration standards configurations, and deviations are self-corrected following inadvertent or unauthorized change.

Optimization is enabled through metrics provided by monitoring tools. IT infrastructure resources such as processing capacity, storage capacity, database capacity, network routing, and load balancing can be adjusted based on feedback to maintain system availability and performance.

IT Infrastructure Monitoring

Monitoring technologies provide real-time feedback on the status of the IT infrastructure environment. Such monitoring includes, but is not limited to:

• Information security vulnerabilities
• IT environment availability
• Database performance
• Infrastructure component failure
• Network connectivity issues
• Application and platform errors
• Virtual environment performance

Machine learning is now being deployed to evaluate data sets (events and logs) generated by monitoring tools. Data trends are analyzed to predict potential IT infrastructure incidents and proactively act to minimize the risk of IT infrastructure failures.

Conclusion

Historical IT infrastructure qualification processes based on paper records can be inefficient. Such approaches often only confirm the correct operation of the IT infrastructure at a point in time and seldom provide assurance that IT controls continue to operate effectively.

Advances in IT infrastructure service models, virtual technologies, automation, monitoring, and self-correction technologies have led to significant improvements in IT governance. Implementation of an IT quality management system based on industry standards, electronic service management tools, and automation is fundamental to managing IT risks.

GAMP Global Leadership strongly supports and endorses the application of these approaches as an effective way of achieving the controls principles defied in the ISPE GAMP® Good Practice Guide: Infrastructure Control and Compliance.2

Buy Now