Pharma 4.0 – Towards IT/OT Architectures for Prescriptive Maintenance

Back to Top

• Table of Contents

  Table of Contents

  1. Introduction

  • 1.1 Background – Maintenance: Status Quo and Future Perspective
  •  1.2 Smart Sensors as Drivers for Future System Architectures
  •  1.3 Aim of the Work

  2. Theoretical Background/Applied Standards

  3. Proof of Concept

  • 3.1 Results and Learnings
  • 3.2 The Asset Administration Shell
  • 3.3 Future Work: Towards an Application Based on the AAS Standard

  4. Conclusion and Outlook

  5. Appendix: Detailed Concept

  6. Acronyms and Abbreviations

  7. Acknowledgments 

  8. References

Download PDF

Abstract

Efficient maintenance is an important factor in pharmaceutical manufacturing to ensure production capacity and avoid unplanned downtime. Currently, most companies perform maintenance activities using time-based cycles dictated by Standard Operating Procedures (SOPs) instead of the actual condition of the equipment. Predictive or prescriptive maintenance offers a new condition-based approach that allows for convenient scheduling of maintenance activities and prevents unexpected equipment failures.

Smart sensors that operate according to this paradigm can deliver condition-based data in addition to the actual real-time sensor values; however, the traditional “automation pyramid” Information Technology/Operational Technology (IT/OT) architecture is not well suited to handle such data. A new architecture approach is needed that can natively integrate the capabilities of smart sensors.

This Concept Paper explores a possible IT/OT architecture for condition-based sensor data exchange in the pharmaceutical industry. It has been tested experimentally in a Proof of Concept (PoC) to determine its practical suitability. The NAMUR Open Architecture (NOA) was chosen as the guideline for the architecture design.

The designed architecture was implemented at a Contract Manufacturing Organization (CMO) on a GMP-compliant bioreactor skid (60 L pilot scale) equipped with a Distributed Control System (DCS) including full batch, Clean in Place (CIP), and Sterilization in Place (SIP) functionality. A smart sensor was added to the bioreactor system and connected to the DCS and an edge device for cloud communication. The PoC demonstrates that existing plants can readily be equipped with additional smart sensors to provide condition-based data and that bidirectional communication between the field and the cloud can be established.

However, maintenance-relevant metadata can still not be provided in an efficient manner with this architecture. To deliver efficient digital support for maintenance tasks, the vendor-independent standard Asset Administration Shell (AAS) is identified as a possible solution to gather and transfer time-based and condition-based data and metadata in a standardized information model. As some of the relevant AAS standards are still in development, the proposed architecture needs to be iteratively updated in future work to fully utilize the benefits of data-driven maintenance.

1 Introduction

Faced with global competition and shrinking margins, (bio)pharmaceutical manufacturers worldwide are urged to increase production efficiency by making better use of time and resources. This is in addition to addressing the unique challenges faced by pharmaceutical manufacturers such as a strictly regulated environment, heterogeneous equipment suppliers, and long adaptation times. Digitalization and Industry 4.0 1 are two concepts that support the achievement of these goals.

Within the framework of Industry 4.0 1, the Industrial Internet of Things (IIoT) offers major opportunities for improved asset performance and operational efficiency, new service offerings, and completely new ways of offering value to customers. The IIoT harnesses the power of smart devices and real-time analytics to leverage the data that is constantly produced in industrial environments. The idea behind IIoT is that smart equipment is not only better than humans at collecting and analyzing data in real time, but also at communicating important information. This information can be used to make faster and more accurate business decisions during operations, maintenance, and calibration strategies.

Through a network of sensors, instruments, and other devices connected into systems that monitor, collect, share, and analyze data, the IIoT leverages smart technology such as machine learning, Artificial Intelligence (AI), and other technologies that can impact every level of the manufacturing ecosystem. An enormous potential for greater efficiency in manufacturing, operation, and maintenance management lies in the orchestration of smart devices’ capabilities with user-oriented software applications. What exactly are the benefits IIoT technologies can bring to biopharmaceutical manufacturers? Which application fields are the most promising? How can maintenance activities be supported?

1.1 Background – Maintenance: Status Quo and Future Perspective

Frequent maintenance activities have an impeding effect on equipment performance and directly impact the Overall Equipment Effectiveness (OEE) 2. However, it is indisputably of importance to ensure production capacity and avoid unplanned downtime. To define maintenance cycles and tasks, most companies rely on Standard Operating Procedures (SOP) based on the supplier’s or Original Equipment Manufacturer’s (OEM) recommendations, regulatory frameworks, and/or internal processes. Most of these concepts are predefined and time cycle-based, contrary to better knowledge that the equipment wear depends on the process cycles, material, type of device, and the medium used. Experience and feedback from users show that a large proportion of the respective maintenance and calibration cycles for sensors are performed too early, resulting in too frequent production stops and high personnel costs. Cutting-edge IIoT applications aim to create a better approach by calculating optimal timeslots for maintenance and calibration.

In general, predictive maintenance is one of the most frequently stated benefits offered by the IIoT. Predictive and prescriptive maintenance are state-of-the-art approaches that rely on the actual condition of equipment rather than average or expected life statistics. By learning the correlations and patterns between the gathered data and the life span of the equipment, better decisions can be made on when to take corrective actions. Once a reliable predictive model is available, measures can even be automated.

This allows for convenient scheduling of maintenance activities and prevents unexpected equipment failures 3. In the field of large molecule production equipment, annual maintenance costs amount to 8–15% of the original equipment investment 4. Current projects show that maintenance costs can be reduced by 15–30% if data and documentation are digitally available for planning and maintenance execution 5. Prescriptive maintenance, therefore, has the potential to generate significant financial savings.

1.2 Smart Sensors as Drivers for Future System Architectures

In terms of sensors, there are a wide range of smart devices available on the market that deliver relevant condition-based data beyond core process control. This includes, for example, information on self-diagnosis, spare parts replacement indication, calibration status, or validation documents. However, traditional Information Technology/Operational Technology (IT/OT) architectures following the ISA95 standard, colloquially called the “automation pyramid” 6, often hinder the efficient transfer of condition-based data to the appropriate target systems. Distributed Control Systems (DCS), which are used as central communication hubs for sensors and actuators on the field level, are not well suited for handling such data, from either a technical or organizational perspective.

From a technical point of view, there is a discrepancy between the one-dimensional, time-based operational data typically used within DCS and the condition-based data, which can be multi-dimensional (images, documents, calibration curves, etc.).

From an organizational point of view, there also is a mismatch. Typically, users of a DCS (operators) are not interested in conditional data, whereas interested users (such as maintenance engineers) might have no access to the DCS at all. To circumvent these issues, suppliers of smart sensors have provided interfaces to directly read the additional data with proprietary companion apps. However, these solutions are also not optimal as they lead to inefficient and poorly scalable system architectures, especially when working with diverse devices from different suppliers.

Therefore, a new vendor-independent architecture approach is needed that can natively integrate the capabilities of smart sensors to fully utilize their potential throughout their lifecycle. The traditional, strictly hierarchical automation pyramid needs to be adapted to facilitate flexible information exchange across all system layers.

1.3 Aim of the Work

This Concept Paper seeks to contribute to the advancement of Industry 4.0 1 within the pharmaceutical industry by exploring a possible IT/OT architecture that enables flexible information exchange of condition-based sensor data. The aim is to leverage existing standards of connectivity and data ingestion and to evaluate their applicability regarding the NAMUR Open Architecture (NOA) 7 concept.

The theoretically developed concept has been tested experimentally in a Proof of Concept (PoC) to determine its practical suitability in GMP-compliant manufacturing. Beyond the technical domain the goal is to evaluate whether the approach of providing condition-based data is sufficient to create value for maintenance applications.

2 Theoretical Background/Applied Standards

The availability of appropriate standards is key to the connectivity of assets and systems and to the realization of new applications. Working groups and organizations such as Plattform Industrie 4.0 1, NAMUR 8and others are driving standardization by leveraging existing principles, including NOA 7 (see Figure 2.1) or the Reference Architecture Model for Industrie 4.0 (RAMI)9. In addition, communication protocols such as OPC UA (Open Platform Communications Unified Architecture) 10or Message Queue Telemetry Transport (MQTT)11are developing as industry standards 12.

The goal of NOA7 is to enable asset monitoring in production environments by integrating secondary communication channels parallel to the existing hierarchical structures. Therefore, the widely accepted advantages of the automation pyramid and the automation system itself are not disrupted. This makes NOA suitable for both existing plants (brownfield) as well as new installations (greenfield) 13.

Figure 2.1: The NAMUR Open Architecture 7

To evaluate NOA’s applicability within the pharmaceutical industry and to demonstrate the value add for maintenance applications, a PoC was established. It aims to showcase in a real plant how production equipment (including a DCS) can be combined with a smart sensor with respect to all IT/OT security needs for GMP-compliant pharmaceutical environments. Based on the NOA concept of Monitoring and Optimization, cloud connectivity is required to establish dashboarding and analytics functionalities in the context of maintenance.

3 Proof of Concept

The realization of the PoC has been established at a CMO site on a GMP-compliant bioreactor skid (60 L pilot scale, see Figure 3.1). It is equipped with a DCS, including full batch, Clean in Place (CIP), and Sterilization in Place (SIP) functionality. In addition to communication through the bus systems of the plant, the DCS is able to exchange data over an OPC interface. See the Appendix for supplementary technical details of the experimental setup.

Figure 3.1: Schematic Representation of the PoC Smart Sensor (left), Bioreactor (center), and

Cloud Cockpit (right)

Center image used with permission from ZETA GmbH, www.zeta.com/en/.

A smart sensor is added to the bioreactor system and connected to the DCS (OPC UA) as well as to an edge device (Bluetooth low energy, OPC UA). To allow connectivity outside the CMO’s manufacturing network, two secure connectivity channels to a cloud are realized. One via an edge device (MQTT) and one via a connectivity server (WebSocket Secure (WSS)), both connected to a cloud environment hosting a platform for data collection, analytics, and data presentation (cockpit).

The cockpit provides the user with the sensor’s health status, facilitating early detection of maintenance cycles and diagnoses of sensor failures. At the same time, real-time bioprocess data is available in the cloud environment and made available to the user. This platform also shares information with smart devices via apps.

The communication paths between the main components of the system (DCS, Smart Sensor, Edge Device, and Cloud) are each implemented redundantly via a secondary interface (as indicated by the dashed arrows in Figure 3.2). The goal is to demonstrate flexible information exchange with a variety of different interfaces/communication protocols.

Figure 3.2: Schematic Representation of the PoC based on NOA

3.1 Results and Learnings

The PoC demonstrates that machines and production lines in existing brownfield plants can readily be equipped with additional smart sensors. Condition-based data generated by the sensors enriches real-time process data and is available for maintenance and operation purposes. Both OPC UA and MQTT proved to be good choices for the communication protocols as they were easy to implement and robust in information transfer. On the other hand, the experience with Bluetooth was mixed. Depending on the environmental configuration (mounting point, nearby metal parts, etc.), the data connection was not always stable; therefore, it is not recommended for transmitting time-critical data.

However, the collected data is still not sufficient to unlock the full potential of a data-driven maintenance approach. Relevant metadata (serial numbers, location within the plant, spare parts list, etc.) is seldom provided by the sensors themselves, although this is essential data for maintenance and calibration applications. This metadata is partly available via the smart sensor supplier’s ecosystem through additional data sources like web tools and/or partly covered in digital engineering systems (Computer-Aided Engineering (CAE) tools). These systems hold the relevant data but usually do not allow for easy integration into the proposed architecture without inefficient custom-made interfaces.

To deliver efficient digital support for maintenance and calibration tasks, a vendor-independent standard of accessing not only real-time and condition-based data, but also metadata is needed. The full financial savings and business value can be realized only if these diverse forms are efficiently utilized to their full potential.

These findings led to the decision to broaden the view within this work and evaluate the Asset Administration Shell (AAS), an existing standard that could contribute to solving the stated challenges.

3.2 The Asset Administration Shell

Being indispensable for interoperability, a well-defined standard for data exchange and mapping is a major requirement in modern IIoT approaches. It is also a precondition for the integration of information provided by different suppliers. The AAS is intended to provide a scalable, standardized, yet flexible framework that enables an exchange between the supplier’s, manufacturer’s, and economic operator’s software systems 14.

As a standardized digital representation of an asset, the AAS is a cornerstone of the interoperability between applications managing a manufacturing system. This standardization allows sharing of engineering information (e.g., 3D models, material specifications), documents regarding electrical planning, qualification, and much more between all parties. Once the envisaged progress has been made in developing a standard for the information model, it is possible to deliver software applications that connect data across all equipment suppliers over the entire lifecycle of an asset. This will enable maintenance and calibration activities to be carried out much more efficiently, and also be a steppingstone for many other data-driven use cases.

This is the predicted future scenario for maintenance and calibration: while the real-time data from machines is used for analysis, AI-based technologies help predict machine and sensor conditions. A smart algorithm informs the plant operator when a sensor needs to be calibrated, serviced, or replaced. Advanced maintenance cycles are calculated at an early stage and initiate a maintenance activity when indicated by the equipment. This way, targeted maintenance work is well coordinated with spare parts management and production schedules, keeping production downtimes, maintenance times, material consumption, and costs low. Eventual machine failures are considered in the overall production planning and scheduled appropriately.

3.3 Future Work: Towards an Application Based on the AAS Standard

The AAS is constituted by sub-models for each respective content-related or functional aspect of an asset. Each of these sub-models is dedicated to a defined purpose (for a sensor these could be maintenance or time series data for example) and should ideally be based on existing and commonly recognized standards. However, many of the relevant sub-models are currently still in development 15. Therefore, the aim for future work is to iteratively update the proposed PoC to integrate time and condition-based data as well as metadata from all relevant sources according to the AAS sub-model standards.

4 Conclusion and Outlook

In order to fully facilitate the benefits data-driven maintenance, three types of data are relevant: time-based, condition-based and metadata. The proposed architecture, based on the NOA, can efficiently handle the first two types of data, but has limitations when working with metadata from remote data sources. Nevertheless, it can be regarded as a reasonable first step as it is easy to integrate with existing architectures (brownfield), utilizes stable communication protocols (OPC UA/MQTT) and is able to provide multiple benefits in comparison to the traditional automation pyramid architecture. Besides maintenance, the bidirectional communication between the field (OT) and the cloud (IT) opens possibilities for improvements in numerous areas such as calibration, documentation, qualification, and many more.

The AAS, with its standardized information model, can be seen as the next step to overcome the current limitations regarding metadata. Standards specifically for maintenance applications are currently in development to enable efficient information transfer – especially across different organizations. Furthermore, it reinforces the focus on considering data along the entire lifecycle of an asset, which enables sustainable data usage in diverse applications. The proposed architecture needs to be adopted according to the AAS standards in future work for unlocking the potential of data-driven maintenance.

It is essential that the useability and feasibility of new standards are proven in cross vendor PoC. The described benefits of the AAS can be fully realized once the standards are accepted widely and supported by system suppliers, machine builders, and end users alike.

5 Appendix: Detailed Concept

All practical tests (PoC) were conducted on a small-scale GMP-grade bioreactor system, a compact 60 L bioprocess system for microbial and cell culture applications. As it includes the same major components as a large volume bioreactor at a small scale, it can serve as representation of a typical pharmaceutical manufacturing plant for research purposes. The bioreactor (fermenter) is equipped with an additional smart sensor (Hamilton Arc), a DCS (SIMATIC PCS 7 16) for process control, and an edge device (Siemens Simatic IPC227E 16) to connect to the cloud platform (PTC ThingWorx® 17); see Figure 5.1.

Figure 5.1: Technical details of the PoC

The smart sensor technology used (includes pH, optical dissolved oxygen, cell density, conductivity, Oxidation Reduction Potential (ORP) and CO2 measurement) provides a new perspective on sensor integration and management by integrating intelligence into the sensor head. The physical sensor seamlessly connects to existing R&D, process development, and production infrastructure at any digitalization level. This helps reduce start-up costs and integration efforts for new projects and expansions. The sensor software used lowers operating costs through laboratory calibration, predictive diagnostics, and automatic documentation.

The edge device ensures connectivity from the sensor via wired (OPC UA) or wireless (Bluetooth low energy) connection to systems like the DCS and the cloud platform. Secondly, it serves as the hub to exchange data between the OT and IT world in a secure manner. The logic of specifying which data to extract from a source and loading it into a target system is orchestrated via the device vendor’s edge management software.

Additionally, edge devices have the ability to preprocess and analyze data before passing it to succeeding systems. This is especially done for computationally intensive calculations that would overload the controllers in the field. This feature was not used, however, as the goal for this PoC was only to exchange information.

The cloud platform is required for bringing together the data from all the diverse sources into one shared location and building IIoT solutions upon this basis. All actions that require a considerable amount of computing power (such as models for predictive or preventive maintenance) are calculated in the cloud. It also serves as a central hub for end users, who access the provided functionalities with apps tailored to the specific needs of the user groups. The goals are to enhance decision making, improve operational efficiency, predict outcomes, and reduce costs.

In this specific PoC, the cloud platform was used to improve maintenance operation and cycles. The cloud platform enables connecting the heterogeneous and inhouse data sources, such as the DCS and smart sensors, as well as organize the data in unified data models, analyze findings, and derive actions that can be deployed either automatically or based on user interaction.

Data protection and security are crucial components of every IT/OT landscape, especially when handling safety-relevant device data. Accordingly, state-of-the-art security mechanisms are implemented as part of the PoC. Communication between the MQTT client and broker leverages SSL/TLS (Secure Sockets Layer/Transport Layer Security) encryption via HTTPS using an inhouse certification authority. Bidirectional communication with the cloud uses the WSS protocol, also implementing TLS. To protect the data housed within the cloud, the cloud platform for rapid development and deployment of smart, connected devices can maintain ISO 27001 18, SSAE SOC 1 and SOC 2 19, FedRAMP® 20, and DFAR 252.204-7012 21certifications.

6 Acronyms and Abbreviations

Acknowledgements

By the ISPE Pharma 4.0™ Plug & Produce Working Group

This Concept Paper represents the outcome of work done by the members of Workstream 3 of the ISPE Pharma 4.0™ “Plug and Produce” Working Group as well as experiences and input from the individuals listed and does not reflect the views of any one individual or company.

Document Authors

Jasmin Basic ZETA GmbH Austria

Jahir Kololli Hamilton Bonaduz AG Switzerland

Joachim Komar ZETA GmbH Austria

Stefan Krauss Siemens AG Germany

Martin Mayer ZETA GmbH Austria

Lukas Schier Parametric Technology GmbH Germany

Josef Trapl Memo3 GmbH Switzerland

Rene Zoelfl Parametric Technology GmbH Germany

Tel: +1-813-960-2105, Fax: +1-813-264-2816

www.ISPE.org