1
|
Pinto J, Ramos JRC, Costa RS, Rossell S, Dumas P, Oliveira R. Hybrid deep modeling of a CHO-K1 fed-batch process: combining first-principles with deep neural networks. Front Bioeng Biotechnol 2023; 11:1237963. [PMID: 37744245 PMCID: PMC10515724 DOI: 10.3389/fbioe.2023.1237963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Accepted: 08/22/2023] [Indexed: 09/26/2023] Open
Abstract
Introduction: Hybrid modeling combining First-Principles with machine learning is becoming a pivotal methodology for Biopharma 4.0 enactment. Chinese Hamster Ovary (CHO) cells, being the workhorse for industrial glycoproteins production, have been the object of several hybrid modeling studies. Most previous studies pursued a shallow hybrid modeling approach based on three-layered Feedforward Neural Networks (FFNNs) combined with macroscopic material balance equations. Only recently, the hybrid modeling field is incorporating deep learning into its framework with significant gains in descriptive and predictive power. Methods: This study compares, for the first time, deep and shallow hybrid modeling in a CHO process development context. Data of 24 fed-batch cultivations of a CHO-K1 cell line expressing a target glycoprotein, comprising 30 measured state variables over time, were used to compare both methodologies. Hybrid models with varying FFNN depths (3-5 layers) were systematically compared using two training methodologies. The classical training is based on the Levenberg-Marquardt algorithm, indirect sensitivity equations and cross-validation. The deep learning is based on the Adaptive Moment Estimation Method (ADAM), stochastic regularization and semidirect sensitivity equations. Results and conclusion: The results point to a systematic generalization improvement of deep hybrid models over shallow hybrid models. Overall, the training and testing errors decreased by 14.0% and 23.6% respectively when applying the deep methodology. The Central Processing Unit (CPU) time for training the deep hybrid model increased by 31.6% mainly due to the higher FFNN complexity. The final deep hybrid model is shown to predict the dynamics of the 30 state variables within the error bounds in every test experiment. Notably, the deep hybrid model could predict the metabolic shifts in key metabolites (e.g., lactate, ammonium, glutamine and glutamate) in the test experiments. We expect deep hybrid modeling to accelerate the deployment of high-fidelity digital twins in the biopharma sector in the near future.
Collapse
Affiliation(s)
- José Pinto
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - João R. C. Ramos
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - Rafael S. Costa
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | | | | | - Rui Oliveira
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| |
Collapse
|
2
|
Wang X, Mohsin A, Sun Y, Li C, Zhuang Y, Wang G. From Spatial-Temporal Multiscale Modeling to Application: Bridging the Valley of Death in Industrial Biotechnology. Bioengineering (Basel) 2023; 10:744. [PMID: 37370675 DOI: 10.3390/bioengineering10060744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 06/13/2023] [Accepted: 06/19/2023] [Indexed: 06/29/2023] Open
Abstract
The Valley of Death confronts industrial biotechnology with a significant challenge to the commercialization of products. Fortunately, with the integration of computation, automation and artificial intelligence (AI) technology, the industrial biotechnology accelerates to cross the Valley of Death. The Fourth Industrial Revolution (Industry 4.0) has spurred advanced development of intelligent biomanufacturing, which has evolved the industrial structures in line with the worldwide trend. To achieve this, intelligent biomanufacturing can be structured into three main parts that comprise digitalization, modeling and intellectualization, with modeling forming a crucial link between the other two components. This paper provides an overview of mechanistic models, data-driven models and their applications in bioprocess development. We provide a detailed elaboration of the hybrid model and its applications in bioprocess engineering, including strain design, process control and optimization, as well as bioreactor scale-up. Finally, the challenges and opportunities of biomanufacturing towards Industry 4.0 are also discussed.
Collapse
Affiliation(s)
- Xueting Wang
- State Key Laboratory of Bioreactor Engineering, East China University of Science and Technology (ECUST), Shanghai 200237, China
| | - Ali Mohsin
- State Key Laboratory of Bioreactor Engineering, East China University of Science and Technology (ECUST), Shanghai 200237, China
| | - Yifei Sun
- Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology (ECUST), Shanghai 200237, China
| | - Chao Li
- State Key Laboratory of Bioreactor Engineering, East China University of Science and Technology (ECUST), Shanghai 200237, China
| | - Yingping Zhuang
- State Key Laboratory of Bioreactor Engineering, East China University of Science and Technology (ECUST), Shanghai 200237, China
| | - Guan Wang
- State Key Laboratory of Bioreactor Engineering, East China University of Science and Technology (ECUST), Shanghai 200237, China
| |
Collapse
|
3
|
Helleckes LM, Hemmerich J, Wiechert W, von Lieres E, Grünberger A. Machine learning in bioprocess development: from promise to practice. Trends Biotechnol 2023; 41:817-835. [PMID: 36456404 DOI: 10.1016/j.tibtech.2022.10.010] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 10/20/2022] [Accepted: 10/27/2022] [Indexed: 11/30/2022]
Abstract
Fostered by novel analytical techniques, digitalization, and automation, modern bioprocess development provides large amounts of heterogeneous experimental data, containing valuable process information. In this context, data-driven methods like machine learning (ML) approaches have great potential to rationally explore large design spaces while exploiting experimental facilities most efficiently. Herein we demonstrate how ML methods have been applied so far in bioprocess development, especially in strain engineering and selection, bioprocess optimization, scale-up, monitoring, and control of bioprocesses. For each topic, we will highlight successful application cases, current challenges, and point out domains that can potentially benefit from technology transfer and further progress in the field of ML.
Collapse
Affiliation(s)
- Laura M Helleckes
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Johannes Hemmerich
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany
| | - Wolfgang Wiechert
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Eric von Lieres
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Alexander Grünberger
- Multiscale Bioengineering, Technical Faculty, Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany; Center for Biotechnology (CeBiTec), Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany; Institute of Process Engineering in Life Sciences, Section III: Microsystems in Bioprocess Engineering, Karlsruhe Institute of Technology, Fritz-Haber-Weg 2, 76131, Karlsruhe, Germany.
| |
Collapse
|
4
|
Nold V, Junghans L, Bayer B, Bisgen L, Duerkop M, Drerup R, Presser B, Schwab T, Bluhmki E, Wieschalka S, Knapp B. Boost dynamic protocols for producing mammalian biopharmaceuticals with intensified DoE—a practical guide to analyses with OLS and hybrid modeling. FRONTIERS IN CHEMICAL ENGINEERING 2023. [DOI: 10.3389/fceng.2022.1044245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
Introduction: For the implementation of robust bioprocesses, understanding of temporal cell behavior with respect to relevant inputs is crucial. Intensified Design of Experiments (iDoE) is an efficient tool to assess the joint influence of input parameters by including intra-experimental changes.Methods: We applied iDoE to the production phase of a monoclonal antibody in a mammalian bioprocess. The multidimensional design space spanned by temperature, dissolved oxygen (DO), timing of change, and growth category was investigated in 12 cultivations. We built ordinary least squares (OLS) and hybrid models (HM) on the iDoE-data, validated them with classical DoE (cDoE)-derived data, and used the models as in silico representation for process optimization.Results: If the complexity of interactions between changing setpoints of inputs is sufficiently captured during planning and modeling, iDoE proved to be valid for characterizing the mammalian biopharmaceutical production phase. For local behavior and flexible composition of optimization goals, OLS regressions can easily be implemented. To predict global and interconnected dynamics while incorporating mass balances, HM holds potential.Discussion: iDoE will boost protocols that optimize inputs for different bioprocess phases. The described key aspects of OLS- and HM-based analyses of iDoE-data shall guide future applications during manufacturing.
Collapse
|
5
|
Bayer B, Duerkop M, Pörtner R, Möller J. Comparison of mechanistic and hybrid modeling approaches for characterization of a CHO cultivation process: Requirements, pitfalls and solution paths. Biotechnol J 2023; 18:e2200381. [PMID: 36382343 DOI: 10.1002/biot.202200381] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 11/07/2022] [Accepted: 11/08/2022] [Indexed: 11/17/2022]
Abstract
Despite the advantages of mathematical bioprocess modeling, successful model implementation already starts with experimental planning and accordingly can fail at this early stage. For this study, two different modeling approaches (mechanistic and hybrid) based on a four-dimensional antibody-producing CHO fed-batch process are compared. Overall, 33 experiments are performed in the fractional factorial four-dimensional design space and separated into four different complex data partitions subsequently used for model comparison and evaluation. The mechanistic model demonstrates the advantage of prior knowledge (i.e., known equations) to get informative value relatively independently of the utilized data partition. The hybrid approach displayes a higher data dependency but simultaneously yielded a higher accuracy on all data partitions. Furthermore, our results demonstrate that independent of the chosen modeling framework, a smart selection of only four initial experiments can already yield a very good representation of a full design space independent of the chosen modeling structure. Academic and industry researchers are recommended to pay more attention to experimental planning to maximize the process understanding obtained from mathematical modeling.
Collapse
Affiliation(s)
| | | | - Ralf Pörtner
- Institute of Bioprocess and Biosystems Engineering, Hamburg University of Technology, Hamburg, Germany
| | - Johannes Möller
- Institute of Bioprocess and Biosystems Engineering, Hamburg University of Technology, Hamburg, Germany
| |
Collapse
|
6
|
Pappenreiter M, Lhota G, Vorauer‐Uhl K, Sissolak B. Antibody glycation during a
CHO
fed‐batch process following a constrained second order reaction. Biotechnol Prog 2022; 38:e3261. [DOI: 10.1002/btpr.3261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Revised: 03/28/2022] [Accepted: 04/13/2022] [Indexed: 11/08/2022]
Affiliation(s)
- Magdalena Pappenreiter
- Innovation Management Bilfinger Life Science GmbH Salzburg Austria
- Institute of Bioprocess Science and Engineering, Department of Biotechnology University of Natural Resources and Life Sciences Vienna Austria
| | - Gabriele Lhota
- Institute of Bioprocess Science and Engineering, Department of Biotechnology University of Natural Resources and Life Sciences Vienna Austria
| | - Karola Vorauer‐Uhl
- Institute of Bioprocess Science and Engineering, Department of Biotechnology University of Natural Resources and Life Sciences Vienna Austria
| | | |
Collapse
|