1
|
Ramos JRC, Pinto J, Poiares-Oliveira G, Peeters L, Dumas P, Oliveira R. Deep hybrid modeling of a HEK293 process: Combining long short-term memory networks with first principles equations. Biotechnol Bioeng 2024; 121:1554-1568. [PMID: 38343176 DOI: 10.1002/bit.28668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 12/22/2023] [Accepted: 01/22/2024] [Indexed: 04/14/2024]
Abstract
The combination of physical equations with deep learning is becoming a promising methodology for bioprocess digitalization. In this paper, we investigate for the first time the combination of long short-term memory (LSTM) networks with first principles equations in a hybrid workflow to describe human embryonic kidney 293 (HEK293) culture dynamics. Experimental data of 27 extracellular state variables in 20 fed-batch HEK293 cultures were collected in a parallel high throughput 250 mL cultivation system in an industrial process development setting. The adaptive moment estimation method with stochastic regularization and cross-validation were employed for deep learning. A total of 784 hybrid models with varying deep neural network architectures, depths, layers sizes and node activation functions were compared. In most scenarios, hybrid LSTM models outperformed classical hybrid Feedforward Neural Network (FFNN) models in terms of training and testing error. Hybrid LSTM models revealed to be less sensitive to data resampling than FFNN hybrid models. As disadvantages, Hybrid LSTM models are in general more complex (higher number of parameters) and have a higher computation cost than FFNN hybrid models. The hybrid model with the highest prediction accuracy consisted in a LSTM network with seven internal states connected in series with dynamic material balance equations. This hybrid model correctly predicted the dynamics of the 27 state variables (R2 = 0.93 in the test data set), including biomass, key substrates, amino acids and metabolic by-products for around 10 cultivation days.
Collapse
Affiliation(s)
- João R C Ramos
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - José Pinto
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - Gil Poiares-Oliveira
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | | | | | - Rui Oliveira
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| |
Collapse
|
2
|
Kuchemüller KB, Pörtner R, Möller J. Implementation of mDoE-methods to a microcarrier-based expansion processes for mesenchymal stem cells. Biotechnol Prog 2024; 40:e3429. [PMID: 38334218 DOI: 10.1002/btpr.3429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 01/01/2024] [Accepted: 01/03/2024] [Indexed: 02/10/2024]
Abstract
The need for advanced therapy medicinal products (ATMPs) has gained increased attention in recent years. In this respect, a well-designed cell expansion process is needed to efficiently manufacture the required number of cells with the desired product quality. This step is challenging due to the biological complexity of the respective primary cell (e.g., mesenchymal stem cells (MSC)) and the usage of microcarrier-based expansion systems. One accelerating approach for process design is model-assisted Design of Experiments (mDoE) combining mathematical process models and statistical tools. In this study, the mDoE workflow was used for the development of an expansion processes with human immortalized mesenchymal stem cells (hMSC-TERT) and the aim of maximizing cell yield assuming only a limited amount of prior knowledge at a very early stage of development. First, suitable microcarriers for expansion in shake flasks were screened and the differentiation of the cells was proven. Second, initial experiments were performed to generate prior knowledge, which was then used to set up the mathematical model and to estimate the model parameters. Finally, the mDoE was used to determine and evaluate the design space to be performed experimentally. Overall, a cell expansion process using microcarriers in a shake flask culture was successfully implemented and a significant increase in cell yield (up to 6,2-fold) was achieved compared to literature.
Collapse
Affiliation(s)
- Kim B Kuchemüller
- Institute of Bioprocess and Biosystems Engineering, Hamburg University of Technology, Hamburg, Germany
| | - Ralf Pörtner
- Institute of Bioprocess and Biosystems Engineering, Hamburg University of Technology, Hamburg, Germany
| | - Johannes Möller
- Institute of Bioprocess and Biosystems Engineering, Hamburg University of Technology, Hamburg, Germany
| |
Collapse
|
3
|
Ebrahimian A, Schalk M, Dürkop M, Maurer M, Bliem R, Kühnel H. Seed Train Optimization in Microcarrier-Based Cell Culture Post In Situ Cell Detachment through Scale-Down Hybrid Modeling. Bioengineering (Basel) 2024; 11:268. [PMID: 38534542 DOI: 10.3390/bioengineering11030268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2024] [Revised: 02/27/2024] [Accepted: 03/05/2024] [Indexed: 03/28/2024] Open
Abstract
Microcarrier-based cell culture is a commonly used method to facilitate the growth of anchorage-dependent cells like MA 104 for antigen manufacturing. However, conventionally, static cell culture is employed for cell propagation before seeding the production bioreactor with microcarriers (MCs). This study demonstrates the effective replacement of the conventional method by serial subculturing on MCs with in situ cell detachment under optimal conditions in closed culture units. This study proves that MA 104 can be subcultured at least five times on Cytodex 1 MC without the need for separating cells and MC after cell harvest. Process parameters impacting cell growth were studied post in situ cell detachment in a scaled-down model. Optimization, using augmented Design of Experiments (DoE) combined with hybrid modeling, facilitated rapid screening of the design space for critical process parameters (CPPs). Optimized conditions included an inoculation density of >16 cells/bead, 3.5-4.5 g/L of Cytodex 1, and a controlled agitation speed, starting at Njs (minimum agitation speed) for the first day with a maximum increase of 25% thereafter. With these design spaces for CPPs, a cell density of 2.6 ± 0.5 × 106 cells/mL was achieved after five days. This refined bioprocess methodology offers a reliable and efficient approach for seed training in stirred tank reactors, which is particularly beneficial for viral vaccine production.
Collapse
Affiliation(s)
- Atefeh Ebrahimian
- ACIB-Austrian Centre of Industrial Biotechnology, Krenngasse 37, 8010 Graz, Austria
- Department of Applied Life Science, Bioengineering, FH-Campus Wien, 1100 Vienna, Austria
- Department of Biotechnology, University of Natural Resources and Life Sciences, 1190 Vienna, Austria
| | - Mona Schalk
- ACIB-Austrian Centre of Industrial Biotechnology, Krenngasse 37, 8010 Graz, Austria
- Department of Applied Life Science, Bioengineering, FH-Campus Wien, 1100 Vienna, Austria
| | | | - Michael Maurer
- Department of Applied Life Science, Bioengineering, FH-Campus Wien, 1100 Vienna, Austria
| | - Rudolf Bliem
- Department of Applied Life Science, Bioengineering, FH-Campus Wien, 1100 Vienna, Austria
- Department of Biotechnology, University of Natural Resources and Life Sciences, 1190 Vienna, Austria
| | - Harald Kühnel
- Department of Applied Life Science, Bioengineering, FH-Campus Wien, 1100 Vienna, Austria
| |
Collapse
|
4
|
Reddy JV, Raudenbush K, Papoutsakis ET, Ierapetritou M. Cell-culture process optimization via model-based predictions of metabolism and protein glycosylation. Biotechnol Adv 2023; 67:108179. [PMID: 37257729 DOI: 10.1016/j.biotechadv.2023.108179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 05/18/2023] [Accepted: 05/21/2023] [Indexed: 06/02/2023]
Abstract
In order to meet the rising demand for biologics and become competitive on the developing biosimilar market, there is a need for process intensification of biomanufacturing processes. Process development of biologics has historically relied on extensive experimentation to develop and optimize biopharmaceutical manufacturing. Experimentation to optimize media formulations, feeding schedules, bioreactor operations and bioreactor scale up is expensive, labor intensive and time consuming. Mathematical modeling frameworks have the potential to enable process intensification while reducing the experimental burden. This review focuses on mathematical modeling of cellular metabolism and N-linked glycosylation as applied to upstream manufacturing of biologics. We review developments in the field of modeling cellular metabolism of mammalian cells using kinetic and stoichiometric modeling frameworks along with their applications to simulate, optimize and improve mechanistic understanding of the process. Interest in modeling N-linked glycosylation has led to the creation of various types of parametric and non-parametric models. Most published studies on mammalian cell metabolism have performed experiments in shake flasks where the pH and dissolved oxygen cannot be controlled. Efforts to understand and model the effect of bioreactor-specific parameters such as pH, dissolved oxygen, temperature, and bioreactor heterogeneity are critically reviewed. Most modeling efforts have focused on the Chinese Hamster Ovary (CHO) cells, which are most commonly used to produce monoclonal antibodies (mAbs). However, these modeling approaches can be generalized and applied to any mammalian cell-based manufacturing platform. Current and potential future applications of these models for Vero cell-based vaccine manufacturing, CAR-T cell therapies, and viral vector manufacturing are also discussed. We offer specific recommendations for improving the applicability of these models to industrially relevant processes.
Collapse
Affiliation(s)
- Jayanth Venkatarama Reddy
- Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, DE 19716-3196, USA
| | - Katherine Raudenbush
- Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, DE 19716-3196, USA
| | - Eleftherios Terry Papoutsakis
- Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, DE 19716-3196, USA; Delaware Biotechnology Institute, Department of Biological Sciences, University of Delaware, USA.
| | - Marianthi Ierapetritou
- Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, DE 19716-3196, USA.
| |
Collapse
|
5
|
Pinto J, Ramos JRC, Costa RS, Rossell S, Dumas P, Oliveira R. Hybrid deep modeling of a CHO-K1 fed-batch process: combining first-principles with deep neural networks. Front Bioeng Biotechnol 2023; 11:1237963. [PMID: 37744245 PMCID: PMC10515724 DOI: 10.3389/fbioe.2023.1237963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Accepted: 08/22/2023] [Indexed: 09/26/2023] Open
Abstract
Introduction: Hybrid modeling combining First-Principles with machine learning is becoming a pivotal methodology for Biopharma 4.0 enactment. Chinese Hamster Ovary (CHO) cells, being the workhorse for industrial glycoproteins production, have been the object of several hybrid modeling studies. Most previous studies pursued a shallow hybrid modeling approach based on three-layered Feedforward Neural Networks (FFNNs) combined with macroscopic material balance equations. Only recently, the hybrid modeling field is incorporating deep learning into its framework with significant gains in descriptive and predictive power. Methods: This study compares, for the first time, deep and shallow hybrid modeling in a CHO process development context. Data of 24 fed-batch cultivations of a CHO-K1 cell line expressing a target glycoprotein, comprising 30 measured state variables over time, were used to compare both methodologies. Hybrid models with varying FFNN depths (3-5 layers) were systematically compared using two training methodologies. The classical training is based on the Levenberg-Marquardt algorithm, indirect sensitivity equations and cross-validation. The deep learning is based on the Adaptive Moment Estimation Method (ADAM), stochastic regularization and semidirect sensitivity equations. Results and conclusion: The results point to a systematic generalization improvement of deep hybrid models over shallow hybrid models. Overall, the training and testing errors decreased by 14.0% and 23.6% respectively when applying the deep methodology. The Central Processing Unit (CPU) time for training the deep hybrid model increased by 31.6% mainly due to the higher FFNN complexity. The final deep hybrid model is shown to predict the dynamics of the 30 state variables within the error bounds in every test experiment. Notably, the deep hybrid model could predict the metabolic shifts in key metabolites (e.g., lactate, ammonium, glutamine and glutamate) in the test experiments. We expect deep hybrid modeling to accelerate the deployment of high-fidelity digital twins in the biopharma sector in the near future.
Collapse
Affiliation(s)
- José Pinto
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - João R. C. Ramos
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | - Rafael S. Costa
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| | | | | | - Rui Oliveira
- LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal
| |
Collapse
|
6
|
Mahanty B. Hybrid modeling in bioprocess dynamics: Structural variabilities, implementation strategies, and practical challenges. Biotechnol Bioeng 2023; 120:2072-2091. [PMID: 37458311 DOI: 10.1002/bit.28503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 07/09/2023] [Accepted: 07/10/2023] [Indexed: 07/30/2023]
Abstract
Hybrid modeling, with an appropriate blend of the mechanistic and data-driven framework, is increasingly being adopted in bioprocess modeling, model-based experimental design (digital-twin), identification of critical process parameters, and optimization. However, the development of a hybrid model from experimental data is an inherently complex workflow, involving designed experiments, selection of the data-driven process, identification of model parameters, assessment fitness, and generalization capability. Depending on the complexity of the process system and purpose, each piece of these modules can flexibly be incorporated into the puzzle. However, this extra flexibility can be a cause of concern to trace an "optimal" model structure. In this paper, the development of hybrid models in a common bioprocess system, selection of data-driven components and their mapping to states, choice of parameter identification techniques, and model quality assurance are revisited. The challenges associated with hybrid-model development, and corrective actions have also been reviewed. The review also suggests the lack of data, and code sharing in communal repositories can be a hurdle in the exploration, and expansion of those tools in a bioprocess system.
Collapse
Affiliation(s)
- Biswanath Mahanty
- Department of Biotechnology, Krunya Institute of Technology and Sciences, Coimbatore, Tamil Nadu, India
| |
Collapse
|