1
|
Saha R, Chauhan A, Rastogi Verma S. Machine learning: an advancement in biochemical engineering. Biotechnol Lett 2024; 46:497-519. [PMID: 38902585 DOI: 10.1007/s10529-024-03499-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 02/24/2024] [Accepted: 05/18/2024] [Indexed: 06/22/2024]
Abstract
One of the most remarkable techniques recently introduced into the field of bioprocess engineering is machine learning. Bioprocess engineering has drawn much attention due to its vast application in different domains like biopharmaceuticals, fossil fuel alternatives, environmental remediation, and food and beverage industry, etc. However, due to their unpredictable mechanisms, they are very often challenging to optimize. Furthermore, biological systems are extremely complicated; hence, machine learning algorithms could potentially be utilized to improve and build new biotechnological processes. Gaining insight into the fundamental mathematical understanding of commonly used machine learning algorithms, including Support Vector Machine, Principal Component Analysis, Partial Least Squares and Reinforcement Learning, the present study aims to discuss various case studies related to the application of machine learning in bioprocess engineering. Recent advancements as well as challenges posed in this area along with their potential solutions are also presented.
Collapse
Affiliation(s)
- Ritika Saha
- Department of Biotechnology, Delhi Technological University, New Delhi, 110042, India
| | - Ashutosh Chauhan
- Department of Biotechnology, Delhi Technological University, New Delhi, 110042, India
| | - Smita Rastogi Verma
- Department of Biotechnology, Delhi Technological University, New Delhi, 110042, India.
| |
Collapse
|
2
|
He Z, Lu Z, Wang X, Xiong Q, Tran KP, Thomassey S, Zeng X, Hong M, Man Y. Multiobjective Optimization of Papermaking Wastewater Treatment Processes under Economic, Energy, and Environmental Goals. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2024. [PMID: 39038180 DOI: 10.1021/acs.est.4c03460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/24/2024]
Abstract
Due to the heterogeneity of recycled paper materials and the production conditions, pollutants in papermaking wastewater fluctuate sharply over time. Quality control of the papermaking wastewater treatment process (PWTP) is challenging and costly. As regulations are also growing about the environmental effects of the PWTP on greenhouse gas (GHG) emission, energy consumption, etc., the PWTP formulates a complex multiobjective optimization problem. This research established a multiagent deep reinforcement learning framework to simultaneously optimize process cost, energy consumption, and GHG emission in the PWTP, subjected to the effluent quality, to realize economic, energy, and environmental (3E) goals. The biological treatment process of wastewater in paper mills was simulated using benchmark simulation model no. 1 (BSM1). The data generated based on the BSM manual was utilized for model training, and real data acquired from a local papermaking factory was used to estimate the model performance. The results show that the proposed method outperforms conventional techniques in identifying the best control strategies for multiple targets.
Collapse
Affiliation(s)
- Zhenglei He
- State Key Laboratory of Pulp and Paper Engineering, South China University of Technology, Guangzhou 510640, China
| | - Zaohao Lu
- State Key Laboratory of Pulp and Paper Engineering, South China University of Technology, Guangzhou 510640, China
| | - Xu Wang
- Institute of Energy Conservation and Environmental Protection, China Center for Information Industry Development, Beijing 100846, China
| | - Qingang Xiong
- State Key Laboratory of Pulp and Paper Engineering, South China University of Technology, Guangzhou 510640, China
| | - Kim Phuc Tran
- Univ. Lille, ENSAIT, GEMTEX-Laboratoire de Génie et Matériaux Textiles, Lille F-59000, France
- International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang 50200, Vietnam
| | - Sébastien Thomassey
- Univ. Lille, ENSAIT, GEMTEX-Laboratoire de Génie et Matériaux Textiles, Lille F-59000, France
| | - Xianyi Zeng
- Univ. Lille, ENSAIT, GEMTEX-Laboratoire de Génie et Matériaux Textiles, Lille F-59000, France
| | - Mengna Hong
- State Key Laboratory of Pulp and Paper Engineering, South China University of Technology, Guangzhou 510640, China
- China-Singapore International Joint Research Institute, Guangzhou 510700, China
| | - Yi Man
- State Key Laboratory of Pulp and Paper Engineering, South China University of Technology, Guangzhou 510640, China
- Pazhou Lab, Guangdong Artificial Intelligence and Digital Economy Laboratory, Guangzhou 510335, China
| |
Collapse
|
3
|
Helleckes LM, Hemmerich J, Wiechert W, von Lieres E, Grünberger A. Machine learning in bioprocess development: from promise to practice. Trends Biotechnol 2023; 41:817-835. [PMID: 36456404 DOI: 10.1016/j.tibtech.2022.10.010] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 10/20/2022] [Accepted: 10/27/2022] [Indexed: 11/30/2022]
Abstract
Fostered by novel analytical techniques, digitalization, and automation, modern bioprocess development provides large amounts of heterogeneous experimental data, containing valuable process information. In this context, data-driven methods like machine learning (ML) approaches have great potential to rationally explore large design spaces while exploiting experimental facilities most efficiently. Herein we demonstrate how ML methods have been applied so far in bioprocess development, especially in strain engineering and selection, bioprocess optimization, scale-up, monitoring, and control of bioprocesses. For each topic, we will highlight successful application cases, current challenges, and point out domains that can potentially benefit from technology transfer and further progress in the field of ML.
Collapse
Affiliation(s)
- Laura M Helleckes
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Johannes Hemmerich
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany
| | - Wolfgang Wiechert
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Eric von Lieres
- Institute for Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH, 52428 Jülich, Germany; RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany
| | - Alexander Grünberger
- Multiscale Bioengineering, Technical Faculty, Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany; Center for Biotechnology (CeBiTec), Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany; Institute of Process Engineering in Life Sciences, Section III: Microsystems in Bioprocess Engineering, Karlsruhe Institute of Technology, Fritz-Haber-Weg 2, 76131, Karlsruhe, Germany.
| |
Collapse
|
4
|
Liu J, Tsai BY, Chen DS. Deep reinforcement learning based controller with dynamic feature extraction for an industrial claus process. J Taiwan Inst Chem Eng 2023. [DOI: 10.1016/j.jtice.2023.104779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2023]
|
5
|
Nikita S, Mishra S, Gupta K, Runkana V, Gomes J, Rathore AS. Advances in bioreactor control for production of biotherapeutic products. Biotechnol Bioeng 2023; 120:1189-1214. [PMID: 36760086 DOI: 10.1002/bit.28346] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 02/08/2023] [Accepted: 02/08/2023] [Indexed: 02/11/2023]
Abstract
Advanced control strategies are well established in chemical, pharmaceutical, and food processing industries. Over the past decade, the application of these strategies is being explored for control of bioreactors for manufacturing of biotherapeutics. Most of the industrial bioreactor control strategies apply classical control techniques, with the control system designed for the facility at hand. However, with the recent progress in sensors, machinery, and industrial internet of things, and advancements in deeper understanding of the biological processes, coupled with the requirement of flexible production, the need to develop a robust and advanced process control system that can ease process intensification has emerged. This has further fuelled the development of advanced monitoring approaches, modeling techniques, process analytical technologies, and soft sensors. It is seen that proper application of these concepts can significantly improve bioreactor process performance, productivity, and reproducibility. This review is on the recent advancements in bioreactor control and its related aspects along with the associated challenges. This study also offers an insight into the future prospects for development of control strategies that can be designed for industrial-scale production of biotherapeutic products.
Collapse
Affiliation(s)
- Saxena Nikita
- Department of Chemical Engineering, DBT Centre of Excellence for Biopharmaceutical Technology, Indian Institute of Technology, Hauz Khas, Delhi, India
| | - Somesh Mishra
- Department of Chemical Engineering, DBT Centre of Excellence for Biopharmaceutical Technology, Indian Institute of Technology, Hauz Khas, Delhi, India
| | - Keshari Gupta
- TCS Research, Tata Consultancy Services Limited, Pune, India
| | | | - James Gomes
- Kusuma School of Biological Sciences, Indian Institute of Technology, Hauz Khas, Delhi, India
| | - Anurag S Rathore
- Department of Chemical Engineering, DBT Centre of Excellence for Biopharmaceutical Technology, Indian Institute of Technology, Hauz Khas, Delhi, India
| |
Collapse
|
6
|
Lim SJ, Son M, Ki SJ, Suh SI, Chung J. Opportunities and challenges of machine learning in bioprocesses: Categorization from different perspectives and future direction. BIORESOURCE TECHNOLOGY 2023; 370:128518. [PMID: 36565818 DOI: 10.1016/j.biortech.2022.128518] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 12/15/2022] [Accepted: 12/17/2022] [Indexed: 06/17/2023]
Abstract
Recent advances in machine learning (ML) have revolutionized an extensive range of research and industry fields by successfully addressing intricate problems that cannot be resolved with conventional approaches. However, low interpretability and incompatibility make it challenging to apply ML to complicated bioprocesses, which rely on the delicate metabolic interplay among living cells. This overview attempts to delineate ML applications to bioprocess from different perspectives, and their inherent limitations (i.e., uncertainties in prediction) were then discussed with unique attempts to supplement the ML models. A clear classification can be made depending on the purpose of the ML (supervised vs unsupervised) per application, as well as on their system boundaries (engineered vs natural). Although a limited number of hybrid approaches with meaningful outcomes (e.g., improved accuracy) are available, there is still a need to further enhance the interpretability, compatibility, and user-friendliness of ML models.
Collapse
Affiliation(s)
- Seung Ji Lim
- Water Cycle Research Center, Korea Institute of Science and Technology, Seoul 02792, Republic of Korea
| | - Moon Son
- Water Cycle Research Center, Korea Institute of Science and Technology, Seoul 02792, Republic of Korea; Division of Energy and Environmental Technology, KIST School, Korea University of Science and Technology (UST), Seoul 02792, Republic of Korea
| | - Seo Jin Ki
- Department of Environmental Engineering, Gyeongsang National University, Jinju 52725, Republic of Korea
| | - Sang-Ik Suh
- Department of Energy System Engineering, Gyeongsang National University, Jinju 52725, Republic of Korea
| | - Jaeshik Chung
- Water Cycle Research Center, Korea Institute of Science and Technology, Seoul 02792, Republic of Korea; Division of Energy and Environmental Technology, KIST School, Korea University of Science and Technology (UST), Seoul 02792, Republic of Korea.
| |
Collapse
|
7
|
Pandey AK, Park J, Ko J, Joo HH, Raj T, Singh LK, Singh N, Kim SH. Machine learning in fermentative biohydrogen production: Advantages, challenges, and applications. BIORESOURCE TECHNOLOGY 2023; 370:128502. [PMID: 36535617 DOI: 10.1016/j.biortech.2022.128502] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 12/11/2022] [Accepted: 12/14/2022] [Indexed: 06/17/2023]
Abstract
Hydrogen can be produced in an environmentally friendly manner through biological processes using a variety of organic waste and biomass as feedstock. However, the complexity of biological processes limits their predictability and reliability, which hinders the scale-up and dissemination. This article reviews contemporary research and perspectives on the application of machine learning in biohydrogen production technology. Several machine learning algorithems have recently been implemented for modeling the nonlinear and complex relationships among operational and performance parameters in biohydrogen production as well as predicting the process performance and microbial population dynamics. Reinforced machine learning methods exhibited precise state prediction and retrieved the underlying kinetics effectively. Machine-learning based prediction was also improved by using microbial sequencing data as input parameters. Further research on machine learning could be instrumental in designing a process control tool to maintain reliable hydrogen production performance and identify connection between the process performance and the microbial population.
Collapse
Affiliation(s)
- Ashutosh Kumar Pandey
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Jungsu Park
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Jeun Ko
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Hwan-Hong Joo
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Tirath Raj
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Lalit Kumar Singh
- Department of Biochemical Engineering, Harcourt Butler Technical University, Kanpur 208002, Uttar Pradesh (UP), India
| | - Noopur Singh
- Dr. A. P. J. Abdul Kalam Technical University, Lucknow, Uttar Pradesh (UP), India
| | - Sang-Hyoun Kim
- Department of Civil and Environmental Engineering, Yonsei University, Seoul 03722, Republic of Korea.
| |
Collapse
|
8
|
Kim JW, Krausch N, Aizpuru J, Barz T, Lucia S, Neubauer P, Cruz Bournazou MN. Model predictive control and moving horizon estimation for adaptive optimal bolus feeding in high-throughput cultivation of E. coli. Comput Chem Eng 2023. [DOI: 10.1016/j.compchemeng.2023.108158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
|
9
|
Lin R, Chen J, Xie L, Su H. Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control. Neural Netw 2023; 158:197-215. [PMID: 36462366 DOI: 10.1016/j.neunet.2022.10.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2021] [Revised: 07/21/2022] [Accepted: 10/13/2022] [Indexed: 11/06/2022]
Abstract
In the context of intelligent manufacturing in the process industry, traditional model-based optimization control methods cannot adapt to the situation of drastic changes in working conditions or operating modes. Reinforcement learning (RL) directly achieves the control objective by interacting with the environment, and has significant advantages in the presence of uncertainty since it does not require an explicit model of the operating plant. However, most RL algorithms fail to retain transfer learning capabilities in the presence of mode variation, which becomes a practical obstacle to industrial process control applications. To address these issues, we design a framework that uses local data augmentation to improve the training efficiency and transfer learning (adaptability) performance. Therefore, this paper proposes a novel RL control algorithm, CBR-MA-DDPG, organically integrating case-based reasoning (CBR), model-assisted (MA) experience augmentation, and deep deterministic policy gradient (DDPG). When the operating mode changes, CBR-MA-DDPG can quickly adapt to the varying environment and achieve the desired control performance within several training episodes. Experimental analyses on a continuous stirred tank reactor (CSTR) and an organic Rankine cycle (ORC) demonstrate the superiority of the proposed method in terms of both adaptability and control performance/robustness. The results show that the control performance of the CBR-MA-DDPG agent outperforms the conventional PI and MPC control schemes, and that it has higher training efficiency than the state-of-the-art DDPG, TD3, and PPO algorithms in transfer learning scenarios with mode shift situations.
Collapse
Affiliation(s)
- Runze Lin
- State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China.
| | - Junghui Chen
- Department of Chemical Engineering, Chung-Yuan Christian University, Taoyuan 32023, Taiwan, ROC.
| | - Lei Xie
- State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China.
| | - Hongye Su
- State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China.
| |
Collapse
|
10
|
Duong-Trung N, Born S, Kim JW, Schermeyer MT, Paulick K, Borisyak M, Cruz-Bournazou MN, Werner T, Scholz R, Schmidt-Thieme L, Neubauer P, Martinez E. When Bioprocess Engineering Meets Machine Learning: A Survey from the Perspective of Automated Bioprocess Development. Biochem Eng J 2022. [DOI: 10.1016/j.bej.2022.108764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
11
|
Modeling of Bioprocesses via MINLP-based Symbolic Regression of S-system Formalisms. Comput Chem Eng 2022. [DOI: 10.1016/j.compchemeng.2022.108108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
12
|
Deep reinforcement learning for automated search of model parameters: photo-fenton wastewater disinfection case study. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-07803-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
AbstractNumerical optimization solves problems that are analytically intractable at the cost of arriving at a sufficiently good but rarely optimal solution. To maximize the result, optimization algorithms are run with the guidance and supervision of a human, usually an expert in the problem. Recent advances in deep reinforcement learning motivate interest in an artificial agent capable of learning to do the expert’s task. Specifically, we present a proximal policy optimization agent that learns to optimize in a real case study such as the modeling of the photo-fenton disinfection process, which involves a number of parameters that have to be adjusted to minimize the error of the model with respect to the experimental data collected in several trials. The expert spends an average of 4 h to find a suitable set of parameters. On the other hand, the agent we present does not require a human expert to guide or validate the optimization procedure and achieves similar results in $$2.5\times$$
2.5
×
less time.
Collapse
|
13
|
Kim JW, Oh TH, Son SH, Lee JM. Primal–dual differential dynamic programming: A model-based reinforcement learning for constrained dynamic optimization. Comput Chem Eng 2022. [DOI: 10.1016/j.compchemeng.2022.108004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
14
|
Dutta D, Upreti SR. A survey and comparative evaluation of actor‐critic methods in process control. CAN J CHEM ENG 2022. [DOI: 10.1002/cjce.24508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- Debaprasad Dutta
- Department of Chemical Engineering Toronto Metropolitan University Toronto Ontario Canada
| | - Simant R. Upreti
- Department of Chemical Engineering Toronto Metropolitan University Toronto Ontario Canada
| |
Collapse
|
15
|
Sabzevari M, Szedmak S, Penttilä M, Jouhten P, Rousu J. Strain design optimization using reinforcement learning. PLoS Comput Biol 2022; 18:e1010177. [PMID: 35658018 PMCID: PMC9200333 DOI: 10.1371/journal.pcbi.1010177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 06/15/2022] [Accepted: 05/06/2022] [Indexed: 11/18/2022] Open
Abstract
Engineered microbial cells present a sustainable alternative to fossil-based synthesis of chemicals and fuels. Cellular synthesis routes are readily assembled and introduced into microbial strains using state-of-the-art synthetic biology tools. However, the optimization of the strains required to reach industrially feasible production levels is far less efficient. It typically relies on trial-and-error leading into high uncertainty in total duration and cost. New techniques that can cope with the complexity and limited mechanistic knowledge of the cellular regulation are called for guiding the strain optimization.
In this paper, we put forward a multi-agent reinforcement learning (MARL) approach that learns from experiments to tune the metabolic enzyme levels so that the production is improved. Our method is model-free and does not assume prior knowledge of the microbe’s metabolic network or its regulation. The multi-agent approach is well-suited to make use of parallel experiments such as multi-well plates commonly used for screening microbial strains.
We demonstrate the method’s capabilities using the genome-scale kinetic model of Escherichia coli, k-ecoli457, as a surrogate for an in vivo cell behaviour in cultivation experiments. We investigate the method’s performance relevant for practical applicability in strain engineering i.e. the speed of convergence towards the optimum response, noise tolerance, and the statistical stability of the solutions found. We further evaluate the proposed MARL approach in improving L-tryptophan production by yeast Saccharomyces cerevisiae, using publicly available experimental data on the performance of a combinatorial strain library.
Overall, our results show that multi-agent reinforcement learning is a promising approach for guiding the strain optimization beyond mechanistic knowledge, with the goal of faster and more reliably obtaining industrially attractive production levels.
Collapse
Affiliation(s)
- Maryam Sabzevari
- Department of Computer Science, Aalto University, Espoo, Finland
- * E-mail: ,
| | - Sandor Szedmak
- Department of Computer Science, Aalto University, Espoo, Finland
| | - Merja Penttilä
- VTT Technical Research Centre of Finland Ltd, Espoo, Finland
| | - Paula Jouhten
- VTT Technical Research Centre of Finland Ltd, Espoo, Finland
| | - Juho Rousu
- Department of Computer Science, Aalto University, Espoo, Finland
| |
Collapse
|
16
|
Rickert CA, Lieleg O. Machine learning approaches for biomolecular, biophysical, and biomaterials research. BIOPHYSICS REVIEWS 2022; 3:021306. [PMID: 38505413 PMCID: PMC10914139 DOI: 10.1063/5.0082179] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 05/12/2022] [Indexed: 03/21/2024]
Abstract
A fluent conversation with a virtual assistant, person-tailored news feeds, and deep-fake images created within seconds-all those things that have been unthinkable for a long time are now a part of our everyday lives. What these examples have in common is that they are realized by different means of machine learning (ML), a technology that has fundamentally changed many aspects of the modern world. The possibility to process enormous amount of data in multi-hierarchical, digital constructs has paved the way not only for creating intelligent systems but also for obtaining surprising new insight into many scientific problems. However, in the different areas of biosciences, which typically rely heavily on the collection of time-consuming experimental data, applying ML methods is a bit more challenging: Here, difficulties can arise from small datasets and the inherent, broad variability, and complexity associated with studying biological objects and phenomena. In this Review, we give an overview of commonly used ML algorithms (which are often referred to as "machines") and learning strategies as well as their applications in different bio-disciplines such as molecular biology, drug development, biophysics, and biomaterials science. We highlight how selected research questions from those fields were successfully translated into machine readable formats, discuss typical problems that can arise in this context, and provide an overview of how to resolve those encountered difficulties.
Collapse
|
17
|
|
18
|
Oh TH, Park HM, Kim JW, Lee JM. Integration of Reinforcement Learning and Model Predictive Control to Optimize Semi‐batch Bioreactor. AIChE J 2022. [DOI: 10.1002/aic.17658] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Affiliation(s)
- Tae Hoon Oh
- School of Chemical and Biological Engineering Institute of Chemical Processes, Seoul National University Seoul Republic of Korea
| | - Hyun Min Park
- School of Chemical and Biological Engineering Institute of Chemical Processes, Seoul National University Seoul Republic of Korea
| | - Jong Woo Kim
- Bioprocess Engineering Technische Universität Berlin Berlin Germany
| | - Jong Min Lee
- School of Chemical and Biological Engineering Institute of Chemical Processes, Seoul National University Seoul Republic of Korea
| |
Collapse
|
19
|
Guan Y, Chaffart D, Liu G, Tan Z, Zhang D, Wang Y, Li J, Ricardez-Sandoval L. Machine learning in solid heterogeneous catalysis: Recent developments, challenges and perspectives. Chem Eng Sci 2022. [DOI: 10.1016/j.ces.2021.117224] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
20
|
Reinforcement Learning for Online Adaptation of Model Predictive Controllers: Application to a Selective Catalytic Reduction Unit. Comput Chem Eng 2022. [DOI: 10.1016/j.compchemeng.2022.107727] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
21
|
Mowbray M, Petsagkourakis P, del Rio-Chanona E, Zhang D. Safe chance constrained reinforcement learning for batch process control. Comput Chem Eng 2022. [DOI: 10.1016/j.compchemeng.2021.107630] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
|
22
|
Mowbray M, Vallerio M, Perez-Galvan C, Zhang D, Del Rio Chanona A, Navarro-Brull FJ. Industrial data science – a review of machine learning applications for chemical and process industries. REACT CHEM ENG 2022. [DOI: 10.1039/d1re00541c] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Understand and optimize industrial processes via machine learning and chemical engineering principles.
Collapse
Affiliation(s)
- Max Mowbray
- The University of Manchester, Manchester, M13 9PL, UK
| | | | | | - Dongda Zhang
- The University of Manchester, Manchester, M13 9PL, UK
- Imperial College London, London, SW7 2AZ, UK
| | | | | |
Collapse
|
23
|
Machalek D, Quah T, Powell KM. A novel implicit hybrid machine learning model and its application for reinforcement learning. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107496] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
24
|
|
25
|
Joshi T, Makker S, Kodamana H, Kandath H. Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107527] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
26
|
Pan E, Petsagkourakis P, Mowbray M, Zhang D, Rio-Chanona EAD. Constrained model-free reinforcement learning for process optimization. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107462] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
27
|
Kim JW, Park BJ, Oh TH, Lee JM. Model-based reinforcement learning and predictive control for two-stage optimal control of fed-batch bioreactor. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107465] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
28
|
Schweidtmann AM, Esche E, Fischer A, Kloft M, Repke J, Sager S, Mitsos A. Machine Learning in Chemical Engineering: A Perspective. CHEM-ING-TECH 2021. [DOI: 10.1002/cite.202100083] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Affiliation(s)
- Artur M. Schweidtmann
- Delft University of Technology Department of Chemical Engineering Van der Maasweg 9 2629 HZ Delft The Netherlands
- RWTH Aachen University Aachener Verfahrenstechnik Forckenbeckstr. 51 52074 Aachen Germany
| | - Erik Esche
- Technische Universität Berlin Fachgebiet Dynamik und Betrieb technischer Anlagen Straße des 17. Juni 135 10623 Berlin Germany
| | - Asja Fischer
- Ruhr-Universität Bochum Department of Mathematics Universitätsstraße 150 44801 Bochum Germany
| | - Marius Kloft
- Technische Universität Kaiserslautern Department of Computer Science Erwin-Schrödinger-Straße 52 67663 Kaiserslautern Germany
| | - Jens‐Uwe Repke
- Technische Universität Berlin Fachgebiet Dynamik und Betrieb technischer Anlagen Straße des 17. Juni 135 10623 Berlin Germany
| | - Sebastian Sager
- Otto-von-Guericke-Universität Magdeburg Department of Mathematics Universitätsplatz 2 39106 Magdeburg Germany
| | - Alexander Mitsos
- RWTH Aachen University Aachener Verfahrenstechnik Forckenbeckstr. 51 52074 Aachen Germany
- JARA Center for Simulation and Data Science (CSD) Aachen Germany
- Forschungszentrum Jülich Institute for Energy and Climate Research IEK-10 Energy Systems Engineering Wilhelm-Johnen-Straße 52428 Jülich Germany
| |
Collapse
|
29
|
Zhou Y, Gao K, Li D, Xu Z, Gao F. Data-Efficient Constrained Learning for Optimal Tracking of Batch Processes. Ind Eng Chem Res 2021. [DOI: 10.1021/acs.iecr.1c02706] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Affiliation(s)
- Yuanqiang Zhou
- Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong999077, China
| | - Kaihua Gao
- Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong999077, China
| | - Dewei Li
- Department of Automation, Shanghai Jiao Tong University, Shanghai200240, China
| | - Zuhua Xu
- National Center for International Research on Quality-Targeted Process Optimization and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou310027, China
| | - Furong Gao
- Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong999077, China
- Guangzhou HKUST Fok Ying Tung Research Institute, Guangzhou511458, China
| |
Collapse
|
30
|
Sachio S, Mowbray M, Papathanasiou M, del Rio-Chanona EA, Petsagkourakis P. Integrating process design and control using reinforcement learning. Chem Eng Res Des 2021. [DOI: 10.1016/j.cherd.2021.10.032] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
31
|
Chen K, Wang H, Valverde-Pérez B, Zhai S, Vezzaro L, Wang A. Optimal control towards sustainable wastewater treatment plants based on multi-agent reinforcement learning. CHEMOSPHERE 2021; 279:130498. [PMID: 33892457 DOI: 10.1016/j.chemosphere.2021.130498] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Revised: 03/25/2021] [Accepted: 04/03/2021] [Indexed: 06/12/2023]
Abstract
Wastewater treatment plants (WWTPs) are designed to eliminate pollutants and alleviate environmental pollution resulting from human activities. However, the construction and operation of WWTPs consume resources, emit greenhouse gases (GHGs) and produce residual sludge, thus require further optimization. WWTPs are complex to control and optimize because of high non-linearity and variation. This study used a novel technique, multi-agent deep reinforcement learning (MADRL), to simultaneously optimize dissolved oxygen (DO) and chemical dosage in a WWTP. The reward function was specially designed from life cycle perspective to achieve sustainable optimization. Five scenarios were considered: baseline, three different effluent quality and cost-oriented scenarios. The result shows that optimization based on LCA has lower environmental impacts compared to baseline scenario, as cost, energy consumption and greenhouse gas emissions reduce to 0.890 CNY/m3-ww, 0.530 kWh/m3-ww, 2.491 kg CO2-eq/m3-ww respectively. The cost-oriented control strategy exhibits comparable overall performance to the LCA-driven strategy since it sacrifices environmental benefits but has lower cost as 0.873 CNY/m3-ww. It is worth mentioning that the retrofitting of WWTPs based on resources should be implemented with the consideration of impact transfer. Specifically, LCA-SW scenario decreases 10 kg PO4-eq in eutrophication potential compared to the baseline within 10 days, while significantly increases other indicators. The major contributors of each indicator are identified for future study and improvement. Last, the authors discussed that novel dynamic control strategies required advanced sensors or a large amount of data, so the selection of control strategies should also consider economic and ecological conditions. In a nutshell, there are still limitations of this work and future studies are required.
Collapse
Affiliation(s)
- Kehua Chen
- Key Lab of Environmental Biotechnology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, 18 Shuangqing Road, Haidian District, Beijing, 100085, China; Sino-Danish Center for Education and Research, University of Chinese Academy of Sciences, Beijing, China
| | - Hongcheng Wang
- Key Lab of Environmental Biotechnology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, 18 Shuangqing Road, Haidian District, Beijing, 100085, China; School of Civil & Environmental Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen, 518055, PR China; State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin, 150001, China
| | - Borja Valverde-Pérez
- DTU Environment, Technical University of Denmark, Bygningstorvet, Building 115, 2800, Kongens Lyngby, Denmark
| | - Siyuan Zhai
- Key Lab of Environmental Biotechnology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, 18 Shuangqing Road, Haidian District, Beijing, 100085, China
| | - Luca Vezzaro
- DTU Environment, Technical University of Denmark, Bygningstorvet, Building 115, 2800, Kongens Lyngby, Denmark.
| | - Aijie Wang
- Key Lab of Environmental Biotechnology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, 18 Shuangqing Road, Haidian District, Beijing, 100085, China; State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin, 150001, China.
| |
Collapse
|
32
|
Schwung D, Yuwono S, Schwung A, Ding SX. Decentralized learning of energy optimal production policies using PLC-informed reinforcement learning. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107382] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
33
|
Smiatek J, Clemens C, Herrera LM, Arnold S, Knapp B, Presser B, Jung A, Wucherpfennig T, Bluhmki E. Generic and specific recurrent neural network models: Applications for large and small scale biopharmaceutical upstream processes. BIOTECHNOLOGY REPORTS (AMSTERDAM, NETHERLANDS) 2021; 31:e00640. [PMID: 34159058 PMCID: PMC8193373 DOI: 10.1016/j.btre.2021.e00640] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Revised: 04/24/2021] [Accepted: 05/27/2021] [Indexed: 01/02/2023]
Abstract
The calculation of temporally varying upstream process outcomes is a challenging task. Over the last years, several parametric, semi-parametric as well as non-parametric approaches were developed to provide reliable estimates for key process parameters. We present generic and product-specific recurrent neural network (RNN) models for the computation and study of growth and metabolite-related upstream process parameters as well as their temporal evolution. Our approach can be used for the control and study of single product-specific large-scale manufacturing runs as well as generic small-scale evaluations for combined processes and products at development stage. The computational results for the product titer as well as various major upstream outcomes in addition to relevant process parameters show a high degree of accuracy when compared to experimental data and, accordingly, a reasonable predictive capability of the RNN models. The calculated values for the root-mean squared errors of prediction are significantly smaller than the experimental standard deviation for the considered process run ensembles, which highlights the broad applicability of our approach. As a specific benefit for platform processes, the generic RNN model is also used to simulate process outcomes for different temperatures in good agreement with experimental results. The high level of accuracy and the straightforward usage of the approach without sophisticated parameterization and recalibration procedures highlight the benefits of the RNN models, which can be regarded as promising alternatives to existing parametric and semi-parametric methods.
Collapse
Affiliation(s)
- Jens Smiatek
- Institute for Computational Physics, University of Stuttgart, D-70569 Stuttgart, Germany
- Boehringer Ingelheim Pharma GmbH & Co. KG, Digitalization Development Biologicals CMC, D-88397 Biberach (Riss), Germany
| | - Christoph Clemens
- Boehringer Ingelheim Pharma GmbH & Co. KG, Focused Factory Drug Substance, D-88397 Biberach (Riss), Germany
| | - Liliana Montano Herrera
- Boehringer Ingelheim Pharma GmbH & Co. KG, Bioprocess Development Biologicals, D-88397 Biberach (Riss), Germany
| | - Sabine Arnold
- Boehringer Ingelheim Pharma GmbH & Co. KG, Bioprocess Development Biologicals, D-88397 Biberach (Riss), Germany
| | - Bettina Knapp
- Boehringer Ingelheim Pharma GmbH & Co. KG, Analytical Development Biologicals, D-88397 Biberach (Riss), Germany
| | - Beate Presser
- Boehringer Ingelheim Pharma GmbH & Co. KG, Analytical Development Biologicals, D-88397 Biberach (Riss), Germany
| | - Alexander Jung
- Boehringer Ingelheim Pharma GmbH & Co. KG, Digitalization Development Biologicals CMC, D-88397 Biberach (Riss), Germany
| | - Thomas Wucherpfennig
- Boehringer Ingelheim Pharma GmbH & Co. KG, Bioprocess Development Biologicals, D-88397 Biberach (Riss), Germany
| | - Erich Bluhmki
- Boehringer Ingelheim Pharma GmbH & Co. KG, Analytical Development Biologicals, D-88397 Biberach (Riss), Germany
- University of Applied Sciences Biberach, D-88397 Biberach (Riss), Germany
| |
Collapse
|
34
|
|
35
|
In silico, in vitro, and in vivo machine learning in synthetic biology and metabolic engineering. Curr Opin Chem Biol 2021; 65:85-92. [PMID: 34280705 DOI: 10.1016/j.cbpa.2021.06.002] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Revised: 05/31/2021] [Accepted: 06/01/2021] [Indexed: 01/29/2023]
Abstract
Among the main learning methods reviewed in this study and used in synthetic biology and metabolic engineering are supervised learning, reinforcement and active learning, and in vitro or in vivo learning. In the context of biosynthesis, supervised machine learning is being exploited to predict biological sequence activities, predict structures and engineer sequences, and optimize culture conditions. Active and reinforcement learning methods use training sets acquired through an iterative process generally involving experimental measurements. They are applied to design, engineer, and optimize metabolic pathways and bioprocesses. The nascent but promising developments with in vitro and in vivo learning comprise molecular circuits performing simple tasks such as pattern recognition and classification.
Collapse
|
36
|
Alshehri AS, You F. Paradigm Shift: The Promise of Deep Learning in Molecular Systems Engineering and Design. FRONTIERS IN CHEMICAL ENGINEERING 2021. [DOI: 10.3389/fceng.2021.700717] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
The application of deep learning to a diverse array of research problems has accelerated progress across many fields, bringing conventional paradigms to a new intelligent era. Just as the roles of instrumentation in the old chemical revolutions, we reinforce the necessity for integrating deep learning in molecular systems engineering and design as a transformative catalyst towards the next chemical revolution. To meet such research needs, we summarize advances and progress across several key elements of molecular systems: molecular representation, property estimation, representation learning, and synthesis planning. We further spotlight recent advances and promising directions for several deep learning architectures, methods, and optimization platforms. Our perspective is of interest to both computational and experimental researchers as it aims to chart a path forward for cross-disciplinary collaborations on synthesizing knowledge from available chemical data and guiding experimental efforts.
Collapse
|
37
|
Oh DH, Adams D, Vo ND, Gbadago DQ, Lee CH, Oh M. Actor-critic reinforcement learning to estimate the optimal operating conditions of the hydrocracking process. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107280] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
|
38
|
Mowbray M, Smith R, Del Rio‐Chanona EA, Zhang D. Using process data to generate an optimal control policy via apprenticeship and reinforcement learning. AIChE J 2021. [DOI: 10.1002/aic.17306] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
- Max Mowbray
- Department of Chemical Engineering and Analytical Science The University of Manchester Manchester UK
| | - Robin Smith
- Department of Chemical Engineering and Analytical Science The University of Manchester Manchester UK
| | | | - Dongda Zhang
- Department of Chemical Engineering and Analytical Science The University of Manchester Manchester UK
| |
Collapse
|
39
|
Abstract
Chemical process engineering and machine learning are merging rapidly, and hybrid process models have shown promising results in process analysis and process design. However, uncertainties in first-principles process models have an adverse effect on extrapolations and inferences based on hybrid process models. Parameter sensitivities are an essential tool to understand better the underlying uncertainty propagation and hybrid system identification challenges. Still, standard parameter sensitivity concepts may fail to address comprehensive parameter uncertainty problems, i.e., deep uncertainty with aleatoric and epistemic contributions. This work shows a highly effective and reproducible sampling strategy to calculate simulation uncertainties and global parameter sensitivities for hybrid process models under deep uncertainty. We demonstrate the workflow with two electrochemical synthesis simulation studies, including the synthesis of furfuryl alcohol and 4-aminophenol. Compared with Monte Carlo reference simulations, the CPU-time was significantly reduced. The general findings of the hybrid model sensitivity studies under deep uncertainty are twofold. First, epistemic uncertainty has a significant effect on uncertainty analysis. Second, the predicted parameter sensitivities of the hybrid process models add value to the interpretation and analysis of the hybrid models themselves but are not suitable for predicting the real process/full first-principles process model’s sensitivities.
Collapse
|
40
|
Real-time optimization meets Bayesian optimization and derivative-free optimization: A tale of modifier adaptation. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107249] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
41
|
Reinforcement learning based optimization of process chromatography for continuous processing of biopharmaceuticals. Chem Eng Sci 2021. [DOI: 10.1016/j.ces.2020.116171] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
42
|
Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2020.107133] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
43
|
Potential of Integrating Model-Based Design of Experiments Approaches and Process Analytical Technologies for Bioprocess Scale-Down. ADVANCES IN BIOCHEMICAL ENGINEERING/BIOTECHNOLOGY 2021. [PMID: 33381857 DOI: 10.1007/10_2020_154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/18/2023]
Abstract
Typically, bioprocesses on an industrial scale are dynamic systems with a certain degree of variability, system inhomogeneities, and even population heterogeneities. Therefore, the scaling of such processes from laboratory to industrial scale and vice versa is not a trivial task. Traditional scale-down methodologies consider several technical parameters, so that systems on the laboratory scale tend to qualitatively reflect large-scale effects, but not the dynamic situation in an industrial bioreactor over the entire process, from the perspective of a cell. Supported by the enormous increase in computing power, the latest scientific focus is on the application of dynamic models, in combination with computational fluid dynamics to quantitatively describe cell behavior. These models allow the description of possible cellular lifelines which in turn can be used to derive a regime analysis for scale-down experiments. However, the approaches described so far, which were for a very few process examples, are very labor- and time-intensive and cannot be validated easily. In parallel, alternatives have been developed based on the description of the industrial process with hybrid process models, which describe a process mechanistically as far as possible in order to determine the essential process parameters with their respective variances. On-line analytical methods allow the characterization of population heterogeneity directly in the process. This detailed information from the industrial process can be used in laboratory screening systems to select relevant conditions in which the cell and process related parameters reflect the situation in the industrial scale. In our opinion, these technologies, which are available in research for modeling biological systems, in combination with process analytical techniques are so far developed that they can be implemented in industrial routines for faster development of new processes and optimization of existing ones.
Collapse
|
44
|
Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process. Processes (Basel) 2020. [DOI: 10.3390/pr8111497] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
One popular method for optimizing systems, referred to as ANN-PSO, uses an artificial neural network (ANN) to approximate the system and an optimization method like particle swarm optimization (PSO) to select inputs. However, with reinforcement learning developments, it is important to compare ANN-PSO to newer algorithms, like Proximal Policy Optimization (PPO). To investigate ANN-PSO’s and PPO’s performance and applicability, we compare their methodologies, apply them on steady-state economic optimization of a chemical process, and compare their results to a conventional first principles modeling with nonlinear programming (FP-NLP). Our results show that ANN-PSO and PPO achieve profits nearly as high as FP-NLP, but PPO achieves slightly higher profits compared to ANN-PSO. We also find PPO has the fastest computational times, 10 and 10,000 times faster than FP-NLP and ANN-PSO, respectively. However, PPO requires more training data than ANN-PSO to converge to an optimal policy. This case study suggests PPO has better performance as it achieves higher profits and faster online computational times. ANN-PSO shows better applicability with its capability to train on historical operational data and higher training efficiency.
Collapse
|
45
|
Teng SY, Yew GY, Sukačová K, Show PL, Máša V, Chang JS. Microalgae with artificial intelligence: A digitalized perspective on genetics, systems and products. Biotechnol Adv 2020; 44:107631. [PMID: 32931875 DOI: 10.1016/j.biotechadv.2020.107631] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2020] [Revised: 09/08/2020] [Accepted: 09/08/2020] [Indexed: 12/18/2022]
Abstract
With recent advances in novel gene-editing tools such as RNAi, ZFNs, TALENs, and CRISPR-Cas9, the possibility of altering microalgae toward designed properties for various application is becoming a reality. Alteration of microalgae genomes can modify metabolic pathways to give elevated yields in lipids, biomass, and other components. The potential of such genetically optimized microalgae can give a "domino effect" in further providing optimization leverages down the supply chain, in aspects such as cultivation, processing, system design, process integration, and revolutionary products. However, the current level of understanding the functional information of various microalgae gene sequences is still primitive and insufficient as microalgae genome sequences are long and complex. From this perspective, this work proposes to link up this knowledge gap between microalgae genetic information and optimized bioproducts using Artificial Intelligence (AI). With the recent acceleration of AI research, large and complex data from microalgae research can be properly analyzed by combining the cutting-edge of both fields. In this work, the most suitable class of AI algorithms (such as active learning, semi-supervised learning, and meta-learning) are discussed for different cases of microalgae applications. This work concisely reviews the current state of the research milestones and highlight some of the state-of-art that has been carried out, providing insightful future pathways. The utilization of AI algorithms in microalgae cultivation, system optimization, and other aspects of the supply chain is also discussed. This work opens the pathway to a digitalized future for microalgae research and applications.
Collapse
Affiliation(s)
- Sin Yong Teng
- Brno University of Technology, Institute of Process Engineering, Technická 2896/2, 616 69, Brno, Czech Republic.
| | - Guo Yong Yew
- Department of Chemical and Environmental Engineering, Faculty of Science and Engineering, University of Nottingham Malaysia, Jalan Broga, 43500 Semenyih, Selangor, Malaysia.
| | - Kateřina Sukačová
- Global Change Research Institute of the Czech Academy of Sciences, Bělidla 986/4a, Brno 603 00, Czech Republic.
| | - Pau Loke Show
- Department of Chemical and Environmental Engineering, Faculty of Science and Engineering, University of Nottingham Malaysia, Jalan Broga, 43500 Semenyih, Selangor, Malaysia.
| | - Vítězslav Máša
- Brno University of Technology, Institute of Process Engineering, Technická 2896/2, 616 69, Brno, Czech Republic.
| | - Jo-Shu Chang
- Department of Chemical and Materials Engineering, College of Engineering, Tunghai University, Taichung 407, Taiwan; Department of Chemical Engineering, National Cheng Kung University, Tainan 701, Taiwan; Research Center for Smart Sustainable Circular Economy, Tunghai University, Taichung 407, Taiwan.
| |
Collapse
|
46
|
Design of control framework based on deep reinforcement learning and Monte-Carlo sampling in downstream separation. Comput Chem Eng 2020. [DOI: 10.1016/j.compchemeng.2020.106910] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
47
|
Kim Y, Lee JM. Model‐based reinforcement learning for nonlinear optimal control with practical asymptotic stability guarantees. AIChE J 2020. [DOI: 10.1002/aic.16544] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
- Yeonsoo Kim
- School of Chemical and Biological Engineering Institute of Chemical Processes, Seoul National University Seoul Republic of Korea
| | - Jong Min Lee
- School of Chemical and Biological Engineering Institute of Chemical Processes, Seoul National University Seoul Republic of Korea
| |
Collapse
|
48
|
|