Nonlinear ill-posed problem analysis in model-based parameter estimation and experimental design

Modeling multi-component separation in hydrophobic interaction chromatography with improved parameter-by-parameter estimation method

Mechanistic models are powerful tools for chromatographic process development and optimization. However, hydrophobic interaction chromatography (HIC) mechanistic models lack an effective and logical parameter estimation method, especially for multi-component system. In this study, a parameter-by-parameter method for multi-component system (called as mPbP-HIC) was derived based on the retention mechanism to estimate the six parameters of the Mollerup isotherm for HIC. The linear parameters (ks,i and keq,i) and nonlinear parameters (ni and qmax,i) of the isotherm can be estimated by the linear regression (LR) and the linear approximation (LA) steps, respectively. The remaining two parameters (kp,i and kkin,i) are obtained by the inverse method (IM). The proposed method was verified with a two-component model system. The results showed that the model could accurately predict the protein elution at a loading of 10 g/L. However, the elution curve fitting was unsatisfactory for high loadings (12 g/L and 14 g/L), which is mainly attributed to the demanding experimental conditions of the LA step and the potential large estimation error of the parameter qmax. Therefore, the inverse method was introduced to further calibrate the parameter qmax, thereby reducing the estimation error and improving the curve fitting. Moreover, the simplified linear approximation (SLA) was proposed by reasonable assumption, which provides the initial guess of qmax without solving any complex matrix and avoids the problem of matrix unsolvable. In the improved mPbP-HIC method, qmax would be initialized by the SLA and finally determined by the inverse method, and this strategy was named as SLA+IM. The experimental validation showed that the improved mPbP-HIC method has a better curve fitting, and the use of SLA+IM reduces the error accumulation effect. In process optimization, the parameters estimated by the improved mPbP-HIC method provided the model with excellent predictive ability and reasonable extrapolation. In conclusion, the SLA+IM strategy makes the improved mPbP-HIC method more rational and can be easily applied to the practical separation of protein mixture, which would accelerate the process development for HIC in downstream of biopharmaceuticals.

Random effects models of tumour growth for investigating interval breast cancer

In Nordic countries and across Europe, breast cancer screening participation is high. However, a significant number of breast cancer cases are still diagnosed due to symptoms between screening rounds, termed "interval cancers". Radiologists use the interval cancer proportion as a proxy for the screening false negative rate (ie, 1-sensitivity). Our objective is to enhance our understanding of interval cancers by applying continuous tumour growth models to data from a study involving incident invasive breast cancer cases. Building upon previous findings regarding stationary distributions of tumour size and growth rate distributions in non-screened populations, we develop an analytical expression for the proportion of interval breast cancer cases among regularly screened women. Our approach avoids relying on estimated background cancer rates. We make specific parametric assumptions concerning tumour growth and detection processes (screening or symptoms), but our framework easily accommodates alternative assumptions. We also show how our developed analytical expression for the proportion of interval breast cancers within a screened population can be incorporated into an approach for fitting tumour growth models to incident case data. We fit a model on 3493 cases diagnosed in Sweden between 2001 and 2008. Our methodology allows us to estimate the distribution of tumour sizes at the most recent screening for interval cancers. Importantly, we find that our model-based expected incidence of interval breast cancers aligns closely with observed patterns in our study and in a large Nordic screening cohort. Finally, we evaluate the association between screening interval length and the interval cancer proportion. Our analytical expression represents a useful tool for gaining insights into the performance of population-based breast cancer screening programs.

Building blocks needed for mechanistic modeling of bioprocesses: A critical review based on protein production by CHO cells

This paper reviews the key building blocks needed to develop a mechanistic model for use as an operational production tool. The Chinese Hamster Ovary (CHO) cell, one of the most widely used hosts for antibody production in the pharmaceutical industry, is considered as a case study. CHO cell metabolism is characterized by two main phases, exponential growth followed by a stationary phase with strong protein production. This process presents an appropriate degree of complexity to outline the modeling strategy. The paper is organized into four main steps: (1) CHO systems and data collection; (2) metabolic analysis; (3) formulation of the mathematical model; and finally, (4) numerical solution, calibration, and validation. The overall approach can build a predictive model of target variables. According to the literature, one of the main current modeling challenges lies in understanding and predicting the spontaneous metabolic shift. Possible candidates for the trigger of the metabolic shift include the concentration of lactate and carbon dioxide. In our opinion, ammonium, which is also an inhibiting product, should be further investigated. Finally, the expected progress in the emerging field of hybrid modeling, which combines the best of mechanistic modeling and machine learning, is presented as a fascinating breakthrough. Note that the modeling strategy discussed here is a general framework that can be applied to any bioprocess.

Fast time-resolved micro-CT imaging of pharmaceutical tablets: Insights into water uptake and disintegration

We use dynamic micro-computed tomography (micro-CT) with a high temporal resolution to visualize water penetration through the porous network of immediate-release pharmaceutical solid tablets and characterize dynamic swelling and disintegration mechanisms. We process the micro-CT images using two theoretical scenarios that reflect different paths of pore structure evolution: a scenario where tablet porosity remains constant during the swelling process and a scenario where the tablet porosity progressively diminishes and eventually closes during the swelling process. We calculate the time evolution of the volume of water absorbed by the tablet and, specifically, absorbed by the excipients and the pore structure, as well as the formation and evolution of cracks. In turn, the three-dimensional disintegration pattern of the tablets is reconstructed. Restricting attention to the limiting scenario where tablet porosity is assumed fixed during the swelling process, we couple liquid penetration due to capillary pressure described by the Lucas-Washburn theory with the first-order swelling kinetics of the excipients to provide a physical interpretation of the experimental observations. We estimate model parameters that are in agreement with values reported in the literature, and we demonstrate that water penetration is dominated by intra-particle porosity rather than inter-particle porosity.

Reaction kinetics determination and uncertainty analysis for the synthesis of the cancer drug lomustine

Fast and reliable model development frameworks are required to support current trends in modernization of pharmaceutical processing, promoting the use of digital platforms to assist process design and operation. In this work, we use a parameter estimation framework built into the PharmaPy library to determine rate parameters and uncertainty regions of different mechanistic and semi-empirical kinetic expressions for the synthesis of the drug lomustine. The parameter estimation procedure was complemented by identifiability analysis, resulting in simplified reaction mechanisms. Comparison of parameters and their uncertainty in process design was demonstrated through design space analysis, showing important differences in model prediction and the extent of their corresponding design spaces. The results of this work can serve to analyze lomustine manufacturing processes that include separation and isolation steps, where parametric sensitivity is expected to propagate along the manufacturing line and impact process feasible operation, and attainment of critical quality attributes of the product.

Modeling the Batch Sedimentation of Calcium Carbonate Particles in Laboratory Experiments-A Systematic Approach

The design of continuous thickeners and clarifiers is commonly based on the solid flux theory. Batch sedimentation experiments conducted with solid concentrations still provide useful information for their application. The construction of models for the velocity of settling allows the estimation of the flux of solids throughout time, which can, in turn, be used to find the area of the units required to achieve a given solid concentration in the clarified stream. This paper addresses the numerical treatment of data obtained from batch sedimentation experiments of calcium carbonate particles. We propose a systematic framework to fit a model that is capable of representing the process features that involve (i) the numerical differentiation of data to generate initial estimates for the instantaneous velocity of settling; (ii) the integration of a differential equation to fit the model for the velocity of settling; and (iii) the assessment of the quality of the fit using common statistical indicators. The model used for demonstration has a theoretical basis combined with an empirical component to account for the effect of the particle concentrations and their state of aggregation. The values of the numerical parameters obtained are related to the characteristic dimensions of the aggregates and their mass-length fractal dimensions.

Industrial ion-exchange chromatography development using discontinuous Galerkin methods coupled with forward sensitivity analysis

In this work, a discontinuous Galerkin method coupled with forward sensitivity analysis (DG-FSA) is presented. The DG-FSA method is used to reduce computational cost required for model-based ion-exchange chromatography development using industrial load samples. As an example, the design of an anion-exchange chromatography step is considered. This step is used to purify an experimental peptide product called Protein G from Novo Nordisk A/S (Bagsværd, Denmark). The results demonstrate, that a fourth order DG-FSA method can reduce computational cost of inverse problems by a factor ×16 compared to a second (low) order DG-FSA method. Furthermore, the fourth-order DG-FSA method enable the computation of probability distributions of optimized processing conditions given uncertainty in model parameters or inputs. This analysis is not possible within a reasonable timeframe when applying the second (low) order DG-FSA method. The design procedure facilitates the optimization of the Protein G purification step. In an experimental validation run, the productivity is increased by 70% while sacrificing 4% yield at a similar purity constraint compared to an experiment with baseline performance.

Semi-mechanistic reduced order model of pharmaceutical tablet dissolution for enabling Industry 4.0 manufacturing systems

We propose a generalization of the Weibull dissolution model, referred to as generalized Weibull dissolution model, that seamlessly captures all three fractional dissolution rates experimentally observed in pharmaceutical solid tablets, namely decreasing, increasing, and non-monotonic rates. This is in contrast to traditional reduced order models, which capture at most two fractional dissolution rates and, thus, are not suitable for a wide range of product formulations hindering, for example, the adoption of knowledge management in the context of Industry 4.0. We extend the generalized Weibull dissolution model further to capture the relationship between critical process parameters (CPPs), critical materials attributes (CMAs), and dissolution profile to, in turn, facilitate real-time release testing (RTRT) and quality-by-control (QbC) strategies. Specifically, we endow the model with multivariate rational polynomials that interpolate the mechanistic limiting behavior of tablet dissolution as CPPs and CMAs approach certain values of physical significance (such as the upper and lower bounds of tablet porosity or lubrication conditions), thus the semi-mechanistic nature of the reduced order model. Restricting attention to direct compaction and using various case studies from the literature, we demonstrate the versatility and the capability of the semi-mechanistic ROM to estimate changes in dissolution due to process disturbances in tablet weight, porosity, lubrication conditions (i.e., the total amount of shear strain imparted during blending), and moisture content in the powder blend. In all of the cases considered in this work, the estimations of the model are in remarkable agreement with experimental data.

State Estimators in Soft Sensing and Sensor Fusion for Sustainable Manufacturing

State estimators, including observers and Bayesian filters, are a class of model-based algorithms for estimating variables in a dynamical system given the sensor measurements of related system states. They can be used to derive fast and accurate estimates of system variables that cannot be measured directly (`soft sensing’) or for which only noisy, intermittent, delayed, indirect, or unreliable measurements are available, perhaps from multiple sources (`sensor fusion’). In this paper, we introduce the concepts and main methods of state estimation and review recent applications in improving the sustainability of manufacturing processes across sectors including industrial robotics, material synthesis and processing, semiconductor, and additive manufacturing. It is shown that state estimation algorithms can play a key role in manufacturing systems for accurately monitoring and controlling processes to improve efficiencies, lower environmental impact, enhance product quality, improve the feasibility of processing more sustainable raw materials, and ensure safer working environments for humans. We discuss current and emerging trends in using state estimation as a framework for combining physical knowledge with other sources of data for monitoring and controlling distributed manufacturing systems.

A protocol for dynamic model calibration

Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and nonmeasurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challenges such as poor parameter identifiability, lack of sufficiently informative experimental data and the existence of local minima in the objective function landscape. These issues tend to worsen with larger model sizes, increasing the computational complexity and the number of unknown parameters. An incorrectly calibrated model is problematic because it may result in inaccurate predictions and misleading conclusions. For nonexpert users, there are a large number of potential pitfalls. Here, we provide a protocol that guides the user through all the steps involved in the calibration of dynamic models. We illustrate the methodology with two models and provide all the code required to reproduce the results and perform the same analysis on new models. Our protocol provides practitioners and researchers in biological modelling with a one-stop guide that is at the same time compact and sufficiently comprehensive to cover all aspects of the problem.

Streamlining tablet lubrication design via model-based design of experiments

In oral solid dosage production through direct compression powder lubrication must be carefully selected to facilitate the manufacturing of tablets without degrading product manufacturability and quality (e.g. dissolution). To do so, several semi-empirical models relating compression performance to process operating conditions have been developed. Among them, we consider an extension of the Kushner and Moore model (Kushner and Moore, 2010, International Journal Pharmaceutics, 399:19) that is useful for the purpose, but requires an extensive experimental campaign for parameters identification. This implies the preparation and compression of multiple powder blends, each one with a different lubrication extent. In turn, this translates into a considerable consumption of Active Pharmaceutical Ingredient (API), and into time-consuming experiments. We tackled this issue by proposing a novel model-based design of experiments (MBDoE) approach, which minimizes the number of optimal blends for model calibration, while obtaining statistically sound parameters estimates and model predictions. Both sequential and parallel MBDoE configurations were compared. Experimental results involving two placebo blends with different lubrication sensitivity showed that this methodology is able to reduce the experimental effort by 60-70% with respect to the standard industrial practice independently of the formulation considered and configuration (i.e. parallel vs. sequential) adopted.

Efficient Two-Step Parametrization of a Control-Oriented Zero-Dimensional Polymer Electrolyte Membrane Fuel Cell Model Based on Measured Stack Data

This paper proposes a new efficient two-step method for parametrizing control-oriented zero-dimensional physical polymer electrolyte membrane fuel cell (PEMFC) models with measured stack data. Parametrizations of these models are computationally intensive due to the numerous unknown parameters and the typically nonlinear, stiff model properties. This work reduces an existing model to decrease its stiffness for accelerated numerical simulations. Subdividing the parametrization into two consecutive subproblems (thermodynamic and electrochemical ones) reduces the solution space significantly. A parameter sensitivity analysis further reduces each sub-solution space by excluding non-significant parameters. The method results in an efficient parametrization process. The two-step approach minimizes each sub-solution space’s dimension by two-thirds, respectively three-fourths, compared to the global one. An achieved R2 value between simulation and measurement of 91% on average provides the required accuracy for control-oriented models.

Model-based workflow for scale-up of process strategies developed in miniaturized bioreactor systems

Miniaturized bioreactor (MBR) systems are routinely used in the development of mammalian cell culture processes. However, scale-up of process strategies obtained in MBR- to larger scale is challenging due to mainly non-holistic scale-up approaches. In this study, a model-based workflow is introduced to quantify differences in the process dynamics between bioreactor scales and thus enable a more knowledge-driven scale-up. The workflow is applied to two case studies with antibody-producing Chinese hamster ovary cell lines. With the workflow, model parameter distributions are estimated first under consideration of experimental variability for different scales. Second, the obtained individual model parameter distributions are tested for statistical differences. In case of significant differences, model parametric distributions are transferred between the scales. In case study I, a fed-batch process in a microtiter plate (4 ml working volume) and lab-scale bioreactor (3750 ml working volume) was mathematically modeled and evaluated. No significant differences were identified for model parameter distributions reflecting process dynamics. Therefore, the microtiter plate can be applied as scale-down tool for the lab-scale bioreactor. In case study II, a fed-batch process in a 24-Deep-Well-Plate (2 ml working volume) and shake flask (40 ml working volume) with two feed media was investigated. Model parameter distributions showed significant differences. Thus, process strategies were mathematically transferred, and model predictions were simulated for a new shake flask culture setup and confirmed in validation experiments. Overall, the workflow enables a knowledge-driven evaluation of scale-up for a more efficient bioprocess design and optimization.

Mathematical Model of Glucagon Kinetics for the Assessment of Insulin-Mediated Glucagon Inhibition During an Oral Glucose Tolerance Test

Glucagon is secreted from the pancreatic alpha cells and plays an important role in the maintenance of glucose homeostasis, by interacting with insulin. The plasma glucose levels determine whether glucagon secretion or insulin secretion is activated or inhibited. Despite its relevance, some aspects of glucagon secretion and kinetics remain unclear. To gain insight into this, we aimed to develop a mathematical model of the glucagon kinetics during an oral glucose tolerance test, which is sufficiently simple to be used in the clinical practice. The proposed model included two first-order differential equations -one describing glucagon and the other describing C-peptide in a compartment remote from plasma - and yielded a parameter of possible clinical relevance (i.e., S_GLUCA(t), glucagon-inhibition sensitivity to glucose-induced insulin secretion). Model was validated on mean glucagon data derived from the scientific literature, yielding values for S_GLUCA(t) ranging from -15.03 to 2.75 (ng of glucagon·nmol of C-peptide^-1). A further validation on a total of 100 virtual subjects provided reliable results (mean residuals between -1.5 and 1.5 ng·L^-1) and a negative significant linear correlation (r = -0.74, p < 0.0001, 95% CI: -0.82 - -0.64) between S_GLUCA(t) and the ratio between the areas under the curve of suprabasal remote C-peptide and glucagon. Model reliability was also proven by the ability to capture different patterns in glucagon kinetics. In conclusion, the proposed model reliably reproduces glucagon kinetics and is characterized by sufficient simplicity to be possibly used in the clinical practice, for the estimation in the single individual of some glucagon-related parameters.

Interleukin-15 Signaling in HIF-1α Regulation in Natural Killer Cells, Insights Through Mathematical Models

Natural killer (NK) cells belong to the first line of host defense against infection and cancer. Cytokines, including interleukin-15 (IL-15), critically regulate NK cell activity, resulting in recognition and direct killing of transformed and infected target cells. NK cells have to adapt and respond in inflamed and often hypoxic areas. Cellular stabilization and accumulation of the transcription factor hypoxia-inducible factor-1α (HIF-1α) is a key mechanism of the cellular hypoxia response. At the same time, HIF-1α plays a critical role in both innate and adaptive immunity. While the HIF-1α hydroxylation and degradation pathway has been recently described with the help of mathematical methods, less is known concerning the mechanistic mathematical description of processes regulating the levels of HIF-1α mRNA and protein. In this work we combine mathematical modeling with experimental laboratory analysis and examine the dynamic relationship between HIF-1α mRNA, HIF-1α protein, and IL-15-mediated upstream signaling events in NK cells from human blood. We propose a system of non-linear ordinary differential equations with positive and negative feedback loops for describing the complex interplay of HIF-1α regulators. The experimental design is optimized with the help of mathematical methods, and numerical optimization techniques yield reliable parameter estimates. The mathematical model allows for the investigation and prediction of HIF-1α stabilization under different inflammatory conditions and provides a better understanding of mechanisms mediating cellular enrichment of HIF-1α. Thanks to the combination of in vitro experimental data and in silico predictions we identified the mammalian target of rapamycin (mTOR), the nuclear factor-κB (NF-κB), and the signal transducer and activator of transcription 3 (STAT3) as central regulators of HIF-1α accumulation. We hypothesize that the regulatory pathway proposed here for NK cells can be extended to other types of immune cells. Understanding the molecular mechanisms involved in the dynamic regulation of the HIF-1α pathway in immune cells is of central importance to the immune cell function and could be a promising strategy in the design of treatments for human inflammatory diseases and cancer.

Monte Carlo Simulations for the Analysis of Non-linear Parameter Confidence Intervals in Optimal Experimental Design

Especially in biomanufacturing, methods to design optimal experiments are a valuable technique to fully exploit the potential of the emerging technical possibilities that are driving experimental miniaturization and parallelization. The general objective is to reduce the experimental effort while maximizing the information content of an experiment, speeding up knowledge gain in R&D. The approach of model-based design of experiments (known as MBDoE) utilizes the information of an underlying mathematical model describing the system of interest. A common method to predict the accuracy of the parameter estimates uses the Fisher information matrix to approximate the 90% confidence intervals of the estimates. However, for highly non-linear models, this method might lead to wrong conclusions. In such cases, Monte Carlo sampling gives a more accurate insight into the parameter's estimate probability distribution and should be exploited to assess the reliability of the approximations made through the Fisher information matrix. We first introduce the model-based optimal experimental design for parameter estimation including parameter identification and validation by means of a simple non-linear Michaelis-Menten kinetic and show why Monte Carlo simulations give a more accurate depiction of the parameter uncertainty. Secondly, we propose a very robust and simple method to find optimal experimental designs using Monte Carlo simulations. Although computational expensive, the method is easy to implement and parallelize. This article focuses on practical examples of bioprocess engineering but is generally applicable in other fields.

Model-Based Methods in the Biopharmaceutical Process Lifecycle

Model-based methods are increasingly used in all areas of biopharmaceutical process technology. They can be applied in the field of experimental design, process characterization, process design, monitoring and control. Benefits of these methods are lower experimental effort, process transparency, clear rationality behind decisions and increased process robustness. The possibility of applying methods adopted from different scientific domains accelerates this trend further. In addition, model-based methods can help to implement regulatory requirements as suggested by recent Quality by Design and validation initiatives. The aim of this review is to give an overview of the state of the art of model-based methods, their applications, further challenges and possible solutions in the biopharmaceutical process life cycle. Today, despite these advantages, the potential of model-based methods is still not fully exhausted in bioprocess technology. This is due to a lack of (i) acceptance of the users, (ii) user-friendly tools provided by existing methods, (iii) implementation in existing process control systems and (iv) clear workflows to set up specific process models. We propose that model-based methods be applied throughout the lifecycle of a biopharmaceutical process, starting with the set-up of a process model, which is used for monitoring and control of process parameters, and ending with continuous and iterative process improvement via data mining techniques.

Parameter identifiability analysis and visualization in large-scale kinetic models of biosystems

Background

Kinetic models of biochemical systems usually consist of ordinary differential equations that have many unknown parameters. Some of these parameters are often practically unidentifiable, that is, their values cannot be uniquely determined from the available data. Possible causes are lack of influence on the measured outputs, interdependence among parameters, and poor data quality. Uncorrelated parameters can be seen as the key tuning knobs of a predictive model. Therefore, before attempting to perform parameter estimation (model calibration) it is important to characterize the subset(s) of identifiable parameters and their interplay. Once this is achieved, it is still necessary to perform parameter estimation, which poses additional challenges.

Methods

We present a methodology that (i) detects high-order relationships among parameters, and (ii) visualizes the results to facilitate further analysis. We use a collinearity index to quantify the correlation between parameters in a group in a computationally efficient way. Then we apply integer optimization to find the largest groups of uncorrelated parameters. We also use the collinearity index to identify small groups of highly correlated parameters. The results files can be visualized using Cytoscape, showing the identifiable and non-identifiable groups of parameters together with the model structure in the same graph.

Results

Our contributions alleviate the difficulties that appear at different stages of the identifiability analysis and parameter estimation process. We show how to combine global optimization and regularization techniques for calibrating medium and large scale biological models with moderate computation times. Then we evaluate the practical identifiability of the estimated parameters using the proposed methodology. The identifiability analysis techniques are implemented as a MATLAB toolbox called VisId, which is freely available as open source from GitHub (https://github.com/gabora/visid).

Conclusions

Our approach is geared towards scalability. It enables the practical identifiability analysis of dynamic models of large size, and accelerates their calibration. The visualization tool allows modellers to detect parts that are problematic and need refinement or reformulation, and provides experimentalists with information that can be helpful in the design of new experiments.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-017-0428-y) contains supplementary material, which is available to authorized users.