Propagation of uncertainty in the mechanical and biological response of growing tissues using multi-fidelity Gaussian process regression

Machine learning modeling of lung mechanics: Assessing the variability and propagation of uncertainty in respiratory-system compliance and airway resistance

BACKGROUND AND OBJECTIVE

Traditional assessment of patient response in mechanical ventilation relies on respiratory-system compliance and airway resistance. Clinical evidence has shown high variability in these parameters, highlighting the difficulty of predicting them before the start of ventilation therapy. This motivates the creation of computational models that can connect structural and tissue features with lung mechanics. In this work, we leverage machine learning (ML) techniques to construct predictive lung function models informed by non-linear finite element simulations, and use them to investigate the propagation of uncertainty in the lung mechanical response.

METHODS

We revisit a continuum poromechanical formulation of the lungs suitable for determining patient response. Based on this framework, we create high-fidelity finite element models of human lungs from medical images. We also develop a low-fidelity model based on an idealized sphere geometry. We then use these models to train and validate three ML architectures: single-fidelity and multi-fidelity Gaussian process regression, and artificial neural networks. We use the best predictive ML model to further study the sensitivity of lung response to variations in tissue structural parameters and boundary conditions via sensitivity analysis and forward uncertainty quantification. Codes are available for download at https://github.com/comp-medicine-uc/ML-lung-mechanics-UQ RESULTS: The low-fidelity model delivers a lung response very close to that predicted by high-fidelity simulations and at a fraction of the computational time. Regarding the trained ML models, the multi-fidelity GP model consistently delivers better accuracy than the single-fidelity GP and neural network models in estimating respiratory-system compliance and resistance (R2∼0.99). In terms of computational efficiency, our ML model delivers a massive speed-up of ∼970,000× with respect to high-fidelity simulations. Regarding lung function, we observed an almost matched and non-linear behavior between specific structural parameters and chest wall stiffness with compliance. Also, we observed a strong modulation of airways resistance with tissue permeability.

CONCLUSIONS

Our findings unveil the relevance of specific lung tissue parameters and boundary conditions in the respiratory-system response. Furthermore, we highlight the advantages of adopting a multi-fidelity ML approach that combines data from different levels to yield accurate and efficient estimates of clinical mechanical markers. We envision that the methods presented here can open the way to the development of predictive ML models of the lung response that can inform clinical decisions.

3D-Printed Functional Hydrogel by DNA-Induced Biomineralization for Accelerated Diabetic Wound Healing

Chronic wounds in diabetic patients are challenging because their prolonged inflammation makes healing difficult, thus burdening patients, society, and health care systems. Customized dressing materials are needed to effectively treat such wounds that vary in shape and depth. The continuous development of 3D-printing technology along with artificial intelligence has increased the precision, versatility, and compatibility of various materials, thus providing the considerable potential to meet the abovementioned needs. Herein, functional 3D-printing inks comprising DNA from salmon sperm and DNA-induced biosilica inspired by marine sponges, are developed for the machine learning-based 3D-printing of wound dressings. The DNA and biomineralized silica are incorporated into hydrogel inks in a fast, facile manner. The 3D-printed wound dressing thus generates provided appropriate porosity, characterized by effective exudate and blood absorption at wound sites, and mechanical tunability indicated by good shape fidelity and printability during optimized 3D printing. Moreover, the DNA and biomineralized silica act as nanotherapeutics, enhancing the biological activity of the dressings in terms of reactive oxygen species scavenging, angiogenesis, and anti-inflammation activity, thereby accelerating acute and diabetic wound healing. These bioinspired 3D-printed hydrogels produce using a DNA-induced biomineralization strategy are an excellent functional platform for clinical applications in acute and chronic wound repair.

Optimizing autoinjector devices using physics-based simulations and Gaussian processes

Autoinjectors are becoming a primary drug delivery option to the subcutaneous space. These devices need to work robustly and autonomously to maximize drug bio-availability. However, current designs ignore the coupling between autoinjector dynamics and tissue biomechanics. Here we present a Bayesian framework for optimization of autoinjector devices that can account for the coupled autoinjector-tissue biomechanics and uncertainty in tissue mechanical behavior. The framework relies on replacing the high fidelity model of tissue insertion with a Gaussian process (GP). The GP model is accurate yet computationally affordable, enabling a thorough sensitivity analysis that identified tissue properties, which are not part of the autoinjector design space, as important variables for the injection process. Higher fracture toughness decreases the crack depth, while tissue shear modulus has the opposite effect. The sensitivity analysis also shows that drug viscosity and spring force, which are part of the design space, affect the location and timing of drug delivery. Low viscosity could lead to premature delivery, but can be prevented with smaller spring forces, while higher viscosity could prevent premature delivery while demanding larger spring forces and increasing the time of injection. Increasing the spring force guarantees penetration to the desired depth, but it can result in undesirably high accelerations. The Bayesian optimization framework tackles the challenge of designing devices with performance metrics coupled to uncertain tissue properties. This work is important for the design of other medical devices for which optimization in the presence of material behavior uncertainty is needed.

Strain energy density as a Gaussian process and its utilization in stochastic finite element analysis: application to planar soft tissues

Data-based approaches are promising alternatives to the traditional analytical constitutive models for solid mechanics. Herein, we propose a Gaussian process (GP) based constitutive modeling framework, specifically focusing on planar, hyperelastic and incompressible soft tissues. The strain energy density of soft tissues is modeled as a GP, which can be regressed to experimental stress-strain data obtained from biaxial experiments. Moreover, the GP model can be weakly constrained to be convex. A key advantage of a GP-based model is that, in addition to the mean value, it provides a probability density (i.e. associated uncertainty) for the strain energy density. To simulate the effect of this uncertainty, a non-intrusive stochastic finite element analysis (SFEA) framework is proposed. The proposed framework is verified against an artificial dataset based on the Gasser-Ogden-Holzapfel model and applied to a real experimental dataset of a porcine aortic valve leaflet tissue. Results show that the proposed framework can be trained with limited experimental data and fits the data better than several existing models. The SFEA framework provides a straightforward way of using the experimental data and quantifying the resulting uncertainty in simulation-based predictions.

Multi-Fidelity Gaussian Process Surrogate Modeling of Pediatric Tissue Expansion

Growth of skin in response to stretch is the basis for tissue expansion (TE), a procedure to gain new skin area for reconstruction of large defects. Unfortunately, complications and suboptimal outcomes persist because TE is planned and executed based on physician's experience and trial and error instead of predictive quantitative tools. Recently, we calibrated computational models of TE to a porcine animal model of tissue expansion, showing that skin growth is proportional to stretch with a characteristic time constant. Here, we use our calibrated model to predict skin growth in cases of pediatric reconstruction. Available from the clinical setting are the expander shapes and inflation protocols. We create low fidelity semi-analytical models and finite element models for each of the clinical cases. To account for uncertainty in the response expected from translating the models from the animal experiments to the pediatric population, we create multifidelity Gaussian process surrogates to propagate uncertainty in the mechanical properties and the biological response. Predictions with uncertainty for the clinical setting are essential to bridge our knowledge from the large animal experiments to guide and improve the treatment of pediatric patients. Future calibration of the model with patient-specific data-such as estimation of mechanical properties and area growth in the operating room-will change the standard for planning and execution of TE protocols.

On modeling the multiscale mechanobiology of soft tissues: Challenges and progress

Tissues grow and remodel in response to mechanical cues, extracellular and intracellular signals experienced through various biological events, from the developing embryo to disease and aging. The macroscale response of soft tissues is typically nonlinear, viscoelastic anisotropic, and often emerges from the hierarchical structure of tissues, primarily their biopolymer fiber networks at the microscale. The adaptation to mechanical cues is likewise a multiscale phenomenon. Cell mechanobiology, the ability of cells to transform mechanical inputs into chemical signaling inside the cell, and subsequent regulation of cellular behavior through intra- and inter-cellular signaling networks, is the key coupling at the microscale between the mechanical cues and the mechanical adaptation seen macroscopically. To fully understand mechanics of tissues in growth and remodeling as observed at the tissue level, multiscale models of tissue mechanobiology are essential. In this review, we summarize the state-of-the art modeling tools of soft tissues at both scales, the tissue level response, and the cell scale mechanobiology models. To help the interested reader become more familiar with these modeling frameworks, we also show representative examples. Our aim here is to bring together scientists from different disciplines and enable the future leap in multiscale modeling of tissue mechanobiology.

A framework to optimize spring-driven autoinjectors

The major challenges in the optimization of autoinjectors lie in developing an accurate model and meeting competing requirements. We have developed a computational model for spring-driven autoinjectors, which can accurately predict the kinematics of the syringe barrel, needle displacement (travel distance) at the start of drug delivery, and injection time. This paper focuses on proposing a framework to optimize the single-design of autoinjectors, which deliver multiple drugs with different viscosity. We replace the computational model for spring-driven autoinjectors with a surrogate model, i.e., a deep neural network, which improves computational efficiency 1,000 times. Using this surrogate, we perform Sobol sensitivity analysis to understand the effect of each model input on the quantities of interest. Additionally, we pose the design problem within a multi-objective optimization framework. We use our surrogate to discover the corresponding Pareto optimal designs via Pymoo, an open source library for multi-objective optimization. After these steps, we evaluate the robustness of these solutions and finally identify two promising candidates. This framework can be effectively used for device design optimization as the computation is not demanding, and decision-makers can easily incorporate their preferences into this framework.

Multiscale mechanobiology: Coupling models of adhesion kinetics and nonlinear tissue mechanics

The mechanical behavior of tissues at the macroscale is tightly coupled to cellular activity at the microscale. Dermal wound healing is a prominent example of a complex system in which multiscale mechanics regulate restoration of tissue form and function. In cutaneous wound healing, a fibrin matrix is populated by fibroblasts migrating in from a surrounding tissue made mostly out of collagen. Fibroblasts both respond to mechanical cues, such as fiber alignment and stiffness, as well as exert active stresses needed for wound closure. Here, we develop a multiscale model with a two-way coupling between a microscale cell adhesion model and a macroscale tissue mechanics model. Starting from the well-known model of adhesion kinetics proposed by Bell, we extend the formulation to account for nonlinear mechanics of fibrin and collagen and show how this nonlinear response naturally captures stretch-driven mechanosensing. We then embed the new nonlinear adhesion model into a custom finite element implementation of tissue mechanical equilibrium. Strains and stresses at the tissue level are coupled with the solution of the microscale adhesion model at each integration point of the finite element mesh. In addition, solution of the adhesion model is coupled with the active contractile stress of the cell population. The multiscale model successfully captures the mechanical response of biopolymer fibers and gels, contractile stresses generated by fibroblasts, and stress-strain contours observed during wound healing. We anticipate that this framework will not only increase our understanding of how mechanical cues guide cellular behavior in cutaneous wound healing, but will also be helpful in the study of mechanobiology, growth, and remodeling in other tissues.

Gaussian Process Surrogates for Modeling Uncertainties in a Use Case of Forging Superalloys

The avoidance of scrap and the adherence to tolerances is an important goal in manufacturing. This requires a good engineering understanding of the underlying process. To achieve this, real physical experiments can be conducted. However, they are expensive in time and resources, and can slow down production. A promising way to overcome these drawbacks is process exploration through simulation, where the finite element method (FEM) is a well-established and robust simulation method. While FEM simulation can provide high-resolution results, it requires extensive computing resources to do so. In addition, the simulation design often depends on unknown process properties. To circumvent these drawbacks, we present a Gaussian Process surrogate model approach that accounts for real physical manufacturing process uncertainties and acts as a substitute for expensive FEM simulation, resulting in a fast and robust method that adequately depicts reality. We demonstrate that active learning can be easily applied with our surrogate model to improve computational resources. On top of that, we present a novel optimization method that treats aleatoric and epistemic uncertainties separately, allowing for greater flexibility in solving inverse problems. We evaluate our model using a typical manufacturing use case, the preforming of an Inconel 625 superalloy billet on a forging press.

Bayesian calibration of a computational model of tissue expansion based on a porcine animal model

Tissue expansion is a technique used clinically to grow skin in situ to correct large defects. Despite its enormous potential, lack of fundamental knowledge of skin adaptation to mechanical cues, and lack of predictive computational models limit the broader adoption and efficacy of tissue expansion. In our previous work, we introduced a finite element model of tissue expansion that predicted key patterns of strain and growth which were then confirmed by our porcine animal model. Here we use the data from a new set of experiments to calibrate the computational model within a Bayesian framework. Four 10×10cm2 patches were tattooed in the dorsal skin of four 12 weeks-old minipigs and a total of six patches underwent successful tissue expander placement and inflation to 60cc for expansion times ranging from 1 h to 7 days. Six patches that did not have expanders implanted served as controls for the analysis. We find that growth can be explained based on the elastic deformation. The predicted area growth rate is k∈[0.02,0.08] [h-1]. Growth is anisotropic and reflects the anisotropic mechanical behavior of porcine dorsal skin. The rostral-caudal axis shows greater deformation than the transverse axis, and the time scale of growth in the rostral-caudal direction is given by rate parameters k1∈[0.04,0.1] [h-1] compared to k2∈[0.01,0.05] [h-1] in the transverse direction. Moreover, the calibration results underscore the high variability in biological systems, and the need to create probabilistic computational models to predict tissue adaptation in realistic settings. STATEMENT OF SIGNIFICANCE: Tissue expansion is a widely used technique in reconstructive surgery because it triggers growth of skin for the correction of large skin lesions and for breast reconstruction after mastectomy. Despite of its potential, complications and undesired outcomes persist due to our incomplete understanding of skin mechanobiology. Here we quantify the deformation and growth fields induced by an expander over 7 days in a porcine animal model and use these data to calibrate a computational model of skin growth using finite element simulations and a Bayesian framework. The calibrated model is a leap forward in our understanding skin growth, we now have quantitative understanding of this process: area growth is anisotropic and it is proportional to stretch with a characteristic rate constant of k∈[0.02,0.08] [h-1].

Multifidelity regression of sparse plasma transport data available in disparate physical regimes

Physical data are typically generated by experiments and computations in limited parameter regimes. When datasets generated using such disparate methods are combined into one dataset, the resulting dataset is typically sparse, with dense "islands" in a potentially high-dimensional parameter space, and predictions must be interpolated among such islands. Using plasma transport data as our example, we propose a multifidelity Gaussian-process regression framework that incorporates physical data from multiple sources at multiple fidelities. The impact of the proposed framework varies from little improvement over simpler approaches to qualitatively changing the prediction with consistently increased confidence in regions lacking high-fidelity data. By varying low- and high-fidelity data sources, we demonstrate an approach for determining when multifidelity Gaussian-process regression adds value over single-fidelity regression and therefore when its additional computational costs are merited. We also examine the case in which the outputs of the low- and high-fidelity models correspond to different physical quantities, one of which may be intrinsically computationally cheaper to compute. We conclude by analyzing strategies for sampling high-fidelity data for use in this framework, and we develop a simple sampling approach for reducing regression error across large gaps in data.

Multiscale simulations of left ventricular growth and remodeling

Cardiomyocytes can adapt their size, shape, and orientation in response to altered biomechanical or biochemical stimuli. The process by which the heart undergoes structural changes-affecting both geometry and material properties-in response to altered ventricular loading, altered hormonal levels, or mutant sarcomeric proteins is broadly known as cardiac growth and remodeling (G&R). Although it is likely that cardiac G&R initially occurs as an adaptive response of the heart to the underlying stimuli, prolonged pathological changes can lead to increased risk of atrial fibrillation, heart failure, and sudden death. During the past few decades, computational models have been extensively used to investigate the mechanisms of cardiac G&R, as a complement to experimental measurements. These models have provided an opportunity to quantitatively study the relationships between the underlying stimuli (primarily mechanical) and the adverse outcomes of cardiac G&R, i.e., alterations in ventricular size and function. State-of-the-art computational models have shown promise in predicting the progression of cardiac G&R. However, there are still limitations that need to be addressed in future works to advance the field. In this review, we first outline the current state of computational models of cardiac growth and myofiber remodeling. Then, we discuss the potential limitations of current models of cardiac G&R that need to be addressed before they can be utilized in clinical care. Finally, we briefly discuss the next feasible steps and future directions that could advance the field of cardiac G&R.

Discriminating Gaussian processes via quadratic form statistics

Gaussian processes are powerful tools for modeling and predicting various numerical data. Hence, checking their quality of fit becomes a vital issue. In this article, we introduce a testing methodology for general Gaussian processes based on a quadratic form statistic. We illustrate the methodology on three statistical tests recently introduced in the literature, which are based on the sample autocovariance function, time average mean-squared displacement, and detrended moving average statistics. We compare the usefulness of the tests by taking into consideration three very important Gaussian processes: the fractional Brownian motion, which is self-similar with stationary increments (SSSIs), scaled Brownian motion, which is self-similar with independent increments (SSIIs), and the Ornstein-Uhlenbeck (OU) process, which is stationary. We show that the considered statistics' ability to distinguish between these Gaussian processes is high, and we identify the best performing tests for different scenarios. We also find that there is no omnibus quadratic form test; however, the detrended moving average test seems to be the first choice in distinguishing between same processes with different parameters. We also show that the detrended moving average method outperforms the Cholesky method. Based on the previous findings, we introduce a novel procedure of discriminating between Gaussian SSSI, SSII, and stationary processes. Finally, we illustrate the proposed procedure by applying it to real-world data, namely, the daily EURUSD currency exchange rates, and show that the data can be modeled by the OU process.

Multiscale modeling meets machine learning: What can we learn?

Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learning to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.

Sensitivity analysis of non-local damage in soft biological tissues

Computational modeling can provide insight into understanding the damage mechanisms of soft biological tissues. Our gradient-enhanced damage model presented in a previous publication has shown advantages in considering the internal length scales and in satisfying mesh independence for simulating damage, growth and remodeling processes. Performing sensitivity analyses for this model is an essential step towards applications involving uncertain patient-specific data. In this paper, a numerical analysis approach is developed. For that we integrate two existing methods, that is, the gradient-enhanced damage model and the surrogate model-based probability analysis. To increase the computational efficiency of the Monte Carlo method in uncertainty propagation for the nonlinear hyperelastic damage analysis, the surrogate model based on Legendre polynomial series is employed to replace the direct FEM solutions, and the sparse grid collocation method (SGCM) is adopted for setting the collocation points to further reduce the computational cost in training the surrogate model. The effectiveness of the proposed approach is illustrated by two numerical examples, including an application of artery dilatation mimicking to the clinical problem of balloon angioplasty.

The Geometry of Incompatibility in Growing Soft Tissues: Theory and Numerical Characterization

Tissues in vivo are not stress-free. As we grow, our tissues adapt to different physiological and disease conditions through growth and remodeling. This adaptation occurs at the microscopic scale, where cells control the microstructure of their immediate extracellular environment to achieve homeostasis. The local and heterogeneous nature of this process is the source of residual stresses. At the macroscopic scale, growth and remodeling can be accurately captured with the finite volume growth framework within continuum mechanics, which is akin to plasticity. The multiplicative split of the deformation gradient into growth and elastic contributions brings about the notion of incompatibility as a plausible description for the origin of residual stress. Here we define the geometric features that characterize incompatibility in biological materials. We introduce the geometric incompatibility tensor for different growth types, showing that the constraints associated with growth lead to specific patterns of the incompatibility metrics. To numerically investigate the distribution of incompatibility measures, we implement the analysis within a finite element framework. Simple, illustrative examples are shown first to explain the main concepts. Then, numerical characterization of incompatibility and residual stress is performed on three biomedical applications: brain atrophy, skin expansion, and cortical folding. Our analysis provides new insights into the role of growth in the development of tissue defects and residual stresses. Thus, we anticipate that our work will further motivate additional research to characterize residual stresses in living tissue and their role in development, disease, and clinical intervention.

Exploring the potential of transfer learning for metamodels of heterogeneous material deformation

From the nano-scale to the macro-scale, biological tissue is spatially heterogeneous. Even when tissue behavior is well understood, the exact subject specific spatial distribution of material properties is often unknown. And, when developing computational models of biological tissue, it is usually prohibitively computationally expensive to simulate every plausible spatial distribution of material properties for each problem of interest. Therefore, one of the major challenges in developing accurate computational models of biological tissue is capturing the potential effects of this spatial heterogeneity. Recently, machine learning based metamodels have gained popularity as a computationally tractable way to overcome this problem because they can make predictions based on a limited number of direct simulation runs. These metamodels are promising, but they often still require a high number of direct simulations to achieve an acceptable performance. Here we show that transfer learning, a strategy where knowledge gained while solving one problem is transferred to solving a different but related problem, can help overcome this limitation. Critically, transfer learning can be used to leverage both low-fidelity simulation data and simulation data that is the outcome of solving a different but related mechanical problem. In this paper, we extend Mechanical MNIST, our open source benchmark dataset of heterogeneous material undergoing large deformation, to include a selection of low-fidelity simulation results that require ≈ 2 - 4 orders of magnitude less CPU time to run. Then, we show that transferring the knowledge stored in metamodels trained on these low-fidelity simulation results can vastly improve the performance of metamodels used to predict the results of high-fidelity simulations. In the most dramatic examples, metamodels trained on 100 high fidelity simulations but pre-trained on 60,000 low-fidelity simulations achieves nearly the same test error as metamodels trained on 60,000 high-fidelity simulations (1 - 1.5% mean absolute percent error). In addition, we show that transfer learning is an effective method for leveraging data from different load cases, and for leveraging low-fidelity two-dimensional simulations to predict the outcomes of high-fidelity three-dimensional simulations. Looking forward, we anticipate that transfer learning will enable us to better capture the influence of tissue spatial heterogeneity on the mechanical behavior of biological materials across multiple different domains.

Acceleration of PDE-Based Biological Simulation Through the Development of Neural Network Metamodels

PURPOSE OF REVIEW

Partial differential equation (PDE) mathematical models of biological systems and the simulation approaches used to solve them are widely used to test hypotheses and infer regulatory interactions based on optimization of the PDE model against the observed data. In this review, we discuss the ability of powerful machine learning methods to accelerate the parametric screening of biophysical informed- PDE systems.

RECENT FINDINGS

A major shortcoming in more broad adaptation of PDE-based models is the high computational complexity required to solve and optimize the models and it requires many simulations to traverse the very high-dimensional parameter spaces during model calibration and inference tasks. For instance, when scaling up to tens of millions of simulations for optimization and sensitivity analysis of the PDE models, compute times quickly extend from months to years for sufficient coverage to solve the problems. For many systems, this brute-force approach is simply not feasible. Recently, neural network metamodels have been shown to be an efficient way to accelerate PDE model calibration and here we look at the benefits and limitations in extending the PDE acceleration methods to improve optimization and sensitivity analysis.

SUMMARY

We use an example simulation to quantitatively and qualitatively show how neural network metamodels can be accurate and fast and demonstrate their potential for optimization of complex spatiotemporal problems in biology. We expect these approaches will be broadly applied to speed up scientific research and discovery in biology and other systems that can be described by complex PDE systems.